[05:06] <Mirv> vila: I didn't get your 'why not a regular MP', when I was suggesting to disable the radeon from the config. you should do the merge proposal against the config but you need to deploy the change regardless of the bzr change.
[05:08] <Mirv> josepht: I don't think you need me to check whether something is running on radeon.. for jobs I guess it's ok if it's still deployed, just look it's idle at http://q-jenkins.ubuntu-ci:8080/view/cu2d/view/Head/? and if you need it for investigations, remove it from the lp:cupstream2distro-config first.
[06:31] <vila> Mirv: because I found it weird that you didn't file the MP. There are enough things that are not (yet!) under version control and are therefore hard to track, to add confusion by not using version control for stuff that is already using it ;)
[06:31] <vila> Mirv: now, what about:
 Mirv: 'long annoying detail of the next job breaking after abort' as in this problem has been know  ? Since when ?
[06:34] <Mirv> vila: I didn't file a MP because I left it for you to decide whether the problems can be solved or if the radeon machine needs to be taken off from rthe grid
[06:34] <Mirv> I just thought that if you need the diff, I'll offer it
[06:35] <Mirv> vila: that problem of error-after-abort has been seen before, but just not fixed. not really sure when it was first noticed.
[06:37] <vila> Mirv: ack, then I've good news, expect it to soon be a thing of the past: http://q-jenkins.ubuntu-ci:8080/job/otto-test-radeon/label=qa-radeon-7750/61/console
[06:37] <vila> Mirv: I'm not fully awake yet and have an urgent dentist appointment to attend 1h30 from now
[06:38] <Mirv> vila: nice!
[06:38] <vila> Mirv: I'll check where this fix need to be deployed then
[06:38] <Mirv> vila: ok, have "fun" there
[06:43] <vila> Mirv: and then I think we need to discuss how to interact around this kind of issues, otto node provisioning is too complex to handle given the occurrence of these incidents. Who is responsible for them is unclear, I think you'll be better served if you could manage them yourself but at the same time there are cases where the ci team need to take control. I don't think this case was one of them.
[06:44] <Mirv> ok
[06:44] <vila> Mirv: so for the next hours, don't abort jobs ;) But lower the timeout ?
[06:46] <vila> Mirv: or as a really dirty trick, if you get stuck, run the job I pointed above, it will stop the container
[06:49] <Mirv> ok, testing running something then using the test job if needed
[07:21] <jibel> Mirv, vila OOC, why do you need to abort jobs?
[07:32] <vila> jibel: Mirv needs to when they are stuck and the timeout is too long for him, hence the advice to reduce it (dunno who set it to 330 mins...)
[07:46] <jibel> vila, but why are they stuck? It usually means something went wrong isn't it?
[07:52] <vila> jibel: AFAIK in this case, the Xserver inside the container crashed, and I think aborting is the wrong idea as it means the Xorg.0.log file is not collected which means Mirv can't provide it upstream
[07:52]  * vila dentist &
[07:53] <jibel> vila, ah, okay, maybe this condition could be checked in the runner and exit when X dies as there is no point in running tests without it
[07:58] <jibel> vila, and I agree 330min is a very long timeout, it used to be 2hours because unity testsuite takes 90min
[08:05] <didrocks> IIRC, it was extended for the Mir guys to debug without needing us to prevent the timeout, I guess it can go back to 2h + the fix jibel mentionned about detecting X
[08:22] <jibel> didrocks, vila on the testbed you could run in background something like "while sleep 60; do pidof X || do_something_to_terminate_cleanly; done"
[08:22] <didrocks> yeah
[08:22] <jibel> or a more elaborate version of that of couse :)
[08:23] <Mirv> well I'm confirming the radeon still breaks, so I'll now remove its usage from config
[08:24] <vila> jibel, didrocks: right, tests should check for X and fail otherwise. That's a more specific fix that the one I'm investigating right now which is: something went wrong, clean the place. And I want that later one in place in any case as it will catch other, yet unknown, failures.
[08:24] <Mirv> (http://q-jenkins.ubuntu-ci:8080/job/autopilot-trusty-daily_release/label=qa-radeon-7750/576/console - X.org probably crashed if one would look at the logs)
[08:27] <vila> Mirv, didrocks: now I'd like to understand what your policy is regarding otto nodes availability. -nvidia absence was a blocker, now qa-radeon presence is a blocker
[08:28] <vila> going with the logic that -nvidia was a blocker as specific failures won't be caught, it seems to me that -radeon should be kept especially when it's not failing for all suites
[08:28] <Mirv> https://code.launchpad.net/~timo-jyrinki/cupstream2distro-config/remove_qa-radeon-7750/+merge/196494
[08:29] <Mirv> vila: we would need all three to be functional in order to get really trustworthy results, also in case one of them breaks we don't want it to be the last one. but having a broken machine enabled blocks more than having 2/3 working.
[08:30] <Mirv> vila: if there'd be a list of stacks it works at every time, then it could be enabled selectively of course. but the crashes might be random too.
[08:30] <didrocks> vila: I mean I want at least 2
[08:30] <didrocks> vila: one isn't enough
[08:30] <Mirv> I proposed now https://code.launchpad.net/~timo-jyrinki/cupstream2distro-config/remove_qa-radeon-7750/+merge/196494 which would remove radeon from all of the stacks for now
[08:31] <vila> Mirv: good, that's what I thought for -nvidia so happy to see some convergence there
[08:33] <vila> Mirv: 'random crashes' indeed is what this catch-all fix is about, needs a bit of refinement as it assumes there is a single container running (and I want a clear error message otherwise)
[08:34] <vila> Mirv, didrocks: only autopilot-*-daily_release jobs ever run otto tests right ?
[08:34] <didrocks> vila: I guess there is another one that jibel put in place for community apps ^
[08:34] <vila> didrocks: on q-jenkins ?
[08:35] <didrocks> IIRC, yeah
[08:35] <vila> didrocks: ack, thanks, I'll followup with fginther about that then
[08:35] <jibel> vila, that was a project to run autopilot tests of desktop application written by the community, but since the move to a queternourly release, there are no slot available anymore to run them
[08:35] <didrocks> thanks
[08:35] <jibel> I'll move that to VMs and hand it over to CI
[08:41] <vila> hioh, since you mention VMs, sorry, never found the time to get back to you about 'vga ??? not supported' remark you made when I talked about pass through
[08:42] <vila> jibel: I'm not sure we're on the same page there, to me, pass through refers to the feature where the host *ignores* the graphics card and let the kvm fully handles it (crashes included). Is that what you were referring to or something different ?
[08:44] <vila> Mirv: timeout set to 120 for now but if you feel you're still  wasting too much time, lower it again.
[08:46] <Mirv> vila: ok, thanks. where it's set, I haven't known that?
[08:47] <vila> Mirv: oh very very sorry, http://q-jenkins:8080/job/autopilot-trusty-daily_release/configure
[08:48] <vila> Mirv: build environment / Absolute / Timeout minutes
[08:49] <didrocks> ogra_: please, kick an image as soon as you can
[08:51] <vila> Mirv: apologies, I shouldn't have assume you knew where that timeout was defined :-( Better repeat assumed knowledge rather giving incomplete hints :-( Painful for all involved in the end...
[08:54] <vila> didrocks: about 'it was extended for the Mir guys to debug without needing us to prevent the timeout', as in: so they can access the otto node while the job was running/suspended but the container still accessible ?
[08:55] <didrocks> vila: they had random races which prevented the session to start
[08:55] <didrocks> vila: so yeah, they wanted to dive in when this happened
[08:55] <didrocks> and most of the time, we noticed after 2 hours
[08:55] <didrocks> so too late, nearly the end of the timeout
[08:56] <vila> didrocks: ack, so in the longer term, we want to give them a way to reproduce that on a different otto node (ideally on *their* machine) so we don't have to interfer with production right ?
[08:56] <didrocks> vila: exactly
[08:57] <didrocks> vila: but as it's racy and hw dependant, we need a way to stress it to reproduce
[08:57] <didrocks> with the same hw
[08:58] <vila> didrocks: yeah, tricky balance to find...
[09:01] <vila> didrocks: but let's start with a solution that preserve production and only fallback to de-provisioning as a last resort (not to mention the landing and ci teams need a lighter process than filing a MP, land it, deploy it)
[09:01] <didrocks> vila: waiting eagerly on your patches :)
[09:02] <vila> didrocks: priorities are 1) fixing 1ss move, 2) ci airline early target(s), 3) the rest AFAIU
[09:03] <vila> didrocks: so stopping the container after the timeout sounds like the best trade-off in the short term
[09:04] <didrocks> vila: I guess it's already what happens (stopping after the timeout)
[09:04] <didrocks> just change the timeout to 2 hours
[09:04] <vila> didrocks: and tuning the timeout to it slowest possible value should also give you faster feedback
[09:04] <vila> didrocks: yes, that's why I say: don't abort !
[09:05] <didrocks> vila: it doesn't abort, it's stopping the container
[09:05] <vila> didrocks: the issue is a container still running when the job is aborted and that happened several times in the last days no ?
[09:06] <didrocks> vila: yeah, but that's not linked to the timeout
[09:06] <didrocks> at least, not to: "vila | didrocks: so stopping the container after the timeout sounds like the best trade-off in the short term"
[09:06] <didrocks> as I said, what you told just above is already done ^
[09:06] <didrocks> another issue is what you work from Friday if I understood correctly
[09:06] <vila> didrocks: yes, but that assumes that nobody abort jobs
[09:06] <didrocks> which is:
[09:07] <didrocks> "if someone abort the jobs, shutdown the container"
[09:07] <didrocks> (nothing to do with timeout)
[09:07] <vila> didrocks: meh, the link between the timeout and somebody aborting the job is that if the timeout is short enough nobody *can* abort the job
[09:08] <didrocks> vila: yeah, so just move the timeout to 2h
[09:08] <didrocks> as we need 90 minutes for the unity7 tests
[09:08] <vila> didrocks: already done
[09:08] <didrocks> so, you're back to what we had initially
[09:08] <didrocks> same functionality, same bugs :)
[09:08] <vila> didrocks: can you guarantee that nobody will abort jobs ? If not I'd rather have a catch-all in place
[09:09] <didrocks> vila: ok, so there is no link if you want to add the safety net
[09:09] <didrocks> between the timeout and this "fix abort" case
[09:10] <vila> didrocks: don't know how to explain it....
[09:11] <vila> didrocks: people have been aborting jobs
[09:11] <didrocks> well, it's easy:
[09:11] <didrocks> - the timeout was too long
[09:11] <jibel> vila, this "jenkins cannot kill tasks it doesn't own" has been there forever, and is not specific to release testing
[09:11] <didrocks> -> just REVERT to 2h, no need for a week to discuss that, it was for a Mir race case, we didn't restore it, just do it…
[09:11] <didrocks> (and discussion closed on that one)
[09:11] <didrocks> - if someone aborts the job, we're screwed
[09:12] <didrocks> -> yeah, valid one, known from the starts, we either decide to fix it or leave it alone
[09:12] <jibel> vila, so if you can fix it for every thing running in jenkins go ahead but it's orthogonal to the daily release IMO
[09:12] <vila> jibel: thanks, didn't know if that was linked to 1ss move or not
[09:12] <didrocks> no need to have discussions on those for days :)
[09:13] <vila> didrocks: but I told you the timeout has been fixed, since you keep discussing I thought you had something else on your mind
[09:13] <didrocks> vila: no, just "the other issue isn't really link if you want a real fix for it"
[09:13] <didrocks> even if I understand we'll get it less, as we did at first
[09:15] <vila> didrocks: right, so several potential fixes have been tried, discussed and agreement found, I came up with a new one yesterday evening and told Mirv this morning. I think that's where we stopped being on the same page ;)
[09:15] <didrocks> I don't have the start of the discussion, what was the new potential fix proposal?
[09:16] <vila> didrocks: I realize there are other fixes, better targeted, but they'll require more time. If you're fine with ensuring the jobs are not aborted anymore, I'm fine leaving that ticking bomb at peace for now.
 Mirv: ack, then I've good news, expect it to soon be a thing of the past: http://q-jenkins.ubuntu-ci:8080/job/otto-test-radeon/label=qa-radeon-7750/61/console
[09:18] <didrocks> vila: how can we ensure?
[09:19] <vila> didrocks: ensure what ? that people won't abort jobs or that we'll stop the container if they do ?
[09:19] <didrocks> vila: I don't know, just repeating your "If you're fine with ensuring the jobs are not aborted anymore" -> how do you do that?
[09:20] <vila> didrocks: that's the question you didn't answer ;)
 didrocks: can you guarantee that nobody will abort jobs ? If not I'd rather have a catch-all in place
[09:21] <didrocks> vila: well, it's a rethorical question I guess… the answer is obvisouly no
[09:21] <didrocks> vila: so, you are going to work on a fix?
[09:22] <vila> didrocks: look at that url above and you'll find a crude fix in the post build part of the job which I need to refine slightly
[09:23] <didrocks> vila: not sure what that means, apart that you detect a container running
[09:27] <vila> didrocks: that's the two encountered cases where a job can't run because the previous one was aborted.
[09:28] <vila> didrocks: or rather, one is when the previous one was aborted, the other is when the current job has started a container and *may* have left it running
[09:28] <didrocks> vila: ok
[09:30] <sil2100> Joining in a moment
[09:33] <sil2100> Damn, hangouts is taking ages today... some plugin problems
[09:35] <sil2100> Ok, getting really irritated here
[09:37] <ogra_> [09:38] <sil2100> Now I have no video from anyone
[09:51] <Mirv> so, radeon was still running after deploying config changes. let's see if it helps now that I removed it also from the daily_release job
[09:55] <ev> jibel: I just closed our task tracking drude and rabisu. Can you confirm that they're working as expected?
[09:55] <ev> (closed because I can confirm they're up)
[09:55] <Mirv> ok everyone FYI, now it should be finally possible to run stacks sanely again and even get autopilot results
[10:13] <jibel> ev, they're working. I reopened the RT because FW rules are incorrect.
[10:14] <jibel> ev, I fixed the problem with the publisher, it is due to an incompatibility in matrix jobs configurations between 1.424 (version before the move) and 1.480, I added info on asana so Larry can notify other people using matrix jobs on d-jenkins
[10:15] <jibel> 1.424, 1.480 = versions of jenkins
[10:15] <jibel> ev, and there is still a ticket open fo the scheduler not working on d-jenkins
[10:16] <jibel> ev, but I think it's a bug in jenkins when a request for shutdown is cancelled and can only be fixed with a restart
[10:17] <ev> jibel: awesome, thank you for all of that!
[10:49] <Saviq> ev, hey, seems the otto runner is b0rked somehow https://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-trusty/957/console
[10:50] <Saviq> ev, twice today it complained about being unable to start the lxc container
[10:51] <ogra_> [10:54] <ev> otp for another 30-45min
[11:09] <popey> wow, when did we add the sound of a 1920's typewriter to the keyboard?
[11:22] <popey> didrocks: RSS Reader is working mostly fine, found a bug and filed it, File manager seems to be an AP issue, most issues relating to pop overs.
[11:23] <didrocks> popey: ok, keep me posting on all other discoveries :)
[11:23] <didrocks> thanks ;)
[11:23] <popey> will do
[11:23] <popey> http://popey.com/~alan/phablet/device-2013-11-25-112323.png is fun
[11:23] <popey> phone got stuck between two scopes
[11:26] <ev> ^ vila is the problem Saviq is experiencing further fallout from what we just discussed on the phone?
[11:27] <Saviq> popey, unity8 crashed?
[11:27] <popey> no
[11:27] <popey> it just got wedged a bit when I searched in the home then while results returned I swiped across
[11:28] <popey> after I took that pick I swiped and it locked on to the scope fine
[11:28] <Saviq> popey, if you can reproduce, please bug
[11:28] <popey> s/pick/pic/
[11:28] <popey> yeah, trying
[11:48] <vila> ev: not exactly and this one seems to be s-jenkins, not q-jenkins
[11:48] <ev> oh right
[11:48]  * ev digs
[12:07] <popey> Saviq: managed to do it again, not quite sure how, will file a bug
[12:10] <popey> Saviq: bug 1254693
[12:10] <popey> Saviq: lemme know if you need any logs off the phone
[12:13] <Saviq> popey, logs likely won't help, steps to reproduce would be good if you find them
[13:05] <ogra_> xnox, did you happen to test the libc6 changes on touch ? (my client disconnected, havent seen the outcome of the conversation on the weekend)
[13:07] <xnox> ogra_: no i did not. plus i don't have mako/maguro.
[13:07] <ogra_> xnox, grouper is a target this cycle ;)
[13:07] <ogra_> ok, then i'll do that testing later today
[13:14] <Mirv> so what was again the status with the unity8 problem?
[13:14] <Mirv> (crasher)
[13:15] <Mirv> I'm wondering whether I could push the mir builds to the PPA or if unity8 needs to be built still which would pull unwanted dependencies in case I'd push for mir
[13:16] <Mirv> either way I can probably work on mir tomorrow morning before the meeting
[13:23] <didrocks> Mirv: oh, you didn't release Mir yet?
[13:27] <Mirv> didrocks: nope, not enough time for today unfortunately to run all the tests anyhow. I'm just thinking whether there was any problem in preparing the PPA now so that I can run tests tomorrow morning.
[13:28] <sil2100> didrocks: and I almost forgot the publish button! Can you ACK a packaging change? http://q-jenkins.ubuntu-ci:8080/view/cu2d/view/Head/view/QA/job/cu2d-qa-head-3.0publish/lastSuccessfulBuild/artifact/packaging_changes_autopilot_1.4+14.04.20131125-0ubuntu1.diff
[13:29] <vila> didrocks, Mirv: catch-all to stop containers on aborted jobs is now in place for http://q-jenkins:8080/job/autopilot-trusty-daily_release/
[13:29] <didrocks> Mirv: good for me to prepare the ppa
[13:30] <didrocks> vila: great!
[13:30] <Mirv> didrocks: thanks. one more thing, is this current manual stack run intentional and is it planned to change now that I disabled radeon again?
[13:30] <Mirv> (ie build_all disabled)
[13:30] <didrocks> sil2100: +1
[13:30] <didrocks> Mirv: you can enable it again I guess
[13:31] <Mirv> ok, enabling
[13:32] <Mirv> updated the stack status page as well
[13:44] <didrocks> thanks Mirv!
[13:57] <fginther> morning
[13:59] <didrocks> hey fginther
[13:59] <fginther> didrocks, morning
[14:03] <dobey> didrocks, lool: do i need to do a landing ask to add a pkg-config file to a -dev package as well, for a source that has some binaries on the touch image? (the -dev package itself isn't on the image of course, and the other binary packages will not be changed)
[14:06] <didrocks> dobey: if nothing else change and you test a rebuild didn't bring anything/change anything, please go ahead
[14:06] <dobey> ok, great. thanks
[14:23] <cjwatson> vila: ddeb -> package containing debug symbols, currently hosted on ddebs.ubuntu.com
[14:23] <cjwatson> we strip those off executables automatically during build
[14:23] <cjwatson> apport-retrace makes use of them when people upload crash reports containing core dumps
[14:24] <cjwatson> or people can run apport-retrace manually to diagnose issues, or even install the ddebs directly
[14:24] <vila> cjwatson: oh ! debug debs, why are they involved for ci ?
[14:24] <cjwatson> because we're copying packages into the primary archive, and therefore ideally need to make sure ddebs are available for those copied binaries
[14:25] <cjwatson> unfortunately ddebs are implemented with a giant hack on the backend right now so that's tricky
[14:25] <cjwatson> the work to fix that has been planned for a while and about 90% implemented
[14:25] <cjwatson> I think it's currently blocked on moving the librarian to swift
[14:25] <vila> cjwatson: oooh, so you want to make sure debs are not blindly copied without their ddeb ?
[14:26] <cjwatson> right, and also: ddebs need to be created in the first place
[14:26] <cjwatson> which can only be done while building the debs
[14:26] <vila> cjwatson: right, ok
[14:26] <cjwatson> wgrant has done a good deal of work on the copy logic etc.
[14:27] <Saviq> ev, any word on the failing containers?
[14:27] <ev> ^ josepht should be able to update you
[14:27] <Saviq> ev, thanks
[14:28] <josepht> Saviq: I'm still looking at it.  It seems jenkins has a gnome-session open but I haven't tracked down exactly why yet.
[14:29] <popey> didrocks: calendar app failing in #29, run tests locally and it fails here too, but the app itself works. so probably ap issue.
[14:29] <Saviq> josepht, thanks
[14:30] <popey> paging balloons.. http://ci.ubuntu.com/smokeng/trusty/touch/mako/29:20131125:20131125/5091/calendar-app-autopilot/ - these failures. can someone take a look at them? Do you need a bug filed?
[14:40] <didrocks> popey: tell him "no thanksgiving for you if not fixed by TOMORROW" ;)
[14:41] <popey> hah
[14:41] <didrocks> popey: I heard there is one every year, they can get into next year one :)
[14:42] <popey> they have way too many holidays
[14:42] <popey> as do you lot!
[14:42]  * popey lumps "europeans" together
[14:42]  * popey is of course just jealous ㋛
[14:47] <balloons> popey,
[14:47] <balloons> popey, https://code.launchpad.net/~nskaggs/ubuntu-calendar-app/fix-new-event-test/+merge/195421
[14:47] <balloons> it was fixed last week
[14:47] <balloons> needs to get pushed to the images
[14:48] <balloons> probably end up with a whole sleuth of stuff to push in again
[14:48] <popey> good stuff
[14:48] <popey> will poke sergio when he is around
[14:48]  * popey hugs balloons 
[14:49]  * balloons peeks and sees what else isn't landed
[14:55] <josepht> Saviq: it looks like it was a gpu lockup
[14:56] <Saviq> josepht, ok, will try to kick a job
[14:56] <popey> balloons: lets get the full list of things that need updating and I'll test them for sergio so we can get them in the next image
[14:57] <josepht> Saviq: wait, I think I'll need to have someone kick the machine first :)
[14:57] <josepht> rfowler: ping
[14:57] <Saviq> josepht, ok
[15:11] <didrocks> balloons: can you check with sergio maybe about those?
[15:11] <balloons> didrocks, yes, popey and I will work with sergio and land stuff :-)
[15:12] <didrocks> balloons: can you tell me once it's done? (even by email)
[15:12] <balloons> popey rejected the fixes last time :-)
[15:12] <didrocks> so that I know what we can expect in the next images
[15:12] <balloons> didrocks, sure, I'll email
[15:12] <didrocks> bad bad popey!
[15:12] <didrocks> ;)
[15:12] <didrocks> thanks
[15:13] <didrocks> ogra_: so, I discussed with asac, and he agrees on the "promote now"
[15:13] <ogra_> ok
[15:14] <didrocks> ogra_: is everything good from your perspective on image 28?
[15:14] <ogra_> just wanted to make sure we dont bypass him
[15:14] <didrocks> ogra_: yeah, talk to various people and they agree I can tell the decision in the future, but let's see as well if asac agrees once back
[15:14] <didrocks> ogra_: let's way for final popey's feedback then and publish if ok
[15:14] <didrocks> wait*
[15:18] <ogra_> didrocks, hmm, the indicators dont actually match the icons ... if i switch to bluetooth in the indicator header, the messaging icon is highlighted
[15:19] <didrocks> ogra_: can you file that as a bug?
[15:19] <ogra_> seems the caption odering doesnt match the icon odering here
[15:19] <didrocks> I saw that in the old days already
[15:19] <ogra_> functionally it seems to be ok though
[15:19] <didrocks> but couldn't reproduce it reliably
[15:19] <didrocks> so maybe just a race
[15:20]  * ogra_ reboots the phone to see if it is still there afterwards
[15:20] <didrocks> ogra_: yeah, if now it happens 100% of the time, I would see that as an improvement (relatively speaking :p)
[15:21] <didrocks> but at least, we'll have a way to fix it more easily ;)
[15:21] <ogra_> definitely reliable
[15:21] <didrocks> it's weird, but I would "\o/"
[15:21] <ogra_> see #ubuntu-touch
[15:21] <ogra_> seems there is even already a bug
[15:22] <didrocks> ok, nice! at least, we'll be able to kill it definitively :)
[15:25] <popey> didrocks: 28 is okay from my perspective
[15:26] <didrocks> popey: great!
[15:26] <ogra_> same here
[15:26] <didrocks> ogra_: please promote, I have an email ready (just need popey to ack it, it's a little bit special and I want to have it right ;))
[15:26] <josepht> rfowler: unping
[15:27] <josepht> Saviq: okay, you are clear to kick a job
[15:27] <Saviq> josepht, thanks
[15:28] <Ursinha> ogra_, didrocks, do you monitor regression bugs whenever checking if a image is good to go or not? The bugs I filed were tagged regression
[15:28] <didrocks> Ursinha: I monitor with tabs
[15:28] <Ursinha> including the indicator bug
[15:29] <ogra_> [15:30] <didrocks> \o/
[15:31]  * didrocks sends email
[15:31] <didrocks> Ursinha: I don't track the tags, but bugs directly TBH
[15:31] <rsalveti> is 28 better than the previous one, even with a few regressions?
[15:32] <rsalveti> in theory we don't want to promote something that we know that has regression
[15:32] <ogra_> rsalveti, it a lot worse
[15:32] <ogra_> *it's
[15:32] <rsalveti> then why did we promote it?
[15:32] <ogra_> http://ci.ubuntu.com/smokeng/trusty/touch/
[15:32] <ogra_> image 10 was the last one ...
[15:32] <ogra_> 28 was just promoted
[15:33] <rsalveti> right, but are you using it as your daily phone?
[15:33] <ogra_> rsalveti, because people are starting to nag i guess
[15:33] <rsalveti> don't push images that are known to be broken
[15:33] <didrocks> rsalveti: it's not worse, the test failures are false positives
[15:33] <rsalveti> it's a pita
[15:33] <rsalveti> Ursinha is using as her daily phone
[15:33] <ogra_> didrocks, it is full of crashes
[15:33] <didrocks> (or negatives)
[15:33] <didrocks> ogra_: the crashes are on exit
[15:33] <Ursinha> didrocks, I found a couple regressions I filed, plus some I still need to file...
[15:33] <didrocks> as discussde this morning
[15:33] <didrocks> discussed
[15:34] <rsalveti> and I stopped counting the amount of times Ursinha complained that something stopped working
[15:34] <rsalveti> and 10 was indeed better
[15:34] <rsalveti> please, don't promote broken stuff
[15:34] <didrocks> Ursinha: the indicator one was a race that happened before (in image 10), I'm not aware/no one mentionned other regressions before we promoted
[15:34] <didrocks> did we?
[15:34] <ogra_> rsalveti, thats why i asked for asac approval for it
[15:34] <didrocks> and asac +1 on those premise
[15:34] <ogra_> i didnt feel like promoting it
[15:34] <rsalveti> well, we all need to agree when to promote something
[15:34] <rsalveti> and not wait for asac to decide
[15:35] <ogra_> rsalveti, clear statement was equal or better than 10
[15:35] <rsalveti> right, and it's not equal, it's worse it seems
[15:35] <ogra_> we didnt have any such image in the last two weeks
[15:35] <Ursinha> ogra_, if it has a regression can I consider it worse?
[15:35] <ogra_> Ursinha, yes
[15:35] <Ursinha> ogra_, so........
[15:35] <rsalveti> still, we don't want to break people dogfooding it
[15:36] <rsalveti> remember, we have people using dogfooding it
[15:36] <didrocks> so, do you have a real list of regressions and why weren't they brought up there?
[15:36] <ogra_> Ursinha, dont look at me ... i wouldnt hve promoted it ... we explicitly asked asac for approval
[15:36] <rsalveti> didrocks: are you using it as your daily phone?
[15:36] <didrocks> rsalveti: I do use it
[15:36] <Ursinha> didrocks, that was my question :) I've been filing bugs and being careful enough to add the regression tag to it so they can be found
[15:36] <rsalveti> didrocks: with trusty?
[15:36] <Ursinha> if they're not, we need to do something about it, because depending on nagging people to have it looked at doesn't scale
[15:36] <didrocks> Ursinha: do you have a link across launchpad, I told I'm tracking bugs that was up here
[15:36] <didrocks> rsalveti: yep
[15:37] <rsalveti> and would you like to use/have new regressions?
[15:37] <didrocks> rsalveti: but I'm using -proposed
[15:37] <rsalveti> or would you like to have a working phone?
[15:37] <rsalveti> right
[15:37] <Ursinha> didrocks, you can have a list of bugs tagged regression for a list of projects, if you need I can craft an script for you that does that
[15:37] <rsalveti> yeah, we need a list of regressions asap
[15:37] <didrocks> rsalveti: again, a list please, all what I need
[15:37] <rsalveti> we shouldn't accept regressions
[15:37] <rsalveti> that's the only way to move forward
[15:37] <Ursinha> we can't consider all bugs tagged regression because that would bring lots of unrelated bugs, but we need to be sure that all the projects important to us are being checked for regression bugs
[15:37] <Ursinha> I can do that
[15:38] <didrocks> from my experience, I just heard here about "the indicator one" that I can reproduce here
[15:38] <Ursinha> didrocks, exactly
[15:38] <didrocks> which I saw before
[15:38] <rsalveti> Ursinha: dogfooding-regression?
[15:38] <didrocks> but only once
[15:38] <didrocks> so it seems a race becoming reliably failing
[15:38] <rsalveti> I was able to reproduce it with 28
[15:38]  * ogra_ too
[15:38] <rsalveti> and we have a bug already, ogra_ as well
[15:38] <rsalveti> so why did we promote 28 hahah
[15:38] <Ursinha> didrocks, I'm trying to create a way of tagging all important bugs in a way no reported and confirmed bugs slip
[15:39] <Ursinha> rsalveti, it was there on 27 already
[15:39] <didrocks> Ursinha: that would be excellent, so that we have a direct access on that list
[15:39]  * ogra_ admits he didnt test r27 ... the tst results looked so bad it didnt look worth it ... 
[15:39] <ogra_> (thugh r28 doesnt actually look much better)
[15:39]  * Ursinha kicks the hell out of ogra_ 
[15:40] <rsalveti> but worse than 10
[15:40] <didrocks> *again*, popey confirmed the test results are false positives
[15:40] <ogra_> yes
[15:40] <popey> hmm?
[15:40] <didrocks> he tried them manually, all after another
[15:40] <ogra_> popey, its all your fault :P
[15:40] <popey> #blamepopey
[15:40] <didrocks> :)
[15:40] <Ursinha> didrocks, I can do that, only need to know the projects that are important to us that should be tracked
[15:40] <Ursinha> all projects and avengers wiki page and what else?
[15:41] <didrocks> Ursinha: well, I guess everything in the phone, so it's a big list :)
[15:41] <didrocks> from lihybris to Unity8
[15:41] <didrocks> or we need a metaproject
[15:41] <didrocks> like affect phone
[15:41] <Ursinha> hmm
[15:41] <ogra_> everything thats seeded
[15:42] <Ursinha> ogra_, touch or all images?
[15:42] <ogra_> touch for us
[15:42] <Ursinha> we need 1) list of packages, 2) list of upstream projects
[15:42] <ogra_> no idea if anyone actuvely looks at desktop regressions that way
[15:42] <Ursinha> there are bugs reported against the projects but not the packages in ubuntu
[15:42]  * ogra_ assumes desktop people know their packages OOTB
[15:42] <davmor2> ogra_: it's always popey 's fault
[15:42] <ogra_> Ursinha, that breaks our bug filing policy
[15:43] <Ursinha> the packages in desktop (and other teams) can be tracked by subscribing a team to them, you have the list of affected packages in that way
[15:43] <Ursinha> but as I said, not only packages need to be tracked, but also projects
[15:43] <Ursinha> ogra_, what's the bug filing policy?
[15:43] <ogra_> Ursinha, bugs *need* to be filed against the ubunu packages ... preferably using ubuntu-bug ... filing them additionally against upstream projects is optionsl
[15:43] <ogra_> *optional
[15:43] <rsalveti> well, we only need to track (from the touch perspective) the packaged affecting the touch seeds
[15:44] <Ursinha> ogra_, okay, so what's broken isn't the policy, but people that are filing bugs need to do that when filing bugs
[15:44] <ogra_> we only care for the bugs in the ubuntu archive .. which means they nneed to be filed against the packages not the project
[15:44] <rsalveti> right
[15:44] <Ursinha> I'm only tracking what exists today, there are people that file bugs only against the upstream projects...
[15:44] <rsalveti> we need to work on disabling the bug tracking system for the upstream projects
[15:44] <rsalveti> and only use the package one
[15:44] <didrocks> rsalveti: +1
[15:44] <Ursinha> or
[15:44] <didrocks> it all just brought confusion
[15:44] <ogra_> rsalveti, well, not really ...
[15:44] <dobey> didrocks: just dput ubuntuone-credentials with the aforementioned .pc file addition; thanks
[15:44] <Ursinha> we need to tell people to file bugs against packages as well
[15:45] <didrocks> dobey: yw, thanks to you!
[15:45] <rsalveti> Ursinha: that doesn't work reliably
[15:45] <ogra_> rsalveti, upstream projects should still be able to use a bugtracker for i.e. wishlist bugs etc
[15:45] <rsalveti> people we then depend on people :-)
[15:45] <Ursinha> rsalveti, so you're suggesting people to file two bugs, one against upstream and another against the package?
[15:45] <rsalveti> ogra_: right, if they want, sure
[15:45] <ogra_> Ursinha, s/as well//
[15:45] <Ursinha> that's what I'm saying, launchpad happily supports that already
[15:45] <rsalveti> ogra_: but I believe we can disable most of them (for the ones we are the upstream)
[15:46] <rsalveti> as we want trunk to always be reflected in the distro
[15:46]  * ogra_ is sure asac wrote a mail about bug filing policies
[15:46] <dobey> the biggest problem i've had with bug filers, is that they file bugs directly rather than using ubuntu-bug
[15:47] <didrocks> rsalveti: just to be clear, I didn't revert the status on the bug, I just renamed the project (and conflicting edits)
[15:47] <dobey> and that's even harder on touch as there's no good way to "report a bug" against a specific package
[15:47] <ogra_> dobey, filing directly is fine as long as the bugs go into the right pocket ...
[15:48] <rsalveti> didrocks: yup, mid-air collision
[15:48] <ogra_> dobey, jzst add +filebug to the url ...
[15:48] <dobey> ogra_: it's not fine, because the reporter tends to not provide valuable information that ubuntu-bug does provide
[15:48] <ogra_> not rocket science
[15:48] <Ursinha> dobey, ogra_ , we need to have a clear policy on how to report these bugs that affect touch packages/upstreams
[15:48] <Ursinha> I'd be happy to do whatever I'm told, but it needs to be told somehow :)
[15:49] <ogra_> Ursinha, eth clear policy is "file against the package in ubuntu, dont file against upstream projects"
[15:49] <ogra_> wether you use ubuntu-bug or not, it needs to go against the package in any case
[15:49] <rsalveti> policy might not work as we expect
[15:49] <rsalveti> people will still use either the package or the upstream project
[15:50] <rsalveti> as we depend on people doing the right thing
[15:50] <Ursinha> ogra_, that's not what https://wiki.ubuntu.com/Avengers says
[15:50] <rsalveti> and that never works as expected :-)
[15:50] <Ursinha> all the filebug links go to the upstream packages, it seems
[15:50] <dobey> it's fine if the bug report contains all the information, filed by someone who knows what they're doing. but random person filing a bug doesn't tend to do that
[15:51] <ogra_> Ursinha, sigh, and one cant even complain about that crap to them because the ML is invite only
[15:51] <Ursinha> popey, ^
[15:52] <popey> hmm?
[15:52] <Ursinha> popey, avengers wiki page points people to file bug against upstream projects, not ubuntu packages
[15:52] <Ursinha> and according to ogra_ that's not the official policy
[15:53] <ogra_> popey, that bug tracking crap poolicy https://wiki.ubuntu.com/Avengersmakes us miss all distro bugs with the tools we use
[15:53] <Ursinha> and maybe because of that my regression bug was missed
[15:53] <popey> ok, it's a wiki, people can edit it if it's wrong
[15:53] <popey> i dont see a problem
[15:53] <ogra_> popey, so you mean i should just delete 90% of the page ?
[15:53] <Ursinha> popey, the problem is if that's is there there's a reason, we won't remove all we think it's wrong before discussing that with people that created that page and the process
[15:53] <ogra_> this is just crap ... we will miss all bugs
[15:53] <popey> delete?
[15:54]  * ogra_ wonders why ev didnt scream out loud 
[15:54] <ogra_> they wont end up on errors.u.c ever
[15:54] <popey> why not edit it, link to the right pages?
[15:54] <didrocks> popey: "exterminate" :)
[15:54] <Ursinha> popey, read my last message :)
[15:54] <ogra_> popey, right, i would just replaces it with a redirect to distro bug tracking policies
[15:54] <popey> that wouldn't be helpful to people who don't know what lives in what package
[15:55] <popey> some of the people filing bugs don't know whether something is a unity bug or indicator bug
[15:55] <ogra_> popey, telling people to file bugs against upstream projects while everyone looks in the distro bugtrackers for bugs is just a mess
[15:55] <popey> hence why there's a nice easy list
[15:55] <popey> I didnt say we should ogra_
[15:55] <Ursinha> good, now we're talking
[15:55] <popey> I said the page should be edited to point to the correct place to file bugs
[15:55] <ogra_> popey, no, the page does
[15:55] <rsalveti> maybe a nice human-readable wiki page pointing upstream projects to packages
[15:55] <popey> yes, and I said it can be corrected
[15:55] <t1mp> https://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-trusty/970/console looks like an issue with jenkins, right?
[15:55] <ogra_> popey, it has upstream "file a bug here" links in the table
[15:55] <rsalveti> and then giving the distro bug tracking policy wiki page link
[15:55] <popey> yes ogra_
[15:56] <popey> you appear not to be reading the lines where I agree with you
[15:56] <ogra_> popey, so people all file their bugs upstream
[15:56] <t1mp> iti s from this MR https://code.launchpad.net/~tpeeters/ubuntu-ui-toolkit/tabbar-closes-toolbar/+merge/196536
[15:56] <popey> yes
[15:56] <popey> I understand the problem and have proposed a simple solution
[15:56] <rsalveti> so people can at least know which package is affected when a scope is not working (or at least how to find the right package)
[15:56] <popey> edit the wiki so it points to the right place to file bugs
[15:57] <Ursinha> popey, can we add to the Avengers page the list of correspondent ubuntu packages for each upstream, and tell them to mark their bugs as affecting distro and the said package?
[15:57] <ogra_> Ursinha, that should rather be by mail imho
[15:57] <rsalveti> Ursinha: we don't want the avengers to be opening bugs against the upstream project
[15:57] <popey> +1
[15:57] <rsalveti> because people are testing what is available in the image
[15:57] <rsalveti> which is just the package
[15:57] <Ursinha> ogra_, mail? wtf?
[15:57] <ogra_> Ursinha, the wikipage needs to lose all the links to the trackers and point to the generic ubuntu bug filing pages
[15:57] <popey> 1. edit page to correct issues, 2. email users of the page. 3. ?? 4. profit!
[15:58] <popey> I would not wipe out the table just to point to one generic bug filing page
[15:58] <ogra_> Ursinha, if upstreams want the bugs additionally in tehir upstream tracker they can indeed do that, but that shouldnt be noted on a wiki
[15:58] <popey> that would be a regression
[15:58] <Ursinha> rsalveti, the package is only a way of distributing the upstream project, how come a bug in a package is unrelated to its upstream?
[15:58] <Ursinha> I might be missing something obvious here
[15:59] <ogra_> popey, well, just the column with all these bug links
[15:59] <ogra_> people will follow them
[15:59] <rsalveti> yeah
[15:59] <popey> ogra_: why cant those bug links be replaced with the right links to the ubuntu bug reporting links in launchpad?
[15:59] <rsalveti> Ursinha: it might be, but not necessarily
[16:00] <popey> i.e. https://bugs.launchpad.net/mir/+filebug?field.tags=avengers replaced with https://bugs.launchpad.net/ubuntu/+source/mir/+filebug?field.tags=avengers
[16:00] <ogra_> popey, why cant they just be replaced with "use ubuntu-bug"
[16:00] <rsalveti> Ursinha: that's why I don't see why we should be linking everything to the upstream project
[16:00] <Ursinha> rsalveti, what's the most common case? a bug being in upstream and therefore in the package?
[16:00] <rsalveti> just use packages, and if the upstream want to link it there, then they can do that
[16:00] <popey> ogra_: if that works sure
[16:00] <rsalveti> Ursinha: yup
[16:00] <Ursinha> or a bug in the package bug upstream already fixed?
[16:00] <Ursinha> so I don't see why forbid people to file bugs against the upstream projects
[16:00] <rsalveti> Ursinha: well, don't have the data to tell
[16:00] <popey> just be aware that sometimes it's not convenient to run "ubuntu-bug packagename" on the phone
[16:00] <rsalveti> not forbid, but we don't necessarily need to force them to do it
[16:01] <Ursinha> rsalveti, we're "forcing" them to file a bug, it doesn't matter the target
[16:01] <rsalveti> in theory our upstreams should be watching for package-related bugs anyway
[16:01] <Ursinha> in launchpad you have one issue and might have several affected pieces
[16:01] <seb128> it makes life easier to "close" the bugs section on the upstream project
[16:02] <seb128> we did that for the settings
[16:02] <Ursinha> so I don't see a reason to prevent people filing bugs against the upstream project, if we can target the bug as affecting the ubuntu package as well
[16:02] <seb128> that was a great move
[16:02] <seb128> you end up with having 2 lists slightly not in sync
[16:02] <seb128> having to change 2 lines every time
[16:02] <seb128> it's just headaches to work with that setup
[16:02] <popey> ogra_: what would you link someone to if they said "how do i file a bug on the device?"?
[16:02] <rsalveti> Ursinha: just because it's usually easier to track it just at one place
[16:02] <rsalveti> such as seb128 said
[16:03] <Ursinha> rsalveti, if upstream people agree on using only the ubuntu package to track problems that's fine
[16:03] <ogra_> popey, i would just say "adb shell ubuntu-bug"
[16:03] <ogra_> popey, instead of looking up a wikipage :)
[16:04] <rsalveti> Ursinha: that should be our default path imho
[16:04] <rsalveti> and if the upstream wants to enable upstream bug tracking, they can do so
[16:04] <Ursinha> rsalveti, so we need people to agree on that and we'll be all fine
[16:04] <rsalveti> up
[16:04] <rsalveti> yup
[16:04] <Ursinha> my suggestion since the beginning was to make default people to file bugs/mark as affecting ubuntu packages
[16:04] <Ursinha> nothing else
[16:05] <Ursinha> that's what is important to you, right? that will (try to) guarantee bugs will be tracked and nothing will be missed
[16:05]  * popey fixes the wiki page
[16:08] <Ursinha> popey, I guess the problem isn't the wiki page per se, but the fact that there's no clear policy on how to file bugs and because of that one regression bug was overlooked...
[16:08] <popey> it is to some degree
[16:08]  * davmor2 wixes the fiki page that popey is on :D  muhahahahaha
[16:10] <Ursinha> didrocks, what can be done is having a team in launchpad then subscribing said team to all the seeded packages, this way you won't miss any important bugs, would that work for you?
[16:10] <didrocks> Ursinha: yeah, I like that plan
[16:11] <didrocks> Ursinha: maybe check with the QA team as well?
[16:11] <didrocks> as they will be the other ones I guess to use that
[16:11] <Ursinha> didrocks, okay, will do
[16:11] <ogra_> popey, i think you need the no_redirect magic in the link too
[16:12] <popey> good call
[16:16] <popey> ogra_: https://wiki.ubuntu.com/Avengers better?
[16:16]  * ogra_ hugs popey 
[16:16] <popey> \o/
[16:16] <ogra_> :)
[16:16] <ogra_> thanks
[16:16] <ogra_> (sorry if i sounded harsh before ...)
[16:17] <popey> Same here.
[16:21] <tedg> So I've got a test failing because of old upstart, but 1.11 is in trusty.  https://jenkins.qa.ubuntu.com/job/upstart-app-launch-trusty-amd64-ci/17/console
[16:21] <tedg> Not sure how that could be.
[16:23] <cwayne> heya, anyone have an idea of why touch_custom tests weren't run?
[16:23] <plars> cwayne: looking
[16:29] <plars> cwayne: looks like we're having launchpad trouble at the moment
[16:32] <plars> cwayne: ok, should be running now
[16:33] <cwayne> plars, wonderful, thanks
[16:43] <Ursinha> didrocks, do you discuss when/if an image is going to be promoted at the daily landing meetings?
[16:43] <didrocks> Ursinha: we discussed it this morning, yeah
[16:44] <Ursinha> maybe having a member of the QA team in that meeting would be useful to point out the relevant bugs when making such decisions (hi jfunk :))
[16:44] <didrocks> Ursinha: they are invited
[16:44] <didrocks> and were coming at some points
[16:45] <jfunk> Ursinha, didrocks -- thanks for the idea, I'll set it up for future
[16:45] <Ursinha> jfunk, I'm setting up a list with all the packages in the ubuntu touch image so it's easier to spot the regressions, that would help
[16:46] <didrocks> thanks Ursinha, that will be really helpful
[16:47] <rsalveti> fginther: what should we do when we get such issues? http://s-jenkins.ubuntu-ci:8080/job/phablet-team-ofono-ubuntu-trusty-armhf-ci/4/console
[16:51] <fginther> rsalveti, I'll file a report for the connection issue. The only thing we can do about the failure is to retry the MP, do you want me to do that?
[16:51] <fginther> plars, what was the launchpad trouble you mentioned?
[16:52] <rsalveti> fginther: no, that's fine, I can do it myself, just checking what would be the procedure :-)
[16:52] <rsalveti> and thanks
[16:52] <fginther> plars, an s-jenkins jobs failed to connect to lp, just want to know if the issues might be related
[16:54] <plars> fginther: temporary issue with dns
[16:55] <plars> fginther: seemed there was a bad line in the resolvconf setup, it's been taken care of now
[16:59] <fginther> plars, did that effect the whole lab?
[16:59] <plars> fginther: should just be kinnara
[16:59] <fginther> ahhh
[16:59] <plars> fginther: were you having problems from there also?
[16:59] <plars> fginther: if so, try again
[16:59] <fginther> plars, no, the failure was on cyclops-node08
[17:08] <robru> didrocks, meeting?
[17:08] <didrocks> robru: joining
[17:08] <sil2100> I think my hangouts is badly broken
[17:08] <didrocks> sil2100: ok
[17:09] <didrocks> cyphermox: joining?
[17:09] <cyphermox> yup
[17:10] <robru> sil2100, do you have an android phone? if so you can join the hangouts with that, might work better
[17:11] <sil2100> robru: I do, maybe next time I'll try that instead, but now I guess it would take too much time to set up, since my phone is a bit oldish
[17:12] <fginther> plars, I'm seeing LP issues on lots of machines
[17:12] <ev> ogra_: what am I supposed to be screaming about? :)
[17:12] <plars> fginther: example?
[17:13] <ogra_> ev, people filing all their bugs in their own upstream projects instead of ubuntu ... so that they wont end up on errors.u.c
[17:14] <ev> what, using bugpatterns to direct crashes away from the ubuntu project in LP?
[17:14] <fginther> plars, cyclops-node0[678]-eth0 so far
[17:14] <fginther> plars, bzr branch fails with "bazaar.launchpad.net: Name or service not known"
[17:15] <plars> fginther: can I ssh to those and take a look somehow?
[17:15] <fginther> plars, yes
[17:19] <fginther> rsalveti, looks like the problem is worse than just one builder. Looking for a resolution
[17:20] <rsalveti> fginther: cool
[17:22] <plars> Saviq: it looks like 26 did not have the unity crash issues, but 27 did. Anything in http://people.canonical.com/~ogra/touch-image-stats/20131120.2.changes that you think could be the culprit?
[17:22] <ogra_> plars, well ... Mir
[17:22] <plars> ogra_: that's what I was thinking, but I defer to the experts :)
[17:23] <Saviq> plars, are those new crash issues?
[17:23] <Saviq> plars, like, does unity8 crash all the time?
[17:23] <plars> Saviq: these are the same crashes we talked about on friday
[17:23] <ogra_> Saviq, on shutdown apparently
[17:23] <plars> Saviq: every single autopilot test, apparently on shutting down unity8
[17:23] <Saviq> plars, so well, basically everything there could cause that
[17:23] <Saviq> plars, that has 'mir' in its name
[17:24] <ogra_> Saviq, sure, but unity produces the crash file :)
[17:25] <ogra_> no doubt a mir change is the root cause :)
[17:25] <Saviq> ogra_, of course
[17:25] <Saviq> ogra_, problem is it's not retrace'able :/
[17:28] <sil2100> Ok, officially I cannot get my hangouts working
[17:28] <sil2100> didrocks: I think I'll need a machine reboot, I'll do it in some moments when I'm finished with something here
[17:30] <cyphermox> didrocks: heads up, I'm freezing today, so don't be surprised if I'm sick tomorrow
[17:31] <cyphermox> I can't seem to manage to get warm :(
[17:31] <didrocks> cyphermox: argh, no worry dude, take care!
[17:31] <cyphermox> still okay today, so this might well just be a false alamr
[17:31] <cyphermox> but hell, it's cold here :/
[18:32] <rsalveti> didrocks: ogra_: Ursinha: another regression, bug 1252737
[18:32] <Ursinha> rsalveti, that was the problem I was facing yesterday, maybe?
[18:33] <ogra_> rsalveti, i thought that was fixed
[18:33] <rsalveti> Ursinha: probably
[18:33] <didrocks> rsalveti: ogra_ told it was fixed
[18:33] <ogra_> (thats at least two weeks old already)
[18:33] <didrocks> wasn't it?
[18:33] <ogra_> and i wasnt able to reproduce it anymore
[18:34] <rsalveti> I can test, but the reporter said he was able to reproduce it with r25
[18:34] <ogra_> didrocks, iirc davmor2 could reproduce it reliably for a few days and also didnt have it anymore
[18:34] <didrocks> exactly
[18:35] <ogra_> and in fact it works fine for me here
[18:35] <ogra_> just dropped off wlan ... which switched it off and got me GSM ...
[18:35] <awe_> om26er was able to reproduce today
[18:36] <rsalveti> you need to remove wifi and reboot
[18:37] <awe_> ogra_, so the question is how did mission control get updated?  We've certainly spent a bunch of time analyzing this, trying to figure out what the heck happened
[18:38] <ogra_> awe_, well, i didnt even know about a bug being filed ... i had my conversations with davmor2 about it
[18:38] <ogra_> and my own tests
[18:38] <ogra_> awe_, that jus proves Ursinha's point
[18:38] <awe_> ogra_, I try to file bugs when people report problems.  ;)-
[18:38] <awe_> not sure what point is proven?
[18:39] <ogra_> awe_, that image releasing needs to be happening based on bugs
[18:39] <Ursinha> ogra_, do you have that mail where people discussed how bugs should be filed?
[18:39] <Ursinha> I'd like to proceed with that discussion
[18:39] <ogra_> instead of the landing team checking some manual testsand the dashboard of some flaky test results
[18:40] <ogra_> Ursinha, there was no discussion, and no, as i said before, i cant find it ... it was a mail from asac
[18:42] <ogra_> hmpf
[18:42] <ogra_> i cant manage to upgrade to r29 from r28
[18:42] <Ursinha> ogra_, what happens?
[18:43] <ogra_> the updater shows nonsense (r) ... (no version) ... and it either doesnt finish the download ortells me there was nothing downloaded
[18:43]  * ogra_ reboots ... lets try a fresh boot 
[18:43] <Ursinha> ogra_, try to close the settings and open again... that happened to me in a previous version...
[18:43] <Ursinha> thought it was a transient bug but surely needs to be investigated
[18:45] <Ursinha> ogra_, didrocks, awe_, and I can confirm bug 1252737 here, currently using r29 on mako
[18:46] <awe_> rsalveti, you overwrote my bug summary!
[18:46] <awe_> title
[18:46] <ogra_> sue him about violating your copyright !!
[18:46] <ogra_> :P
[18:46] <rsalveti> awe_: mid-air collision
[18:47] <ogra_> Ursinha, upgrade worked fine after a reboot now
[18:47] <ogra_> something to keep an eye on i guess
[18:47] <Ursinha> ogra_, yes, we should file a bug...
[18:47] <Ursinha> against the ubuntu package :P
[18:48] <awe_> Ursinha, can you try: https://bugs.launchpad.net/ubuntu/+source/ofono/+bug/1252737/comments/12
[18:48]  * Ursinha looks
[18:48] <awe_> Ursinha, originally the bug was filed against ofono (Ubuntu), I added a task for telepathy-ofono (Ubuntu), and marked the ofono task Invalid after we diagnosed the problem
[18:49] <ogra_> Ursinha, this is pointless without being able to attach some data ... i usually file such non reproducable bugs the next time they occur so i can attach logs etc
[18:50] <Ursinha> ogra_, but I don't even know which data should I look for, I don't feel ignoring it and not reporting the bug is the right approach
[18:50] <ogra_> Ursinha, i'm not ignoring it :)
[18:50] <ogra_> and it is all data thats related to system-image :)
[18:50] <rsalveti> someone removed the ofono bugtask
[18:50] <rsalveti> argh
[18:51] <rsalveti> should be fine to keep it as invalid (for ofono)
[18:51] <Ursinha> ogra_, sorry, I didn't mean to imply that :) I meant it's more useful to  have the issue reported so other people can observe if they have the same behavior
[18:51] <rsalveti> just flashed 29 with -b, no signal
[18:52]  * ogra_ is just booting 29 with wifi off before the reboot 
[18:52] <ogra_> *twiddle* ... the boot takes a century
[18:52] <rsalveti>         Attached = 0
[18:52] <rsalveti> let me connect to a wifi ap
[18:52] <rsalveti> aaaaaaaargh
[18:52] <ogra_> confirmed
[18:52] <rsalveti> keyboard has sound now
[18:52] <Ursinha> rsalveti, I told you lol
[18:53] <didrocks> yeah, confirmed as well
[18:53] <ogra_> GSM is fine afte disabling WLAN
[18:53] <rsalveti> that clearly shows that nobody tested 29 with bootstrap and no backup
[18:53] <rsalveti> lol
[18:53] <ogra_> GRRR
[18:53] <didrocks> and yeah, empathy upload to ubuntu date matches
[18:53] <rsalveti> Ursinha: so annoying
[18:53] <ogra_> that indicator bug is super annoying !
[18:54] <Ursinha> hehe
[18:54] <Ursinha> how come no one noticed that before?
[18:54] <ogra_> ah, finally found the network tab
[18:54] <Ursinha> ogra_, click on the words and go directly to the tab you want otherwise is just crazy
[18:54] <didrocks> Ursinha: always starting my phone when flashing at home, which has WLAN…
[18:54] <ogra_> rsalveti, WOAH ... i wouldnt bother about sound from the kbd ... but it is LOUD !!!!
[18:55] <Ursinha> hahahaha
[18:55] <ogra_> and makes it really slow
[18:55] <Ursinha> you are all a funny bunch :)
[18:55] <rsalveti> yeah
[18:55] <didrocks> yeah, I would +1 on the "not fan on keyboard sound"
[18:55] <Ursinha> didrocks, ah, I meant the indicator bug
[18:55] <rsalveti> seems I'm typing on a typewriter
[18:55] <Ursinha> it's been there since 27 at least
[18:55] <ogra_> not only that
[18:55] <didrocks> Ursinha: really, I never use those settings, do you?
[18:55] <ogra_> try typing fast
[18:55] <Ursinha> rsalveti, a really     slow    one
[18:55] <rsalveti> hahaha, yeah
[18:55] <ogra_> you only get sound for evey other keypress
[18:56] <Ursinha> didrocks, everytime I want to check Incoming, or change brightness
[18:56] <Ursinha> or connect to wifi, or enable/disable bluetooth
[18:56] <rsalveti> wonder if the one who did the sound change actually uses the phone
[18:56] <Ursinha> or raise volume
[18:56] <didrocks> Ursinha: ah, I don't use sms ;)
[18:56] <Ursinha> I listen to music on this little thing :)
[18:56] <ogra_> rsalveti, he tested on a PC :P
[18:56] <rsalveti> might be
[18:56] <ogra_> (or she ... who knows)
[18:56] <didrocks> Ursinha: and just turn on/off the phone screen
[18:56] <rsalveti> bfiller: ^^ :-)
[18:57] <didrocks> it's clearly developer's fault, all Bill's :)
[18:57] <rsalveti> super-annoying-low-and-slow keyboard sound
[18:57] <rsalveti> *high
[18:57] <rsalveti> :-)
[18:57] <bfiller> rsalveti: really bad, should be off by default. MR in progress for that
[18:57] <rsalveti> bfiller: *thanks* :-)
[18:57] <didrocks> bfiller: oh, it wasn't intended to be on by default?
[18:57] <bfiller> that's what you get when a developer bangs something in before leaving the company (:
[18:57] <bfiller> didrocks: no
[18:58] <rsalveti> hahah
[18:58] <didrocks> bfiller: ahah, "let's push the bomb"
[18:59] <bfiller> rsalveti: DONE https://code.launchpad.net/~thomas-moenicke/ubuntu-keyboard/ubuntu-keyboard-sound-off/+merge/196580
[18:59] <Ursinha> awe_, that command made the GSM signal appear even with wifi off
[18:59] <rsalveti> Ursinha: yeah
[18:59]  * rsalveti hugs bfiller 
[18:59] <rsalveti> didrocks: now please, land this asap
[18:59] <rsalveti> :P
[18:59] <didrocks> Ursinha: awe_: confirmed :)
[19:00] <didrocks> rsalveti: ahah, yeah, will do with the GSM/Unity8 and ubuntu-ui-toolkit fixes. Same level of severity :)
[19:00] <bfiller> rsalveti: you can turn it off by running "gsettings set com.canonical.keyboard.maliit key-press-feedback false"
[19:00] <rsalveti> didrocks: cool
[19:00] <rsalveti> bunch of regressions
[19:00] <rsalveti> bfiller: cool
[19:00] <didrocks> so, back on the empathy one
[19:01] <ogra_> rsalveti, didrocks, and dont forget about "not able to make the image writable anymore" one too
[19:01] <didrocks> ogra_: what? which one? didn't hear (yet) about it
[19:01] <didrocks> for empathy: should we change the settings key? I would prefer having a seb around to know this use-conn impacts…
[19:01] <rsalveti> stgraber is pushing a fix for that
[19:01] <ogra_> didrocks, was discussed the last hours in #phablet ... (and is already fixed in archive)
[19:02] <rsalveti> still waiting another android upload
[19:03] <rsalveti> with the system.img and swap security fix as well
[19:03] <didrocks> ogra_: ok, backlogged
[19:04] <didrocks> so, all that planned for tomorrow, the Mir team is looking at the unity8 fix, I don't have feedback from the toolkit team yet though (for the indicator one)
[19:04] <didrocks> we'll get a promoted image that should fix all those
[19:05] <rsalveti> we should build another image later today though
[19:05] <ogra_> ++
[19:05] <rsalveti> instead of waiting for tomorrow
[19:05] <didrocks> rsalveti: +1 as soon as you have this android thingy done
[19:05] <ogra_> once the android and kbd fixes are in
[19:05] <rsalveti> alright
[19:05] <rsalveti> and how to fix the 3g/wlan regression?
[19:05] <Ursinha> didrocks, there's bug 1253703 as well
[19:05] <didrocks> rsalveti: I told as well to kenvandine and robru to ping you for a rekick if they have the toolkit landed
[19:05] <rsalveti> great
[19:06] <robru> didrocks, rsalveti: still waiting to hear from kaleo about landing that fix.
[19:06] <didrocks> rsalveti: for the 3g/wlan, I would go for patching empathy, but I would prefer to have a desktop team member around for this key
[19:06] <didrocks> robru: yeah, do not hesitate to ping him to get progress
[19:06] <rsalveti> didrocks: alright
[19:06] <robru> didrocks, yeah, did already a couple times ;-)
[19:07] <didrocks> Ursinha: yeah, on my list to land
[19:07] <didrocks> tedg: btw, who is reviewing https://code.launchpad.net/~ted/upstart-app-launch/uri-splitting/+merge/196316? This will be done by tomorrow?
[19:07] <didrocks> (europe tomorrow)
[19:07] <Kaleo> robru, if we get a fix, can we land the previously released toolkit with just that added fix as opposed to the latest trunk?
[19:08] <tedg> didrocks, I haven't asked anyone specifically to review it.
[19:08] <didrocks> tedg: can you please?
[19:08] <tedg> didrocks, Sure
[19:08] <robru> Kaleo, uh.... no. trunk needs to be releasable
[19:08] <rsalveti> tedg: also marked as critical as it's a regression
[19:08] <Kaleo> robru, ok
[19:08] <didrocks> robru: feel free to distro-patch directly
[19:08] <Kaleo> didrocks, thanks
[19:08] <didrocks> I would prefer as Kaleo told, just having one commit
[19:08] <didrocks> tedg: thanks!
[19:09] <robru> didrocks, can that be done in jenkins? even if I add a distropatch, jenkins still pulls that from trunk
[19:09] <Kaleo> didrocks, me too, we have 18 commits in just a few days
[19:09] <didrocks> robru: no, you need to apt-get source
[19:09] <didrocks> and patch
[19:09] <didrocks> that's the easiest
[19:09] <didrocks> robru: push to the ubuntu-unity ppa to get an armhf build
[19:09] <didrocks> to test
[19:09] <robru> didrocks, that's way harder! then I need somebody to review and sponsor the upload... ken is nearly EOD, it would never happen. building trunk with jenkins is way easier.
[19:10] <didrocks> robru: ok, you need to retargeted the sdk branch
[19:10] <didrocks> retarget*
[19:10] <didrocks> and rekick a build
[19:10] <robru> didrocks, what do you mean 'retarget'
[19:10] <didrocks> robru: change source_branch= in the config
[19:10] <didrocks> deploy that in jenkins
[19:10] <didrocks> build it
[19:10] <didrocks> publish it
[19:11] <didrocks> and finally redeploy the revert to trunk
[19:11] <didrocks> do you feel able to do it?
[19:11] <robru> didrocks, change to what? i don't understand what you're asking for. you mean make my own branch only with the fix?
[19:11] <didrocks> robru: yeah
[19:11] <didrocks> robru: do you know about cupstream2distro-config?
[19:11] <didrocks> and target_branch?
[19:11] <robru> didrocks, of course I know about target_branch, I do that many times.
[19:11] <didrocks> robru: so, just change that for the ubuntu-ui-toolkit
[19:12] <didrocks> to point to your branch
[19:12] <didrocks> (under the same owner or ~ubuntu-unity)
[19:12] <didrocks> then deploy the config in jenkins
[19:12] <didrocks> with cu2d-update-stack
[19:12] <robru> Kaleo, ok, yes it is possible to release just this patch if you feel trunk is not currently safe to release. but in future please keep trunk releasable.
[19:12] <didrocks> build the sdk stack
[19:12] <didrocks> publish it
[19:12] <didrocks> once tested
[19:12] <didrocks> and revert your config change
[19:12] <didrocks> robru: when deploying, you need to only deploy with -U
[19:12] <didrocks> not -US
[19:13] <robru> didrocks, yeah, that's how i usually do it
[19:13] <didrocks> (as you need for merging back to have the right bzr branch target
[19:13] <didrocks> ok, great!
[19:13] <didrocks> kenvandine: robru: can you send my way an email for tomorrow morning?  with all what was done/what's remaining?
[19:13] <didrocks> that would be awesome
[19:13] <robru> didrocks, ok
[19:13] <Kaleo> robru, it's not that
[19:13] <didrocks> thanks :)
[19:13] <Kaleo> robru, it's more like if anything happens, god forbid, I don't want the team to spend the night on it
[19:15] <robru> Kaleo, ok, here is I think the best plan: land your fix in trunk, then I will quickly build & test that. if there are any regressions, I will backport your fix to the existing package in the image and then you can worry about making trunk releasable later
[19:15] <didrocks> robru: put Mirv into the loop as well please :)
[19:15] <robru> didrocks, ok
[19:16] <didrocks> thanks
[19:16] <Kaleo> robru, nah
[19:16] <Kaleo> robru, you cannot detect all possible regressions
[19:16] <Kaleo> robru, therefore there is still a chance of spending the night
[19:16] <Kaleo> robru, therefore I'd rather be 100% safe
[19:16] <Kaleo> robru, and just patch the package
[19:17] <Kaleo> robru, it's already past 9pm for some of us debugging the issue
[19:17] <robru> Kaleo, ok, do you have a patch ready? is there a branch? please land it in trunk anyway (for future releases) and then once I see it in trunk I will begin backporting it to the released package.
[19:20] <didrocks> robru: or you can ask rsalveti to sponsor your distro-patch
[19:20] <didrocks> rsalveti: would you be able to help on that? ^
[19:20] <rsalveti> sure, just point me to the debdiff
[19:20] <didrocks> thanks man :)
[19:20] <robru> rsalveti, ok, how long will you be around for?
[19:20] <Kaleo> robru, no it's not ready at all yet
[19:20] <didrocks> rsalveti is always around :)
[19:20] <rsalveti> robru: 3-4 hours
[19:21] <rsalveti> but yeah, I'm always coming back
[19:21] <Kaleo> robru, will let you know
[19:21] <robru> rsalveti, ok, I am still waiting for the patch before I can even start making the debdiff ;-)
[19:21] <didrocks> "he's coming BACK" :)
[19:21] <rsalveti> :-)
[19:21] <didrocks> ok, sent some instructions for the morning to Mirv
[19:21] <didrocks> wife really unhappy let's fix that :)
[19:21]  * didrocks waves good evening
[19:39] <balloons> ohh fginther is even vanguard today :-) Welp, I think the core apps jenkins might not be happy atm
[19:39] <balloons> fginther, http://91.189.93.70:8080/job/generic-mediumtests-trusty/241 and http://91.189.93.70:8080/job/generic-mediumtests-trusty/247/console
[19:40] <fginther> balloons, looking
[19:50] <fginther> balloons, ugh, that machine has almost no free memory
[19:51] <balloons> fginther, I hope that helps explain why things come out funny on it
[19:53] <fginther> balloons, that would do it
[19:53] <balloons> hehe :-)
[20:01] <fginther> balloons, I restarted the node, it's looking better now
[20:01] <balloons> ty fginther .. I'll be pushing it again this afternoon, so we'll see how it holds up
[20:29] <Kaleo> robru, quick'n'dirty patch proving that the analysis is correct works
[20:29] <Kaleo> robru, working on better patch now
[20:30] <robru> Kaleo, great to hear, let me know when it's in trunk.
[20:44] <cwayne> plars, ping
[20:52] <fginther> robru, got a moment to talk about webapps SRUs?
[20:52] <robru> fginther, yeah sure. did you see the failures from the last build i kicked?
[20:53] <fginther> robru, no, do you have a link?
[20:53] <robru> fginther, the prepare failures here: http://q-jenkins.ubuntu-ci:8080/view/cu2d/view/Saucy/view/WebApps/job/cu2d-webapp-saucy-1.0prepare/
[20:54] <robru> fginther, it's some kind of infrastructure issue, i don't understand it. meant to file a bug / ping you earlier, but it got away on me
[20:55] <fginther> robru, looking at the timestamps, I know that the daily-release-executor slave died about the same time. That would explain these errors
[20:55] <robru> fginther, so i should just retry?
[20:55] <fginther> robru, yep
[20:56] <plars> cwayne: hi
[20:56] <cwayne> fginther, hey, any reason i can't remove webbrowser-autopilot tests as being part of touch_custom?
[20:56] <cwayne> plars, ^ same question :)
[20:57] <robru> fginther, great. so what was it you wanted to ask about SRUs?
[20:57] <fginther> robru, you sent me email on Friday  about testing 13.10 SRUs
[20:58] <plars> cwayne: I think maybe those were requested?
[20:58] <robru> fginther, oh right. yeah?
[20:58] <plars> cwayne: whatever makes sense to you guys to run, fine with me
[20:58] <cwayne> plars, i'll double check, but if it was requested it probably shouldn't have been.. they're not quite relevant
[20:59] <robru> fginther, so what I'm thinking is we want a -ci job that will take commits to the saucy branches of webapps, build, install, and test those on saucy itself, so that the test log can be shown in the SRU bug.
[20:59] <robru> fginther, (i'm not very familiar with what -ci is currently doing for saucy so maybe it already does this? not sure)
[21:01] <fginther> robru, no, I don't think we're already doing this. At least not as part of testing an MP
[21:01] <plars> cwayne: it's easy to remove, just let me know
[21:01] <cwayne> plars, sure, just waiting to hear from sfeole, i'll ping you when I get some info
[21:01] <fginther> robru, I assume victor would be the expert on how to run the tests, I'll have to talk to him
[21:01] <robru> fginther, does it make sense to do it at the MP level? What I need is the test to happen ASAP during the SRU process so that I can just link the log and the SRU people can have more confidence to release my SRU.
[21:02] <robru> fginther, yeah, thought so. I CC'd him on the mail
[21:02] <fginther> robru, do current SRU developers have to run these tests manually?
[21:02] <fginther> robru, in other words, is this duplicating something they already have to do, or does this free them from the task?
[21:02] <robru> fginther, oh yeah, current SRUs are *very* manual. webapps are much simpler than normal SRUs that need to get tested, so I'm trying to automate away the testing step. it's the only way I can possibly be responsible for the next five years of webapps SRUs in trusty.
[21:03] <robru> fginther, it frees them (me) from the task
[21:03] <fginther> robru, thanks, I think I have the context now
[21:03] <robru> fginther, great, thanks
[21:04] <fginther> cyphermox, ping
[21:05] <robru> fginther, can you take a look at http://q-jenkins.ubuntu-ci:8080/job/cu2d-webapp-saucy-1.1prepare-unity-webapps-facebookmessenger/129/console ? it downloads the source package from saucy, then creates what the updated version should be, then it fails because the updated version is lower than the version in trusty. what?
[21:06] <robru> fginther, same here too: http://q-jenkins.ubuntu-ci:8080/job/cu2d-webapp-saucy-1.1prepare-unity-chromium-extension/99/console
[21:06] <fginther> robru, looking
[21:07] <robru> fginther, thanks
[21:11] <fginther> robru, https://bugs.launchpad.net/ubuntu-ci-services-itself/+bug/1254876
[21:12] <robru> fginther, that sounds about right, thanks.
[21:14] <robru> fginther, [the other thing we're taling about]'s probably a cu2d-config problem that forgot to specify 'saucy' at some point in the stack config, but I'm not sure where it would be missing from. it generated the correct version number to use for the saucy package (2.4.16+13.10.20131125-0ubuntu1, which is higher than latest saucy and lower than trusty), but for some reason it's comparing it against the latest trusty version and sayi
[21:14] <robru> ng "it's lower! fail!" when really that's fine.
[21:16] <fginther> robru, yep, I'm looking for a missed update
[21:26] <fginther> robru, can you look at https://code.launchpad.net/~robru/unity-webapps-facebookmessenger/icon-sru/+merge/196223
[21:26] <fginther> robru, was it supposed to bump the version to 14.04?
[21:27] <robru> fginther, no, absolutely not. that's an SRU to saucy and the version number is supposed to be 13.10+DATE
[21:28] <robru> fginther, oh, I see, *I* am the one who bumped the version number in the merge itself.
[21:28] <fginther> robru, can you fix that?
[21:29] <robru> fginther, yep
[21:29] <fginther> robru, the unity-chromium-extension is a different problem
[21:29] <fginther> robru, thanks
[21:31] <fginther> robru, ~webapps/unity-chromium-extension/13.10 also has 14.04 in the changelog, but I haven't figured out how it made it there yet
[21:32] <robru> fginther, my guess would be that there was already a trusty release before that branch got branched for saucy
[21:34] <robru> fginther, ok, i think i fixed facebook, gotta break for lunch though. will look at chromium in a bit.
[21:34] <robru> fginther, thanks for looking
[21:34] <fginther> robru, thank you
[21:40] <fginther> robru, I think you're right about chromium, I found the MP that bumped the version, but it was targeted at trunk: https://code.launchpad.net/~ps-jenkins/unity-chromium-extension/latestsnapshot-2.4.8+14.04.20131108.1-0ubuntu1/+merge/194594
[21:40] <fginther> robru, leads me to believe that 13.10 was branched afterwards
[21:42] <balloons> fginther, I get to ping you again about core apps box... I'm not seeing any of the autolanding builds triggering
[21:42] <fginther> balloons, ack, I disabled that while working on the earlier problem. I'm about ready to re-enable
[21:43] <balloons> fginther, ahh, kk, I'm not crazy
[21:45] <fginther> balloons, not this time
[21:50] <fginther> balloons, back up
[22:50] <wgrant> cjwatson: FWIW my belief is that ddebs are 100% implemented; it's just a librarian space issue today. I've done extensive testing on DF and it's all fine.
[23:04] <fginther> robru, have you had a chance to look at unity-chromium-extension?
[23:12] <fginther> robru, can you review? https://code.launchpad.net/~fginther/cupstream2distro-config/add-unity-voice/+merge/196633
[23:16] <rsalveti> robru: hey, any news for the toolkit release?
[23:17] <rsalveti> just to know if I need to sponsor something today still
[23:45] <Kaleo> robru, around?
[23:45] <Kaleo> rsalveti, fix is 99% ready
[23:45] <rsalveti> Kaleo: what is missing still?
[23:48] <Kaleo> rsalveti, a landing )
[23:48] <Kaleo> rsalveti, timp is happroving as we speak
[23:48] <rsalveti> cool
[23:49] <Kaleo> rsalveti, if robru is not here it's going to be harder though
[23:50] <Kaleo> rsalveti, any backup?
[23:50] <rsalveti> Kaleo: maybe cyphermox
[23:50] <Kaleo> cyphermox, around /°
[23:50] <Kaleo> ?