/srv/irclogs.ubuntu.com/2014/11/25/#ubuntu-ci-eng.txt

=== johnlage_partyha is now known as johnlage
robrui.... what?01:55
robrubut platform-api is in the ppa... it built!01:57
imgbot=== trainguards: IMAGE 32 building (started: 20141125 02:05) ===02:04
robruwell the train's gone and shit itself good this time02:11
robruhmmmmmm02:15
robruAlbertA: Ok I think that worked. there seems to be some kind of bug where the train can't handle the idea that a silo is being published for the second time. I dug in and removed the evidence of the initial publication and now it looks like it's working. I gotta run out but I'll be back in a few hours and double check that this published for real.02:16
robruoh that's a good sign02:17
AlbertArobru: thanks dude!02:45
imgbot=== trainguards: IMAGE 32 DONE (finished: 20141125 03:25) ===03:24
imgbot=== changelog: http://people.canonical.com/~ogra/touch-image-stats/32.changes ===03:25
=== chihchun_afk is now known as chihchun
=== chihchun is now known as chihchun_afk
Mirvmorning05:22
robruAlbertA: yep it definitely published. you're welcome. looks like it's still stuck in proposed due to the binary package from the previous build though, I think if we get an archive admin (cjwatson?) to delete the old arm64 build it should be able to get through this time.05:24
robruMirv: heya05:25
robruMirv: found a bug in the train, but it's a rare corner case. if you rebuild packages after they've been published, it can't publish them again. You have to go delete ~/silos/ubuntu/landing-XXX/*.project_* manually in order to publish. This regression is probably from some changes I made weeks ago, so it'll be difficult to just revert to a working state. Not05:28
robrusure how long it'll take me to get a fix out (definitely won't be tonight). Hopefully all your publishings succeed on the first try, but if anything gets caught in proposed you'll have to be aware of this.05:28
Mirvrobru: hmmkay05:41
Mirvrobru: it's quite rare indeed05:41
MirvI don't remember myself ever doing that05:42
Mirvunless of course it happens also wen publish fails, then build/watch_only and pubish again, which is more common05:42
robruMirv: I've done it a few times over the years. it definitely used to be possible, as things used to get stuck in -proposed more often in the past05:42
Mirvoh, yes, those rare cases. I remember a couple.05:42
robrunot sure if sarcasm ;-)05:43
Mirvhaha :D no, they are rare.05:48
=== chihchun_afk is now known as chihchun
=== chihchun is now known as chihchun_afk
=== chihchun_afk is now known as chihchun
bzoltanMirv: My question on #ci did not provoke anybody to respond .. who do you know from CI who I can direct ping?07:45
Mirvbzoltan: I tend to point to fginther for solving CI issues but that's unfair since he is simply too good in achieving/fixing things :)08:03
Mirvbzoltan: so I don't kind of have clear ideas who to ping / how they delegate tasks08:04
Mirvbzoltan: they should respond to cihelp, but I haven't gotten any response when I've pinged that nick in my morning hours08:04
MirvI mean, it does not look like they really properly backlog the requests08:04
bzoltanMirv:  with "how" i can  help... i do not know who is at CI who is actually really responsible to keep the machine running.08:04
Mirvwhich is probably partially true for trainguards too - when the ping is long enough time ago, one may assume that "someone else" took care of it08:05
robrubzoltan: I think a lot of CI people are off at some kind of sprint/training right now.08:05
robrubzoltan: if your issue is with s-jenkins I can poke at it but I don't have a lot of expertise there.08:06
bzoltanrobru: Our staging landings are tottaly blocked... we have 40+ MRs blocked by a CI problem. . according to this logs - https://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-runner-mako/6179/console  the build rootfs does not seem to be pure vivid.08:07
robruhmmm08:07
vilabzoltan, robru, Mirv : fginther *is* working on it and tracks it in asana08:07
bzoltanrobru:  It would be great if you could help. But of course I can wait if it beyond your reach08:07
bzoltanvila: ohh, really? So fginther knows aboout this issue... good to know.08:08
vilawell, AFAIK he is, he did update the ticket 4 hours ago08:09
robrubzoltan: yeah that log doesn't have enough info to go off. when you see 'but it isn't going to be installed' it usually means there's a problem with some other dep that isn't even mentioned. You have to reproduce it locally and then try forcibly installing everything it complains about until it finally tells you the real problem. real frustrating... and it's08:09
robrumidnight here ;-) I'll let fginther lead that one08:09
vilabzoltan: in any case, pinging cihelp is the way to go08:11
vilabzoltan: hold on, it's on a mako ? Then it's probably plars, there have been issues about installing images related to a pretty convoluted set of blockers08:15
bzoltanvila:  the logs say it is on mako08:15
vilabzoltan: that's where I am yes. No idea if it's related to the issues I mentioned above though08:16
bzoltanvila: OK, thanks... the other option is to disable AP tests for the UITK for the time this issue is fixed.08:21
bzoltanMirv: the silo16 is good to go08:21
vilabzoltan: meh, the job you're pointing above was "Started by upstream project "generic-deb-autopilot-utopic-touch" build number 6616" *utopic* not vivid, is that expected ?08:22
Mirvbzoltan: ok!08:22
Mirvvila: I'm sure it's not expected, everything should be running vivid when it comes to merging vivid branches08:23
vilahttps://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-utopic-touch/ doesn't look like a job that succeeded recently %-/08:23
vilabzoltan: Do you when this started to fail ? (An url for a successful run will help)08:25
vilabzoltan: or at least the project name (generic jobs are a pain :-/)08:26
bzoltanvila:  it is the lp:ubuntu-ui-toolkit/staging branch where we target our MRs08:26
=== chihchun is now known as chihchun_afk
vilabzoltan: sounds like http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sdk-team-ubuntu-ui-toolkit-staging-check/ then las success on Nov 10 ?08:29
vilabzoltan: http://s-jenkins.ubuntu-ci:8080/job/generic-deb-autopilot-runner-mako/6104/consoleFull looks vivid to me... (despite being "Started by upstream project "generic-deb-autopilot-utopic-touch" build number 6527")...08:33
vilabzoltan: so no MPs has landed since Nov 10 ? Or am I looking at the wrong place ?08:42
vilabzoltan: according to the branch history, the last commit was done yesterday... revno 1339,  timestamp: Fri 2014-11-21 18:12:18 +0200, message: Sync with trunk, is from you, how did you land it ?08:48
bzoltanvila: this is our staging https://code.launchpad.net/~ubuntu-sdk-team/ubuntu-ui-toolkit/staging08:52
bzoltanvila:  some MRs do land occasionally .. But we up tracking down the issues when 95% of the jenkins job fail on some cryptic crap...08:53
vilabzoltan: as far as I can find my way into that cryptic thing you're mentioning above, it seems to me http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sdk-team-ubuntu-ui-toolkit-staging-utopic-amd64-autolanding/635/consoleFull did land revno 1341 12 hours ago08:56
vilai.e. gating is done for utopic on armhf, amd64 and i386 : http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sdk-team-ubuntu-ui-toolkit-staging-autolanding/691/console08:57
vilahttp://s-jenkins.ubuntu-ci:8080/job/ubuntu-sdk-team-ubuntu-ui-toolkit-staging-autolanding/691/parameters/? says utopic too (parent of the job above)09:00
vilaMirv: correct me if I'm wrong, those gating rules come cu2d-config right ? ubuntu-ui-toolkit is defined in stacks/head/sdk.cfg AFAICS, am I looking at the wrong place again ?09:05
Mirvvila: I don't unfortunately have a clue whether cu2d-config plays a role still. I kind of assumed it was not used at all anymore, but I might be wrong... if it is in use, that would explain something!09:12
vilacrickets09:13
Mirvvila: looking at the changelog, it does seem to be in use still09:13
Mirvvila: I think we can fix it ourselves if that's the ase, let me give you a branch to approve..09:13
vilaMirv: thanks, that's useful feedback, I'll try to get feedback from fginther (new ticket created in asana so it is tracked)09:14
Mirvvila: oh, actually no branch that I could create, it seems to be all vivid for ubuntu-ui-toolkit from what I can see..09:14
Mirvvila: thanks for creating a ticket09:15
vilaMirv: ha, great, gee, so you end up with the same understanding ? I.e. it should be vivid but it's still utopic ?09:15
Mirvvila: yes, looks like it09:16
vilaMirv: pfew. THanks !09:17
=== chihchun_afk is now known as chihchun
brendandogra_, no psivaa-holiday so who can help us from ci?09:24
brendandogra_, seems security and sdk suites didn't run for some reason09:25
psivaa-holidaybrendand: sorry i am not on holidays, this is irccloud madness09:25
=== psivaa-holiday is now known as psivaa
ogra_psivaa, to late ... now you have to take off09:25
brendandpsivaa, ok :)09:25
psivaa:D09:26
ogra_You have not informed bzr of your Launchpad ID, and you must do this to09:26
ogra_write to Launchpad or access private data.  See "bzr help launchpad-login".09:26
ogra_thats the issue it seems09:26
psivaabrendand: ogra_: it is lp temp issue '503', let me run those again09:27
sil2100;)09:27
=== chihchun is now known as chihchun_afk
Mirvsil2100: we've vivid autopkgtests failing a lot, not sure what could be done about it09:46
sil2100ogra_: oh, btw.! How are those adbd changes going?09:46
sil2100Mirv: for which packages?09:47
Mirvsil2100: about everything that runs some :) for example see http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html#ubuntu-download-manager09:47
sil2100uh09:47
MirvI just don't see any clear causer of it09:47
ogra_sil2100, rolled back ... they break the lab which is stuck on an old UDF until we can land a one line change in krillins recovery to touch the override file when usin --developer-mode09:47
ogra_needs to wait til next week ,when we can land stuff in RTM again09:48
Mirvsil2100: but since it's there in stuff like binutils too, I'd guess foundations people would be aware and might know where to find the actual culprit09:48
ogra_(assuming you mean the new lock-screen-check feature)09:48
Mirvsil2100: correction, I see it has been already discussed 2h ago, but more people are needed09:57
Mirvso let's assume the people will be found and vivid autopkg test issues fixed by eod or so09:57
bzoltanMirv:  could you please gently kick the UITK package in the proposed pocket. It thinks that it is regressing on ubuntuone-credentials  what is BS10:12
sil2100bzoltan: yeah... see Mirv's messages above ^10:13
sil2100bzoltan: it seems autopkgtests are failing for vivid now and we're waiting for more people to help10:13
sil2100Not much we can do here :/10:13
ogra_did someone ping pitti ?10:13
bzoltansil2100: Ahh... sorry I have not read the logs10:14
LaneyThat one is fixed: https://jenkins.qa.ubuntu.com/job/vivid-adt-ubuntuone-credentials/lastBuild/10:14
Mirvogra_: yes, or he himself pinged mvo and mvo waits for barry :)10:24
MirvLaney: great, it looks like it might fix uitk migration, although the system-image problem hits eg. http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html#ubuntu-download-manager10:24
LaneyThere are other failures, but I think that they are independent10:25
Laneye.g. the kate ones seem like a packaging error10:25
cjwatsonrobru,AlbertA: Yes, removing mir-graphics-drivers-android/arm64 from vivid-proposed was the right answer here.  (This was only a problem because there was a previous unmigrated version in -proposed.)  I've done that now.10:47
pstolowskitrainguards, hey, may I ask for reconfiguring landing-013 if needs be (one extra MP added to unity-scopes-shell), and rebuilding of *just* unity-scopes-shell (it takes ages to rebuild entire silo again)?11:42
sil2100pstolowski: k11:44
sil2100pstolowski: in case of adding MPs to the list of already configured projects you can reconfigure yourself actually, but let me do that this time11:45
sil2100pstolowski: done11:47
sil2100pstolowski: it's building11:47
pstolowskisil2100, thanks; is rebuilding of just single project possible?11:47
sil2100pstolowski: yes, when you press the build button, you need to select the project you want to rebuild in the PACKAGES_TO_REBUILD11:48
sil2100pstolowski: but in theory when you just add one merge to the silo, CI Train by default will only rebuild the project that changed (e.g. had new merges)11:48
sil2100But to be perfectly safe you can include the name in PACKAGES_TO_REBUILD ;)11:48
pstolowskisil2100, awesome, thanks!11:49
sil2100yw!11:49
pstolowskipete-woods, ^ you may want to know that as well :)11:49
pete-woods:)12:01
ollihiho12:43
olliogra_, sil2100, jibel, how are things looking?12:43
olliare we still playing audio on the latest build ;)12:43
ogra_olli, we have some more info about the issue12:44
ogra_indeed we do :P12:44
ogra_it is still open if we will have a fix ... but if there is a silo ready by tonight we could still get it in ...12:45
ogra_if not the agreeent was to not do anything and ship 16912:45
ollisounds good, thx for the update12:46
=== alan_g is now known as alan_g|lunch
sil2100olli: still nothing concrete though ;)12:52
* ogra_ wouldnt say that ... we knoe there is a long running connection which cgproxy isnt designed for 12:55
ogra_*know12:55
ogra_and that some process sends 13 times the same request over this connection12:55
ogra_which shouldnt happen12:55
ogra_we just need someonne to hit the issue again and collect the requested data and we should know more12:55
ogra_GRRRR !12:56
* ogra_ hates adbd 12:56
ogra_#define open ___xxx_open12:56
ogra_#define write ___xxx_write12:56
ogra_lovely ... aint it ?12:57
barryMirv: that looks like typical timeout errors.  you just have to retry the build12:59
jibelogra_, I attached the data this morning t 1394919, is there anything else to provide?12:59
jibel*to12:59
ogra_jibel, nope, lets wait for stgraber12:59
cwayneheyhey12:59
cwaynefreeze is tomorrow right12:59
davmor2cwayne: depends if the fix for cgmanager arrives there will be another spin tomorrow otherwise it is 169 which is already released, why?13:01
cwaynedavmor2: had a bugfix for https://bugs.launchpad.net/hanloon/+bug/139576713:02
ubot5Error: launchpad bug 1395767 not found13:02
ogra_cwayne, no more landings unless you get special approval from olli, victorp or pmcgowan13:02
cwayneack13:03
pmcgowanthat is indeed a bad one, see that all the time13:03
victorpcwayne, you will need a click which only has that change in from the current today13:05
victorpseems like alow of stuff has gone in for testing13:05
victorpso 1st prepare a branch for that13:05
victorpthen pmcgowan I think let it in if we end up rebuilding today, but not sure we should rebuild just for that one13:05
olliyeah13:06
Mirvbarry: I already reran it twice13:06
ogra_right, i would put that one in the langpack category13:06
Mirvso job #3, 4 and 5 failed within 12h13:06
cwaynei'm ok if we do it OTA, just thought the freeze was on wedsnesdays now (oops!)13:06
ogra_if we rebuild and it is available it can go in13:06
pmcgowanok with me13:06
cwayneok, ill get a branch with just that fix and make a click with it13:07
cwayneand go from there13:07
barryMirv: there are times when i have to re-run it 2-3 times to get a clean build.  if it's failing with timeout errors more often then that, then there could be a regression in udm, since si hasn't changed in vivid (there's a newer version in rtm, but nothing new in vivid yet).  is this with mandel's latest udm upload?13:12
mandelbarry, is with the latests, but did you see the errors? it complains about dry runs..13:13
barrymandel: look higher up13:13
barryi am testing it locally13:14
mandelack13:15
* sil2100 lunch o/13:20
=== alan_g|lunch is now known as alan_g
kenvandinefginther, jgdx said you were helping him with some issues with the settings tests run on otto last week, what's the status of that?14:24
fgintherkenvandine, the test environment was updated and the tests were passing again. Have they regressed yet again?14:26
kenvandineyeah, tedg has been trying to get CI passes and it's blowing chunks14:26
kenvandinehttps://code.launchpad.net/~ted/ubuntu-system-settings/silent-mode-trunk/+merge/24170914:27
kenvandinein otto there are crashes and everything fails14:27
kenvandinefginther, but there is one test failure in generic-deb-autopilot-runner-vivid-mako14:28
kenvandinewhich is a traceback that looks like something i see all over the otto job logs14:28
kenvandinebut they aren't all failing, just one14:28
kenvandinethe otto failures are significant that i would say we should just land his branch, if the others test pass14:29
kenvandinebut the other job has 1 failure in the same panel as his change14:29
kenvandineso i'm reluctant to land it14:29
kenvandinebut the traceback looks like an unrelated problem14:29
kenvandineand... the tests pass locally on mako14:29
fgintherkenvandine, we can update the otto environment and try again.14:30
fgintherkenvandine,although  the mako testing is a lot more reliable. It's doing the same testing the smoke testing runs, with the addition of the MP packages on top of the latest image14:31
kenvandinefginther, do those tracebacks look like something you saw last week?14:31
kenvandineright, which is why i hesitate to land14:31
fgintherkenvandine, I'll look14:32
kenvandinethx14:32
=== fginther changed the topic of #ubuntu-ci-eng to: Need a silo? ping trainguards | Need help with something else? ping fginther | Train Dashboard: http://bit.ly/1mDv1FS | QA Signoffs: http://bit.ly/1qMAKYd | Known Issues: RTM Archive frozen (no new silos landing) ! RTM cron builds disabled
fgintherbzoltan, the issue with the uitk staging branch running mako tests on non-vivid has been resolved. A test run using trunk is now passing: https://jenkins.qa.ubuntu.com/job/ubuntu-sdk-team-ubuntu-ui-toolkit-staging-ci/1268/14:34
kenvandinetedg, i went ahead and prepared a silo for your silent-mode branch, but won't publish anything until i have a better sense of the CI failure14:37
kenvandinefginther, ^^14:37
tedgkenvandine, K, cool!14:37
kenvandinetedg, don't get too excited, i'm pretty worried about that mako failure14:38
kenvandinei don't want to break smoke testing14:38
kenvandinetedg, but i'm pretty suspicious that it's something outside of settings... we'll need fginther's magic there :)14:39
fgintherkenvandine, should have a retest started in a few minutes14:39
kenvandinethx14:40
kenvandinewe'll get silo testing started soon, tedg has been trying to land this branch for weeks :/14:40
bzoltanfginther: \o/ thank you a bunch15:02
AlbertAtrainguards: silo 009 is still stuck in migration, any ideas why?15:06
sil2100AlbertA: let me take a look at that15:06
cjwatson    * amd64: mir-graphics-drivers-desktop, ubuntu-touch15:07
cjwatson    * arm64: mir-graphics-drivers-desktop15:07
cjwatson    * armhf: mir-graphics-drivers-desktop, ubuntu-touch15:07
cjwatson    * i386: mir-graphics-drivers-desktop, ubuntu-touch15:07
cjwatsonIt apparently makes those packages uninstallable15:07
cjwatsonAlbertA,sil2100: looks like a hardcoded and now-incorrect dependency in mir-graphics-drivers-desktop on libmirplatform3driver-mesa rather than libmirplatform4driver-mesa15:09
AlbertAcjwatson: oh yeah!15:10
sil2100Yeah, saw this in update_output.txt but didn't check the sources in time ;)15:10
sil2100cjwatson: thanks!15:11
kenvandinetedg, fginther: vivid smoke testing for settings is 100% currently, so this must be somehow related to tedg's branch... i just can't seem to see how15:11
cjwatsonWould be nice to unhardcode that, since it should be entirely possible to generate that dependency on the fly15:11
kenvandinefginther, is there any difference in how the smoke testing is run?15:11
cjwatson(If necessary, via a manual substvar)15:11
sil2100AlbertA: is it hardcoded in the end?15:11
AlbertAcjwatson: how should I do that?15:11
AlbertAcjwatson: sil2100: I believe the intention we had was to use those meta packages so that they could be listed in the seeds15:12
cjwatsonAlbertA: few minutes please15:12
AlbertAand the alternatives wouldn't have to be changed every release15:12
cjwatsonAlbertA: Something like http://paste.ubuntu.com/9233922/ should do it15:15
cjwatsonAlbertA: You still have to hardcode the binary package names, but that should at least somewhat reduce the potential for error15:15
cjwatson(binary package names> by which I mean the contents of Package fields)15:15
AlbertAcjwatson: cool thanks!15:16
Saviqtrainguards ↑ please :)15:47
sil2100ogra_: heeeeey, can I go for practice to day? Can I? Caaan I?16:19
ogra_hmm16:19
sil2100ogra_: (will you lead the meeting today ;) ? )16:19
* ogra_ ponders16:19
sil2100ogra_: pliiiiiz16:20
ogra_lol. ok ok16:20
ogra_:)16:20
sil2100Thanks ;)16:20
sil2100Anyway, I guess we only have the cgmanager thing to discuss... oh, and the sanity tests for #169 still running16:21
ogra_yep16:21
ogra_and checking the two missing smoke results16:21
sil2100I see management is leaning towards using #169 for the GM image anyway - and indeed I think we won't be able to get a fix till EOD :|16:23
ogra_unless stgraber or ted have any brioght ideas16:23
seb128did anyone notice a screen flicker on apps switch since 169?16:24
ogra_its not since 16916:25
seb128when I right swipe between apps the screen flickers16:25
sil2100Oh, I thought that was only because of debugging16:25
seb128could be since > 16516:25
ogra_thats old and there is a bug (and iirc a fix in vivid) for it16:25
seb128not sure I upgraded during the w.e16:25
seb128well, it started today for me, was not there with friday's image16:25
seb128and it persists accross reboots16:25
ogra_well, it is definitely an old bug16:26
AlbertAcjwatson: sil2100: so how are the packaging issues like I had in mir caught? is there something we can run in our CI to catch these sort of issues?16:27
Saviqtrainguards, can someone please publish vivid silo 30 for me :|16:27
cjwatsonAlbertA: not really I'm afraid16:30
cjwatsonAlbertA: one of the wishes for the new CI engine is to be able to run those tests per-silo though, in general; not sure how far that's got16:31
sil2100Saviq: doing! Been in a meeting!16:34
Saviqsil2100, you're always in a meeting! ;P16:34
sil2100I know! ;(16:34
sil2100Saviq: top approve plz!16:34
Saviqsil2100, OOPz16:34
tedgseb128, bug 139462216:36
ubot5bug 1394622 in ubuntu-app-launch (Ubuntu) "0.4+15.04.20141118~rtm-0ubuntu1 causes flickering on spread "alt+tab" gesture" [High,Fix released] https://launchpad.net/bugs/139462216:36
sil2100Saviq: ready?16:36
tedgseb128, Started with 166, fixed in vivid.16:36
seb128tedg, great, thanks16:36
seb128ogra_, sil2100, ^ it might not be an old bug?16:37
Saviqsil2100, just confirming why it was not top-ack, gimme a minute16:37
seb128tedg, do you know when the issue started?16:37
ogra_tedg, oh, wow, i thought it was older16:37
ogra_maybe i dreamt that16:37
tedgseb128, UAL version 0.4+15.04.20141118~rtm-0ubuntu116:37
seb128ogra_, ^16:37
Saviqsil2100, ready16:38
seb128sil2100, ogra_, olli, imho that should be fixed in rtm, that's a recent regression is quite visible16:38
ogra_seb128, well ...16:38
ogra_tedg, feel like preparing a silo for that ... in case mgmt decides we allow a rebuild16:39
ogra_?16:39
tedgseb128, Only do long right swipes, it doesn't happen there :-)16:39
fgintherkenvandine, there is no difference in how the test_suite is executed during MP and smoke testing. The only differences I can come up with are different devices and during smoke testing, other test suites run on the device (but the device is rebooted between suites)16:39
tedgogra_, Not allowed to have a silo unless it is critical and rtm.16:39
ogra_tedg, nonsense ... you can have silos as much as you want16:39
fgintherkenvandine, I've kicked off another test run on a mako. The updated otto run didn't look much better.16:40
tedgogra_, Hah, YOU can have silos as much as you want. :-)16:40
ogra_tedg, weather they *land* lies in the hands of ... well ... these three guys16:40
tedgThere's a have and have-nots of landing.16:40
ogra_but if we dont have a silo at all it surely wont land at all16:40
ogra_not sure if olli did a bug meeting today, i would have brought that issue up there16:41
kenvandinefginther, :(16:45
ollipmcgowan, ^16:45
pmcgowanoo that is bad, how did I miss that16:49
davmor2bfiller: open messaging, swipe up for a new message, add a contact/number, click on the camera bottom left, add a photo, then try an type in a message, the keyboard disappears after a letter or so16:49
pmcgowansil2100, how could that have been introduced with 166? what did we land there16:49
bfillerdavmor2: ack, bug that please16:50
ogra_pmcgowan, ubuntu-app-launch16:50
davmor2bfiller: wilko16:50
pmcgowanogra_, we obviously got more changes than we expected16:51
ogra_yep16:51
ogra_as usual :P16:51
ogra_the shiny world of sideefects ...16:51
ogra_olli, pmcgowan, in case you want to land this, we should let the apprmor fix (that is in the same u-a-l upload in vivid) in as well, so we dont get system slowdown when apparmor tries to collect the cproxy data (and fails)16:53
tedgapport :-)16:54
ogra_sigh16:54
ogra_things starting with "A"16:54
ogra_:)16:55
cwayneif we do that, can i also land my custom fix? :P16:55
tedgBetter than the number of things starting with "U" ;-)16:55
ogra_true16:55
ogra_i guess it is my subconcious that wants me to ping jamie all the time in a subtle way16:57
davmor2bfiller: https://bugs.launchpad.net/messaging-app/+bug/139624816:57
ubot5Launchpad bug 1396248 in messaging-app "Keyboard disappears when adding a image to an mms message" [Undecided,New]16:57
bfillerdavmor2: thanks16:57
davmor2title is bad I'll mode it16:58
tedgogra_, I think that jamie hates your subconscious :-)16:58
ogra_lol16:58
brendandci-help - i have some merge requests that aren't running tests in jenkins, what do i need to do? https://code.launchpad.net/~brendan-donegan/ubuntu-clock-app/wait_for_bottomedgetip_visible/+merge/242792 & https://code.launchpad.net/~brendan-donegan/reminders-app/test_add_notebook_must_append_it_to_list_swipe_to_bottom/+merge/24280816:59
brendandcihelp ^ ?17:00
fgintherbrendand, looking17:00
pmcgowantedg, if that fix was done last friday why are we only discussing today?17:06
fgintherbrendand, jenkins is finding your branches now17:11
tedgpmcgowan, I'm confused, what are you expecting? It's listed as a bug in RTM, and marked High.17:14
tedgThere are a bunch of those.17:15
pmcgowantedg, not regressions from the handful of landings we just did17:16
pmcgowanas a regression we should have been all over this17:16
pmcgowanmoot point now17:16
balloonsohh fginther lucky man today. So can you have a look at this autolanding job? http://91.189.93.70:8080/job/ubuntu-calculator-app-vivid-amd64-autolanding/2/console. It failed do to apt-get update (hash sum mismatch), aka the indexes were updating.17:24
balloonsWe shouldn't fail the job because of that; you should be able to just keep going17:24
cjwatsonballoons,fginther: You should apply the same workaround launchpad-buildd does17:24
pmcgowantedg, in the meantime, it would make sense to prepare an rtm silo if we do not have one yet17:25
cjwatsonballoons,fginther: End of http://bazaar.launchpad.net/~canonical-launchpad-branches/launchpad-buildd/trunk/view/head:/update-debian-chroot - in practice that's an extremely reliable workaround17:25
cjwatsonAs in I used to be forever retrying spurious failures due to that, now I can't remember the last time I had to17:26
cjwatson(OK, it's a bit different against archive.ubuntu.com, but should still help a lot)17:26
ogra_tedg, if you do that, please only th eone MP that fixes the issue ...17:26
ogra_in case there is a re-spin the fix can still be considered17:27
tedgpmcgowan, K, also bug 1394919 recoverable error fix?17:27
ubot5bug 1377332 in cgmanager (Ubuntu) "duplicate for #1394919 [TOPBLOCKER] UI randomly freezes" [Critical,Confirmed] https://launchpad.net/bugs/137733217:27
tedgWell, the bug got dup'd.17:27
ogra_tedg, if you do that, do it in two silos17:27
tedgogra_, It's already in a separate MR17:27
ogra_right17:27
ogra_but if mgmt decides they only want one of the two we cant easily unbundle it17:28
ogra_would need new QA etc etc17:28
tedgWell, you can't have the same project in separate silos.17:28
ogra_(if mgmt decides at all :) )17:28
tedgThe lock each other out.17:28
balloonscjwatson, awesome, thanks17:28
ogra_tedg, well, then only the flicker17:29
balloonscjwatson, since I have you btw, are you the right person to ask about supporting multi-arch builds with click automagically? Perhaps I specify the schroots (or click manages them)17:30
cjwatsonballoons: can I redirect you to mvo, unless he can't handle it?  I'm moving out of click development17:30
balloonscjwatson, right, I couldn't remember if that was on the list or not.. I thought it might be17:31
tedgtrainguards, rtm silo for line 57 please17:32
ogra_cjwatson, what ? you mean you want deploy LP in click packages ? :P17:32
cjwatsonogra_: some day you'll quit trolling and the world will implode ;-)17:32
ogra_lol17:32
davmor2cjwatson: pfff the world, it's the universe getting sucked into the blackhole that the world imploding causes that is more worrying ;)17:53
kenvandinetedg, which vivid build did you run these tests on?17:59
kenvandinetedg, i just ran it on image 32 mako, 6 of the 8 sound tests failed18:00
* kenvandine tries with vivid version18:00
=== alan_g is now known as alan_g|EOD
kenvandineinteresting, i downgraded to the vivid version and also had 6 failures in the sound tests18:06
kenvandinebut ran them again and had no failures18:07
* kenvandine tries silo again18:07
kenvandinefginther, is there any chance the mako devices in the data center have the wrong orientation?18:07
kenvandinewhen i had the failures system-settings was started rotated each time18:08
kenvandinei turned it for the second run, when they passed18:08
kenvandinecrap, one failure this time,  test_keyboard_sound_switch18:09
kenvandinefginther, ok, i have 3 runs in a row with the same 6 failures in the sound panel tests when in landscape18:14
kenvandineand 3 runs in portrait with just 1 failure each18:15
kenvandinewe clearly need to be more resilient to orientation in testing18:15
kenvandinetedg, adding a sleep(0.2) before the assert in test_keyboard_sound_switch gave me repeated passes18:19
kenvandinein portrait :/18:19
kenvandineon a complete autopilot test run i had 117 failures when in landscape18:20
kenvandinelooks like all of our tests need some love to make sure they work there18:21
kenvandine:(18:21
fgintherkenvandine, orientation could be a concern as I don't know if this is always guaranteed to be portrait when the device first comes up. The devices in the lab lay flat, nothing is moving them except for the occasional device that must be manually reset18:27
kenvandinefginther, i thought so18:28
kenvandinei'm getting the same number of failures as the otto job i was looking at earlier18:28
kenvandine11718:28
kenvandinewhen run in landscape18:28
kenvandinethat's a lot of tests we should fix to work in landscape :/18:29
kenvandinei guess being better at following the page object model guidelines could improve that18:31
fgintherkenvandine, hmm. I didn't even consider the desktop tests running under a different orientation.18:32
kenvandineis otto desktop?18:32
kenvandinei thought the job had "mako" in the name18:32
kenvandineoh it doesn't18:33
fgintherkenvandine, yes. otto basically runs a x86 desktop iso18:33
kenvandinequite a coincidence then that i had 117 failures on mako in landscape18:33
kenvandinedesktop it would always be portrait18:34
kenvandineso that can't be it18:34
fgintherkenvandine, if the desktop tests aren't useful right now, we could disable them. but I'll leave that up to the project team18:34
kenvandineseb128, ^^ what do you think?  the otto job runs the tests on desktop18:34
kenvandinehow much value do you think we get from that now?18:34
kenvandineseb128, it feels like that's where we spend the most time tracking down infrastructure/environment related failures18:35
kenvandineit'll be more important with convergence, but for now i see little value18:35
kenvandinepmcgowan, ^^ thoughts?18:38
ogra_sounds rather poointless to me18:39
kenvandineogra_, yeah, someday sure18:39
ogra_you miss armhf ... you miss the android container backends ...18:39
kenvandinebut many of our tests are around ofono, etc18:39
ogra_yeah and ofono18:40
kenvandinefor convergence we'll need to ensure we do desktop testing18:40
ogra_right18:40
kenvandinebut it'll be based on what is needed for desktop18:40
ogra_but thats still far out18:40
kenvandineright now it doesn't seem like it adds value18:40
kenvandineand we spend a TON of time chasing these problems18:40
ogra_ugh18:40
pmcgowankenvandine, I thought settings was soon to be the solution on desktop18:41
ogra_do you ever chase the ones in touch smoketesting at all then ?18:41
kenvandineogra_,  we try :)18:41
ogra_these are the only important ones currently18:41
kenvandineright now we're 100%18:41
pmcgowankenvandine, is the issue x86 or landscape? trying to follow the backscroll18:41
kenvandinepmcgowan, at some point, sure18:41
kenvandineneither18:41
kenvandineit's that right now we have most tests failing in otto, which is run based on the x86 desktop iso18:42
kenvandineand we're not really sure why18:42
kenvandinei happened to hit the same number of failures on my mako when run in landscape18:42
kenvandinebut on the desktop it would be portrait18:42
kenvandineso that can't be it18:42
pmcgowanwas it all the tests?18:42
kenvandinei'm filing a bug now to fix our tests so they don't fail in landscape18:42
kenvandineno18:42
kenvandineover 90% though18:42
kenvandineand we're still unsure why18:43
kenvandinethey pass on mako18:43
kenvandinebut totally busted on otto18:43
kenvandineand we're questioning the value those even bring us right now18:43
pmcgowanlast time ths happened the otto iso was old18:43
kenvandineyeah, fginther updated it18:43
kenvandinestill blows chunks18:43
pmcgowanlets check with seb on timeframe for settings app on desktop, I thought it was maybe 15.0418:44
kenvandinejgdx and i have sunk a ton of time chasing this18:44
kenvandineeven if it is 15.04, i think we need to look at the tests more for desktop18:44
pmcgowanand the failures dont tell us the reason?18:44
ogra_kenvandine, fyi test_language_page_title_is_correct fails in smoketesting18:44
kenvandinenot really... different things crashing... tracebacks, etc18:44
kenvandineogra_, i just looked this morning...18:44
ogra_(or failed today)18:44
kenvandinewell, i was checking vivid mako18:45
kenvandineto compare these results18:45
ogra_right, we dont look at amko18:45
ogra_:)18:45
kenvandineok... fair enough18:45
ogra_since thats not our product18:45
kenvandinei used that as an example to compare the CI runs18:45
ogra_nor do we check vivid18:45
kenvandinei'll check that out18:45
ogra_though system-settings occasionally has one or two failures18:45
kenvandinebut see... this issue testing desktop distracted from smoketesting tests :)18:45
kenvandineyeah, often transient18:46
kenvandinenext run will pass18:46
ogra_unlike camera-app that constantly is bad ...18:46
kenvandinei usually don't look closely until i see repeated failures18:46
* ogra_ looks at bf18:46
ogra_iller18:46
ogra_... who isnt here :P18:46
fgintherkenvandine, do the uss tests need to touch the bottom edge?18:46
ogra_kenvandine, yeah, if we see them in the daily LT review we usually ping someone18:47
kenvandinefginther, nope18:47
fgintherkenvandine, looking at the videos on the otto jobs, it appears that the bottom edge is off screen18:48
fgintherkenvandine, k. I have to run to an appointment, will be back in about an hour18:48
kenvandineyeah, i think it's just that we need to be better at scrolling to where we need to touch18:49
=== fginther changed the topic of #ubuntu-ci-eng to: Need a silo? ping trainguards | Need help with something brelse? ping cihelp | Train Dashboard: http://bit.ly/1mDv1FS | QA Signoffs: http://bit.ly/1qMAKYd | Known Issues: RTM Archive frozen (no new silos landing) ! RTM cron builds disabled
kenvandinefginther, thanks!18:49
seb128kenvandine, sorry, was at dinner, what's the issue/question?18:54
kenvandineseb128, how much value do you think we get from the settings otto tests for desktop?18:55
kenvandinei know with convergence we'll need to make sure we have good tests for desktop18:55
kenvandinebut right now i don't see a lot of value from those tests running on otto18:55
kenvandineand we sink a ton of time in trying to figure out what's broken18:56
robruAlbertA: disregard citrain failure message, looks like your package was uploaded just fine.18:56
robruAlbertA: let me know when you're ready to publish that18:56
seb128kenvandine, what are "otto tests"? running the autopilot tests on an otto setup?18:56
kenvandineseb128, and when we do put system-settings on the desktop, what we test will probably be different... so those desktop tests will be more about testing it for the desktop18:56
kenvandinenot testing the phone settings on x86 desktop iso18:56
kenvandineyeah, autopilot tests run on a base x86 desktop iso18:57
ogra_robru, in case you missed it, see teds trainguads ping above please18:57
AlbertArobru: yeah it's still building armhf in the ppa18:59
robrutedg: sorry for the delay, rtm218:59
kenvandineseb128, part of our problem could be the state of mir, unity8, etc for desktop in vivid19:00
kenvandinethose tests run with all that19:00
seb128kenvandine, I think we should start running desktop tests if we can, we don't especially want to stop on failures though19:02
seb128just get things going19:02
kenvandineso we should keep investigating what's causing the otto failures?19:03
kenvandineat least some of them seem to include unity8 crashes19:03
kenvandinebut others don't, with similar results19:03
kenvandineso actually, these tests don't test against desktop-next19:03
kenvandinethey test on the stock x86 desktop iso19:03
kenvandineperhaps otto should be using the desktop-next iso?19:04
seb128well, not sure19:05
seb128I though CI wanted to replace otto19:05
seb128or provide a new infra for desktop testing19:05
seb128ev said that was on the roadmap for septembre19:05
seb128I guess that's got delayed19:05
seb128but it's worth checking with them19:05
seb128in any case we need to start testing unity8 desktop-next19:05
seb128and apps in desktop mode19:05
kenvandineseb128, yeah, but is testing settings on the desktop iso add value right now?19:06
kenvandinewe don't even know why it's broken yet...19:06
kenvandinewho knows, maybe it's because we aren't using desktop-next :)19:07
seb128kenvandine, not really, it's going to fail because things like indicator-network are not running19:07
seb128or do we mock those?19:07
kenvandineah... not really19:07
seb128well in any case we want to test those under desktop-next19:08
kenvandinewe need to19:08
kenvandinebut jgdx struggled a bit with that19:08
kenvandinetedg's branch we are trying to land includes mocking the sound indicator19:08
kenvandineseb128, the otto tests do run unity8 though19:09
kenvandineso it should have the indicators19:09
seb128well, in any case testing under unity7/otto is better than nothing19:09
kenvandineunless something has changed about the way that runs on desktop19:09
kenvandineit's not unity719:09
seb128hum, k19:09
seb128so dunno19:09
seb128I don't know enough about the otto setup19:09
kenvandinewe see lots of dbus errors in the logs about qtmir19:09
seb128can we run that locally to debug?19:10
kenvandinedidrocks knows how to :)19:10
kenvandinei don't19:10
kenvandinehttps://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-vivid-fjg/4/console19:10
seb128my understanding was that otto was not properly supported by CI19:10
kenvandineseb128, ^^ a log19:10
seb128so maybe that's something that needs to be resolved on the CI side first19:10
seb128like they need to provide us a framework we can run desktop tests on19:10
kenvandineseb128, yeah, so i still think in it's current state, it doesn't provide as much value as it takes to make it work19:12
seb128right19:12
kenvandinewe clearly need something for desktop mode testing though19:12
seb128we need some CI infra that allow us to boot unity8-mir and run tests19:12
seb128otto is vm based right?19:12
seb128and mir doesn't run in vms?19:12
kenvandineseb128, so is that a +1 to disable the current otto test until we can sort out the right solution?19:12
seb128how do we manage to run unity8?19:12
kenvandineno idea :)19:13
kenvandinethese passed in utopic though19:13
kenvandinemaybe something mir related has changed and unity8 isn't starting right?19:13
kenvandineor we're missing depends?19:13
seb128well, mir never worked in VMs afaik19:13
kenvandinethe logs aren't very useful :(19:13
kenvandineyeah, so not sure how this worked19:13
seb128but yeah, +1 from me to disable tests19:13
seb128we need to resolve the desktop testing problem19:14
seb128but we need to start with a clean board19:14
seb128with step1 having a proper infra booting a desktop next iso19:14
kenvandinefginther, ^^ lets disable them19:14
kenvandineseb128, also in the settle tests i see the load is quite high in the otto tests19:14
seb128ok, on that note need to go19:14
kenvandinecould be related, something spinning out of control19:14
kenvandinegood night seb128!19:14
kenvandinethanks19:14
seb128yeah, it feels like we don't have a solid base19:15
seb128difficult to have datas19:15
seb128let's reboot that setup and see where we get19:15
kenvandinepmcgowan,  bug 139630519:16
ubot5bug 1396305 in ubuntu-system-settings (Ubuntu) "[autopilot] tests expect orientation to be portrait" [Undecided,New] https://launchpad.net/bugs/139630519:16
kenvandinewe should take care of that when we refactor our testing over time19:16
AlbertArobru: ok mir package has been built - sanity checked, ready to publish (silo 009)19:40
robruAlbertA: alright, here goes...19:40
fgintherkenvandine, ack19:41
fgintherkenvandine, thanks for digging on that19:41
kenvandinefginther, np, we surely appreciate all your help too19:42
kenvandinetedg, now manually testing the silo i notices i get no sound... but now that i think about it i noticed a call the other day where the ringer never rang, i just saw it19:43
kenvandineanyone know if there is a known problem with sound on vivid/mako?19:43
robruAlbertA: yeah I'm gonna have to wrangle this. gimme a few minutes...19:47
=== fginther changed the topic of #ubuntu-ci-eng to: Need a silo? ping trainguards | Need help with something else? ping fginther | Train Dashboard: http://bit.ly/1mDv1FS | QA Signoffs: http://bit.ly/1qMAKYd | Known Issues: RTM Archive frozen (no new silos landing) ! RTM cron builds disabled
tedgrobru, Thanks, building!19:52
robrutedg: you're welcome19:54
sil2100ogra_: any news regarding the cgproxy issue?19:58
sil2100jibel: ^ ?19:58
robruAlbertA: ok I think it's happening...20:07
jibelsil2100, nothing from me. I cannot reproduce with gdb attach20:08
jibeled20:08
robruAlbertA: I gotta step out for lunch. When you see version ...1125 show up at http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html#mir then we're golden.20:08
AlbertArobru: thanks!20:09
sil2100olli, jibel: so I guess we stick with #169, right? Did sanity testing pass on this? Did we start regression testing?20:13
olliyes, that's the plan of record20:14
sil2100olli, davmor2, jibel: so, I see #169's mako equivalent failed sanity testing...20:21
sil2100Ah, wait, no20:21
sil2100Correction20:21
sil2100The es image failed sanity testing20:21
sil2100What does that mean? Do we need a fix landing before we can promote?20:22
ollisil2100, otp, give me a few min20:22
sil2100Sure20:22
ollisil2100, done21:02
ollisil2100, so do we know why this only happens in the ES img?21:04
robrusil2100: you still around? can you help me interpret mir in http://people.canonical.com/~ubuntu-archive/proposed-migration/update_output.txt ? I guess we need a seed update? does that sound right?21:32
jhodapprobru, can you please publish vivid silo #1?21:40
robrujhodapp: please approve your merges: https://ci-train.ubuntu.com/job/ubuntu-landing-001-2-publish/51/console21:41
jhodapprobru, oh hrm...21:42
robrujhodapp: oh yeah, one's superceded. I guess we have to reconfigure and rebuild with the new MP.21:42
jhodapprobru, let me double check, one sec21:43
jhodapprobru, where are you seeing a superceded MR?21:45
robrujhodapp: https://ci-train.ubuntu.com/job/ubuntu-landing-001-2-publish/51/console one of these two has been superceded.21:46
jhodapprobru, argg, you're right...this silo will never land! ;)21:49
AlbertA2robru: that's strange....why does ubuntu-touch have a dep on libmirplatform3driver-android ?21:52
robruAlbertA2: I dunno, maybe indirectly? maybe some other component deps on it? can you check the rdeps and see if you can find it? I'm a bit busy21:53
AlbertA2robru: yeah I'll check it21:54
robruAlbertA2: thanks21:54
jhodapprobru, I need justinmcp_ to make a change before we can finish landing, so no worries for right now21:59
robrujhodapp: ok cool21:59
AlbertA2robru: it does look related to the seeds. I'll ping ogra_22:12
sergiusensrsalveti: line 59 on the sheet23:33
sil2100robru: update_output says that ubuntu-touch becomes uninstallable for the main archs23:48
robrusil2100: that's what I thought but I wasn't sure.23:49
robrusil2100: thanks for clarifying, that file is a mess.23:49
sil2100robru: yeah... there's not much documentation for it as well sadly23:50
sil2100o/23:50

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!