/srv/irclogs.ubuntu.com/2016/03/24/#ubuntu-ci-eng.txt

=== chihchun is now known as chihchun_afk
=== chihchun_afk is now known as chihchun
SaviqMirv, morning... any idea where unity8 from https://requests.ci-train.ubuntu.com/#/ticket/1138 ended up on xenial? the silo was force-merged so that we could speed up other landings, but a few packages seem to have ended up in oblivion :(08:00
Saviqalso, any idea about the "blanket FFe" we got before on touch components? is it still in effect or should we actually open up xenial overlay already?08:02
jibelMirv, can you help vicamo with the publication of silo 70. The is a packaging change which I think need a ack from someone08:33
jibelthere*08:34
robruSaviq: I'm not sure what happened specifically in that case, but anything force merged while a silo is in "UNAPPROVED queue" would wind up in oblivion, yes. force merging is only safe for packages in proposed pocket.08:47
robruSaviq: indeed http://paste.ubuntu.com/15485856/ the ones listed as "UNAPPROVED queue" are lost and you can blame kenvandine for force merging08:48
Saviqrobru, yeah was worried that was what happened08:49
robruSaviq: best course of action I guess is to hurry up and finish the next silo and get it published, assuming it has all the same packages. if it doesn't, you'll need a new silo with null merges in order to get trunk released to xenial08:50
Saviqrobru, can we not recover the packages in the silo and publish?08:50
Saviqundelete/copy to a new silo08:50
Saviqthey're all there after all08:51
robruSaviq: yeah I suppose you can fish them out: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-013/+packages?field.name_filter=&field.status_filter=&field.series_filter=08:51
Saviqyup that was what I was thinking08:51
robruSaviq: probably easiest to get a core dev to just copy them directly to -proposed, less work than starting a new silo and copying to a new PPA and then publishing all over again08:52
Saviqrobru, yup08:52
Saviqsil2100, morning!08:52
Saviqyou awake yet? ;)08:52
sil2100Saviq: morning!08:52
robrusil2100: so the new source package build code is live and has had a dozen or so successful builds today, including GLES packages, so I don't expect you'll see too many surprises at this point. watch out for unity8, mediascanner2, unity-scopes-api and unity-scopes-shell, those ones will now require the branches I submitted to them already.08:59
Saviqsil2100, so... because we jumped the gun yesterday, some packages from silo 13 ended up in oblivion (UNAPPROVED queue, so never got into proposed)08:59
Saviqsil2100, http://pastebin.ubuntu.com/15485893/08:59
Saviqany chance you could fish them out from the PPA and into proposed?09:00
sil2100Saviq: let me take a look09:03
sil2100uh, unity8 landed in the UNAPPROVED queue?09:04
Saviqthat's what the train said, yeah09:05
Saviqand then, because Ken force-merged it (because we had no idea it wasn't good until things are in proposed), they just went POOF09:05
* Saviq thought UNAPPROVED is *in* proposed09:06
sil2100I thought things would be good as well - it doesn't see them as valid anymore?09:06
sil2100Since it should still be visible as a sync in UNAPPROVED, so in theory it should migrate to -proposed once someone pushes the button09:07
Saviqnot sure where UNAPPROVED is, though09:07
sil2100cjwatson: hey! An archive-related question: we have a bunch of uploads that went to the UNAPPROVED queue (CI Train copy-package from a silo PPA) and then we got the silo freed (with the packages removed) - did the package removal cause any issues here, or are we still safe?09:09
robrusil2100: what? no. merging in UNAPPROVED has never been legit.09:12
sil2100robru: what's happening then?09:13
robrusil2100: Saviq: "pockets" are places where packages actually live, eg proposed pocket and release pocket. "queues" are like TODO lists "eventually copy this somewhere". if you delete a package from a PPA and it's just in UNAPPROVED queue, then when they go to copy it they find the source PPA is empty09:14
Saviqah09:14
sil2100robru: deleted packages are still in the PPA, they don't disappear completely ever09:14
sil2100So LP is not able to find them just because of that?09:14
Saviqbut the copy probably looks for one in Published state09:14
sil2100That's a bit sad09:14
robrusil2100: well however queues are managed they don't see deleted PPA packages. this was always the case. every time somebody force merges packages in UNAPPROVED eventually some archive person pings me (like months later) saying "hey what happened to this sync, the source disappeared"09:15
Saviqsil2100, maybe easiest to copy the deleteds to a new silo and publish once more?09:15
robrusil2100: this situation will only get worse as we move to ephemeral PPAs and delete the entire PPA rather than just delete packages from the PPA.09:15
robrueg the PPA will really be deleted and it won't be possible to just "copy the deleted package"09:16
Saviqso maybe force_merge should not allow that when copys are in a queue09:16
MirvSaviq: jibel sorry, on holiday today09:17
SaviqMirv, go away then ;P09:17
robruSaviq: well at one point all the train guards knew this, which is why we made it so that only train guards can force merge, I'm not sure why everybody forgot this recently.09:17
jibelMirv, np sil2100 is on it. Enjoy your holidays09:17
sil2100Saviq: yeah, that's the plan I'm realising ;)09:17
Saviqrobru, they probably forget every time a new release is open so no UNAPPROVED ;)09:18
Saviqsil2100, thanks09:19
robruSaviq: I guess. ken should have known this too. argh.09:20
Saviqrobru, now I know :)09:20
oSoMoNMirv, do you have rights to publish oxide-qt? silo 80 is ready to land09:22
sil2100oSoMoN: I'll be on it once I deal with this aftermath here09:23
oSoMoNsil2100, thanks09:24
davidbarththanks indeed, it's ready and hopefully the last release for that ota09:24
robrusil2100: sounds like you have everything under control. goodnight!09:28
om26erdobey, Hi! What should I look out for in silo 66 ?09:28
jibelom26er, the U1 login page is blue and it shouldn't09:31
jibelI think that's the visible part of the buf09:31
jibelbug*09:31
om26erjibel, that rightly is Fast track09:32
jibelom26er, https://launchpadlibrarian.net/248086202/screenshot20160314_120118243.png is the bug09:33
om26erjibel, does 'Fast track' tag also means we don't need to go through the TestPlan or is it still required to run the test plan ?09:34
jibelom26er, run the part of the test plan affected by the change only, verify that the issue is fixed09:35
morphissil2100: ping09:36
Saviqrobru, the gles refactor in train is not yet in effect, is it?09:36
sil2100morphis: pong09:37
morphissil2100: you talked with QA about https://requests.ci-train.ubuntu.com/#/ticket/1145 ?09:37
sil2100morphis: yes, they should be aware09:38
morphissil2100: great, thanks09:38
morphisjibel, davmor2: any reason why I don't see https://requests.ci-train.ubuntu.com/#/ticket/1145 on the QA board yet?09:38
=== king-afk is now known as cking
davmor2morphis: because we hate you ;) I can create a ticket but I think jibel can set it in the system so we get all the info from bileto09:41
morphisdavmor2: that is the reason I see :-)09:42
morphisdavmor2: more to come :-)09:42
davmor2morphis: no you only get one for ota10 final freeze and all that :P09:43
morphisdavmor2: I am just development others take the decision what to push and what not :-D09:46
* davmor2 makes note to block everything morphis lands in future :P09:47
morphis:-)09:47
jibelmorphis, proposed migration failed with unsatisfiable deps09:54
jibelmorphis, https://requests.ci-train.ubuntu.com/static/britney/vivid/landing-028/excuses.html09:54
morphisjibel: yes, but that is just because vivid itself misses thigns and britney doesn't know about the overlay ppa properly (that is what sil2100 told me)09:56
morphisjibel: we ignore that execuse last time09:56
jibelmorphis, ^10:01
morphisjibel: thanks!10:03
sil2100Saviq: did you get my earlier message?10:23
robruSaviq: the recent rollout did affect how gles are handled but you still need to supply null merges for gles packages. Only difference is the get-orig-source target is ignored & handled by train now10:33
robru(I'm totally not here tho)10:33
robruSaviq: the larger refactoring where gles is totally handled for you is caught up with some other big changes and will take a bit longer. For now, enjoy the faster build times ;-)10:37
cjwatsonsil2100: Deleted packages are remembered forever to the extent that you can't ever reuse a version number in the same archive, but the actual files are garbage-collected after a short delay (a day or two?  I forget) after deletion10:38
cjwatsonIn an ideal world the copy request would hold a reference such that they aren't GCed10:38
cjwatsonBut it needs quite a lot of complex refactoring before that's possible, unfortunately10:39
Saviqsil2100, don't think I did10:45
sil2100Darn VPN issues10:45
sil210011:20 < sil2100> Saviq: I got preempted for a bit from your silo (emergency in the turbo10:45
sil2100                 world), but I'll get back to re-adding the remvoed packages once I can10:45
Saviqsil2100, nw, thanks10:45
SaviqI know what's the prio10:45
sil2100Saviq: I'm actually doing some copies in the background to silo 57 already10:46
Saviqtx10:47
=== chihchun is now known as chihchun_afk
=== chihchun_afk is now known as chihchun
=== _salem is now known as salem_
bzoltansil2100: Something is fishy here - https://launchpadlibrarian.net/249786544/buildlog_ubuntu-xenial-armhf.ubuntu-ui-toolkit_1.3.1908+16.04.20160324.2-0ubuntu1_BUILDING.txt.gz11:38
bzoltansil2100: `dbus-test-runner --task gdb [...]` segfaults11:39
=== chihchun is now known as chihchun_afk
sil2100cjwatson: thanks - another quick question: the package publisher in LP, how much time does it actually need for a single package?12:13
sil2100Since I made a snapshot (so a lot of copies from one PPA to another) and I'm waiting for them to publish since over 30 minutes12:13
cjwatsonsil2100: That's not a meaningful question, because it doesn't operate at the single-package level; it basically runs in a loop republishing all PPAs that have changed.  So it depends on other activity in the system.12:20
cjwatsonsil2100: That said, it's spent the last hour or more working on nothing but your snapshot.12:21
cjwatsonsil2100: Looks like it's maybe two-thirds of the way there or so?  I can only guess12:22
cjwatsonYou're asking for it to copy an awful lot of data around over the network, so it takes quite a while.12:24
cjwatsonIt's literally been streaming data out of the librarian for basically an hour.12:24
sil2100hah12:25
sil2100Thanks ;)12:25
cjwatsonThen it will have to think for quite a while to generate the indexes.12:25
cjwatsonNot sure how long that will take.12:26
Saviqsil2100, mterry's around now so we should be able to untangle the situation ourselves12:55
mterrySaviq: I don't have UNAPPROVED rights12:56
popeyjibel: do you have someone assigned / planned to test dekko for OTA-10?12:56
popeyjibel: I don't see a card in https://trello.com/b/AE3swczu/qa-testing-requests-for-questions-ping-ubuntu-qa-on-ubuntu-ci-eng12:56
mterrySaviq: I think that's release team?12:56
mterrySaviq: no.  archive admins12:56
Saviqmterry, as long as we start with putting the packages in the new PPA and publishing them, we can then start chasing them12:57
sil2100mterry, Saviq: no worries, I almost have all the packages in the PPA12:57
sil2100Will publish in a moment12:57
Saviqktx12:57
mterrySaviq: oh wait, maybe I misunderstood.  I assumed the packages were sitting in the UNAPPROVED xenial queue.  You're talking like they're lost and we're doing archeology12:57
Saviqmterry, UNAPPROVED queue is just that - a queue, the packages can't sit there12:58
Saviq(I learned this morning)12:58
Saviqit's just a reference to packages that are to be copied12:58
mterrySaviq: well they can if the queue isn't processed right?12:58
Saviqmterry, they still need to remain in where *from* they are to be copied until they're copied12:59
mterrySaviq: oh....12:59
Saviqmterry, queues don't actually hold source packages12:59
mterrySaviq: so deleting the PPA was bad then.  I get it12:59
Saviqyup12:59
mterrySaviq: that's funny12:59
sil2100mterry, Saviq: all packages in silo 57 now12:59
Saviqcoolz12:59
cjwatsonsil2100: Oh, didn't actually take too long after I said that.13:00
sil2100cjwatson: indeed :)13:00
cjwatsonmterry: It's basically a Launchpad bug, but a very difficult one to fix.13:00
sil2100cjwatson: waiting for a few others to get published that I had to copy afterwards13:01
cjwatsonSaviq: Not even a proper reference; it just names the things to copy.  If it were a reference then this would be easily fixable.13:01
Saviqright, depends on the definition of a reference :)13:01
jibelpopey, if there is no request there won't be any card on the board13:03
chrisccoulsonCan I have a silo to land the fix for bug 1559428 ?13:05
ubot5bug 1559428 in Canonical System Image "requesting location updates in oxide webview triggers memory corruption" [High,Confirmed] https://launchpad.net/bugs/155942813:05
greyback_trainguards: hey, what have I got wrong to cause this: https://ci-train.ubuntu.com/job/ubuntu-landing-013-0-status/9529/consoleFull13:11
kenvandineoh geez... i had no idea... if UNAPPROVED is just a reference what happens to packages that were dput?13:11
kenvandineSaviq, robru: sorry about that... i thought that was ok13:12
Saviqgreyback_, that's likely the same issue - packages not published to xenial13:14
Saviqgreyback_, but it shouldn't be fatal13:14
sil2100greyback_: eh, you got silo 13 huh?13:14
greyback_sil2100: yeah13:15
greyback_good luck 13, eh?13:15
dobeyom26er: check for regressions and that the colors are correct13:15
Saviqright, that might be interesting :)13:15
sil2100greyback_: I suppose this will clear out once we get the packages re-published13:15
dobeyoh i guess jibel answered13:16
kenvandinei didn't read all the scrollback, did someone figure out how to resurrect the packages?13:16
om26erdobey, approved that already13:16
sil2100Saviq: could you confirm https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-057 has all packages we need?13:16
greyback_sil2100: Saviq, ok thanks13:16
Saviqkenvandine, yeah, copied to silo 57 and publishing again13:16
kenvandinewhew13:16
dobeyom26er: yeah, just saw. thanks13:16
Saviqsil2100, UNAPPROVED queue (geonames/xenial, indicator-datetime/xenial, indicator-session/xenial, qtmir-gles/xenial, qtmir/xenial, qtubuntu-gles/xenial, qtubuntu/xenial, unity-api/xenial, unity8/xenial).13:17
Saviqso yeah, looks fine13:17
Saviqsil2100, but waiting to publish - you saturated LP with the snapshot ;)13:17
sil2100...;)13:17
sil2100Yeah, sorry about that13:18
Saviqsil2100, btw, are we opening xenial overlay any time soon?13:19
dobeysil2100: can you publish/pkg ack silo 66?13:19
sil2100We could, that would be nice to discuss and decide for next week13:19
cjwatsonkenvandine: unapproved is just a quasi-reference in the case of copies.  it's somewhat more concrete in the case of direct uploads.13:21
kenvandinecjwatson, noted... i didn't realize it was handled differently for copies13:22
sil2100dobey: will do, got preempted to turbo agian13:28
Saviqmterry, can you please publish https://requests.ci-train.ubuntu.com/#/ticket/1179 - the PPA seems published now14:12
mterryk14:12
Saviqsil2100, FYI ↑14:12
mterrySaviq: it says "UNAPPROVED queue"?  makes me think it already got published?14:13
Saviqmterry, well, I'm not sure what will happen - the packages were added to the queue, but then disappeared from the PPA14:13
Saviqit might be we'll actually need to no-change rebuild them, otherwise archive might reject since it's the same version14:14
Saviqmterry, or we ask someone with the powers to actually copy to proposed14:14
mterryseb and didier aren't around...  let me see who else to bother14:16
sil2100Just re-publish14:16
mterryok14:16
sil2100We allow re-publishing sources from the same silo, the same should apply if it's from a different silo14:16
mterrydone14:17
sil2100mterry: thanks!14:19
Saviqmterry, looks a bit no-op... https://ci-train.ubuntu.com/job/ubuntu-landing-057-2-publish/6/consoleFull14:22
Saviqok so how do we now ask for these to be copied to proposed? we need FFe?14:23
mterrySaviq: well we got an FFe before...  that was before final freeze hit.  But I think we should be fine.  Just need to bug an archivee admin14:25
Saviqmterry, can you please take care of that?14:27
mterrySaviq: ok14:28
cjwatsonSaviq: version checks don't apply to things in the queue FWIW14:31
cjwatsonthis may or may not be considered a feature14:31
Saviq:)14:35
pstolowski_sil2100, hey, any idea why this keeps failing https://launchpadlibrarian.net/249805433/buildlog_ubuntu-xenial-amd64.unity-scopes-shell_0.5.7+16.04.20160324.3-0ubuntu1_BUILDING.txt.gz ? note http://pastebin.ubuntu.com/15487628/14:47
charleshm15:33
charlesindicator-display test failure, let's see what happened...15:33
pstolowski_cjwatson, hey, any idea about the issue I mentioned above (1h ago)?15:49
pstolowski_cjwatson, ah, i just noticed the comment from robru under this silo...15:53
robrupstolowski_: I'm not sure if my comment would result in that failure, but if you look at the diffs that were generated, your debian/control file is wrong because you're missng my branch15:54
pstolowski_robru, i've just added your MP to my silo, retrying15:54
jibelKaleo, bfiller_ silo 30 approved15:59
sil2100jibel, davmor2: if you guys don't mind, let's skip today's LT meeting - all is clear I suppose nayway16:00
bfiller_jibel: I saw, thank you16:00
jibelsil2100, +1 to skip16:00
davmor2+1000 to skip it16:00
cjwatsonpstolowski_: doesn't look especially exotic, you're missing a build-dep16:00
cjwatson(on cmake)16:00
robrupstolowski_: diffs look good with my branch in place, good luck with your build failure!16:01
pstolowski_cjwatson, i've cmake in build-deps (note, we have control.in and control is generated). it's most likely the pre-release-hook change from robru that's now needed16:03
cjwatsonI guess16:03
pstolowski_yeah i think it looks good so far in the silo16:04
charlesmorphis, ondra, this is a bit of a tracer so I wouldn't be surprised if this one fails too16:17
morphischarles: aye16:18
Saviqrobru, here's my approach https://code.launchpad.net/~unity-team/qtmir/inline-gles-quilt/+merge/29006116:44
charleswell, ok then16:47
Kaleojibel, thanks!16:50
jhodapprobru, so what's the process with britney for silo 53...it's been approved by everybody including QA, ready to land, but now qtubuntu-camera needs a rebuild because another qtubuntu-camera change landed first17:00
robrujhodapp: nothing to do with britney... You'll need to wait for the other silo to merge, rebuild qtubuntu-camera, resubmit to qa, and stop submitting conflicting silos to qa as it's a total waste of their time.17:02
jhodapprobru, well I had no way of knowing there was another qtubuntu-camera in another silo17:03
jhodapprobru, also, who's to say that I wasn't first? QA just completed theirs before my silo17:03
jhodapprobru, this process feels like it could be improved a bit is all I'm trying to say17:04
robrujhodapp: you should be coordinating this with other landers. Once upon a time the train wouldn't even allow you to have the same package in two silos, do i need to bring that back?17:04
jhodapprobru, I think a simple warning detecting the process would be useful, I don't think you need to make it mandatory17:04
jhodapprobru, I was coordinating with the owner of qtubuntu-camera17:05
jhodapprobru, that warning could be useful in these times when there's a higher potential for overlapping commits right before an OTA freezes17:06
robrujhodapp: I'm not sure what a good solution is, the thing is this is an inherent problem of vcs systems, not some train specific deficiency. You have to merge the other silo that was published first otherwise your publish will revert the other one17:07
jhodapprobru, right and that's why I think a warning or other type of way to present to someone who is creating a new request to the same project might be useful. Help them be aware of it. The main list of active silos is large and hard to read IMO so that info will be lost there.17:08
robrujhodapp: there used to be code to iterate over silos and detect conflicts and warn about them but it doesn't scale when you have so many silos, it was taking forever to run17:10
jhodapprobru, that would run when submitting a new silo request?17:10
robrujhodapp: at the time it ran during every build and added many minutes to the build process17:10
jhodapprobru, yeah, maybe it could run once on demand when someone requests a silo17:11
robrujhodapp: i think what we really need is some way to track it on the qa side, "this package cannot be qa'd again until the last one that got approved merges"17:11
robrujhodapp: that doesn't work because the set of packages in a silo isn't known at assign time. Also the concept of assignment is on the way out17:12
jhodapprobru, I don't think it has to be that heavy handed...in my case if I could simply know that there already is an existing landing request again qtubuntu-camera I'd know to go coordinate with that person17:12
jhodapprobru, ok how about run it right after assignment happens17:13
robrujhodapp: because packages can change over the life of a silo, it would need to be updated periodically17:13
jhodapprobru, that wouldn't matter how fast it is then, because how many people request a silo, get it assigned and then immediately want to land it17:13
jhodapprobru, so run it any time the package list changes17:13
jhodappthat'd be a perfect trigger17:14
robrujhodapp: please file a bug against lp:bileto and specifically mention that qa people are wasting time reviewing stuff that can't possibly land.17:14
jhodapprobru, will do man, thanks!17:15
robruYou're welcome17:15
jhodapprobru, apologies if I came off annoyed to you, I was frustrated17:15
robrujhodapp: yeah sorry my first message was overly accusatory17:15
jhodapprobru, np, I'm interested in improving the process so that's why I ping you17:16
robrujhodapp: yeah there's lots of improvements to be had. I'm working on parallelizing builds now so they'll be even faster17:16
jhodappthat's awesome17:16
robruWish I'd thought of it sooner!17:17
jhodappincremental improvements :)17:17
robruYep17:17
jhodapprobru, here's the bug I just filed: https://bugs.launchpad.net/bileto/+bug/156167318:15
ubot5Launchpad bug 1561673 in Bileto "Warn the lander if there's the same project set to land in their silo as in another lander's silo" [Undecided,New]18:15
jhodapprobru, what do you think of that last paragraph idea?18:15
jhodapprobru, so basically that's a way to synchronize overlapped landings in a first-come-first-serve kind of way automatically18:16
robrujhodapp: yeah that's an interesting idea, as britney is already gating what goes to QA or not.18:17
jhodappexactly18:17
jhodapponce the other silo has landed, then britney can be tried again18:17
jhodappthat could be automatic but doesn't have to be18:17
robrujhodapp: something to think about. britney wouldn't be able to store that state but we have a little wrapper script around britney that could do it...18:18
jhodappoh awesome18:18
jhodapprobru, but that would at least make it impossible for two things that have the same package in it to show up on the QA board, and I'm sure they'd be happy with that18:18
robruyeah18:30
camakorobru, the MP you added to silo 80, is it needed even though this is a test silo (not meant to be landed)?19:39
robrucamako: yep, trust-store doesn't build right without it (the train changed, this brings trust-store up to speed with the train changes). particularly the vivid packages will be built wrong. it'll be needed in all trust-store silos until one of them eventually lands on trunk19:40
camakotvoss, ^19:42
robrucamako: if you saw my 'ACTION REQUIRED' email on ubuntu-phone this week, trust-store is one of the affected packages.19:44
tvossrobru, please elaborate19:44
camakorobru, ack I saw... It's just that this silo is not to land19:44
robrutvoss: https://lists.launchpad.net/ubuntu-phone/msg19015.html19:44
robrucamako: doesn't matter... the package won't actually build correctly without it. the SONAME in vivid will be wrong19:44
camakorobru, ok gotcha19:45
camakothanks for your quick action19:45
robrucamako: you're welcome. I've been monitoring the train for affected packages because it's a tricky change.19:46
robruSaviq: you need my unity8 branch in https://requests.ci-train.ubuntu.com/#/ticket/1186 or you won't get translations updated19:51
Saviqrobru, I know, that's ok, there's no translation changes there20:10
Saviqwanted to keep this silo as small as possible20:10
salem_trainguards: hi, can anybody trigger a rebuild of messaging-app/xenial/powerpc and ppc64el on silo 42?21:32
robrusalem_: on it21:35
salem_robru, thanks21:35
robrusalem_: you're welcome21:38
=== davmor2 is now known as davmor2_HOLS

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!