[05:54] <michi> trainguards: Could someone assign me a silo please? Ticket 970
[05:55] <robru> michi: you should be able to do it
[05:55] <michi> How?
[05:55] <michi> The build said out of silos
[05:56] <robru> michi: oh there's none left
[05:56] <michi> yep :)
[05:56] <robru> michi: i recently lifted the restriction on users assigning the last five silos, hopefully i don't regret that
[05:57] <michi> Well, so we are physically out of build machines now?
[05:57] <michi> In other words, I can’t have a silo?
[05:57] <robru> michi: not physically no, just PPAs. I'll have to free one or ten
[05:58] <michi> Can’t we just create more PPAs?
[05:59] <robru> michi: that's an option if we are often running low but I'd prefer not to reach for that the first time we run out
[05:59] <michi> Ah, I see.
[05:59] <michi> Well, I’m relaxed about it. But it’s not great if I can’t get a silo when I need it.
[05:59] <michi> I don’t mind asking for help here, that’s cool.
[05:59] <michi> And you are super-responsive each and every time!
[06:00] <michi> I’m just pointing out that not getting a silo at all is serious in terms of blocking progress.
[06:00] <michi> I did a horse trade of my thumbnailer silo last night with Pawel because he couldn’t get a silo either.
[06:00] <michi> Rather than bugging train guards, I just gave him mine because I didn’t need it for the moment :)
[06:03] <robru> michi: ok I just freed one, go ahead and assign yours
[06:03] <michi> Awesome, thank you!
[06:03] <michi> Hmmm… Maybe keeping a handful in reserve isn’t such a bad idea?
[06:03] <michi> I don’t know.
[06:03] <robru> michi: yeah there's plans to make the system more flexible, creating ppas dynamically as needed, but it's a ways off (lots of prereqs need to get shuffled around first)
[06:04] <michi> Ah.
[06:04] <michi> Sounds like you are on top of it :)
[06:04] <michi> I’m still in love with the train, BTW. It really works well!
[06:04] <robru> michi: yeah, it's not a bad idea, but the implementation was problematic so I regressed it while cleaning up some code. will probably have to reimplement it soon
[06:04] <robru> thanks
[06:05] <michi> robru: I noticed that it’s even more clever now.
[06:05] <robru> michi: what's more clever?
[06:05] <michi> When I hit “Mine”, I no longer have to edit the URL from michihenning to michi to see the results :)
[06:05] <michi> ^
[06:06] <robru> michi: oh right, yeah bdmurray fixed up the IRC names for the ticket creation & Mine link.
[06:06] <michi> \o/
[06:06] <michi> That was worth doing.
[06:06] <michi> Little things like that are not a big deal, but the cumulative friction over time is large.
[06:07] <robru> michi: yeah it was on the list for a long time but wasn't a high priority unfortunately
[06:07] <michi> Hey, I survived quite well until now, even without it. But I really appreciate your attention to detail!
[06:07] <robru> michi: unfortunately there's lots of big architectural shifts that need to happen that aren't very visible to users, so my time for papercut fixing isn't as much as I'd like
[06:08] <michi> Yes, well, I suspect the big shifts will also provide a much bigger bang, so they are worth doing first.
[08:08] <jibel> trainguards, on https://requests.ci-train.ubuntu.com/#/silo/048 tests are marked as still running, but I cannot find them in the running tests, could someone have a look?
[08:10] <robru> jibel: in running.html just ctrl f for landing-048, i see 4 different things queued
[08:11] <robru> jibel: also the excuses page shows which ones are running
[08:12] <robru> jibel: also, "running" is a misnomer, it also includes stuff that didn't even start yet
[08:12] <jibel> robru, I must not be looking at the right running.shtml, which link do you use?
[08:13] <robru> jibel: I'm not aware of any others: http://autopkgtest.ubuntu.com/running.shtml
[08:14] <robru> jibel: on the ticket you can click "excuses" and then click and if the blue "in progress" to get there
[08:14] <jibel> robru, ah it's queued, I was looking at the currently running tests
[08:15] <robru> jibel: that's why i said "ctrl f", comes right up
[08:15] <jibel> robru, thanks for the clarification
[08:15] <robru> You're welcome
[09:30] <pstolowski> hey trainguards, may i ask for removal of thumbnailer from ppa 24?
[09:47] <sil2100> pstolowski: on it
[09:49] <sil2100> pstolowski: removed from the PPA
[09:49] <pstolowski> sil2100, thanks!
[09:49] <sil2100> yw!
[10:18] <oSoMoN> trainguards: is it true that there’s no silo available?
[10:18] <Mirv> oSoMoN: it's true :(
[10:18] <oSoMoN> bleh :/
[10:18] <Mirv> we're hitting this wall now quite a bit
[10:19] <Mirv> I'm even building some builds in my other silos just because I can't get a new one
[10:19] <robru> Mirv: you should check for stale ones to free
[10:20] <Mirv> robru: I did yesterday, there are none that are >3 weeks old and even the oldest was being pinged about when I asked. I managed to get permission to free up 2 "in proposed" SRU:s but they were filled up immediately
[10:20] <Mirv> sil2100: help welcome in the hunt for freeable silos
[10:21] <sil2100> Mirv: let me try hopping on that in a moment
[10:21] <robru> Might be time to get forcible
[10:21] <Mirv> oSoMoN: there are 5 webbrowser-app silos btw, anything that can be considered?
[10:22] <Mirv> I think I'll move my vivid Qt 5.5 backport somewhere else since there is not much demand for that at the moment. else == my own PPA.
[10:22] <robru> sil2100: if this remains a problem all day I'll create some more during my shift tomorrow
[10:22] <oSoMoN> Mirv, 4 of them are actually webapp-container things, dbarth_ can you provide an update on those? can any of them be freed?
[10:38] <jamesh> I'm also looking for a silo, but I understand if that's going to be difficult short term
[10:59] <dbarth_> oSoMoN: hmm, yes probably, let me have a pass
[10:59] <dbarth_> i have only 3 silos on my list, 1 is webbrowser-app
[11:01] <oSoMoN> dbarth_, silos 13, 16, 21, 38
[11:02] <oSoMoN> indeed 3 of them are property of alex-abreu
[11:02] <dbarth_> oSoMoN: 21 could be freed i think
[11:22] <Mirv> dbarth_: oSoMoN: alex-abreu: ok, freeing up silo 21 https://requests.ci-train.ubuntu.com/#/silo/021 , that is https://code.launchpad.net/~abreu-alexandre/webbrowser-app/container-extension/+merge/278870 - take a note and land it later when it's time for it
[11:23] <Mirv> dbarth_: alex-abreu: I meant to link the ticket https://requests.ci-train.ubuntu.com/#/ticket/878 (which stays visible even after silo is freed)
[11:23] <dbarth_> Mirv: yup note
[11:23] <dbarth_> d
[11:26] <oSoMoN> huh, silo 21 was freed, and I got silo 27, looks like we were not completely short of silos after all…
[11:27] <Mirv> oSoMoN: yeah I noticed there was also 1 other silo freed
[11:27] <Mirv> I think it was the one that got published by sil2100 2h ago
[11:27] <oSoMoN> cool
[11:27] <Mirv> I'm "working" on freeing up my vivid Qt 5.5 silo but I've a build ongoing in that silo for a few hours still. but I'm moving it to https://launchpad.net/~canonical-qt5-edgers/+archive/ubuntu/ubuntu1504-qt551 for now.
[11:44] <Mirv> jamesh was the other lucky one to get a silo :)
[12:09] <jamesh> silo 21 should be up for grabs again shortly: I've combined its contents with another team member's related landing
[12:48] <kgunn> cjwatson: afternoon, i figure you'll know about this...question, is it normal for queue lengths on http://autopkgtest.ubuntu.com/running.shtml
[12:48] <kgunn> to be 0 for most archs/distros, but ~320 tests on ppc64...230 just on ppc64 xenial
[12:48] <kgunn> ?
[12:49] <cjwatson> kgunn: http://irclogs.ubuntu.com/2016/02/05/%23ubuntu-devel.html#t05:31
[12:49] <cjwatson> kgunn: https://rt.admin.canonical.com/Ticket/Display.html?id=88522
[12:50]  * kgunn reads
[12:50] <kgunn> thanks
[12:52] <kgunn> oh and jibel ^
[12:56] <kgunn> trainguards ^^ since there's an issue with ppc64 autopkg tests running, is there a manual override to the "automated" signoff?
[12:56] <jibel> kgunn, okay, there is a workaround
[12:58] <jibel> kgunn, ^
[12:59] <kgunn> jibel: thanks!
[12:59] <kgunn> jibel: all just to help actually find a bug :D
[12:59] <jibel> kgunn, it'll block again in xenial-proposed
[13:00] <kgunn> jibel: but will migrate to overlay right?
[13:00] <jibel> yes
[13:00] <kgunn> cool
[13:03] <cjwatson> kgunn: they're supposed to be not considered right now
[13:04] <cjwatson> kgunn: at least as I understand pitti's remarks
[13:05] <Mirv> kgunn: no packaging changes, I think you could just hit Publish?
[13:05] <kgunn> cjwatson: yeah no prob, it was knock on effect of bileto looking at the queue....an "automated" signoff was added recently
[13:05] <kgunn> Mirv: dang it...thanks!
[13:13] <morphis> sil2100: ping
[13:13] <sil2100> morphis: pong
[13:14] <morphis> sil2100: are we in a freeze for the overlay ppa now with 9.5 being imminent?
[13:14] <morphis> or you will just cherry-pick?
[13:15] <sil2100> morphis: hm, not really, I will be cherry-picking when needed
[13:15] <sil2100> But so far all landings were for 9.5
[13:15] <morphis> sil2100: so I can go ahead and push things for ota 10 timeline?
[13:16] <sil2100> morphis: hm, I suppose so, just need to remember to build out of the snapshot
[13:16] <sil2100> morphis: remember the silo needs sign-off by QA anyway
[13:17] <morphis> sil2100: sure
[13:17] <morphis> I just don't want to spend that time when we're in a freeze anyway
[13:19] <sil2100> kgunn: could you get https://code.launchpad.net/~mir-team/mir/0.19/+merge/284772 reviewed/approved?
[13:19] <Mirv> kgunn: did you notice it failed on no top-approve
[13:20] <Mirv> ah, :)
[13:42] <alex-abreu> Mirv, oSoMoN ack & sorry there have been some delays in the landing
[14:01] <kgunn> Mirv: gah!
[14:02] <alex-abreu> Mirv, I dont seem to be able to manually run britney on the excuses pages, ... "You are not allowed" ... for https://requests.ci-train.ubuntu.com/static/britney/xenial/landing-038/excuses.html
[14:02] <Mirv> kgunn: ok publishing in process now
[14:02] <kgunn> thanks...
[14:02] <kgunn> was one of those week
[14:02] <kgunn> s
[14:02] <Mirv> alex-abreu: yes, it's according to upload rights
[14:03] <Mirv> alex-abreu: I've clicked through those now
[14:04] <alex-abreu> Mirv, thx
[14:15] <Mirv> NOTE: 1 silo free, first one gets it! :)
[14:16] <Mirv> ok, now I'm able to free my vivid-overlay-qt-5.5 silo after moving it to canonical-qt5-edgers
[14:17] <Mirv> NOTE: 2 silos free, the fastest ones win :)
[15:35] <dobey> no silos :(
[15:54] <kenvandine> has anyone else run into a problem with xenial builds?  it's failing to install ofono during the build for all arches
[15:56] <kenvandine> Setting up ofono (1.17.bzr6910+16.04.20160115.3-0ubuntu1) ...
[15:56] <kenvandine> invoke-rc.d: unknown initscript, /etc/init.d/ofono not found.
[15:56] <kenvandine> dpkg: error processing package ofono (--configure):
[15:56] <kenvandine> but ofono hasn't changed in a couple weeks
[15:58] <dobey> oh ffs, why can't people consolidate their silos
[16:01] <Mirv> dobey: freeing up one of mine that FTBFS:s and waits on other landing too
[16:02] <Mirv> dobey: oh btw there are free silos otherwise too now, assigning
[16:02] <dobey> Mirv: thanks, but there are still people who have multiple silos for the same project, which could probably be combined into single silos instead
[16:04] <Mirv> dobey: please feel free to ping directly people asking whether they could do that. there are various reasons people have, and generally it's ok to have staging silos but overuse is a bit problematic until we lift the limit on the silos.
[16:04] <dobey> Mirv: yeah, but for things with the same target and are successfully built, and the MPs are all approved, having multiple silos doesn't make much sense :)
[16:05] <Mirv> the current situation is that all 60 silos are in more or less active use at least judging from when last built - https://requests.ci-train.ubuntu.com/#/silo/landing is sorted according to last touched, oldest have builds from 3 weeks ago
[16:05] <Mirv> dobey: that's very true
[16:06] <dobey> doesn't help that all of QA is stuck on not testing the landings at the moment, too, of course
[16:32] <Saviq> trainguards, please delete qtubuntu{,-gles} in silo 57
[16:32] <sil2100> Saviq: on it
[16:33] <Saviq> sil2100, jibel, FYI we're cutting our losses on the visibility thing for now, we have the feature working but there's unstable bits
[16:55] <Saviq> sil2100, jibel, I'm preparing to land 57 with minimal, important, fixes
[16:56] <jibel> Saviq, ack
[16:56] <sil2100> Saviq: k, does it have the window hint parts?
[16:56] <Saviq> sil2100, <Saviq> sil2100, jibel, FYI we're cutting our losses on the visibility thing for now, we have the feature working but there's unstable bits
[16:56] <Saviq> that's the window hint
[16:57] <sil2100> Ah, sorry, missed that message after deleting the packages ;)
[16:57] <Saviq> so *almost* there, but not stable enough IMO
[17:20] <Saviq> sil2100, jibel, FYI, isn't looking good, xenial dependency broken atm https://launchpadlibrarian.net/236411680/buildlog_ubuntu-xenial-amd64.unity8_8.11+16.04.20160205.1-0ubuntu1_BUILDING.txt.gz
[17:20] <Saviq> +chain
[17:20] <Saviq> so yeah... looks like Monday
[17:21] <sil2100> Grrrr
[18:38] <Wellark> sil2100: hi
[18:38] <Wellark> we want to force xenial-overlay to have the current version of packagekit
[18:38] <Wellark> as main will transition to 1.0 which breaks our .click installation
[18:39] <Wellark> in my opinion this branch should be uploaded manually to the overlay ppa and using the epoch we guarantee that the main version will never override the overlay version
[18:39] <Wellark> http://bazaar.launchpad.net/~unity-api-team/packagekit/xenial-overlay/revision/95
[18:39] <Wellark> what do you think?
[18:40] <Wellark> (that branch is based on wily, btw)
[18:41]  * ogra_ wouldnt use an epoch ... an epoch should always be the very last resort if nothing else works 
[18:43] <Wellark> ogra_: only for the overlay.
[18:43] <Wellark> where we can even delete packages if necessary
[18:43] <ogra_> still, if peope actually use apt on the phone you add one more breakage
[18:44] <Wellark> I don't want to get to the dance of uploading a new (same old) version of packagekit to the overlay every time main gets a new PA 1.0 package
[18:44] <ogra_> use the archive version string and attach something like "~but-actually-the-wily-version1" or some such crap
[18:44] <Wellark> ogra_: there is breakage already anyway
[18:44] <ogra_> thats better than an epoch
[18:46] <Wellark> and you have to remember to upload a new version to the overlay each time PA 1.0 gets an update or otherwise phone xenial images break
[18:46] <Wellark> for the phone, we freeze to 0.x
[18:47] <Wellark> as long as phone components depend on PA to install .clicks
[18:47] <Wellark> and I fail to see how this breaks apt on the phon
[19:08] <Wellark> sil2100: we would do the same epoch pinning for click as well
[19:09] <Wellark> both are in maintenance and we are not expecting changes to them
[19:09] <robru> Wellark: better would be a version number like '+16.05.YYYYMMDD', that way xenial releases using '+16.04...' will never be higher than that, but once the next release starts the packages from that series will "win" instead of having to have an epoch forever. once you add an epoch you can never remove it
[19:10] <robru> Wellark: also it's not clear to me that xenial images are even built with the overlay PPA, I thought that was just a vivid thing
[19:12] <Wellark> robru: the epoch only lives in the overlay
[19:12] <Wellark> and yes, we start building xenial images with the overlay very soon
[19:13] <robru> Wellark: yeah but eventually, like on xenial+1, you'd want the archive version to override what's in the overlay, and the only way to do that is to put the epoch on it. you'll never get a non-epoch version that installs over an epoch version, which is the whole point of epochs
[19:13] <Wellark> robru: no, we don't want xenial+1 to override
[19:14] <robru> Wellark: so the phone will just be stuck on this lower version *forever*
[19:15] <Wellark> robru: and that is exactly what we want. or we can just delete the packages from the ppa
[19:15] <Wellark> PA dropped support for plugins in 1.0 and they are not coming back
[19:15] <robru> Wellark: that's not how this works.
[19:15] <Wellark> and we depend on those plugins on the phone
[19:15] <alecu> robru: yes, we'll be stuck on the old version of PackageKit, until the phone moves to snaps
[19:16] <alecu> Wellark: s/PA/PK
[19:16] <robru> Wellark: if you "just delete the version from the ppa", apt doesn't just downgrade to whatever's available. apt only installs higher versions over lower versions
[19:16] <Wellark> *PK yes
[19:17] <alecu> robru: I guess this is not installed by apt, but instead by the phone image installer.
[19:17] <alecu> I mean, the system-image updater.
[19:17] <robru> alecu: Wellark: I guess you should talk to barry about how system-image would handle this situation
[19:19] <Wellark> robru: I don't understand what I should ask
[19:22] <Wellark> robru: we are not using apt on the phone. we deliver system-images that are freshly build against the overlay + main. dropping a package from the overlay with epoch will then be replaced by whatever is in the main archive
[19:22] <robru> Wellark: ask what the implications of using an epoch version in the overlay ppa are, what would happen in phone images xenial+1 that don't want those packages anymore, if they can just be deleted or if the archive version will need to take on the epoch version as well in order to be installed over the ppa version
[19:23] <robru> Wellark: I'm not familiar enough with system-image to know if that would actually work or not
[19:24] <robru> Wellark: this is the first I've heard anybody say that lower versions can be installed over higher versions
[19:26] <sil2100> Wellark: hm, why not just upload the current packagekit and click packages to the overlay and simply not update those?
[19:26] <alecu> sil2100: that sounds like what we want to do
[19:27] <robru> sil2100: it's a versioning issue. they want to epoch the version so that the xenial version number will never be higher
[19:27] <Wellark> sil2100: don't the never versions from main override them?
[19:27] <sil2100> Wellark: we shouldn't need to use an epoch on it, just upload the current version to the overlay and because we use pinning the builders will always use the version that's in the overlay
[19:27] <sil2100> No
[19:27] <Wellark> ok. that makes it simpler then
[19:27] <sil2100> Wellark: that's why we have the PPA pinning
[19:27] <alecu> wonderful
[19:27] <robru> heh, I forgot about the ppa pinning
[19:27] <sil2100> Wellark: anything that's in the overlay PPA will always override what's in other archives, unless you put a higher pin priority on those
[19:27] <alecu> sil2100: thanks for pointing it out!!!
[19:27] <robru> sil2100: I just made 20 new PPAs btw
[19:27] <Wellark> I thought you said yesterday that you indeed want newer versions from the main to override
[19:27] <sil2100> alecu: no problem :)
[19:27] <sil2100> robru: wooo! 81 silos then? :)
[19:27] <Wellark> but I misunderstood
[19:28] <robru> sil2100: yep!
[19:28] <sil2100> Wellark: no no, I just said we would need to remember to update the packages that are in the overlay when some security fixes are released in main
[19:29] <sil2100> Wellark: just so that we don't forget about the packages in the xenial-overlay
[19:29] <sil2100> Wellark: by 'update' I mean 'cherry-pick'
[19:29] <sil2100> And re-release on top of what's in that archive
[19:29] <sil2100> robru: excellent, thanks!
[19:31] <robru> yw
[19:49] <dobey> what's up with xenial builds? proposed is busted?
[20:09] <Wellark> sil2100, robru: could one of you do a manual upload of the current click and packagekit (from and to) stable-phone-overlay for xenial
[20:14] <robru> Wellark: what do you mean "from and to"?
[20:19] <sil2100> robru: from xenial to xenial in the overlay :)
[20:21] <robru> ok I can do that
[20:22] <sil2100> Thanks!
[20:23] <robru> sil2100: Wellark: here we gooooooo https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/stable-phone-overlay/+packages?field.name_filter=&field.status_filter=published&field.series_filter=xenial
[20:25] <Wellark> robru: awesome!
[20:26] <Wellark> alecu: --^
[20:29] <alecu> YAY!
[20:39] <cjwatson> robru: I've bumped authorized_size and relative_build_score on those new silos.
[20:39] <cjwatson> Wellark: when you say "not expecting changes to click", you're aware that there's in fact a change in a silo right now awaiting QA?
[20:39] <robru> cjwatson: oh, thanks.
[20:40] <robru> heh
[20:40] <cjwatson> Wellark: and it's not that long since we had a critical security fix there :)
[20:40] <cjwatson> Wellark: it would be nice to know what the process is there, because now click is going to be out of step with how its uploads were previously managed
[20:41] <robru> cjwatson: well, you'd do a dual silo as usual and then after it's done, copy from xenial archive to stable overlay ;-)
[20:50] <dobey> robru: i don't think it'll be that simple
[20:50] <dobey> because the point is that xenial is diverging and won't have the packagekit plug-in, while the overlay will keep it
[20:51] <robru> dobey: right, there'll be cherry-picking involved for overlay builds.
[20:51] <robru> dobey: cjwatson: also train can't be set to release a dual silo into xenial overlay (duals are hard-coded as xenial archive + vivid overlay), so you'd have to build a dual & then do manual copying from the silo ppa to the overlay ppa.
[20:52] <dobey> robru: i mean, i don't think "dual" will work any more
[20:53] <robru> dobey: well it depends, if you're working on a click branch "for the phone" you probably want the same thing released to vivid overlay and xenial overlay, right?
[20:53] <dobey> at least not without changes to click
[20:53] <dobey> robru: right, the problem is xenial non-overlay
[20:57] <cjwatson> yeah, exactly
[20:57] <cjwatson> I guess that can just be uploaded to the archive in the normal (non-train) way
[20:58] <cjwatson> until xenial+1 opens
[20:58] <robru> cjwatson: there's no reason you couldn't build it in a silo and publish to the archive.
[20:58] <cjwatson> robru: is, because that would build against stable-phone-overlay
[20:58] <cjwatson> robru: and thus link against PK
[20:58] <robru> cjwatson: like you could have one xenail (non dual) silo targeting overlay and a different silo targeting archive
[20:58] <robru> oh right
[20:58] <cjwatson> robru: well, it's true that I could manually reconfigure a silo to not do that
[20:59] <robru> cjwatson: we can manually remove the overlay from the landing PPAs on a case by case basis ;-)
[20:59] <robru> yeah
[20:59] <cjwatson> it's error-prone, but it's possible
[20:59] <robru> brb
[21:00] <cjwatson> robru: um, you didn't get these new silos configured for all arches ...
[21:00]  * cjwatson goes to do that
[21:00] <dobey> sigh, whey did they have to break packagekit
[21:00] <cjwatson> PK plugins were a bit awful in fairness
[21:04] <cjwatson> robru: (fixed)
[21:29] <robru> cjwatson: what? I asked webops to devirt and I then I went through and enabled all the arches already?
[21:31] <robru> cjwatson: like I specifically checked that the +edit page had everything checked already. where are you looking?
[22:22] <dobey> https://launchpadlibrarian.net/236529470/buildlog_ubuntu-xenial-amd64.unity-scope-click_0.1.1+16.04.20160205.1-0ubuntu1_BUILDING.txt.gz :(
[22:24] <dobey> i guess this is because of new apt in proposed?