[07:26] <dbarth_> sil2100: hey, just to confirm: there's a xenial overlay now, right?
[07:26] <dbarth_> ie, if we land a dual silo with oxide, it's not going to touch the archive for real?
[07:32] <sil2100> dbarth_: hey! Yes
[07:32] <sil2100> dbarth_: the phone is operating on the xenial-overlay now in dual landings
[07:35] <dbarth_> sil2100: cool, thanks
[07:36] <dbarth_> oSoMoN: so I think we're good to go with the silo now
[07:37] <oSoMoN> dbarth_, except for the fact that the xenial build of oxide on armhf failed in the silo (and consequently the corresponding webbrowser-app build failed too)
[07:37] <oSoMoN> dbarth_, we should also put oxide 1.15.3 in the silo
[07:38] <oSoMoN> sil2100, would it be ok to do a binary copy of oxide packages from https://launchpad.net/~oxide-builds/+archive/ubuntu/oxide-next-for-stable-phone-overlay/+packages to silo 3 ?
[07:39] <oSoMoN> (that PPA is built against the overlay)
[07:43] <sil2100> oSoMoN: the problem might be the lower version of the one in the PPA
[07:43] <sil2100> oSoMoN: let me delete the silo 3 oxide and try to bin-copy the one from the PPA
[07:43] <sil2100> But I anticipate the version number to be a problem
[07:44] <sil2100> oSoMoN: you ok with me deleting the old oxide in silo 3?
[07:44] <oSoMoN> sil2100, that’s fine, but I’m not sure I understand the issue? the version in the oxide-next-for-stable-phone-overlay PPA is higher than the version in the silo, isn’t it?
[07:45] <sil2100> Oh
[07:45] <sil2100> Sorry, right!
[07:46] <sil2100> I didn't read the main version correctly, yeah, .2 here and .3 there
[07:47] <sil2100> Copying
[07:48] <oSoMoN> thanks!
[09:01] <Mirv> sil2100: bzoltan: where the meeting?
[09:01] <bzoltan> Mirv: sil2100: https://plus.google.com/hangouts/_/canonical.com/liberating-the?authuser=0&hceid=em9sdGFuLmJhbG9naEBjYW5vbmljYWwuY29t.bj95cmk4g3496q75m0jr9e2pvo
[09:30] <sil2100> bzoltan: crap!
[09:30] <bzoltan> sil2100:  no worries :)
[09:48] <Mirv> sil2100: I was trying to explain that once I get my Meizu Pro 5 tomorrow I should hopefully have fast unbreaking 4G connection even outside :)
[09:49] <Mirv> ok, now I really continue to the lunch
[09:56] <sil2100> Mirv: hah!
[09:56] <sil2100> :)
[09:58] <Mirv> I'm eagerly waiting for it also because that frees up my krillin for development and I can finally do side-by-side comparisons (with mako)
[09:59] <Mirv> it has arrived to Finland now
[11:27] <sil2100> dbarth_: hey! Did you guys have the oxide weekly meeting already?
[11:56] <Saviq> rvr, jibel, looks like 58 request got duped on your Trello board
[11:56] <rvr> Saviq: Let me see
[11:56] <Saviq> rvr, there's one at the top of Ready and one at the bottom of Need QA
[11:57] <jibel> Saviq, it happens when a silo is rebuilt and ready for qa again
[11:57] <rvr> New card removed
[11:57] <Saviq> ack, tx
[11:58] <Saviq> thought it checked for an existing card (probably does, in Need QA)
[11:59] <jibel> Saviq, it doesn't, this trello board and the bot were not supposed to live that long
[11:59] <jibel> the bot needs some love now
[11:59] <Saviq> nothing is
[11:59] <Saviq> (supposed to live long)
[12:19] <dbarth_> sil2100: we did, that was on tuesday; what's up?
[12:37] <sil2100> dbarth_: any news on the arm64 oxide bits?
[12:49] <Trevinho> sil2100: hey, how can I get rid of unity-settings-daemon pacakges from the silo https://requests.ci-train.ubuntu.com/#/ticket/1419
[12:49] <Trevinho> ?
[12:52] <pmcgowan> sil2100, thats going to take some time, just started investigating
[12:53] <dbarth_> sil2100: yes, chrisccoulson is going to look into that; it's not a simple build option to turn on, considering all of the bit that go into blink itself, but he feels it's doable in a reasonable timeframe
[12:53] <dbarth_> sil2100: what is reassuring is that there are arm64 builds of chrome already
[12:53] <pmcgowan> dbarth_, do you have a 15.2 silo coming?
[13:04] <sil2100> Trevinho: hey! Let me try helping you
[13:04] <sil2100> dbarth_, pmcgowan: phew, thanks, since that will be one of the biggest xenial blockers right now
[13:05] <sil2100> dbarth_, pmcgowan: the other big one, platform-api, already has a fix in a silo
[13:07] <pmcgowan> dbarth_, maybe santosh can get that going unless you already thought of that
[14:01] <dbarth_> pmcgowan: it's going to be chris initially mostly; santosh is up next on a memory optimization bug
[14:01] <pmcgowan> dbarth_, ack
[14:01] <dbarth_> https://bugs.launchpad.net/oxide/+bug/1501297
[14:46] <Trevinho> sil2100: have you been able to get rid of that pkg? :)
[14:47] <sil2100> Trevinho: aaaaaa... no, but I removed it now :)
[14:47] <Trevinho> sil2100: thanks
[14:47] <sil2100> Sorry!
[14:47] <Trevinho> sil2100: np... you'll have to publish that silo soon anyway :=
[14:47] <Trevinho> :)
[14:48] <Trevinho> sil2100: was just removing it from the ppa enough, or is there something to do at ci-train level too?
[14:49] <sil2100> Trevinho: no, it should be gone now on the next silo refresh
[14:49] <Trevinho> k
[14:49] <seb128> so just ppa delete?
[14:49] <sil2100> (as long as no merges for it are added)
[14:49] <Trevinho> seb128: FYU ^
[14:49] <seb128> I could have done that
[14:49] <seb128> but I though it would confuse the CI train
[14:49] <Trevinho> seb128: damn, you've eyes everywhere!
[14:49] <seb128> that at least the silo would need to be reconfigured
[14:49] <seb128> Trevinho, ;-)
[14:50] <Trevinho> and... s/FYU/FYI/
[15:09] <robru> sil2100: I see you merged the branches, did you get webops to do the rollout
[15:09] <sil2100> robru: not yet, wait a moment with that if you can
[15:10] <sil2100> I wanted to do the batch copy first but I see that it would overwrite some bits so I need to consult slangasek first before proceeding
[15:10] <robru> sil2100: yeah so the thing is, lp:cupstream2distro auto-rolls-out every hour but lp:bileto doesn't, so currently in production dual silos will now publish xenial to SRUs
[15:10] <sil2100> Oh, it does? Oh man, I forgot about that
[15:10] <robru> sil2100: I guess it's fine as long as nobody published anything
[15:11] <sil2100> No, not that I know of at least
[15:12] <robru> sil2100: ok, I need breakfast, we probably should rollout bileto sooner rather than later.
[15:12] <sil2100> Yeah, I guess everything will be good after the meeting is over
[15:28] <robru> sil2100: what was the issue? some yakkety uploads conflict with what's in xenial overlay?
[15:30] <sil2100> robru: yeah, I can't do a full batch binary copy of all xenial overlay to yakkety as yakkety moved forward - this means that core-devs had to no-change rebuild some of them due to toolchain changes
[15:30] <sil2100> So when I would overwrite those with our old binaries, those packages would be again stuck in limbo as they would dep on old libraries and binaries
[15:30] <oSoMoN> ubuntu-qa: what’s the current throughput of the QA team for silo validation? any chance silo 53 will be tested before EOW ?
[15:31] <rvr> oSoMoN: It's on the queue
[15:31] <rvr> Not sure it will land this week, but we'll do our best
[15:31] <sil2100> robru: will have to probably skip those packages that are conflicting and do source-copies with version numbers changed
[15:31] <sil2100> But I want to confirm with Steve first
[15:34] <robru> sil2100: yeah source copy with bumped version number sounds like the right approach to me
[15:40] <sil2100> robru: ok, so anyway, if you have a moment I guess it's good to update bileto
[15:40] <sil2100> I'll be performing the batch copy in a moment but I guess having bileto switched won't make a difference
[15:49] <oSoMoN> rvr, thanks! you’re doing a great job, much appreciated
[17:04] <kdub> I think my autopackage tests got stuck somehow... looks like its stuck downloading something? https://requests.ci-train.ubuntu.com/#/ticket/1379
[17:36] <Saviq> robru, hey, I think something went wrong after trying to make this silo triple https://requests.ci-train.ubuntu.com/#/ticket/1426
[17:37] <robru> wat
[17:38] <robru> Saviq: just abandon and reassign
[17:38] <Saviq> ack
[17:38] <Saviq> mterry, FYI ↑
[17:38] <mterry> Saviq, hrm
[17:39] <Saviq> I think it tried to get the highest version from yakkety
[17:39] <Saviq> and apply to overlay, which was already ahead
[17:40] <robru> kdub: not sure what makes you think that. grep for 069 in http://autopkgtest.ubuntu.com/running.shtml and it shows a test running with some failures
[17:40] <robru> Saviq: oh that's possible. yeah the versioning would go by yakkety and then it would decrease the series value for xenial and vivid.
[17:40] <Saviq> not series, but $date.$rebuild
[17:41] <kdub> robru, yeah, you're right, it just paused for a long time, but looks like it got going again
[17:41] <robru> Saviq: no but I mean when it decides what $date.$rebuild to use, it looks exclusively at yakkety, and then for xenial and vivid it literally just s/16.10/$series/
[17:41] <Saviq> right, yes
[17:41] <dobey> robru: i guess the overlay xenial packages haven't been copied to yakkety?
[17:41] <robru> dobey: sil is working on it, I dunno how far along he is.
[17:42] <Saviq> dobey, still would be an issue if you've rebuilt in silo
[17:42] <Saviq> robru, you might wanna mention that Re: your trio announcement
[17:42] <robru> Saviq: your other option is to wait for tomorrow then $date will be one higher and it'll work ;-)
[17:42] <Saviq> lol
[17:42] <dobey> well, i tried to rebuild. but libual isn't new enough
[17:42] <robru> Saviq: actually it goes by UTC so UTC tomorrow is only a few hours away
[17:43] <Saviq> meh, looks like I got the same silo :P
[17:43] <dobey> and apparently somoene did a manual upload to y for another thing
[17:43] <robru> Saviq: same silo is fine, all the old packages will be deleted.
[17:43] <Saviq> still need to wait for them to be deleted, no?
[17:43] <Saviq> i.e. it will take an hour or so
[17:44] <robru> Saviq: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-059/+packages lgtm
[17:44] <dobey> *sigh*
[17:44] <Saviq> robru, https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-059/+packages?field.name_filter=&field.status_filter=&field.series_filter=
[17:44] <mterry> Saviq, are you going to triple-ize silo 58?
[17:44] <Saviq> mterry, no
[17:45] <robru> Saviq: oh yeah right. you might have to wait for utc tomorrow.
[17:45] <robru> Saviq: or get a dummy upload to yakkety with that version number so that it knows to +.1 it
[17:45] <Saviq> robru, or, since it uploads fine for yakkety, we just need to try twice more ;D
[17:45] <robru> heh
[17:46] <Saviq> mterry, 58 is already queued for QA, don't wanna mess with that
[17:49] <robru> Saviq: oh that'll be tricky because if you try to publish a xenial+vivid, xenial is now an SRU (the silo logic is that the primary one goes to main archive and any +series go to overlay)
[17:49] <robru> Saviq: could manually copy to overlay I guess.
[17:49] <Saviq> robru, does your new'n'shiny bileto jenkins replacement build straight away if it's never been built? waiting for it to realize it needs to build is a meh
[17:49] <robru> Saviq: best if you rebuild so that we have yakkety binaries that can be published
[17:50] <Saviq> robru, hmm ok, it's probably not gonna be touched before tomorrow anyway
[17:50] <robru> Saviq: well right now it builds everything straight away, that "detect new commits" thing isn't implemented
[17:50] <Saviq> wfm ;D
[17:50] <robru> Saviq: but it'll have lots of speedups once I implement that again, doing things in parallel is absurdly faster at everything.
[17:51] <Saviq> cpt. Obvious
[17:51] <Saviq> mterry, FYI ↑ we're triplifying 58 after all, robru forgot to mention in his email that's roughly mandatory ;P
[17:52] <mterry> heh
[17:52] <Saviq> I think I rebuilt it today so we'll have the same issue with yakkety vs. overlay versions
[17:52] <Saviq> oh maybe not, it was last night
[17:53] <robru> Saviq: mterry: "so if you currently have any active dual silos, you need to edit your ticket, switch to yakkety+xenial+vivid and rebuild."
[17:53] <Saviq> "need to" sounds like "if you want to have it trio-ed"
[17:53] <Saviq> but yeah I was probably just hoping ;)
[17:53] <robru> Saviq: I dunno, seems pretty clear to me. "if you have a dual silo, you need to edit it"
[17:54] <Saviq> potehto, potahto
[18:01] <Saviq> grr 503
[18:17] <Saviq> gaaarhghahghra
[18:18] <Saviq> robru, we really need overlay copied to yakkety, we're missing dependencies in there
[18:20] <Saviq> IMO should've happened before trio silos :[
[18:20] <seb128> Saviq, the overlay was copied over this afternoon I think (from yakkety-changes list at least it looks like it was)
[18:21]  * Saviq gets a yakkety chroot
[18:23] <dobey> trainguards: can someone click retry for https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-056/+build/9772513 please?
[18:24] <dobey> ppc64el898 jobs (2 hours)
[18:24] <dobey> meh. 8 builders and 98 jobs in queue
[18:27] <seb128> dobey, there is some infra issues since this afternoon, #is has been looking at it
[18:28] <dobey> seb128: sure. i'm just lamenting for my poor silos :)
[18:29] <dobey> and foundations people creating more work for me without warning
[18:31] <oSoMoN> robru, re your last e-mail to bileto-announce, can you confirm that dual silos currently awaiting QA verification need to be rebuilt as trio silos?
[18:32] <Saviq> oSoMoN, yes
[18:33] <oSoMoN> that’s what I feared
[18:33] <Saviq> oSoMoN, dual landings would mean SRU now
[18:38] <oSoMoN> ubuntu-qa: https://trello.com/c/I5HJ3brK/3224-1387-ubuntu-landing-053-webbrowser-app-osomon-dbarth can be deleted as I’m rebuilding the silo as 3×landing
[18:40] <robru> Saviq: talk to sil2100, he flipped the switch on trio silos because he was ready to copy xenial overlay into yakkety. He mentioned having some conflicts but I'm not following him that closely, I dunno why he isn't finished yet.
[18:40] <Saviq> robru, maybe it is, trying to find out what's going on now
[18:40] <robru> oSoMoN: yeah sorry, it's hard to coordinate this switch, lots of moving pieces
[18:41] <oSoMoN> robru, I understand, no worries
[18:56] <Saviq> hmm no lxd images for yakkety yet ¿?
[18:57] <Saviq> ohnoes overlayfs not supported on zfs 8-O
[19:11] <Saviq> robru, sil2100, ok found the culprit - mir didn't get into yakkety https://launchpad.net/ubuntu/+source/mir
[19:11] <Saviq> and a lot of http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html is waiting for that
[19:13] <robru> jhodapp: one more should do it
[19:14] <jhodapp> robru, hehe ok :)
[19:14] <jhodapp> robru, you were faster to the draw than me, was about to contact you :)
[19:16] <robru> jhodapp: just happened to see the irc logs of the failure and checked the PPA contents
[19:16] <jhodapp> awesome thanks
[19:26] <Saviq> mterry, can you do anything about mir and platform-api (at least) missing from the overlay → yakkety copy? or would you rather not touch it without sil2100?
[19:27] <mterry> Saviq, uh...  just needs a copy?
[19:28] <dobey> no
[19:28] <Saviq> no?
[19:28] <dobey> there have been manual uploads of mir to y
[19:28] <dobey> so the conflicts need resolved
[19:29] <Saviq> oh fook and they were not put back into trunk?
[19:29] <mterry> platform-api looks fine though, right?  It's already in sync between yakkety and overlay?
[19:30] <dobey> well platform-api hasn't had any manual uploads to y
[19:30] <dobey> but it might depend on mir or something?
[19:30] <Saviq> mterry, hmm maybe it's supposed to be that old
[19:31] <dobey> ugh
[19:31] <Saviq> right, with the exception there's a failed upload in the overlay
[19:31] <dobey> it looks like lukascz did a no-change rebuild upload of platform-api to the overlay
[19:31] <Saviq> https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/stable-phone-overlay/+sourcepub/6391641/+listing-archive-extra
[19:31] <dobey> and it blew up
[19:31] <Saviq> that FTBFS'd
[19:32] <dobey> why was that not done through a silo
[19:32] <Saviq> ah I think it might've been the libhybris fiasco
[19:32] <Saviq> that made it fail
[19:32] <dobey> bloody arm64
[19:32] <Saviq> this is a mess
[19:32] <dobey> indeed
[19:32] <dobey> didn't we invent this whole ci train thing to avoid most of this mess?
[19:33] <Saviq> apparently not
[19:41] <cyphermox> Saviq: dobey: seems to me like platform-api FTBFS there in tests because there's something different between the overlay and the real xenial archive.
[19:41] <Saviq> there was also a libhybris release since then that fixed a similar issue, maybe we could just retry the mir build
[19:48] <sil2100> Saviq: I'm working on it
[19:48] <sil2100> mterry, Saviq: just give me a minute
[19:48] <sil2100> THe package copies are still in progress
[19:48] <Saviq> ack
[19:49] <sil2100> The mir packages need a source copy so it takes a while, had a bunch of packages that needed uploading like that
[19:50] <sil2100> (and I need to preserve the mir changes)
[19:50]  * sil2100 hopes xnox pushed all those changes upstream
[19:51] <robru> dobey: indeed, but if there's a manual upload to the overlay PPA that failed, that's not a train failure, that's a failure to use the train properly.
[19:52] <robru> sil2100: uh? Core devs uploading to devel never sync to trunk, that's the landers job to notice and sync trunk
[19:53] <sil2100> Well, that's not how the archive should work, for every project in Ubuntu we always first upstream the changes and then release downstream
[19:53] <sil2100> So I would expect the same policy for people using the train
[19:53] <sil2100> But if people are used to that, then good
[19:53] <dobey> robru: of course
[19:54] <robru> sil2100: i dunno where you're getting this idea from. I've butted heads with release team on several occasions, they upload direct to distro and don't care at all about upstream trunks.
[19:54] <robru> sil2100: this is why train has "... missing from changelog" check, so the lander knows to sync distro back to trunk
[19:55] <sil2100> robru: then those are bad practices, even doko pokes me everytime (or most of the times) he uploads a change to a train-managed project
[19:55] <sil2100> With all +1 maintenance it's always: "be sure the patch is submitted upstream"
[19:55] <robru> sil2100: good luck convincing them of that
[19:55]  * sil2100 sighs
[19:55] <dobey> sil2100: how did you get doko to do that?
[19:56] <sil2100> dobey: he was doing it himself! I seem to be his point of contact for all train projects though - still, that's better than not caring at all ;)
[19:56] <dobey> sil2100: well, there's a difference to a patch (things in debian/patches/) and packaging changes
[19:56] <dobey> sil2100: "push packaging changes upstream" has never been a thing
[19:56] <robru> sil2100: i hope you haven't been clobbering yakkety uploads with old xenial ones, we need those diffs committed to trunk
[19:56] <sil2100> Right, but for instance the mir changes I'm seeing now are mods in the code contents, not packaging
[19:57] <robru> dobey: the difference now is that we have packaging in the "upstream"trunks when we are upstream
[19:58] <robru> Maybe that was a mistake, i dunno
[19:58] <dobey> sil2100: well i don't know what exactly changed in mir, but i've never been consulted about manual uploads to any train-managed projects i manage
[19:58] <sil2100> That's so sad
[19:58] <dobey> robru: well, sure, but there are other upstreams that have done that before us too.
[19:58] <dobey> but yeah, it's a mess
[19:59] <robru> sil2100: i wanted to auto sync distro uploads back to trunks but slangasek veto'd because he didn't want silos that were more than "trunk + MPs"
[20:00] <sil2100> I remember being yelled at when I once released a package without mirroring the same change in the given project's branch
[20:00] <sil2100> During my +1 maintenance
[20:00] <robru> sil2100: new rule maybe? That's not consistent with my experience of distro uploads
[20:02] <sil2100> robru: looks like a very old rule, even the CodeReview page mentions this when sponsoring someone's work: "make sure to ask the patch author to submit the patch to Debian and/or Upstream. "
[20:02] <robru> sil2100: in fact it used to be part of the official train docs that you'd put a comment in the packaging that said "feel free to do manual distro uploads, upstream will notice and sync back to trunk for you"
[20:03] <sil2100> Well, anyway, I certainly would prefer this to work the 'proper' way, but yeah
[20:04] <robru> sil2100: yeah i used to get mad at uploaders but gave up years ago, now it's just a fact of life that distro sometimes has surprise non-train uploads in it and landers have to deal wth that
[20:07] <dobey> sil2100: well that comment in the codereview page is still about patches, not packaging. i think right now foundations is just trying to get yakkety rebuilt, and concentrating on fixing things in the archive, rather than pushing any patches necessary for fixing FTBFS upstream
[20:07] <dobey> sil2100: the more annoying thing to me is all the little things like 'enable arm64 builds'
[20:09] <sil2100> No-change rebuilds are understandable and not require any upstreaming, but code changes should always (IMO) be upstreamed
[20:14] <sil2100> Ok, but anyway, I think I copied (and rebuilt) what I could
[20:14] <sil2100> Now I need to go AFK to regular life business, I'll be around in an hour still if anything
[20:52] <robru> Saviq: any luck with mir? ^
[20:54] <Saviq> robru, can you retry these builds https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-058/+sourcepub/6439825/+listing-archive-extra
[20:55] <Saviq> the rest should resolve itself
[20:56] <Saviq> not sure why these decided to be FTBFS and not dep waits
[21:00] <robru> Saviq: done, will try to keep an eye on it
[21:01] <Saviq> robru, not sure why https://requests.ci-train.ubuntu.com/#/ticket/1426 was so unhappy (mismatches, ready to builds etc.), ran a diff only now
[21:02] <Saviq> ah but it's also missing a few packages there, running a normal build then
[21:02] <robru> Saviq: version mismatch means the branch it pushed to lp doesn't match what's in the ppa, which could be a result of the versions isues you were having earlier.
[21:03] <Saviq> ack, let's see what's gonna happen
[21:04] <dobey> bah i give up for today
[21:15] <Saviq> robru, ah, too early - it's still building https://launchpad.net/ubuntu/+source/mir/0.22.1+16.04.20160516.2-0ubuntu2
[21:15] <robru> Saviq: OIC.
[21:59] <robru> goddamn 503s, why, every time.