=== salem_ is now known as _salem
=== chihchun_afk is now known as chihchun
=== chihchun is now known as chihchun_afk
=== chihchun_afk is now known as chihchun
[09:43] <Mirv> sil2100: are you available normally for the next few hours? I'd use your newly gained super powers to publish three main packages from another silo after I've seen the main silo publishing correctly
[09:44] <Mirv> sil2100: well I guess I'd need you 3-4h from now or so, so after your lunch probably
[09:44] <Mirv> assuming there are no new problems of course
[09:46] <sil2100> Mirv: o/
[09:47] <sil2100> Yeah, I should be around, even during lunch preparation I can still do some publishings in-between
[09:49] <rvr> tvoss: ping
[09:49] <tvoss> rvr, o/
[09:49] <rvr> tvoss: Hi. About silo 21 ... [4.] Turn GPS off from the indicator (but leave location on!) <- That switch doesn't exist anymore.
[09:50] <tvoss> rvr, okay, just leave location on then
[09:51] <rvr> tvoss: In System Settings, there is no option to disable GPS
[09:51] <tvoss> rvr, that's fine, I did not account for that as the change landed only a few days ago
[09:52] <robru> sil2100: Mirv: can somebody publish https://requests.ci-train.ubuntu.com/#/ticket/616 for dobey ?
[09:52] <rvr> tvoss: I got location quickly running the command indoor, so I guess the test case passes.
[09:53] <tvoss> rvr, yup
[09:53] <sil2100> Ah, xenial only landing
[09:53] <sil2100> robru: looking at it now
[09:53] <Mirv> robru: sil2100: it's published ages ago, the new Qt module is just in NEW
[09:53] <sil2100> Oh, actually, maybe Mirv you'd like to take that one? It's a Qt landing
[09:53] <Mirv> robru: sil2100: or is there a new build or something to do republishing of?
[09:54] <robru> sil2100: Mirv yeah one package had to be rebuilt due to being stuck in proposed
[09:54] <Mirv> robru: ah ok..
[09:56] <Mirv> hmm
[09:57] <robru> Mirv: publish with ignore
[09:57] <Mirv> robru: right
[09:58] <Mirv> robru: bug found, it tries to push something that is in NEW queue again. probably harmless but still https://ci-train.ubuntu.com/job/ubuntu-landing-015-2-publish/136/artifact/packagelist_rsync_ubuntu-landing-015/*view*/
[09:58] <Mirv> well probably so harmless that it doesn't need to get fixed even
[09:59] <robru> Mirv: yeah it skips publishing stuff already in the archive, but NEW isn't in the archive
[10:11] <robru> Yay
[10:17] <sil2100> bzoltan_, zsombi, Mirv: hey guys! Just a quick request - when you see an UITK bug that someone from your team is working on, could you also make sure to open up the Ubuntu RTM task for it in the bug?
[10:18] <sil2100> bzoltan_, zsombi, Mirv: this would make progress tracking much easier since (as per my few announcement e-mails ;p) on overlay-ppa releases the Ubuntu RTM bug will close if the bug number is mentioned in the changelog
[10:21] <sil2100> bzoltan_, zsombi: while we're at it, a quick question: did the fix for LP: #1512924 get released already? Since someone mentioned rc-proposed is good now
[10:21] <ubot5> Launchpad bug 1512924 in ubuntu-ui-toolkit (Ubuntu) "Timestamps not localized in notifications" [High,In progress] https://launchpad.net/bugs/1512924
[10:21] <sil2100> But I don't see it released through the bug
[10:22] <Mirv> sil2100: yeah I think Zoltan & Zsombor handle the bugs generally, but I'll also keep that on mind when touching bugs
[10:22] <Mirv> oh, I'm at 26.1GB / 30.0GB in the Qt PPA because of the rebuilds
[10:47] <rvr> tvoss: Silo 21 approved
[10:47] <tvoss> rvr, thanks
[10:47] <tvoss> sil2100, anything I need to do?
[10:47] <tvoss> mardy, ^
[10:57] <sil2100> tvoss: no, I think it's on our side now
[10:58] <sil2100> tvoss: just remember about syncing those changes to xenial later
[10:58] <tvoss> sil2100, sure
[10:58] <sil2100> tvoss: (note down all the changes that aren't released to xenial for later or Steve will kill me)
[10:58] <sil2100> ;)
[10:59] <tvoss> sil2100, sure, a simple diff is good enough, though
=== _salem is now known as salem_
=== chihchun is now known as chihchun_afk
=== chihchun_afk is now known as chihchun
=== alan_g is now known as alan_g|lunch
[13:48] <sil2100> Mirv: can I publish silo 59?
[13:48] <Mirv> sil2100: yes, I was just about to say that please do, the 12 rsync looks good https://ci-train.ubuntu.com/job/ubuntu-landing-012-2-publish/91/artifact/packagelist_rsync_ubuntu-landing-012/*view*/
[13:51] <Mirv> sil2100: the UITK patches are already their staging, so we discussed I'll do a fake sync of the changelog entry to their trunk and they can just ship their next release normally as dual landing
[13:52] <sil2100> Mirv: ok, makes sense, silo 59 is now doing a diff of the contents (oxide takes ages)
[13:53] <Mirv> oxide does
=== alan_g|lunch is now known as alan_g
[14:00] <sil2100> Mirv: "tar: oxide-qt-1.10.3/third_party/chromium/src/chrome/test: Cannot mkdir: No space left on device" :<
[14:00] <sil2100> Oxide might have killed the  train
[14:00] <sil2100> Yeah, I see multiple dead jenkins executors, eh
[14:00] <sil2100> robru: ^
[14:01] <Mirv> sil2100: uh oh :(
[14:01] <sil2100> Not sure what happened, I see errors all around now
[14:01] <sil2100> df: ‘/var/cache/pbuilder/build/cow.18519/run/shm’: No such file or directory
[14:01] <sil2100> df: ‘/var/cache/pbuilder/build/cow.18519/var/lib/jenkins/silos/ubuntu/landing-052’: No such file or directory
[14:01] <sil2100> df: ‘/var/cache/pbuilder/build/cow.1500/run/shm’: No such file or directory
[14:01] <sil2100> df: ‘/var/cache/pbuilder/build/cow.1500/var/lib/jenkins/silos/ubuntu/landing-046’: No such file or directory
[14:01] <sil2100> WTF
[14:02] <Mirv> maybe some symlinks or such
[14:02] <sil2100> Anyway, let me try publishing, maybe it'll work, I wonder if it's not super broken
[14:03] <Mirv> sil2100: you could also copy-package to xenial-proposed and then try to clean the 059 so that it'd clean out the oxide diff?
[14:03] <Mirv> I'm glad I managed to get 012 out then first, properly
[14:03] <sil2100> Let's see, if it doesn't work then I copy-package
[14:04] <sil2100> Mirv, robru: I'm just worried that we have currently 16 dead executors
[14:05] <sil2100> There seems to be enough space for normal usage currently, but hmm...
[14:05] <Mirv> sil2100: whenever there's a little bit of space available, one can restart them via the UI
[14:05] <Mirv> a couple executors are enough for jenkins to function
[14:19] <sil2100> Mirv: ok, silo 59 looks like published... I mean, the rsync looks sane
[14:20] <Mirv> sil2100: ok, great! it'll probably soon show up at https://lists.canonical.com/archives/xenial-changes/2015-December/thread.html
[14:40] <Mirv> sil2100: yep, it's all in. there might be something to see regarding autopkgtest results before you're off, but more likely the full situation is seen after the night
[14:44] <Saviq> robru, hey, could you please have a look at why I can't copy phablet-tools to wily https://launchpad.net/~phablet-team/+archive/ubuntu/tools/+packages?field.name_filter=phablet-tools&field.status_filter=published
[14:49] <Saviq> robru, also, https://code.launchpad.net/~saviq/unity8/in-train-pot-update/+merge/279100 - seems to be doing what's needed, now who do we need to convince it's not completely dumb and stupid?
[14:58] <Elleo> trainguards: Heya, anyone have any idea why build dependencies are suddenly failing for ubuntu-keyboard builds on xenial? https://launchpadlibrarian.net/228101612/buildlog_ubuntu-xenial-armhf.ubuntu-keyboard_0.99.trunk.phablet2%2B16.04.20151201-0ubuntu1_BUILDING.txt.gz
[15:01] <Mirv> Elleo: possibly because Qt 5.5.1 is just landing and it'll take a bit of time before LP is happy about everything being in proposed pocket properly. if you mean within last 1h.
[15:02] <Elleo> Mirv: ah, okay; thanks
[15:20]  * Mirv out to team dinner
[15:22] <sil2100> Mirv: o/
[15:22] <sil2100> Have fun!
[15:48] <pstolowski> sil2100, hey, any idea what's the failure here https://ci-train.ubuntu.com/job/ubuntu-landing-004-1-build/lastBuild/console ?
[15:49] <sil2100> pstolowski: hmm, could you retry to check if it's not a single case?
[15:50] <pstolowski> sil2100, sure
[15:50] <sil2100> We had some free-space issues, maybe this caused those?
[15:50] <pstolowski> tsdgeos, ^
[15:50] <tsdgeos> let's hope :)
[17:02] <tvoss> sil2100, you around?
[17:21] <dobey> slangasek: hey, now proposed migration is complaining about the old binaries after i changed ubuntu-push-autopilot to only build on the three main archs. is that something we can manually work around, or do i need to make more changes in packaging?
[17:21] <slangasek> dobey: that requires an archive admin to delete the old ("NBS", in parlance) binaries; I'll do that now
[17:22] <dobey> slangasek: ok, thanks
=== alan_g is now known as alan_g|EOD
[18:44] <robru> Saviq: you're trying to binary copy phablet tools from xenial to wily within the same ppa? That can never work because the version number is already used, by itself. You need to download the package, change the version number, and re upload
[19:17] <robru> wat
[19:28] <camako> robru, argh... can I get help with this silo..  It got stuck in a bad state  ---------> https://requests.ci-train.ubuntu.com/#/ticket/669
[19:28] <robru> camako: you mean 'bad merges'?
[19:29] <camako> robru, no I got past that
[19:29] <camako> ended up canceling it due to something else
[19:29] <camako> it was before point of no return
[19:29] <robru> camako: strange, the logs aren't loading for me
[19:29] <dobey> slangasek: thanks. looks like ubuntu-push made it through now. but now pay-service says on excuses that it has an unsatisfiable depends on ubuntu-push-client on those 3 archs, but that binary package does exist there. is this just a timing issue with archive sync, or something else we need to deal with?
[19:30] <robru> camako: if you have a log open can you pastebin it for me?
[19:30] <camako> robru, you can't see this : https://ci-train.ubuntu.com/job/ubuntu-landing-019-1-build/308/console
[19:30] <camako> ok I will
[19:31] <camako> robru, http://pastebin.ubuntu.com/13604515/
[19:31] <robru> camako: huh ok that one I can see but when I click through from bileto it just says "no such file /path/to/this/log/file"
[19:32] <robru> camako: one sec, I can fix that, but unfortunately without the previous log I'll never know how that broke
[19:32] <kdub> what am I doing wrong when trying to assign a silo?: https://ci-train.ubuntu.com/job/prepare-silo/6595/console
[19:33] <camako> robru, you're right.. This one I can't see either : https://ci-train.ubuntu.com/job/ubuntu-landing-019-1-build/306/console
[19:33] <camako> that's the one I cancelled
[19:34] <robru> kdub: "Name or service not known" hopefullly that's transient
[19:36] <kdub> thanks robru
[19:37] <robru> kdub: camako: I'm asking #webops about this because apparently everything is exploding now
[19:37] <camako> Oo
[19:37] <camako> thanks robru
[19:40]  * dobey wonders if this jenkisn job is stuck or what
[19:40] <dobey> it's been building this source package for 30 minutes already :-/
=== robru changed the topic of #ubuntu-ci-eng to: Train trouble? ping trainguards | CI problems? ping cihelp | Train: http://bit.ly/1hGZsfS | QA Signoffs: http://bit.ly/1qMAKYd | Known Issues: train has completely imploded, robru is working on it
[19:48] <slangasek> dobey: ubuntu-push-client> checking
[19:53] <mterry> robru, sil2100: how are we doing on silos?  Is there a spare for me to put some testing code?
[19:53] <mterry> Oh, I see the title  :)
[19:54] <robru> dobey: mterry: camako: kdub: now'd be a great time to go for a long walk
[19:54] <mterry> heh
[19:54] <dobey> oh
[19:54] <camako> :-)
[19:54] <mterry> robru, I'm used to seeing that phrase followed by "off a short pier"
[19:54] <dobey> i *just* clicked build again too. maybe you want to kill that one? :)
[19:55] <robru> mterry: no no, we like you. but the train has completely shat itself, don't hold your breath on this oe
[19:55] <dobey> mterry: i think he's saying the train did that already
[19:56] <mterry> robru, I knew what you were saying, just the phrasing made me double check   ;)
[19:56] <slangasek> dobey: I don't see anything related to ubuntu-push blocking pay-service on http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html#pay-service; I do see an autopkgtest failure on armhf
[19:57] <dobey> slangasek: oh, so must have been a timing issue then. i'm not sure what that armhf fialure is. seems to be for an older version of unity-scope-click for some reason, and the test setup failed to install all the dependencies. can you retrigger that to run again against the newer version or something?
[20:04] <slangasek> dobey: looking
[20:06] <dobey> hmm, or is excuses lying about the version there
[20:07] <slangasek> dobey: excuses isn't lying; but the right answer would be for a fresh test to be triggered with -proposed pay-services when new unity-scope-click landed in xenial, and I'm not sure if that happened automatically or not
[20:08] <slangasek> dobey: so I don't know why excuses reports the version that it does; I do see that unity-scope-click 0.1.1+16.04.20151124-0ubuntu1 did get tested for the new pay-service (https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/armhf/u/unity-scope-click/20151201_072158@/log.gz) and did fail
[20:08] <slangasek> so yes, that result *should* be reported on excuses
[20:09] <slangasek> but perhaps it doesn't report the newer version number, just the first version number where the regression was seen
[20:09] <slangasek> and the unity-scope-click autopkgtest is failing a *lot*
[20:09] <slangasek> when triggered by multiple dependencies
[20:09] <slangasek> so probably a flaky test
[20:09] <dobey> lots of things seem to be failing to install there, including bluez :-/
[20:10] <dobey> and for some reason, unity-scope-click passed twice on sunday
[20:12] <dobey> i wonder what broke it, and why it didn't break on amd64/i386
[20:16] <dobey> dpkg: error processing package bluez (--configure): subprocess installed post-installation script returned error exit status 1
[20:17] <dobey> hmm, so bluez :-/
[20:17] <dobey> Job for bluetooth.service failed because the control process exited with error code. See "systemctl status bluetooth.service" and "journalctl -xe" for details.
[20:19] <Saviq> robru, tried a source copy, actually... what do you mean that "version is used by itself"? is it because changelog says "xenial" still in that case?
[20:19] <dobey> ah, and the bluez armhf test is flagged as [always failed] :(
[20:19] <dobey> that's not good
[20:19] <slangasek> but bluez didn't change in xenial since Nov 9, so shouldn't account for this flakiness
[20:20] <dobey> hmm
[20:22] <dobey> slangasek: hmm, this error seems to have been happening before then too: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/armhf/u/unity-scope-click/20151026_110844@/log.gz
[20:23] <dobey> probably some flakiness with systemd inside a chroot?
[20:24] <dobey> or inside an lxc even
[20:24] <dobey> i've seen similar flakiness with upgrading some packages inside my own local development lxc containers
[20:26] <robru> Saviq: I mean that PPAs enforce a 1:1 relationship between version numbers and package contents. So when you try to copy a package from one PPA to the same PPA, it doesn't work because the version number is already taken by the package you're trying to copy. You need to change the version number.
[20:26] <Saviq> robru, even across series?
=== salem_ is now known as _salem
=== _salem is now known as salem_
=== salem_ is now known as _salem
=== _salem is now known as salem_
=== salem_ is now known as _salem
[20:39] <Saviq> robru, ok, tweaked the version to say 15.10, that got accepted, tx
[20:40] <Mirv> robru: regarding the topic, I did publish 012.... and when sil2100 published the helper silo 059, jenkins ran out of space
[20:40] <robru> Saviq: yw
[20:40] <robru> Mirv: yeah this is unrelated, basically I did a pbuilder --clean and it decided to delete ALL of the files.
[20:41] <Mirv> robru: ah, ok
[20:43] <Mirv> very clean!
[20:44] <robru> Mirv: i can't even
[23:49] <robru> dobey: mterry: camako: kdub: are any of you still around? can you please try again what you were doing before?
[23:50] <camako> robru, sure
=== robru changed the topic of #ubuntu-ci-eng to: Train trouble? ping trainguards | CI problems? ping cihelp | Train: http://bit.ly/1hGZsfS | QA Signoffs: http://bit.ly/1qMAKYd | Known Issues: train has been restored from backup; ping robru if you have any issues
[23:50] <robru> camako: backup is a day old so you'll likely need to rebuild anything that you built today
[23:51] <camako> robru, ah ok.. was wondering..
[23:51] <robru> camako: I'm working to make this a bit more robust...
[23:51] <robru> so errors like that ^^ are going to mean 'this didn't get restored from the backup' I think
[23:54] <camako> robru, I issued a 'build' for mir but I don't see it on the 'build history' https://ci-train.ubuntu.com/job/ubuntu-landing-019-1-build/build
[23:55] <robru> camako: yeah that's just jenkins being garbage, click the most recent one and then click 'next' and it should appear. also you can click 'status' from bileto and it'll link to the right log
[23:57] <camako> robru, o I think it went back in time to the back up time... latest build is supposed to be #308 but it's chewing on #299.
[23:57] <camako> https://ci-train.ubuntu.com/job/ubuntu-landing-019-1-build/299/console
[23:58] <robru> camako: well any non-backed-up job logs would be lost...