/srv/irclogs.ubuntu.com/2015/12/02/#ubuntu-ci-eng.txt

camakorobru, sure... But it "remembers" today's build history.. But assigns the next job a number from yesterday I think : https://ci-train.ubuntu.com/job/ubuntu-landing-019-1-build/build00:00
robrucamako: yeah I can't explain that00:01
robrucamako: but your job is running, i think that's the best we can hope for00:01
camakorobru, yeah it seems to be working00:01
camakothanks00:01
robrucamako: you're welcome! and sorry for the hassle!00:01
robrucamako: ^^ so your status being all "...1125 not in PPA", I'm assuming that's due to stale backups being restored, don't worry about that, I'm working on a fix for that, just leave it for now00:46
camakorobru, yea I was ignoring those00:47
=== chihchun is now known as chihchun_afk
veeberstrainguards Hi all, it's been a while since I've used the train. After creating a request what's my next step? Do I assign it or is that used for something else?02:18
renatutraingruards, hey I am getting this error: 2015-12-01 21:24:58,594 WARNING Silo name list not found, run setup_citrain.py02:24
renatutraingruars, on silo 45, https://ci-train.ubuntu.com/job/ubuntu-landing-045-1-build/114/console02:24
veebersrenatu: Hi, you may know :-) After requesting a silo, do I hit the Assign button or something else?02:26
renatuveebers, yes, and it was building before02:26
veebersrenatu: sorry, I meant for my own silo. It's been a while since I've used it and things have changed02:27
renatuveebers, yes you need to hit the assign button02:28
renatuveebers, the silo should change to 'Ready to build"02:28
veebersrenatu: ah I see, awesome thanks for the help :-)02:28
robrurenatu: the train imploded earlier today, how old is that log? Should work if you try again02:33
renaturobru, I am trying and nothing happens02:33
robrurenatu: what do you mean "nothing happens"? Doesn't look like you ran the job02:35
renaturobru, I click on build button and the screen refresh without any new job02:35
robrurenatu: one sec02:36
robrurenatu: it's something https://ci-train.ubuntu.com/job/ubuntu-landing-045-1-build/114/console02:40
robruBrb, eating02:40
dobeyrobru: uhm, how are there two tickets assigned to silo 21?02:49
=== chihchun_afk is now known as chihchun
dobeyi just tried to build https://requests.ci-train.ubuntu.com/#/ticket/724 but it is building location-service instead02:49
dobeywhich i see is already built in the ppa02:50
robrudobey: the train experienced a catastrophic failure today and we restored from a backup.02:58
robrudobey: was one of them landed already?02:59
robrudobey: what ticket numbers?02:59
dobeyi don't know about location-service03:01
dobeyhttps://requests.ci-train.ubuntu.com/#/ticket/724 i created earlier today03:01
dobeyjust before the implosion03:01
dobeyhttps://requests.ci-train.ubuntu.com/#/ticket/678 seems to be the one for location-service, and the MPs do say "Merged" on it03:02
robrudobey: https://ci-train.ubuntu.com/job/ubuntu-landing-023-1-build/176/console you're building in 23 ho03:08
robrunow03:08
dobeyok03:08
robrudobey: sorry for the hassle.03:09
renaturobru, the build started but got a very strange error03:16
renaturobru, https://requests.ci-train.ubuntu.com/#/ticket/72203:17
robrurenatu: yeah that's because the backup is probably missing address-book-app. If you want to fix that quickly you can rebuild it, or wait some hours as I'm working on a permanent fix.03:17
renaturobru, ok I will rebuild it, thanks03:18
robrurenatu: ok, sorry03:18
=== chihchun is now known as chihchun_afk
=== chihchun_afk is now known as chihchun
=== chihchun is now known as chihchun_afk
robruoh fun04:19
robruhere we go!06:21
Mirvyay for working backups07:23
robruMirv: yay even more for pushing branches to lp so that backups aren't even necessary ;-)07:33
Mirvrobru: yeah, I read your e-mail, it sounds good07:33
robruMirv: I'm just going through and double checking that the silo states make sense and are consistent with reality. so far it seems good07:34
Mirvrobru: nice. thanks for your hard work.07:36
Mirvit seems the Qt migration autopkgtests will take a while... this is 18h after the publishing: http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html#qtbase-opensource-src07:36
robruMirv: you're welcome. also sorry for breaking everything constantnly07:37
robruMirv: yes I've heard that autopkgtest infra is strained07:37
Mirvrobru: well when you break everything in every possible way, you eventually get a really tested and bulletproof system :) and the problems haven't like caused huge, longstanding problems.07:37
robruMirv: heh, thanks. once it stabilizes I think I need to take a month off ;-)07:38
Mirvrobru: yes, please eventually have a good, long break and we will also then ban you from IRC this time :)07:39
robrulol07:39
robruMirv: ugh, silo 12 status reporting is broken, which means auto merge is also broken. looking into it.07:40
Mirvrobru: thanks. a working merge / marking as Landed would be nice.07:45
=== chihchun_afk is now known as chihchun
robruMirv: this error is very strange though as it seems to be an error in requests library itself.07:46
robruMirv: it seems to only happen with qtdoc-opensource-src. do you know anything unusal about that one?07:49
Mirvrobru: it's a package that had Ubuntu changes but now is a sync from Debian, that's the only thing special I can think of07:51
robruMirv: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-012/+sourcepub/5736443/+listing-archive-extra hummmm no signer!!! dun dun dun!07:51
Mirvrobru: right, if you do a copy-package from Debian to PPA or archive it doesn't have a signer indeed07:51
robruMirv: also no changes file?07:52
Mirv(or so the LP says in the PPA)07:52
Mirvrobru: I guess it's because it's never uploaded to Ubuntu, it's a LP-made sync directly from Debian07:52
robruMirv: what's happening is that the train is trying to download the changes file from the ppa and from ubuntu and confirm that the contents of the changes file from both matches (ie, that the version in proposed is really our version from the ppa and not some other package with the same version number)07:53
Mirvrobru: there would have been many other ones, but the others were already auto-synced from Debian to -proposed and I made a "build1" of them07:53
robruMirv: ok, pushed a fix, we'll see how it goes in about 12 minutes.08:05
Mirvrobru: ok!08:19
robruMirv: so the train thinks your "pre-merge commit" is a new commit08:22
robruMirv: if possible I recommend deleting your commit, and then pushing https://code.launchpad.net/~ci-train-bot/qtubuntu-camera/qtubuntu-camera-ubuntu-xenial-landing-012 to lp:qtubuntu-camera instead. then the train will recognize it's own commit and report correctly that a rebuild is not needed08:24
robruMirv: as it stands, this 'new commit' warning will prevent the auto merge.08:24
Mirvrobru: ok, trying out08:25
MirvI tried to help Kaleo in working on his silo08:25
Mirvrobru: done08:26
robruMirv: ok, let's see if it works ;-)08:27
Mirvno idea why I thought a manual merge would be a good idea. it was 1 min before leaving for team dinner..08:27
robruhehe08:27
Mirvrobru: it worked!08:29
robruMirv: blam08:29
Mirvrobru: that's nice actually, that train supports this pre-pushing while being able to track the migration still08:30
=== chihchun is now known as chihchun_afk
robruMirv: it didn't used to. before it would track tip commit id's and complain if there was any change. I fixed it to use 'bzr missing' and parse the output, so now it's much smarter about if commits are missing or not08:31
Mirvthat's smart indeed08:32
robruMirv: did silo 12 used to contain oxide and now it doesn't?08:33
Mirvrobru: correct. I moved the main packages to 059 for sil2100 to publish them.08:33
robruok08:33
Mirvwell, more exactly non-train main packages that needed to have a manual upload instead of no-change MP.08:34
Mirvmoved at the very last minute, worked well and thankfully sil2100 got his core dev rights.08:35
=== chihchun_afk is now known as chihchun
robruMirv: no worries, just found an old copy of the source tree from the last time oxide was diffed, just cleared it out08:36
robruMirv: https://ci-train.ubuntu.com/job/cyphermox-test/24/console how's that for a train full of silos? ;-)08:36
Mirvrobru: sweet! :)08:42
=== robru changed the topic of #ubuntu-ci-eng to: Train trouble? ping trainguards | CI problems? ping cihelp | Train: http://bit.ly/1hGZsfS | QA Signoffs: http://bit.ly/1qMAKYd | Known Issues: train experienced a catastrophic failure but has been restored to normal operation. any problems, just rebuild your silo
=== chihchun is now known as chihchun_afk
=== chihchun_afk is now known as chihchun
=== chihchun is now known as chihchun_afk
=== chihchun_afk is now known as chihchun
=== chihchun is now known as chihchun_afk
=== chihchun_afk is now known as chihchun
=== _salem is now known as salem_
=== xavigarcia is now known as xavigarcia_lunch
* sil2100 off to lunch12:27
=== alan_g is now known as alan_g|lunch
rvrkenvandine: Silo 14 approved13:35
=== alan_g|lunch is now known as alan_g
=== chihchun is now known as chihchun_afk
morphisMirv, sil2100: one of you have time for some uploads?14:27
sil2100morphis: I'm here o/14:27
morphisawesome14:28
=== barry` is now known as barry_
mardyMirv: hi! For some reason whenever I build it, it tells me that it needs to be rebuilt: https://requests.ci-train.ubuntu.com/#/ticket/69514:40
Mirvmardy: funny. the only thing I see is that there's some disparency between your MP branch and the target - like trunk is at 11, the MP says unmerged commits start from 13. so maybe try doing a new MP based on trunk where you apply the diff in a single commit or something (just a guess based on seeing there's some history with the branch)15:04
=== chihchun_afk is now known as chihchun
=== chihchun is now known as chihchun_afk
dobeyslangasek: hi again. can you poke qtpurchasing-opensource-src through the NEW queue in xenial?15:56
dobeywhee netsplits16:00
dobeyit'll unfortunately end up blocked in proposed though, i think, because of qt5.516:04
slangasekdobey: if I were reviewing this package in a silo, I would reject it for16:33
slangasekdobey: if I were reviewing this package in a silo, I would reject it for multiple packaging issues; but since it's in the NEW queue, I guess it's going through...16:34
slangasek(and Mirv isn't around for me to poke)16:34
dobeyslangasek: hmm, ok. yes, please bug mirv about that, as i'm sure the same issues need to be fixed in the vivid overlay version too16:36
slangasekdobey: ok, he wasn't in this channel but his irc client is connected so I've braindumped to him and accepted the package16:41
dobeyslangasek: ok, thanks16:42
dobeyseems freenode is having lots of issues today16:43
mterryrobru, sil2100: is there a spare silo I could take for a bit for testing some code changes?17:28
=== alan_g is now known as alan_g|EOD
robrumterry: yeah there's like 15 free. You don't need to ask, generally ;-)18:42
mterryrobru, yeah but there was that email a bit back where you said you folks were full up18:43
mterrydidn't know if we were still hard up18:43
robrumterry: what? That must have been over a week ago...18:43
mterryrobru, yeah...  :)18:44
mterryPoint is, I'll make  a silo18:44
robrumterry: it says how many are in use on requests.c.u.c, just subtract from 60 total18:44
mterrySo just 6 free18:44
robrumterry: OK, you can make a request and assign it yourself, just let me know if there's any trouble18:44
mterryrobru, that total would be interesting data to surface.  like 54 / 60 Assigne18:45
robrumterry: requests page unfortunately doesn't know how many total there are, unless I hard coded it. Eventually we'll move to a system of unlimited ppas, so it won't even matter18:46
robruThere were 15 free when I went to bed last night, people got greedy in my sleep ;-)18:47
renaturobru, xenial is having problems with qtdeclarative5-* packages. Do you know what is happening?18:57
renaturobru, full build log: https://launchpadlibrarian.net/228221248/buildlog_ubuntu-xenial-amd64.address-book-app_0.2%2B16.04.20151202.4-0ubuntu1_BUILDING.txt.gz18:57
robrurenatu: no, what? is there an error log?18:57
mterryrobru, I ruined everything: https://requests.ci-train.ubuntu.com/#/ticket/73718:57
robrumterry: your merge is superceded: https://code.launchpad.net/~mterry/ubuntu-app-launch/warn-on-xapp/+merge/27849718:58
mterryrobru, ugh right18:58
mterryrobru, oh18:59
mterryrobru, did you see the "fail train" page?18:59
robruMirv: you still around? can you help renatu with this qt explosion? ^^18:59
robrumterry: I *made* the fail train page... what are you seeing now?18:59
mterryrobru, well I was seeing that on that page (737) when I said I ruined everything.  But now I don't see it.  So I dunno19:00
robrumterry: dunno, could be a transient network error or something. did it say an error code before it disappeared?19:01
mterryrobru, no19:01
robruthen i dunno19:01
mterryrobru, again: http://imgur.com/VuwWyBQ19:02
mterryOh, at least queuebot tells me error19:03
ChrisTownsendHi, the ppc64el build on Xenial failed earlier today for https://requests.ci-train.ubuntu.com/#/ticket/726 and according to the build log, it looks like python3.5 seg faulted during setup.  Any one know of this issue?19:03
ChrisTownsendBuildlog: https://launchpadlibrarian.net/228197665/buildlog_ubuntu-xenial-ppc64el.libertine_0.99.6%2B16.04.20151202-0ubuntu1_BUILDING.txt.gz19:03
robrumterry: uh well if you see {{'s like that it means javascript isn't running to fill out the template. are you running noscript or something?19:03
mterryrobru, not intentionally  :)19:03
mterryrobru, huh.  If I refresh the page, it doesn't go away. But if I press enter in the location bar, it does19:05
robrumterry: try it in firefox. I think it's a problem on your end, it's fine for me19:06
robrurenatu: so long story short, there's a problem with qt and it'll take some time to fix. are you able to work on something else for a day or two?19:15
robruChrisTownsend: first I've heard of it19:16
renaturobru, can we land it?19:16
ChrisTownsendrobru: Hmm, I'll just do a rebuild and see what happens then.19:16
renaturobru, we will be able to land it, when  it get approved, or we will need to wait for the fix?19:17
robruChrisTownsend: I'd report a python3.5 segfault directly to barry ;-)19:17
robrurenatu: well you can't land a package that won't build, no.19:17
ChrisTownsendrobru: Ok, I'll see if it happens again, and if so, poke him.19:17
renaturobru, ok19:18
renatubfiller, ^^19:18
ChrisTownsendrobru: Thanks19:18
robruyw19:18
bfillerrenatu: what silo are you talking about19:19
renatubfiller, all address-book silos19:19
barryChrisTownsend, robru: it's more likely of course that some extension is misbehaving19:19
bfillerrobru: we'll need to get that sorted rather quickly as it will potentially block a lot of silos19:23
bfillerwhich I'm sure you already know19:23
dobeytrainguards: for xenial FTBFS fixes on dual landed packages, is it required to go through the full QA process to land a trivial fix?19:23
ChrisTownsendHrrm, now all build are failing due to a different reason...19:26
dobeyi'll presume not for now19:28
dobeyheh19:35
robru_hates_ircmterry: just a note, your qtmir-gles build will fail because you are trying to build qtmir 0.4.7 but qtmir-gles is still 0.4.6. If you're going to bump the upstream version number you have to do it in both branches19:35
mterryrobru_hates_irc, ugh -- I was just trying to avoid the error about not having qtmir-gles19:35
mterryrobru_hates_irc, I didn't want to bother making a real qtmir-gles branch19:36
cjwatsonChrisTownsend: next time please don't do a full source rebuild for a single-arch failure.  we can retry single architectures19:36
cjwatsonChrisTownsend: random segfaults on ppc64el are likely a known (but fortunately quite rare) bit of guest memory corruption19:36
robru_hates_ircmterry: no big deal, just s/0.4.6/0.4.7/ on the first line of the changelog in qtmir-gles19:36
robru_hates_ircmterry: most of the -gles pain has been automated away19:37
ChrisTownsendcjwatson: Ok, how do I just do a single arch rebuild?  I don't see any obvious option for that.19:37
robru_hates_ircChrisTownsend: you have to ask me and I can do it19:37
cjwatsonChrisTownsend: anyone with direct upload access to the silo can do it, so anyone in ~ci-train-ppa-service19:38
ChrisTownsendrobru_hates_irc: Hmm, well, now I know19:38
ChrisTownsendcjwatson: Ok, thanks19:38
robru_hates_ircChrisTownsend: there's an open bug about adding a train feature so people can retry ppa builds but it hasn't been a priority unfortunately.19:39
ChrisTownsendcjwatson: robru_hates_irc: Since I did do the full source rebuild, any ideas why all xenial builds are failing?19:40
dobeyerr, xenial archive wonky at the moment?19:40
robru_hates_ircChrisTownsend: what silo #? i lost my scrollback19:40
ChrisTownsendrobru_hates_irc: 04419:41
dobeyhttps://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-010/+packages has all build failures on xenial due to unable to install build deps19:41
ChrisTownsendYeah, maybe wonky archive.19:41
dobeyi don't know when will be an appropiate time to have someone poke the retries though19:42
cjwatsonhaha yeah that was some amusement with archive admin19:42
cjwatsona certain person deleted qtbase-opensource-src from xenial-proposed19:42
cjwatsonslangasek was just in #launchpad-ops asking for undeletion instructions, which I provided19:42
robru_hates_irccjwatson: is anybody working on a fix for that? timo just signed off...19:42
dobeyah19:42
robru_hates_ircnice19:42
cjwatsonso it should be back after the next publisher cycle19:42
dobeyfun19:43
robru_hates_irccjwatson: how long are publisher cycles?19:43
cjwatsonas long as they take19:43
cjwatsonbut typically 15 minutes ish19:43
robru_hates_ircah ok19:43
slangasekyes; fixed on the archive side, and I'm addressing the removal with the responsible AA so that we don't end up in a revert war.19:43
robru_hates_ircslangasek: I liked Mirv's plan to fix the contested issue in the next upload19:44
robru_hates_ircrather than breaking the entire world19:44
ChrisTownsendrobru_hates_irc: Well, I'll need a xenial only rebuild for all archs for the silo when this is fixed:)19:44
robru_hates_ircChrisTownsend: will do19:45
ChrisTownsendrobru_hates_irc: Thank you!19:45
robru_hates_ircyw19:45
dobeyi guess a bunch of things will need that :)19:46
cjwatsonok, I retried that seconds too early, trying again20:07
robru_hates_irccjwatson: oh are you doing the retries? thanks20:08
cjwatsonwell, some20:10
cjwatsonrobru_hates_irc: I've done 10 and 44, but I haven't been keeping an eye on what else might need it20:11
cjwatsonrobru_hates_irc: also haven't touched the train (can't remember if that's needed nowadays), just silos20:11
robru_hates_irccjwatson: nah these days it'll notice things on it's own20:14
cjwatsonthought so, good20:14
robru_hates_ircrenatu: ok, the issue with qt should be resolved, I've retried your builds20:23
renaturobru_hates_irc, thanks20:23
renaturobru_hates_irc, was quick than a day :D20:23
robru_hates_ircrenatu: yeah thanks to slangasek fixing it quickly, I was afraid it would take longer (qt is usually quite slow to move)20:23
robruthere we go20:34
=== salem_ is now known as _salem
dobeyhmm21:38
ogra_this channel has turned into a logfile21:38
dobeyi guess i can't do that :-/21:38
dobeyrobru: so having upload rights isn't enough to be able to publsih something?21:42
=== ubot5` is now known as ubot5
robrudobey: nope, it published. There's just a race condition between the status updater and the publisher, so it reverted to "successfully built" status temporarily before noticing the publish was successful22:25
dobeyrobru: ah, ok22:27
robruogra_: yeah i increased the granularity of the status reporting (now reporting per-arch statuses) and the bot went crazy pinging constantly. I'm not sure how to get it under control again22:27
dobeyrobru: fix the code to wait until all statuses are either failed or success, until reporting in channel?22:36
dobey(not sure how hard that would be, of course)22:36
robrudobey: well the problem with that I'd that i want the first failure reported as early as possible so people can respond to it22:38
robruI'd have to come up with a regex that ignores statuses that only have some combination of building/built but reports all failures22:39
dobeywell, i mostly ignore the bot anyway22:39
dobeyi should probably just actually /ignore it, so it doesn't even show up for me22:39
robrudobey: right, it'd be worth paying attention to if it didn't spam constantly ;-)22:39
dobeywell i just pay attention to the PPA page and the requests page22:40
robrudobey: right but the advantage of the bot is that it's a push notification instead of having to reload the page constantly22:41
robruAlso i personally love the bot because i need to keep an eye on all failures, and investigate ones that are train problems22:42
dobeyrobru: implement actual push notifications support, and write a simple app for the phone? :)22:42
robrudobey: ugh, push notifications ;-) I'd rather use cross platform tech like sending you emails22:43
dobeyrobru: there's a W3C proposal for push notifications :)22:43
robruOoooh22:43
dobeyof course, i'm sure it's not implemented anywhere useful22:44
robruI'll add support for w3c push and let the phone adapt to me ;-)22:45
=== elopio_ is now known as elopio

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!