robruTrain going down in 15!00:44
robruit's alive!02:05
=== robru changed the topic of #ubuntu-ci-eng to: Train trouble? ping trainguards | CI problems? Use JenkaaS: http://bit.ly/jenkins-docs | Train: http://bit.ly/1hGZsfS | QA Signoffs: http://bit.ly/1qMAKYd | Known issues: diffs missing, robru is regenerating
=== chihchun_afk is now known as chihchun
=== robru changed the topic of #ubuntu-ci-eng to: Train trouble? ping trainguards | CI problems? Use JenkaaS: http://bit.ly/jenkins-docs | Train: http://bit.ly/1hGZsfS | QA Signoffs: http://bit.ly/1qMAKYd | Known issues: if your ticket says "diffs mising", that should fix itself next build
robruTrevinho: congrats on being the only tester and the first person to do a build in production ;-)03:33
Trevinhorobru: yeah, I noticed that... Pretty nice :-)03:35
Trevinhorobru: I noticed just one small thing: I just added to my silo a new MP which affects a new compnent that wasn't listed before... Then when I did the build thing it wasn't listed03:36
=== chihchun is now known as chihchun_afk
robruTrevinho: yeah, that's expected. that list is based off the sources list which wouldn't be updated until after the build. the problem is that identifying source names from MPs is quite slow so it's really not feasible to detect that right away03:37
Trevinhorobru: like: the silo had unity and compiz MPs. I added a new BAMF MP and the did the build thing, But bamf checkbox wwasn't there in the build page03:37
robruTrevinho: if you add bamf to the list of sources by hand, it'd show that option. or run a diff03:37
Trevinhorobru: I suppose yes03:38
Trevinhorobru: not a big deal though...03:38
Trevinhorobru: very nice stuff!03:38
robruTrevinho: thanks! I'm noticing other small issues, I'll have to push some fixes ;-)03:38
Trevinhorobru: indeed there will be somehting but overall the experiecne is nice03:39
robruTrevinho: thanks!03:40
Trevinhorobru: I actually was wondering that things had been switched because of the new favicon :)03:41
robruTrevinho: yes, as we phase out the "train" we had to do away with train favicon, there's also a new error image but hopefully you don't see that too soon;-)03:42
Trevinhorobru: i actually saw it :)03:44
Trevinhorobru: not sure how it happened but while loading the page...03:44
Trevinhoerr, when opnenig the ticket from the build page03:44
=== chihchun_afk is now known as chihchun
robruTrevinho: that's odd, when did www-data start contributing to this project? http://bazaar.launchpad.net/~ci-train-bot/compiz/compiz-ubuntu-yakkety-landing-012/revision/4050#debian/changelog04:30
robru(le sigh)04:30
=== chihchun is now known as chihchun_afk
robruTrevinho: this is happening because smspillaz' bzr is misconfigured: https://code.launchpad.net/~smspillaz/compiz/compiz.animationaddon-returns/+merge/295497 (he somehow doesn't have angle brackets around his email in the commit logs, so parsing is failing)04:37
robruTrevinho: I've pushed an experimental fix, try again in 20 minutes and see if that helps (should at least use the raw name rather than www-data)04:41
=== chihchun_afk is now known as chihchun
tvossrobru, you still around?06:44
robrutvoss: hi06:44
tvossrobru, hey there :) I'm trying to build https://requests.ci-train.ubuntu.com/log/1449/build/ but fails due to lack of permissions. I'm logged in to the train, though06:45
robruThat's not06:47
tvossrobru, ? :)06:48
robrutvoss: is your email address set in launchpad? That is some kind of lp error, nothing to do with permissions as far as bileto is concerned06:49
tvossrobru, yup, set06:50
robrutvoss: no it isn't? https://launchpad.net/~thomas-voss06:51
robrutvoss https://www.irccloud.com/pastebin/CX93BshO/06:53
tvossrobru, well, it's set but hidden :)06:53
tvossrobru, try again06:54
robrutvoss: can you make it public? I'm not sure how that would have worked before. it needs to know that so that it knows what to put in the changelog message06:54
tvossrobru, done06:54
robrutvoss: ok, looks good now. try the job again (I already built it but if you want your name in the changelog build it again006:54
tvossrobru, ack06:56
robrutvoss: http://bazaar.launchpad.net/~phablet-team/location-service/trunk/view/head:/debian/changelog#L10 A HA! Apparently old code fell back on default email address since your address was hidden all this time. Let me know if you want me to restore that or if you prefer having your address in changelogs now ;-)07:01
tvossrobru, I'm fine with my mail address being in the changelog07:02
robrugreat, less work for me, yippee!07:02
marcustomlinsontrainguards: Something odd about silo 31, it's been "running" autopkgtest for 24 hours now07:07
robrumarcustomlinson: hmm indeed something looks goofy there. please ping pitti to dig into that07:08
robruno mention of landing-031 at http://autopkgtest.ubuntu.com/running.shtml07:09
marcustomlinsonrobru: will do thanks07:10
robruyou're welcome07:12
robruOK, getting sleepy...07:16
tvosstrainguards, trying to kick a rebuild for https://requests.ci-train.ubuntu.com/#/ticket/1449 never ends in the silo09:11
tvossrobru, ^ :)09:22
pstolowskirobru, hey, getting python exception in the silo - http://pastebin.ubuntu.com/16886214/09:30
sil2100Ok, I got an exception while trying to rebuild a silo as well09:41
pete-woodsso it seems that building has changed hugely in citrain10:50
pete-woodsI've added a new package to my silo10:51
pete-woodsbut it's not available in the list of checkboxes10:51
tvossMirv, sil2100 any more insight into the build failures?10:51
pete-woodsmanually added to the source package list helped..10:51
pete-woodsgreat, I seem to no longer have permission to build the silo10:51
pete-woodstrainguards: http://paste.ubuntu.com/16887307/10:52
pete-woodsam I the first to start seeing this one?10:53
pete-woodsI see it's a lot of people10:57
pete-woodssilly me didn't read bak10:57
Mirvtvoss: the best guess is the big rollout as announced by robert today in an e-mail11:09
Mirvrobru noticed a problem regarding the "0replaceme" functionality, but this permission problem is apparently another11:10
rvrMirv: Did you tell me that silo 9 was going to be reworked or did I imagine it?11:22
Mirvrvr: yes I did, and it's built, I was under impression mzanetti was still going to think a bit about it though. it did fix the problem to at least some extent, he was wondering why needs two app restarts instead of just one when the problem happens.11:23
mzanettiyeah, didn't make much progress, but I haven't managed to see a .lock file since11:25
mzanettirvr, when you test it, keep on running this:11:25
mzanettirvr, find ~ -name *.lock11:25
mzanettirvr, before restarting the app. if you find a .lock file in e.g. ~/config/com.ubuntu.music/ and then on the next start the music app works fine, we're good11:26
rvrmzanetti: What I did was to create the lock file manually to see whether music app would start or not, and it did not.11:27
mzanettirvr, did you try a second restart?11:27
mzanettiif I created the lock file with touch, that silo would get over it on the second run only - which is what I don't understand properly yet11:28
alf_trainguards: Hi! Some time ago there were discussions about supporting MPs for git branches in ci-train requests. Are there any news about this?11:44
Mirvalf_: robert will answer you in a few hours but I believe that's still some time off even though this big rollout of features brings it closer11:56
alf_Mirv: thanks11:56
=== seb128_ is now known as seb128
=== _salem is now known as salem_
=== chihchun is now known as chihchun_afk
alf_trainguards: Hi! I am not able to rebuild packages in ci-train PPA. Only way I have found to rebuild is to abandon request and create a new one. Any idea what's going, or what I am doing wrong? See https://requests.ci-train.ubuntu.com/#/ticket/148713:33
Mirvalf_: probably fallout from the big rollout of new train code as announced by robert. he'll answer questions and start fixing in a few hours.13:34
Mirvalf_: some discussion a few hours ago on this channel above13:34
alf_Mirv: ok, I will just fall back to the new request method for now13:35
dobeyoh fun13:35
dobeyalf_: you mean rebuild individual binary packages?13:36
alf_dobey: I haven't tried individual packages, but a full rebuild doesn't work for me. ci-train doesn't complain is just that no new source packages reach the PPA to get built.13:38
dobeyalf_: it tells you why though13:39
Trevinhorobru: hey, it seems I can't force rebuild and it doesn't go... Weird, it was working tonight13:39
dobeyTrevinho: you mean on the new build page with the one big button?13:40
alf_dobey: Perhaps the problem is that rebuilds don't get proper new package versions (in the past they got .1,.2 etc suffixes), so the PPA ignores them?13:40
Trevinhodobey: yep13:40
Trevinhodobey: it used to work tonight, as I've been the first user of it according to robru :). And I also did a rebuild13:40
dobeyTrevinho: yeah i just got an error with that anyway. but if you go to jenkins and find the build job for your silo #, you can do it the "old" way there13:41
Trevinhomight be... it maybe doesn't get the .X thing13:41
Trevinhooh, thanks13:41
dobeyalf_: well, one problem is that there was a manual upload to yakkety for u-s-c, and you need to merge those changes back to trunk for it13:41
alf_dobey: thanks, I will fix that...13:42
dobeyalf_: and repowerd failed to build: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-013/+packages13:43
alf_dobey: Yes, that's why I want to rebuild after making some updates to the MP in the ci-train, but that doesn't work13:43
dobeyalf_: ah, it seems to perhaps not be adding the .N13:45
dobeyalf_: build from https://ci-train.ubuntu.com/job/ubuntu-landing-013-1-build/build instead perhaps13:46
alf_dobey: thanks, I will try that13:47
Trevinhothe thing is that if you tried to build with the new system, then you've to force a rebuild in jenkins since.... The system thinks ti has already built the last src13:51
robrupstolowski: make your email public14:10
pstolowskirobru, oh.. how/where?14:10
robrupstolowski: in lp. The error is that it doesn't have permission to access your email address for the changelog entry14:14
robrudobey: also^^14:14
robruPreviously it would just default to its own email address, I had no idea so many people had private emails14:15
robruOK I'll investigate this other build issue in a sec, breakfast first14:15
dobeyrobru: fix it to work for people with private e-mails. :)14:16
pstolowskirobru, 'hide my email addresses from other LP users', is that it?14:19
dobeypstolowski: yes14:19
pstolowskidobey, hmm i just changed it and it still fails14:19
dobeybut it's all or none14:19
dobeyand it shouldn't matter if it's private or not14:20
=== chihchun_afk is now known as chihchun
dobeyi think what it needs here, is the openid needs to have a way to allow you to select which e-mail address to make visible to the citrain site, and citrain should use that e-mail address if it must use the address of the person who clicked 'build'14:21
robrudobey: right, good call. I can do that. but since this issue has a workaround let me fix this .X thing first14:24
dobeywell, building via the jenkins job is a workaround for both :)14:24
robrupstolowski: hmm? tvoss had the same issue and he fixed it by going public14:25
tvossrobru, help would also be appreciated for: https://requests.ci-train.ubuntu.com/#/ticket/144914:26
robrutvoss: yep, i have an idea about that, will have to write some code though.14:27
dobeyhmm, though it complains about diff missing14:28
pstolowskirobru, not sure what's going on.. i got the exception, but it's building14:31
robrupstolowski: log14:32
pstolowskirobru, i closed it, sorry. it was the same as before afaict14:33
robrupstolowski: what ticket?14:33
pstolowskirobru, https://requests.ci-train.ubuntu.com/#/ticket/97914:34
robrupstolowski: I dont' see the exception I just see it working: http://bazaar.launchpad.net/~ci-train-bot/unity-scopes-shell/unity-scopes-shell-ubuntu-yakkety-landing-065/revision/32714:35
pstolowskirobru, weird.. browser caching error?14:39
robrupstolowski: ah, could be14:40
pstolowskianyway..nvm if it works then good :)14:40
rvrSaviq: Silo 59 approved14:42
Saviqrvr, you really wanted to prove us wrong, didn't you ;)14:44
Saviqmterry, please do the honors :) ↑↑14:44
mterrySaviq, ooh14:45
mterrySaviq, let me go through all branches and confirm approval then press button14:45
Saviqmterry, I don't think train or rvr would have let it through unapproved14:46
mterrySaviq, that's a check made at publish time I believe14:46
mterrySaviq, (I know that one or two are unapproved, but trivially, will chase people to give final thumbs)14:47
Saviqmterry, ack14:47
robrumterry: Saviq if MPs were anything other than Work In Progress or Approved, train would say 'Bad Merges' state, and when you click publish it also checks that everything is Approved for you14:47
mterryrobru, is that...  is that a golden ticket for the web page icon?  :)14:50
robrumterry: yes! ;-)14:50
mterryrvr, thanks for the silo 59 review, I know it was a lot of tiny things to check14:55
rvrmterry: Yeah, I prefer smaller ones.14:55
mterryrvr, we're in a big backlog right now14:55
rvrSaviq: I tried to prove you wrong but had no luck :D14:57
dobeyrobru: why would ticket say "diff missing" if it shows the diffs in the artifacts list?15:01
robrudobey: because it doesn't check the artifacts list, it checks swift directly. sometimes swift can be inaccessible and that can come back 'diffs missing'. Also I just changed the naming scheme of the diffs so it's possible you have old-style diffs (though unlikely if you just built)15:02
robrudobey: which ticket?15:02
robrualf_: pstolowski: dobey: tvoss: ok, the issue with the versions not incrementing .X should be resolved, please try your builds again15:03
dobeyrobru: https://requests.ci-train.ubuntu.com/#/ticket/143515:03
robrudobey: yeah those are old-style diffs. series/source_name/content.diff should be series_source_name_content.diff15:04
robrudobey: I'm just testing the fix for the email then in a few minutes you should be able to use the new jobs again15:05
renaturobru, hey could you help me to understand what is happening on silo 73: https://requests.ci-train.ubuntu.com/#/ticket/103715:16
renaturobru, I updated the changes but it is not pushing it to ppa15:16
robrurenatu: you mean "destination version missing from changelog"?15:17
renaturobru, fail to build15:17
renaturobru, I fixed the build, but it is not pushing the changes to ppa15:17
renaturobru, the ppa still sayint that it failed 1h ago15:18
robrurenatu: oh, I *just* fixed that, please try again15:18
renaturobru, nice, thanks15:18
robrurenatu: you're welcome15:18
robrurenatu: sorry for the disruption!15:18
renaturobru, np, the new build interface is much better now15:19
robrurenatu: thanks! yeah no more auto-log-out-after-10-seconds ;-)15:19
robrutvoss: dobey: pstolowski: ok bileto is now grabbing emails from your SSO session rather than from LP, so if you set your emails private it should still succeed in putting your email in the changelog.15:26
pstolowskirobru, cool, tx15:27
robrupstolowski: you're welcome15:27
robrutvoss: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_39a8dbb93caf4ec889f8a1b7f69885db/bileto-1449/2016-06-01_09:01:38/vivid_net-cpp_packaging_changes.diff this diff is kinda funny16:04
robrutvoss: first, it seems you're clobbering a release that's already in vivid overlay, and it seems it has the same/similar changelog message. so I guess this is a duplicate release?16:05
robrutvoss: second, you shouldn't use bullet points in the Commit Message field on the MP. only the first line is used and it's passed to dch which prepends the bullet point for you, so you're getting duplicate bullet points there.16:05
robrutvoss: third, you should use the lp feature of linking bugs to the branches, those will be included in the message automatically.16:06
dobeyrobru: but it can only use the e-mail that is set as the "default" in lp right?16:21
robrudobey: well it can only use the email that is given to it by SSO, so and SSO (at least for me) only shows one email address, so presumably that is only the default, yes16:22
robrudobey: log out and log back in to see which email that is. or do a build and see what shows up in the changelog ;-)16:23
dobeyoh, so it's not even the lp e-mail16:23
dobeyit's the one you use to log in to sso16:23
robrudobey: right, because I don't have magical admin powers to access your private emails in your lp profile; this way is the only way that I can get an email from you16:25
dobeyrobru: well, still, what e-mail i use to log in to sso, and which one should appear in the changelog entries, aren't the same.16:27
robrudobey: I'm not really sure what you're wanting from me here.16:27
dobeyrobru: well, when i figure out if it's possible, i'll let you know16:27
robrudobey: ok. I guess you want it to query lp first and then fall back on SSO if that fails, but you still need to make your lp email public for that to work16:28
dobeyno, that's not what i want. that would still present the same issue, because i'm pretty sure you'd only grab the "preferred" address from LP16:29
robrudobey: well, yeah. why is your preferred address not the one you prefer?16:32
dobeyrobru: well it is the one i prefer. however, the one i prefer for me in general, and the one that is preferred for canonical projects, are not the same.16:33
robrudobey: I'm not sure where else I'd get your email from. Maybe make a new SSO using the email you want, and then use that account exclusively for bileto builds?16:34
robrudobey: in the old system you would have just been "Rodney Dawes <ci-train-bot@canonical.com>" so surely your SSO email address is an improvement there.16:36
dobeyrobru: well, it seems that the openid bits can't do what i was hoping it could do. so i'm asking to get requests.ci-train.u.c added to a special list where sso will always return the foo.bar@canonical.com address for accounts that have one defined, regardless of what e-mail was used to log in, for the bileto site16:40
robrudobey: well if that works, then go for it. does that special list already exist?16:40
robrudobey: while you're at it, also get bileto.ubuntu.com added to that list, we'll start using that domain eventually16:41
dobeyrobru: seems so.16:41
Trevinhorobru: internal server errror :o16:47
Trevinhorobru: on requesting a buld16:48
robruTrevinho: where?16:48
Trevinhohttps://requests.ci-train.ubuntu.com/log/1481/build/ check compiz, build16:48
Trevinhorobru: I've commented a MP in the list, but it shouldn't matter, right?16:48
robruTrevinho: yes theoretically commented MPs are ignored correctly16:49
robruwell, a bunch of "diff missing" implies that swift is down ^, possibly other services busted16:49
robruTrevinho: https://requests.ci-train.ubuntu.com/log/1481/build/latest working now, momentary hiccup I guess?16:50
robruTrevinho: wait, what? this is some unicode bs16:51
robruTrevinho: https://pastebin.canonical.com/157824/ (\xf1 is your funny n)16:52
robruTrevinho: but this makes no sense because a) it used to work and b) that shouldn't be forcing ascii at all wtf16:52
robruTrevinho: eh, I have an idea. apparently the thing I did to fix emails now broke unicode characters in environment variables, fun times.16:55
=== alan_g is now known as alan_g|EOD
Saviqrobru, getting 500 when trying to build this https://requests.ci-train.ubuntu.com/log/1483/build/latest :/17:02
robruSaviq: try again, I just pushed a fix17:03
Saviqrobru, I *just* tried, and tried again17:03
Saviq500, even17:03
robruSaviq: ok, it's because bileto is trying to store your name in an env var and for some reason it's not happy with the unicode.17:05
robruSaviq: I kicked the build for you for now, trying to work out a proper fix17:05
Saviqack, tx17:05
robrustupid python defaulting to ascii env vars IN TWENTY SIXTEEN17:05
robruSaviq: Trevinho: ok I have a fix confirmed working in staging, just waiting for #webops to roll it out (guy's on lunch, could take a while), in the mean time just ping me if you need any builds done17:14
Saviqthat's just unicodism I tell you17:15
Saviqit's just like racism, but about unicode chars17:15
robruSaviq: ok well production has gone and shat itself now17:23
Saviqsounds like fuyn17:24
robruSaviq: HTTP500 on some endpoints but not all, and nothing in the logs. great. also can't reproduce in staging.17:24
Trevinhorobru: new gif for me too :)17:27
Saviqrobru, when it's back, if you could please rebuild https://requests.ci-train.ubuntu.com/#/ticket/1483 again, thanks17:38
Saviqcimi, actually it might be !dead for you, since it seems to be robru's unicodism kicking in again17:39
cimiSaviq, sorry in other words why is only dead for me?17:39
robruSaviq: yeah i dunno i asked them to restart it to get some new environment vars and now the whole thing's dead, and guy went on lunch17:39
cimiah ok, well I go afk a bit, will be back later to check17:41
cimithank you17:41
robrugreat I can't even log in now17:47
robruTrevinho: Saviq: just for shits and giggles can I get one of you to run this staging job: https://requests.ci-train.staging.ubuntu.com/log/1186/build/ just to confirm the fix really works18:00
Trevinhorobru: done18:01
robruTrevinho: thanks18:01
Saviqrobru, https://requests.ci-train.staging.ubuntu.com/log/1186/build/latest seems to be running18:01
robruSaviq: Trevinho: ok, looks really good, thanks guys. I have no idea why production exploded, just waiting for #webops guy to come back and reboot it basically, exact same code is working perfect in staging.18:02
Trevinhoyay nice when it happens18:02
robruSaviq: apologies, seems to be running now: https://requests.ci-train.ubuntu.com/log/1483/build/418:42
robruliterally just rebooted it. no idea what went wrong18:42
robrucimi: we're back up, please retry18:51
=== chihchun is now known as chihchun_afk
robruMuch diff missing. Wow.22:01
=== salem_ is now known as _salem

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!