/srv/irclogs.ubuntu.com/2014/08/25/#ubuntu-ci-eng.txt

imgbot=== trainguard: IMAGE 207 building (started: 20140825 02:05) ===02:05
imgbot=== trainguard: IMAGE 207 DONE (finished: 20140825 03:35) ===03:35
imgbot=== changelog: http://people.canonical.com/~ogra/touch-image-stats/207.changes ===03:35
Mirvveebers: https://code.launchpad.net/~veebers/autopilot/legacy-lp1352889/+merge/230241 not approved04:45
veebersMirv: ah rats, not the first time I've done that either :-P Thanks for the heads up, sorting it now04:46
Mirv;)04:46
veebersMirv: Have approved now, is there a button that I need to (re)push?04:47
Mirvveebers: no buttons needed, thanks!04:48
veebersMirv: nw, thanks for sorting that out04:48
Mirvnow we just need a core-dev to approve the pkg changes :S but I'll have it done today.04:48
veebersMirv: awesome, thanks again :-)04:49
mardysil2100: hi! Do you know why silo 15 hasn't yet landed?07:00
Mirvmardy: there seems to have been some sort of build error according to the spreadsheet07:03
Mirvmardy: or maybe that thing is an erronous report in itself, but that's why it's red in the spreadsheet so no-one has noticed it has actually been tested..07:03
Mirvmardy: I think that's a false alarm that's totally Wellark's fault :) https://ci-train.ubuntu.com/job/ubuntu-landing-015-1-build/9/console - he probably erronously tried to build a wrong silo07:04
mardyMirv: yes, I've seen the error message, but I honestly didn't understand what it means :-) The silo was built and tested07:04
Mirvmardy: I'm running 'watch only' build on it now so that the status gets refreshed correctly07:04
mardyMirv: cool, that seems to have worked, thanks a lot! :-)07:12
Mirvmardy: no problem :) but the MP:s are not approved07:17
Mirvhttps://ci-train.ubuntu.com/job/ubuntu-landing-015-2-publish/5/console07:17
Mirvor at least not top approved07:17
Mirvdbarth: could you top-approve also those in that console log ^ ? you've already approved them in comments.07:18
mardyMirv: I just top-approved them myself :-)07:19
Mirvdbarth: unping :)07:19
Mirvso, now only packaging acks07:20
Mirvmardy: hey. unfortunately the libaccounts-glib needs changes. the diff https://ci-train.ubuntu.com/job/ubuntu-landing-015-2-publish/lastSuccessfulBuild/artifact/packaging_changes_libaccounts-glib_1.17+14.10.20140819.1-0ubuntu1.diff reveals that manual uploads from spring enabling multi-arch did not get merged to trunk, and they'd need to be added back (including changelog entries) :(07:24
Mirvhttps://launchpad.net/ubuntu/+source/libaccounts-glib/+changelog07:24
Mirvubuntu2 was just a rebuild, but ubuntu3 added the two multi-arch lines. so it should be just syncing changelog and adding back those two multi-arch lines07:25
mardyMirv: ah, OK, I'll try to fix that; the multi arch lines should be already in trunk, only the changelog is missing07:25
Mirvmardy: the diff claims for some reason that the multi-arch lines are being removed in the current build07:26
mardyMirv: what diff?07:26
* mardy checks07:27
mardyMirv: yes, I guess you are right; I'll add those lines07:27
mardyMirv: is it important that I add the changelog entries as well?07:28
Mirvmardy: thanks. it's a bit unfortunate this happens every now and then.07:28
Mirvmardy: yes it is, since we don't want to erase history :) possibly easiest to copy paste from https://launchpad.net/ubuntu/+archive/primary/+files/libaccounts-glib_1.15%2B14.04.20131126.2-0ubuntu3.debian.tar.gz (open in file-roller, open debian/changelog in gedit)07:28
mardyMirv: I pushed a couple of commits fixing that, can you please check?07:36
dbarthMirv: hi; while you're on webapps stuff, silo 2 can be freed; it's not ready for re-testing07:37
dbarthMirv: and line 12 is obsolete; can i just remove it from the spreadsheet?07:38
Mirvmardy: looks perfect, thanks!07:48
mardyMirv: cool! So, what is the next step?07:48
Mirvmardy: rebuilding the libaccounts-glib only07:49
Mirvdbarth: ok, freed. should it be marked as "Ready?" "No" also?07:49
Mirvdbarth: you can simply remove the line 12, go ahead07:49
Mirvmardy: I just kicked a libaccounts-glib build, then07:52
mardyMirv: thanks07:54
tvosstrainguards, can I have silos for line 39 and line 40?08:17
sil2100tvoss: sure o/08:18
tvosssil2100, thanks08:18
sil2100tvoss: those are for utopic for now, yes?08:18
sil2100Mirv: I see you're ssigning a silo for oSoMoN ? ;)08:21
Mirvsil2100: yes :)08:22
oSoMoNthanks guys :)08:23
Mirvmardy: landing-015 is now built again, you maybe want to have a quick smoke test? the diff was of course only packaging and doesn't affect functionality at all.08:23
mardydbarth_: What do you think? I don't see the need to re-test it ^08:29
dbarth_Mirv: i can do a quick smoke test no worries08:30
Mirvdbarth_: thanks!08:30
Mirvomg, meeting08:31
sil2100It will be a very sad meeting08:31
dbarth_hi trainguards; can i have a silo for line 29; thanks08:35
Mirvdbarth_: utopic, yes? not rtm/14.0908:53
Mirv(assuming)08:54
=== tvoss is now known as tvoss|test
=== tvoss|test is now known as tvoss
bzoltanMirv: sil2100: I do not know if anybody is actively working on this issue -> https://bugs.launchpad.net/ubuntu/+source/phablet-tools/+bug/1360582 I have not seen mvo online yet. But this bug killed the SDK, like totally. Not little, not just a bit, but ultimately.09:06
ubot5Ubuntu bug 1360582 in phablet-tools (Ubuntu) "Can't manually install clicks "Signature verification error" since #205" [Undecided,Confirmed]09:06
sil2100bzoltan: we discussed that on the meeting, we're waiting for mvo and or jdstrand, although it seems to be caused by mvo's landing09:08
bzoltansil2100:  OK, I will make a simple AP tests for the SDK tools what people could run before landing something what effects the SDK09:11
Mirvdbarth_: we'll need to finish the 015 and 010 landings first before line 29.09:14
Mirvthostr_: pstolowski: please coordinate with Saviq on whether you can build, test and publish your line 34 before his landing-017 or not09:19
thostr_Mirv: will do09:19
dbarth_Mirv: 15 smoke tested; you can land09:20
dbarth_Mirv: for 10, that's fine; i can do a rebuild if oSoMoN lands first09:20
Mirvdbarth_: thanks, we'll just need packaging acks still09:20
satorisAnyone have an idea what is causing this failure: https://jenkins.qa.ubuntu.com/job/thumbnailer-utopic-amd64-ci/18/console09:25
pstolowskiMirv, it's fine, Saviq will remove it from his entry09:27
Mirvsatoris: looks like a jenkins problem and something for cihelp, but I'm not sure what's their availability today09:29
Mirvpstolowski: ok then09:29
satorisThanks.09:30
WellarkMirv, mardy: whoops, sorry!09:42
Wellark15 and 12 are not that far of each other :)09:43
Mirv:)09:44
dbarth_Mirv: cool, silo 15 is migrating, can we have a new silo then?  ^^ ;)09:50
dbarth_remember i freed silo 2 as well, as a special bonus09:50
Mirvdbarth_: done now10:21
Mirv015 not yet merge&cleaned though since it's still in proposed10:23
dbarth_Mirv: ty!10:24
=== tvoss is now known as tvoss|lunch
asac12:40 < ogra> heh, so nothing landed in RTM since my last upload on friday ...10:58
asacsil2100: ^10:58
ogra_asac, it is painful ans awful for people using bzr atm10:59
asaci know, doesnt change a thing though10:59
ogra_we need that fixed before we force *everyone* into having to maintain two out of sync branches10:59
ogra_which is something we wont be able to easily revert later11:00
asacwell, whatever isnt landing in stable now, isnt in stable11:00
ogra_(and which contradicts the whole plan we initially had)11:00
asacdoesnt help11:01
ogra_asac, right, because you only can land something in stable if you create a diverged branch atm11:01
asacthen do that11:01
sil2100Well, *theoretically* you can already do this:11:01
ogra_i wont, i will rather use debs now ... like most of us11:01
sil2100(sneakily)11:01
sil2100Fill in a landing with MR's for a ubuntu11:02
sil2100And then below fill in the same landing for ubuntu-rtm but without any MPs, just source package names listed11:02
ogra_then pull the source package out of the silo and re-land that in rtm11:02
ogra_right11:02
asacyeah, then do that11:02
ogra_thats what we need11:02
asacif you dont land you will not land11:02
sil2100Yeah, troublesome (I'll try to make that much easier today) but still11:02
ogra_and thats where we need to work towards in automation11:02
asacdont hold your breath for automation. people that land in ubuntu only now are risking that their changes won't make it11:03
asac-> diverging the baseline further, making live even harder for everyone who wants to land in both directions11:04
asaclfie11:04
ogra_asac, well, i think we need some wiggle room anyway ... we dont have rtm tests for krillin (well, we dont even have automated tests yet, its all manual)11:05
ogra_so if we want to target that, there is still a good bunch of things missing11:05
asacthats going to happen today11:06
ogra_i would expect a few days til everything is fallen in place on all sides ... and while thats goinng on sil2100 can work on that extra PPA love11:06
asacnow that the channel is avail11:06
asacogra_: we wont get the stuff over then11:06
asacthats the point. land your stuff in stable with N4 targetted then11:06
sil2100I'll be loving it much more after lunch11:06
asacwe shouldnt allow folks to shoot themselves even more11:06
ogra_asac, right, it is tricky to get anything over if you dont even have a reference image test11:06
asacwe have N4 and later today krillin too11:07
ogra_right11:07
ogra_anyway, deb landing is trivisal in both ... the bzr/MP landing needs some love first else we paint ourselves in an RTM corner with forcing two branches for each landing11:08
asacwe have the trick sil said11:08
ogra_(in both -> ubuntu and rtm)11:08
asacno need to be stuck11:08
ogra_right11:08
asacwe need to stop now and replay whoevfer landed without landing in ubuntu-rtm11:08
ogra_i am not stuck11:08
ogra_but i think many people hold their feet still hoping for improvement first11:08
asacthats not coming11:09
asacif it comes it comes, but dont wait11:09
ogra_i guess someone should outlin the source package step on the ML ;)11:09
ogra_since up to now people onlykone the MP way11:09
ogra_*only know11:09
asacright, and so their changes are not landing in stable11:09
ogra_yes ... since they dont know that process is also allowed11:10
ogra_(see the ML discussion from the weekend)11:10
asacfor me those changes landed today are not meant for stable then. we certainly won't do the replaying11:10
asacfor them11:10
=== mandel is now known as mandel|lunch
=== renato is now known as Guest9436
* jdstrand notes that bug #1360582 is not his bug (I got pinged earlier)12:14
ubot5bug 1360582 in phablet-tools (Ubuntu) "Can't manually install clicks "Signature verification error" since #205" [Undecided,Confirmed] https://launchpad.net/bugs/136058212:14
ogra_jdstrand, yes, thats mvo's12:16
ogra_but it was fixed afaik12:16
jdstrandok cool12:16
* jdstrand is curious how12:16
ogra_dunno, he had me test it and later provided a new landing12:16
ogra_(which makes me assume he attempted to fix it with tht)12:17
jdstrandok, thanks12:17
ogra_oh, thats the --allow-unauthenticate bit12:18
ogra_the issue i tested for was "no clicks can be installed at all because click tries to create some dirs in /root/"12:18
ogra_i know he fixed that one ... the one above was likely fallout of this12:19
sergiusensogra_: that one's not really fixed; the workarount is to use click install directly which I was told we shouldn't do12:22
ogra_how else would you sideload a click package ?12:22
ogra_jedi waves over your screen ?12:22
sergiusensogra_: pkcon install-local ...12:24
ogra_oh12:24
ogra_that, yeah12:24
ogra_(in fact thats what i use locally in my scripts ... thats what you get using scripts and never looking what you programmed initially :P )12:25
sergiusensthat's the dream; fire and forget and work always :-)12:25
ogra_yeah12:26
ogra_thats what makes my dev mode changes so hard :(12:26
thostr_can anybody reconfigure silo 12, please?12:41
om26erMirv, hey is there an image to test ?12:43
om26erMirv, on mako I mean12:43
sil2100Saviq: hey!12:57
Saviqsil2100, elo12:58
sil2100Saviq: so, I see you have a silo ready for releasing12:59
Saviqsil2100, actually...12:59
Saviqsil2100, I need to take it back and push one small change and rebuild :/12:59
Saviqsil2100, sorry for the noise, will have it ready in ~30mins13:00
sil2100Saviq: I was thinking... I know that you have already published something to RTM for unity8 (and did a separate unity8 branch probably), but I was thinking if maybe we could allocate you an RTM silo to which we'll do a srccopy from the ubuntu one?13:00
sil2100Saviq: no worries :)13:01
Saviqsil2100, yeah, let's do that13:01
Saviqsil2100, I need to look back for dependencies, though13:01
Saviqsil2100, I'll let you know when we're ready13:01
sil2100Saviq: for now all manual, so I'll help you out with the copies and such, just tell me when you13:01
sil2100are ready13:01
Saviqsil2100, will do13:01
sil2100(premature newline)13:01
sil2100We'll get a silo for you then13:01
sil2100I just need to consume my lunch and I'll be back13:02
=== tvoss|lunch is now known as tvoss
Mirvom26er: which kind of mako image you'd want? :)13:20
Mirvom26er: I think we're not considering mako promotions today at least13:20
Mirvom26er: and there are enough unsolved blockers in the image now (apparently some new over the weekend even)13:21
ralsinasil2100: good morning, can I get a reconfigure in silo 1, please?13:25
om26erMirv, alright, I just wanted to inquire.13:27
Mirvralsina: I can reconfigure it13:32
ralsinaMirv: awesome, thanks!13:32
asacSaviq: i would recommend to not publish your ubuntu silo if you plan to get that into stable (using sil2100's new trick) as is as we will have QA double check for stable silo, so you might have to iterate there one more time before publishing13:36
asac... which probably means you want to iterate with devel in sync too (and stay in sync)13:36
Mirvralsina: uh oh, the csv output is again broken at Google, which means many functionalities including reconfigure is broken at the moment :( a manual reconfiguration worked however but hopefully google will fix itself soon13:36
Mirvsil2100: ^ csv again broken, prepare-silo-manual still works13:36
ralsinaMirv: oops, thanks!13:36
ogra_asac, stable = rtm ?13:36
asacyeah13:37
Saviqasac, I don't have a devel, not sure what you mean there13:37
* ogra_ thinks our terms get heavily overused :(13:37
asacSaviq: you have a silo targetted normal utopic, no?13:37
brendandogra_, do you know how to get add-apt-repository to recognize 14.09 as a distro?13:37
bzoltansil2100: dholbach:  we have not seen mvo so far today. He might be on vacation.13:37
Mirvsil2100: unping, a couple of minutes later csv works again, thanks google13:37
ogra_brendand, should just work on rtm installs13:37
* asac will try to stick to long form: "ubuntu" "ubuntu-rtm" in near future13:37
ogra_brendand, if not, file a bug :)13:38
brendandogra_, well /etc/lsb-release is not correct in 14.09 so it's not13:38
asacSaviq: devel i used as synonym for -> ubuntu ... stable for -> ubuntu-rtm13:38
ogra_brendand, oh, that, yeah13:38
brendandogra_, where's the best place for that to go?13:38
Saviqasac, if you want to say that I should file separate MPs into ubuntu and ubuntu-rtm, that's not gonna happen, I don't have the time to retarget a dozen or more MPs13:39
ogra_brendand, in lsb i guess13:39
asacSaviq: sil has a trick he wanted to try with your silo that uses source copy13:39
Saviqasac, please don't use synonyms like that unless we rename the channels as well, we have devel-proposed, devel, stable, rtm, ubuntu etc.13:39
asacSaviq: but you can only use that if you havent gotten out of sync yet13:39
Saviqasac, I don't think that's a problem13:40
asacSaviq: yeah, i will use devel-propose and rtm13:40
asacthe rest doesnt matter now13:40
ogra_right13:40
asacrtm-proposd actually13:40
Saviqasac, with the first srccopy landing it will get back in sync13:40
ogra_asac, rtm-proposed doesnt exist for devs ... thats happening automatically (like uploading to ubuntu ending up in ubuntu-proposed)13:41
brendandogra_, or maybe against 14.09, if possible13:41
ogra_Saviq, the plan was some kind of auto-copy of the source packages to the rtm silo ... so you dont need two branches unless you do dircet rtm uploads13:41
asacSaviq: cant say for sure - you are most likely right. the further things diverge the more stepping stones you might encounter. anyway, sil think he has a trick to do these with source copy's... would be good to try so we can announce that more widely as an option.13:42
Saviqogra_, yes, that's exactly what I want indeed13:42
ogra_right13:42
ogra_everyone does :)13:42
asac"auto" is not avail, but semi-auto13:42
bzoltanogra_: have you guys heard about that the SDK tools are broken since the  #205? https://bugs.launchpad.net/ubuntu/+source/phablet-tools/+bug/1360582 The developers can not run their apps on the device.13:42
asacis what we want to try NOW13:42
ubot5Ubuntu bug 1360582 in phablet-tools (Ubuntu) "Can't manually install clicks "Signature verification error" since #205" [Undecided,Confirmed]13:42
ogra_sil2100 is fixing :)13:42
asacauto will come later if that really turns out feasible13:42
ogra_bzoltan, yes, we need mvo to fix that13:42
ogra_bzoltan, and like most debianners he is at debconf this week13:43
Saviqogra_, asac was just saying I shouldn't do that because I already have an rtm branch, which I don't see as a problem (I will just drop the rtm serie for now or just keep it manually updated)13:43
bzoltanogra_: mvo is no around13:43
ogra_bzoltan, right13:43
bzoltanogra_:  would this situation qualify as a reason for reverting?13:43
ogra_bzoltan, he might be later today ... as he is (like many from UE, is in portland)13:43
ogra_bzoltan, but it didnt land in rtm yet13:43
asacSaviq: ah so you have branched and will use branches, then yes, you are not a good target to try; remember to deliver early and often though so you dont risk big issues with this13:43
ogra_so no worries13:43
ogra_bzoltan, shouldnt even affect you atm13:44
bzoltanogra_: and should not13:44
Saviqasac, no I won't13:44
Saviqasac, use branches13:44
ogra_well, it wont until mvo lands it there13:44
bzoltanogra_:  it does13:44
ogra_bzoltan, how, it didnt land yet13:44
sil2100ogra_, Saviq: the auto-mechanism will give the ability to request dual-silos (if there are silos free) that would basically build stuff for ubuntu and just with every build do the same for the RTM silo13:44
ogra_bzoltan, note that we dont really care for ubuntu anymore13:44
sil2100Just source-only for the RTM ones13:44
bzoltanogra_:  it is on the image since #20513:44
ogra_bzoltan, it isnt in image 613:44
ogra_bzoltan, ignore ubuntu ...13:45
ogra_rtm is our target13:45
ogra_file a bug (which already happened i think) and wait for mvo to fix ubuntu and then to land in rtm13:45
asacSaviq: right, then what i said was that you could try landing in rtm with source copying now; just talk to sil </EOF>13:46
bzoltanogra_: so that is the ubuntu-touch/ubuntu-rtm/14.09-proposed13:46
ogra_bzoltan, exactly13:46
asacyes13:46
bzoltanogra_:  thank you13:46
ogra_:)13:46
ogra_bzoltan, i agree its a bit annoying that we leave ubuntu broken now though13:46
ogra_but there are rtm channels for all devices13:47
asacbroken?13:47
ogra_asac, yep13:47
asacwe dont want to ignore ubuntu13:47
bzoltanasac: https://bugs.launchpad.net/ubuntu/+source/phablet-tools/+bug/136058213:47
ubot5Ubuntu bug 1360582 in phablet-tools (Ubuntu) "Can't manually install clicks "Signature verification error" since #205" [Undecided,Confirmed]13:47
asacbzoltan: how does this block you?13:48
bzoltanasac: since the ubuntu image 205 developers can not run their apps on the device13:48
ogra_asac, you need to hack round it by breaking security13:48
ogra_you can install clicks locally ... but not by using the right tools13:48
asacright13:48
asacso ignoring ubuntu doesnt help as we have the same on rtm i am sure13:49
bzoltanogra_:  as root and not as phablet13:49
ogra_asac, only once that landed in rtm13:49
ogra_which didnt happen13:49
ogra_bzoltan, yeah13:49
bzoltanogra_: and we do not do that :)13:50
ogra_bzoltan, well, as a workaround you can use sudo ... surely not a long term solution :)13:50
ogra_but if i would be a dev iÄd rather use sudo than waiting a day for a fix :)13:50
ogra_bzoltan, i think we sadly cant roll back ... so we need mvo to fix it13:51
bzoltanogra_: it is not about what a dev would do... it is about releasing a known to be broken and super ugly hack to LTS13:51
ogra_the server signs the packages now13:51
ogra_so rolling back might mmake them uninstallable13:51
ogra_bzoltan, oh, i didnt mean you should hack your scripts ... just tell devs how to work around it manually13:52
bzoltanogra_:  I would roll back without hesitation. I respect and love mvo .. but even he should consider what effects the changes he commits have on the app development.13:52
ogra_a fix has to happen and will eventually13:52
ogra_bzoltan, but rolling back will break even worse13:52
ogra_making the store unusable13:52
ogra_at least thats what i suspect13:53
asacso if w dont have this on stable13:53
asacit means you cannot install apps on stable?13:53
asacerr13:53
asacrtm :)13:53
bzoltanogra_:  it is not about working around .. our customers expect the "green triangle" run the app they edit. Most of them do not even know what adb and click tools are.13:53
ogra_:)13:53
* asac hits is heads13:53
ogra_asac, can you ? try it13:53
asacogra_: well, if it works, we can back it out13:53
asacno?13:53
ogra_asac, right, someone on rtm needs to install a click from the store and verify13:54
ogra_if that works fine we can also roll back13:54
=== mandel|lunch is now known as mandel
asacogra_: does it matter whether its a webapp or a native?13:54
bzoltanogra_: let's hope that not so many developers update to the 205+ image13:54
ogra_https://lists.ubuntu.com/archives/rtm-14.09-changes/2014-August/thread.html shows the new click definitely didnt land yet13:54
ogra_asac, no, just a click package with the new gpg signing from the store13:54
popeybzoltan: we've already had our own core apps developers hit by this13:54
popeypeople working on our own apps13:54
asacogra_: which one can i pick?13:55
popeyasac: ^13:55
asacdo you know?13:55
ogra_asac, any ?13:55
=== Guest9436 is now known as renatu
asaci installed delta13:55
asacwebapp13:55
bzoltanpopey: yeps..13:55
ogra_if click doesnt bail you shoudl eb fine13:55
asacand that worked on r613:55
ogra_cool13:55
ogra_well, then we should be able to roll back13:55
asaccan someone check that its really signed?13:55
ogra_beuno should be able tto tell you i think13:55
asacpopey: do you have a better example i could validate on rtm branch to be sure that the backout will not break app install by default?13:55
popeyasac: a better example of what?13:56
ogra_/me thinks all packages are auto-signed now on the store side13:56
ogra_its just the question if click bails on that or not13:56
asacpopey: of an app that might not be installable with the old click that doenst check signatures13:56
ogra_but it seems it doesnt13:56
asacyeah13:56
asaci could install the "delta webapp" on r613:56
ogra_so we should be fine13:56
asacack13:56
sil2100Is it about reverting click?13:56
popeyasac: no, it's that people developing our apps locally in qtc are having difficulty with their apps13:57
ogra_(and hope that ,mvo shows up soon :P )13:57
ogra_sil2100, yes13:57
popeyasac: e.g. clock, calendar, weather, calculator, music etc13:57
ogra_popey, they shouldnt even have noticed13:57
popeyif you get trunk and develop in qtc, then push to phone to debug13:57
sil2100If yes then I would ne +1 if it doesn't break anything in the store, which I was afraid from the start13:57
popeyogra_: why?13:57
ogra_popey, nothing of that is in rtm13:57
ogra_:P13:57
ogra_means they use the wrong image13:57
popeyuh, hang fire a moment13:57
asacright13:57
ogra_:P13:57
popeythis bug is on #205, the nexus 4 image13:57
asacso 1. lets back this out from devel13:57
ogra_popey, right13:58
asac2. lets mvoe devs to rtm13:58
popeywhich _all_ the developer use13:58
ogra_popey, devs should target rtm since last week13:58
asacpopey: app devs should move to rtm now13:58
asacor soonish13:58
ogra_since the announcement that we all switch our focus to rtm13:58
popeywell, it would be nice if we didn't have an utterly confused messaging in that regard!13:58
asacsome can stay on devel of course to continue dogfooding that it doesnt break13:58
ogra_popey, did we ?13:58
sil2100ogra_: I can prepare the source packages with the revert - do we have certainty that a revert will not cause any new issues?13:58
* ogra_ saw several announcements 13:58
popeywhere?13:58
ogra_on tubuntu-phone ?13:59
popeylink?13:59
ogra_[Ubuntu-phone] ANNOUNCEMENT: Landing team - RTM landings now officially open!14:00
ogra_thats the subject14:00
popeyNothing in there suggests that core apps developers should reflash their phone.14:00
popeyunless I missed it14:00
ogra_well, probably asac could send a clearification mail then14:01
ogra_as a higher entity :)14:01
sil2100My e-mail was only from the landing perspective ;)14:02
popeygiven none of the clicks are in the archive or via ci train, the mail only addresses individuals affected by those changes.14:02
popeyexactly14:02
boiko_robru: sil2100: is there a way to use that citrain tool with rtm silos?14:03
asacpopey: ogra_: i will wait until the initial dust around rtm/devel fork has settled; then send a call to recommend switching channel for dogfooding and app development target baseline14:03
popeyok14:03
sil2100I didn't send out an official annoucement since back then I didn't have clarification of what the procedures are to be14:03
asacpopey: ogra_: in the end it shouldnt' matter much; popey could tell them that rtm branch might be better from stability point of view etc. as landings get more thorough testing there14:04
asacalso app devs probably never wanted to run devel-proposed btw14:04
popeyyeah, they totally do!14:04
asacpopey: they run devel-proposed?14:04
popeye.g. clock depends on new / fixed features in EDS14:04
ogra_asac, they have to14:04
popeyyeah, all of them.14:04
popeydevel is like debian stable to them14:04
sil2100ogra_: so, I'll prepare the revert packages in case we want that, and then you can simply pull that in and dput to the archive14:05
asacright, thats something we should fix. folks should only temporarily switch to-proposd14:05
ogra_asac, how else would you verify your changes against the latest image ?14:05
asacto validate that a fix in there helps14:05
ogra_that would only work if we had way more promotions14:05
asacbesides that dont be on -proposed as that is wild west with reressions potentially busting you14:05
ogra_sil2100, ok14:05
asacand yes, more promotions could help, but then they could also temporarily switch to -proposed if they need sometihng now14:05
asacanyhow, later14:06
* popey gets back to holiday14:06
ogra_asac, if the last promotion is a week old you are way to much behind to make sure your changes work against latest14:06
ogra_and nobody switches temporary :)14:06
asacogra_: thats good feedback; folks said we wanted to allow longer periods of non-promotion s they can self organize to fix their issues14:07
asacbut thef act that we never had any promotion without TRAINCON-014:07
ogra_well, we had a pretty bad cycle wrt promotions and traincon14:08
ogra_that needs to become better next round14:08
asacsows taht their i can self organize claim is true on a micro level, but not on a golballevel14:08
asacwe need someting beteween -2 and -014:08
ogra_and promote even more broken with more exceptions ?14:08
asacthat doesnt slow down folks that are having no problems, but gives incentives for those that have problems to bring things back to green14:08
asacno14:08
ogra_our prob is that we never had a green image in utopic14:09
asacpart of it was an experiment14:09
ogra_and whitelist because of time pressure to get something out14:09
ogra_last cycle we blocked hard til it was green14:10
asacyes14:10
asacthis cycle was experiment based on what upstreams wanted14:10
ogra_right14:10
asaci think we need something in between and then its a touch down and perfect14:10
ogra_and i dont think there is any sane middle ground :(14:10
asaci think there is :)14:11
ogra_heh, ok14:11
* ogra_ will trust you 14:11
asacthere are multiple knobs to tune still14:11
asacthat we bounced to the other extreme this time14:11
asacbtw sent mail to foundations folks14:11
ogra_great14:11
asactelling them we consider backing out their click14:11
asacif they have better guidance they should let us know14:12
asacnot sure how long to wait14:12
asacmaybe half a day14:12
sergiusenstraincon 0 should be per project team14:12
sergiusensno reason why non related components should be blocked14:12
asacsergiusens: right, we have that basically in traincon-1 ... where slow down only happens for those that are affected14:12
asacand that is supposed to start sooner14:12
sergiusensand should be triggered that same day14:12
asacright, we have that, just not implmeneted14:13
sergiusensI've suffered enough this cycle, more so because I don't have the staging branch14:13
asacafter 2 days without promotions, components with troubles would get a similar treatment as traincon-0 while the rest can continue14:13
asaconly if that isnt envough for a week or so we bring down the big axe14:13
sergiusenswhich just allows people to delay their fixes and kill the ones with package == trunk14:13
Saviqsil2100, ok, I'm ready for publishing, what do I do?14:14
sil2100Saviq: one moment :)14:14
Saviqsil2100, sure, whenever you're ready14:14
sil2100sergiusens: TRAINCON-0 is a means that blocks projects that are currently broken and makes sure (or decreases the risk of) no new regressions appearing14:14
sil2100sergiusens: that's why TRAINCON-0 affects everyone14:15
asacsil2100: right, but they say we should do -1 :)14:15
asacsil2100: earlier, so we dont need to do -014:15
* ogra_ didnt say that :O 14:15
sergiusenssil2100: but if I have no involvement; I just get to stop working14:15
sergiusensor miss the feature freeze14:15
ogra_i dont think we can find a sane middle ground ... (but happy to be proven wrong)14:15
asacright, hence we had a middle stage where we just slow those with issues14:15
asacwhich is -1 :)14:15
asaclol14:15
sil2100Sure, we can do that, but it won't really help in this particular case, as we're getting constantly blocked by new things14:15
sil2100-1 doesn't protect us at all from new blockers appearing14:16
ogra_right14:16
sergiusenssil2100: look historically and you will see that the things that cause traincon live around the same projects14:16
sergiusensand everyone gets to suffer14:16
asacsil2100: we havent tried, so we can only guess with gut feeling. we can try and see14:16
sil2100I know from experience that everytime we're not able to get a promotion it's because when we fix one blocker, suddenly another appears in a different project14:17
sergiusenssil2100: asac I also want to see a correlation with the traincon 0 causers and the projects with staging branches14:17
asacsil2100: we should try to make statistics  :) ... also there is a psychological component that another, earlier alert level might mean earlier peer pressure and earlier cautious behaviour.14:18
asacsergiusens: that would be intersting :)14:18
ogra_heh14:18
ogra_you are assuming peer pressure wheer we have none14:18
asacand direct uploads14:18
ogra_thats the issue14:18
sil2100sergiusens, asac: we're having incident reports for TRAINCON-0's, but I do not have an 'archive' of issues that are blockers14:18
asacwe have some imo14:18
sil2100SInce this list would be really big14:19
ogra_people wont put pressure on the person causing the issue14:19
sil2100I might consider archiving those in the issue tracker though14:19
ogra_they just lean back and moan but dont pester the peer specifically14:19
asacsil2100: we have the mails and can check which issues were blockers when we announced TRRAINCON14:19
sil2100asac: I'm not saying -1 is a bad idea, I think it's good, but I just know it won't stop us from getting to -014:19
ogra_(because next time it coulld be them causing -0)14:19
sil2100asac: right, that's true, but parsing that would be a pain ;)14:20
asacsil2100: it will not stop us forever, sure. heence there is still -0. however, if we manage to not get to -0 before making a promotion 50% of time we would have onw14:20
asacwon14:20
asaceven once would be good :)14:20
asachehe14:20
asacnever seen us promote without -0 :/14:20
asacanyway14:21
asaclets focus on RTM14:21
sil2100Hey, we did promote many times without -014:21
sergiusenssil2100: asac so think about this, when in traincon 0; projects with trunk == package can't merge anymore; but projects (which caused traincon 0) can continue working as it didn't exist; so the full  team doesn't focus on traincon 014:21
sil2100We only started having TRAICON-0's all the time since like 1-2 months14:21
ogra_asac, we did promote without 0 in trusty ... several times actually14:22
sergiusenswhich is what you want with traincon 014:22
asacin trusty yes14:22
asacin utopic --> very rare14:22
ogra_asac, but that was because we had one green image we could refer to14:22
asaccertainly not in last 3 month14:22
ogra_which we never managed in utopic14:22
sil2100sergiusens: right, that shouldn't happen, managers should make sure that when in TRAINCON-0 everyone is working on getting things unblocked14:22
sil2100asac: no no, in utopic as well14:22
asacsergiusens: so thats true and thats why i said they shouldnt do it14:22
sil2100I'm sick of everyone forgetting the good promotions in utopic ;p!14:22
asacsergiusens: hwoever, they have also pressure to land stuff, which is when they get hurt too14:23
* sil2100 looks for some data14:23
asacsergiusens: ;)14:23
asacerr14:23
asacsil2100: :)14:23
asacsil2100: dont worry; also its not your fault anyway!14:23
asac:)14:23
asacyes we had an image that was almost green14:23
asacexcept 1 failure :/14:23
ogra_http://ci.ubuntu.com/smokeng/utopic/touch/ some data :P14:23
asacthat was cool and very close14:23
ogra_all non-green except for the ones that were identical to trusty14:23
sil2100That doesn't mean any promotions ;)14:23
asacwe should make nice marker which image got promoted14:23
asacso you can see easily the stream of images and promotions there14:24
ogra_asac, http://system-image.ubuntu.com/ubuntu-touch/utopic/mako/14:24
ogra_there you have your list :)14:24
asacogra_: yeah, now just a nice overlay for the dashboard made out of that14:25
sergiusensasac: I have pressure too; but I don't do the staging branches as it wasn't the recommended way14:25
sergiusensasac: so it's screw the people who work by the book14:25
ogra_asac, yeah14:25
dobeyso for ubuntu-rtm one has to create the "rtm-14.09" branch for a project, by branching trunk at the revision last synced to ubuntu-rtm, and then make an MP for trunk -> rtm-14.09 to get things back up to date?14:25
ogra_sergiusens, just go back to debs ... so much easier :P14:25
sil2100dobey: we're trying out something easier14:26
ogra_dobey, no, wait ... sil2100 is working on a better process14:26
asacsil2100: maybe dobey can help trying? :)14:26
sil2100Saviq: ok, so I'm working on this - just one question, does your landing require the new click?14:26
=== boiko_ is now known as boiko
ogra_dobey, you only land your MP in utopic ... and then when creating an rtm silo your source package gets copied over and re-built14:26
dobeyto just sync debs from ubuntu? or what/ :)14:26
sil2100dobey: yeah, that's the plan ;)14:26
Mirvoh, ok, sil2100 can handle that one :) pkging changes anyhow.14:27
dobeyoh ok14:27
ogra_dobey, only if you want rtm only changes you need an rtm branch then14:27
sil2100Mirv: wait with Saviq's changes ;)14:27
Mirv+1 for marking promoted images clearly at http://ci.ubuntu.com/smokeng/utopic/touch/14:27
dobeyogra_: that would be excellent14:27
ogra_yep14:27
ogra_send flowers to sil2100 ;)14:27
asacsergiusens: so you say we should work something into the rules that gives incentives for components that have trunk == archive?14:28
dobeysil2100: you want me to test this? :)14:28
sergiusensasac: well that is the model, right?14:28
ogra_dobey, Saviq is already ... but you could be another datapoint14:28
sil2100dobey: we're testing it manually on Saviq's branch now, but you can be first for the automatic one!14:28
asacsergiusens: i think the model is: deliver early and often, otherwise your landing is more risky and you will have troubles landing without regressions14:28
sergiusensasac: and if it isn't I'll just dput every now and then as it would be the same14:29
asacwith correlary: if you needf a staging branch you might not do it right14:29
dobeysil2100: ok, just ping me and let me know what to do, when ready for me to test it14:29
sergiusensasac: yeah; break often and fast is also missed14:29
asacsergiusens: we want to land through silos to ensure we get testplan systematically run14:29
sergiusenswhich means, when it breaks; it's hard to fix14:29
sergiusensideally no breakages; but they should be easy to identify14:29
sergiusenstraincon 0 just makes me create megaMRs14:29
sergiusensas I can't land anything14:29
=== balloons_ is now known as balloons
=== alecu_ is now known as alecu
=== psivaa_ is now known as psivaa
asacsergiusens: can you tell me what makes it hard for you to land your stuff during TRAINCON-0? is it hte QA sign off step that takes too long?14:33
=== cprov__ is now known as cprov
ogra_asac, it is the fact that his branch has to be identical to whats in the archive14:34
asacsergiusens: by design what should happen is: you can just land your stuff as usual and if you have good quality stuff, QA will come around and sign it off and it just goes into the image14:34
ogra_so if he wants to move forward he cant push stuff into his branch and things pile up14:35
asacsergiusens: can you tell me if thats the case or if there are other roadblocks that are not in that line14:35
ogra_i.e. the feature branches will grow14:35
asacogra_: thats what i am saying; traincon allows you to land14:35
ogra_or you get a bigger amount of branches to merge into trunk etc14:35
asacits designed to allow everything to land still except if your component has a blocker issue, then you cannot land anything that doesnt fix that issue14:35
sergiusensasac: requiring QA sign off last time took 3 to 4 days for a packaging change14:36
asacogra_: i understand that thats the observed symptom, asking him why that ends up to be the case exactly14:36
ogra_right14:36
asacsergiusens: ok, so its just that?14:36
ogra_QA ia busy trying to help to get the blockers fixed14:36
asac"just" :)14:36
sil2100Saviq: hah, so it seems the PPA copy-package view in LP needs to be updated, as right now it doesn't list the other's distribution seriesses14:36
ogra_so your stuff gets lower prio14:36
sil2100Saviq: from the automatic-dputting view it's not a problem, but doing it manually is more irritating14:36
sergiusensasac: mostly; and silos get full as everything is mostly blocked on the same bottleneck14:36
asacok, so a smarter/fairer queue management and QA time allocation might help14:36
asacright14:37
ogra_asac, i personally gave up trying to land during traincon ... just to not cause more work ...14:37
asacthats an outcome14:37
asacright14:37
ogra_and i know in one/two days i can land without having caused extra QA work14:37
sil2100asac: the biggest problem we noticed during traincon is also the lack of silos14:37
asacso i think we should really try the middle step whhere we only put folks with issues and risky landings (e.g. huge merges from staging branches migth often be risky) through QA sign off14:37
ogra_but that only works if you have enough other stuff to work on and if your future features dont depend on what you try to land14:37
sil2100asac: since things get slowed down natually, landings pile up and we can't even assign silos14:37
asacand then see if we can be fairer/better in case yo uneed QA sign off14:38
asacsil2100: right.14:38
sergiusensasac: right; I didn't break the image in the first place; and I do't think I have caused any blocker issue from history; but I get exta qa even though I'm careful not to break14:38
sergiusensI test on 3 to 4 devices!14:38
sergiusensI might be a special case; but why would I need QA signoff if I wouldn't outside of traincon 014:38
asacwell,14:39
asacthe state of highest alert wants to prevent new regressions14:39
pstolowskiguys, any idea why one package from row #34 failed with "'libunity-scopes>=0.6.2' not found", if it's supposed to be in the same silo (and that dependency check passes for me locally)?14:39
asacso that has to stay14:39
sil2100sergiusens: will you guarantee that you won't cause an regression with your landing?14:39
pstolowskisil2100, ^14:39
asacwhat is missing is something in the middle14:39
asacreally14:39
asac:)14:39
sergiusenssil2100: when you set testing to Yes; I expect people to guarantee that14:39
sergiusenssil2100: there have been regressions with QA Sign off14:39
sil2100sergiusens: but they don't14:39
asacfrom macro perspective that seems not the case14:40
sergiusensso why would you expect that14:40
asachence we needed to introduce qa sign off14:40
asacin the end we all are humans14:40
sil2100sergiusens: there have been hundrets of landings that were set to testing done and caused regressions, some of them weren't even properly tested since somethings were skipped14:40
sergiusenssil2100: asac the best way to make this easy is if any ci train changes would also go through ci train14:40
sergiusensthen you would get a feel for the process and it's pain14:40
sergiusenssil2100: you need history and to track the lander that set it to yes; after a couple of red flags; they get QA signoff required always14:42
sergiusensperiod14:42
sergiusensthis is like the anti good behavior; you follow the process and suffer due to other people14:42
ogra_but it makes you god like !14:43
sil2100TRAINCON-0 is supposed to be an emergency case where we're making sure nothing bad happens, not trusting anyone - TRAINCON-0 by principle should happen rarely14:43
ogra_:)14:43
sergiusensasac promised me that good behaviored citizents of the train would get benefits14:43
Saviqsil2100, aham14:43
asacyes, we are working on it14:43
asac:)14:43
Saviqsil2100, if we srccopy, is QA signoff blocking landing into utopic as well>?14:43
asacsergiusens: problem is that all think they are good citizen if you ask them :)14:43
asacso we need to be smarter, implicit etc.14:44
asacthats the tricky part of adjusting these rules14:44
sergiusensasac: that can only be proven by numbers14:44
asacSaviq: yes14:44
sil2100Saviq: no no, to utopic it can go normally, only the RTM bits need to move - but well, it'll block you to proceed further if your main target is RTM14:44
sergiusensasac: I expect them to be collected14:44
asacSaviq: whats the whole point. you need to go through qa for rtm14:44
ogra_asac, uh, no14:44
ogra_asac, utopic landings should juet be landings ... you want to have QA for the rtm one though14:45
ogra_*just14:45
asacogra_: yes, thats what i said14:45
* Saviq thinks we should batch RTM landings14:45
ogra_no need to QA twice in different context14:45
asacif you land your stuff in RTM you need QA14:45
Saviqasac, I was asking whether it's blocking landing into *utopic*14:45
ogra_asac, right, but only on the RTM silo14:45
ogra_else you have to do it twice14:46
asacSaviq: we assume that you land them in both at the same time. because of that assumption we said we wont need qa sign off for normal utopic and would also not do traincon business there14:46
asacif behaviour goes too far off we have to change that14:46
ogra_that wont work unless you rais the QA team headcount massively :)14:47
ogra_*raise14:47
asacwe have put more folks in there14:47
ogra_still, if you want every landing for both targets QAed thats like 300% more than we had before14:47
ogra_adding a few people wont cut it14:47
asacogra_: no... we want landing in both at same time14:48
asacbut only the landing in the rtm silo gets QA14:48
ogra_(up to now we only have QA signoff at all during traincon-0)14:48
Saviqtrain → halt14:48
ogra_asac, right14:48
sil2100asac: I wouldn't block the utopic landing until RTM lands - if it fails QA sign-off for RTM then we would just reject the RTM part of the landing, request a fix for utopic then publish it for utopic and RTM again14:48
ogra_asac, i'm just saying we dont have the resources to QA both landings14:48
asacsil2100: i didnt say we block utopic landing; what i said we would like to see QA sign off on the RTM silo which hopefully will exist at same time as the utopic landing silo14:49
asacogra_: and we are not doing that14:49
ogra_you just said worst case we'd have to :)14:49
sil2100asac: right ;)14:49
ogra_thats what i commented on14:49
sil2100asac: I think what Saviq wanted to know is if his utopic part of the landing will be blocked from releasing until the RTM version of the landing gets qa sign-off14:50
sil2100Which I think shouldn't be the case14:50
asacsil2100: i dont think it needs to be forced unless it makes it hard for Saviq to iterate on the fixes tahht QA fill find and also put them into utopic14:50
Saviqsil2100, asac, ogra_, still, that sounds to me like we need to batch ubuntu landings for QA sign off into ubuntu-rtm14:50
Saviqbecause they will die trying to sign off every landing separately14:51
asacSaviq: well, just land whatever you want now in obht and we see how good QA is at catching up14:51
asaclet me know14:51
asacthe better your quality delivered is the faster we can go14:51
asacif QA is the bottleneck then let me know asap14:51
asacjfunk: is QA ready to do silo sign off for stable?14:54
asac:)14:54
asacerr for rtm14:54
jfunkasac: they are in place and ready to act14:54
asacack14:55
jfunkasac: but there was some blockers14:55
asacjfunk: are those resolved?14:55
asacor is it about krillin rtm images not there yet?14:55
asacSaviq: ^^ go and grab those for your first landings. whatever is in, is in and doesnt need to be batched anymore I would say :P14:55
ogra_asac, lsb info being wrong is another one14:59
ogra_asac, i.e. you cant easily add ppas14:59
=== plars changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | CI Train support: trainguards | Vanguard (general help): plars | CI Train Status: #203 promoted | CI Train Dashboard: http://bit.ly/1mDv1FS | Known issues: citrain struggles with source packages, don't WATCH_ONLY until *after* the source is built in PPA. http://youtu.be/-Rnw0D2AdYU.
asacogra_: who  should fix lsb?15:01
asacogra_: isnt that you?15:01
asacogra_: you cannot add ppas?15:02
ogra_asac, i was taking a brief look but dev mode is as important15:02
asaci dont understand what lsb has to do with that15:02
asacogra_: yeahh was kidding. does it really need to happen urgently?15:02
ogra_the tool checks for the distro name in lsb-release as i understood15:02
asacwhich tool?15:02
ogra_add-apt-repository15:02
asacand where do we use that?15:02
thostr_sil2100: can you reconfigure silo 3 please15:02
asacduring testing?15:03
ogra_to enable silos for testing15:03
asacso if i install a rtm image15:03
ogra_you shoudl be able to hack around that ... but i havent found the time to look yet15:03
asaci cannot enable the rtm silos?15:03
ogra_right15:03
asacsure?15:03
ogra_they tool looks for ubuntu. not ubuntu-rtm15:03
sergiusensogra_: what about ubuntu-bug?15:03
sergiusensdoes it use that as well?15:04
ogra_yeah, same thing15:04
ogra_probably15:04
asacsergiusens: ogra_: can you file a bug with that info? want to do a batch mail with issues for colin later today15:04
ogra_asac, there is one15:04
ogra_there is even a ML discussion15:04
ogra_asac, i'll try to take a look later today15:05
thostr_plars: could you reconfigure silo 3?15:05
sil2100thostr_: sure15:05
thostr_plars: ^ sil2100 already took care..15:06
plarsthostr_:  you need trainguards for that15:06
sil2100ogra_: reminding you about poking plars ^ ;)15:06
asacogra_: thanks. let me know if i shall inlcude that in the batch mail for steve/colin to fix15:06
asacogra_: is there a way to workaround that we can tell landers to use?15:06
ogra_sil2100, already done :)15:06
ogra_asac, for sure, but i need to test it before i suggest ... after the meeting mumbo jumbo in 2h15:07
asacthanks15:07
sil2100ogra_: so, no revert? ;)15:08
ogra_sil2100, thats about lsb15:08
ogra_not about click15:08
sil2100Ah, ok ;)15:08
ogra_sil2100, for click lets try to catch mvo first, then revert and give him a chance to fix etc15:08
ogra_(perhaps there is justa  one line that fixes the issue ... so i'd like to get feedback frist)15:09
Saviqsil2100, anything I need to do for silo 4 packaging ACK?15:21
sil2100Saviq: no, I'll ACK it in a moment, want to push the RTM versions first, so it'll be published in ~5 mins15:21
Saviqsil2100, ok thanks15:21
sil2100Saviq: ok, so I got some XP points from this, let me now publish your silo15:23
Saviqsil2100, :)15:29
sil2100ogra_: ok, the revert package is ready - just tell when we proceed15:36
sergiusenssil2100: for silo 1 which belongs to ralsina and me, can we get an rtm silo?15:39
sil2100sergiusens: sure, are the packages in that silo final and good? If yes, I'll assign a silo and push the src packages from it to the RTM one15:41
sergiusenssil2100: so I need to test first on regular and then on rtm?15:41
sil2100Since I suppose both projects are RTM specific, right?15:41
sergiusenssil2100: everything we do is rtm specific15:41
sergiusensthe people not rtm specific are outliers actually15:42
sil2100That would be best, yeah... since it would be good if everything is still working on utopic as well15:42
sil2100sergiusens: I'll prepare a silo for you and make sure your packages get there15:42
sergiusenssil2100: well, I'll test on regular ubuntu and have qa test on rtm; I can be flashing all day15:42
sil2100sergiusens: and promise it will be automatic tomorrow15:42
ogra_sil2100, dont give him to much, next time he will expect you to have it ready in 24h :)15:45
sil2100;p15:46
sil2100The ETA would be like that, yes!15:47
* sil2100 is an optimist15:47
ogra_be more like mr. scott :)15:48
ogra_"it will take three days" ....15:48
ogra_"oh look, i made it work in two" ...15:48
bfillersil2100: are we suppose to be testing ubuntu silos against ubuntu image and rtm silos agains rtm images?15:50
sil2100bfiller: yes...15:51
sil2100ogra_: btw. did you confirm if we have ubuntu-rtm images for krillin?15:51
ogra_sil2100, we do15:51
ogra_sil2100, see janis mail :)15:52
bfillersil2100: hmnn, that's not good15:52
sil2100Ah! I see it15:52
bfillermeans a reflash to land something in both places15:52
sil2100bfiller: I know :/ If you have mako and krillin then I guess you can have krillin on ubuntu-rtm and mako on ubuntu15:53
sil2100And at least test it this way15:53
sil2100The situation is worse if you only have one of those...15:53
sil2100But there's not much we can do here, this whole split is sadly very painful in many ways15:54
kenvandineat least for me, lately everything i do has to be tested on krillin15:54
sil2100Since it would be nice to still have ubuntu up to shape15:54
=== gatox is now known as gatox_lunch
sil2100kenvandine: right, make sure you test it on ubuntu-rtm on krillin though15:54
ogra_bfiller, you only need QA signoff on the rtm ones15:54
kenvandineyeah... so much time flashing back and forth :/15:55
ogra_bfiller, test the ubuntu ones on your own ... then poke QA for rtm15:55
sil2100ogra_: well, I think the idea is not only leaving the tests to QA, but I can understand that it might be redundant15:55
ogra_there is no point in testing rtm yourself15:55
ogra_since QA needs to sign it off anyway15:55
ogra_sil2100, oh, then i misunderstood15:56
sil2100Right, not sure what are the exact rules for ubuntu-rtm landings, but when we have TRAINCON-0 and expect QA sign-off, we actually want landers to test their stuff and QA just does a 'safety check'15:56
bfillerogra_: that will be quick :)15:56
sil2100ogra_: no no, maybe you're right, I don't know ;) asac might have to shed some light on this15:57
ogra_sil2100, well, i was called a "bad ogra" when i tzested my rtm silo myself for my last landing15:57
ogra_so if Qa has to test it anyway, why shoudl i15:57
ogra_i can then invest my time into testing ubuntu15:57
sil2100Then maybe it's indeed like that, I only know what we do on TRAINCON-015:57
sil2100Not sure here15:57
ogra_asac, ^^^^^ his masters voice please15:58
ogra_:)15:58
ogra_whats the exact test plan15:58
sil2100In overall the idea of the double QA sign-off was always to put an additional pair of eyeballs looking at the landing15:58
sil2100Well, in case of RTM, I'm indeed not sure if that's needed ;)15:58
ogra_right, but double QA just wastes time15:58
ogra_test yourself in ubuntu, have QA test for RTM ... and we should be safe15:58
ogra_and traincon should anyway only target RTM15:59
sil2100Right, might be enough here, since we can more or less basically say that ubuntu and ubuntu-rtm are rather similar15:59
sil2100ogra_: but then...15:59
bfillerogra_: no offense to QA but I like to test all of our releases myself15:59
ogra_sil2100, no we cant say that15:59
sil2100ogra_: if we only force landers to test on ubuntu, this means they basically develop for ubuntu... where the target should be ubuntu-rtm, right?15:59
ogra_how do you know there isnt a minor toolchain change in ubuntu next week that we wont have in rtm16:00
ogra_sil2100, well, then dont test ubuntu at all16:00
ogra_but lets not duplicate the work for all of us16:00
sil2100Right, but then we might end up having stuff broken in ubuntu, which will be sad16:00
sil2100It's actually a very hard topic ;)16:00
ogra_harps !!16:01
rsalvetiin the end I personally like bfiller's original idea16:02
rsalvetilanding first in RTM (and testing more there)16:02
rsalvetithen syncing in Ubuntu (like we do with debian)16:03
ogra_cant do that16:03
ogra_ubuntu is upstream16:03
rsalvetiright, that's why we sync there later16:03
ogra_you need to land there first16:03
rsalvetilike we do for debian16:03
ogra_we dont do that usually16:03
rsalvetiogra_: that's a rule we created16:03
ogra_only for very specific bits developed only in ubuntu16:03
ogra_the default is the other way round16:03
rsalvetiright, but here it seems we care the testing more when landing on ubuntu16:03
rsalvetiwhile RTM is for sure the higher priority here16:04
ogra_right16:04
rsalvetiI'd just land on rtm by default and sync from rtm to ubuntu from time to time16:04
ogra_but whats the issue with having the silos created at the same time16:04
rsalvetias we don't actually care much if ubuntu is worse than rtm atm16:04
ogra_and then only test the one we care for16:04
rsalvetimore work basically16:04
ogra_we will have to drop non rtm image builds at some point anyway16:04
rsalvetiwe can have folks doing the sync from rtm to ubuntu from time to time16:04
ogra_unless we add more builders16:05
rsalvetias we have people working on upstreaming ubuntu changes in debian16:05
rsalvetiwould just be faster16:05
rsalvetiogra_: right, before we get to that point we just sync everything :-)16:05
bfillerI think the model we used to do in OEM worked very well: freeze automatically syncing ubuntu into RTM release, put current work into RTM and test there, backport changes in ubuntu at some point decoupled from RTM release16:05
ogra_might be ok for a one time exceptio16:05
ogra_but surely not as general rule16:05
rsalvetibfiller: yeah, would be way easier and faster16:05
bfillerrsalveti: worked great for all of our custom OEM projects for many years16:05
rsalvetiadding more work at the same time we're already suffering to find time to finish the rest of the stuff is not ideal16:06
ogra_bfiller, except that your changes only returned to ubuntu a release later (if at all)16:06
ogra_once we release rtm we need to switch back to ubuntu16:07
ogra_rsalveti, but it isnt more work16:07
rsalvetiwe only care about rtm now anyway16:07
rsalvetiogra_: of course it is16:07
ogra_how can it be ?16:07
bfillerogra_: yup, so there is overhead there but it's less of a priority imo, we are developing for the phone and not the desktop right now16:07
rsalvetieven if QA on both sides, because we're kind of doing the sync at the same point we're landing rtm related stuff16:07
ogra_you have to test once today you will have to test once in the future16:07
ogra_why would you QA on both sides at all16:08
rsalvetiogra_: because that's the current process16:08
rsalvetiland on ubuntu, qa, land on rtm, qa16:08
rsalvetiwhile we could only be doing land on rtm, qa now :-)16:08
ogra_rsalveti, who said so ?16:08
rsalvetiand care about landing on ubuntu later on16:08
awe_ogra_, so should I still ask sil2100 to setup a rtm silo for my ofono changes???  Seems like the discussion is still happening...???16:08
rsalvetiogra_: that's the process16:08
ogra_the current process is test yourself in ubuntu and have QA sign off rtm16:08
rsalvetiwe're not landing on rtm without testing16:08
rsalvetiand we're not landing on ubuntu without testing16:08
ogra_yes16:08
awe_that's totally broken ogra_16:08
awe_we *care* about rtm16:09
ogra_huh ???16:09
awe_per asacs mail16:09
awe_I don't want to leave QA to test my detailed ofono/urfkill cases16:09
ogra_awe_, you need QA signoff in *any* case for the rtm silo16:09
awe_that's fine, but I'm going to do more in-depth testing than they are16:09
rsalvetithat's why I'd just land stuff in rtm, make it easier for our devs16:09
rsalvetibut well :-)16:09
ogra_and if you know it works in ubuntu that should be relatively safe16:09
bfillerogra_: yes you need signoff (another slowdown) but that's in addition to develper testing (or should be)16:10
awe_so is RTM traincon-0 by default?16:10
* rsalveti lunch16:10
ogra_safe enough for QA to pick it up in rtm then16:10
ogra_awe_, yes16:10
awe_sigh16:10
ogra_awe_, well, QA needed by default16:10
ogra_not traincon ... that might even become worse :)16:10
awe_so back to my original question... you mentioned that I should ask sil2100 to setup a rtm silo based on silo-020 ( which contains ofono ).  Is that still a legit request?16:11
ogra_yes16:11
ogra_awe_, if it doesnt work yet someone who can needs to pull your silo sources and dpout for you in the rtm silo ... as interim solution16:12
awe_ogra_, ok16:12
awe_how will I know if it doesn't work?16:12
ogra_sil will tell you :)16:15
ogra_once we are out of that meeting i guess :)16:15
Wellarksil2100: bug #1355130 seems to have slipped to the (promition) blocker list on Friday by accident16:16
ubot5`bug 1355130 in indicator-network (Ubuntu) "indicator-network crashing during dialer-app and default tests on smoketesting" [Low,New] https://launchpad.net/bugs/135513016:16
Wellarkplease remove it16:16
ogra_Wellark, but the crash is still there ?16:16
ogra_http://ci.ubuntu.com/smokeng/utopic/touch/mako/207:20140825:20140811.1/9930/dialer_app/16:17
Wellarksure it is16:17
ogra_thats the lates dialer test16:17
Wellarkit's a Low priority bug.16:17
ogra_it is preventing a green image16:19
ogra_which is a pretty high prio nowadays16:19
Wellarkogra_: oh, how come? as it's explained in the bug that the issue has absolutely no side effects on any tests that are being run16:21
Wellarksil2100: could I have your comment on this16:21
sil2100Wellark: hey, one moment :) In a meeting16:22
sil2100Wellark: sooo, reading up what happened16:23
sil2100Wellark: the thing is, sure, this wasn't top-priority, but since we reported this 10 days ago we started counting the so called timer - we have a policy that every non-impacting bug that appears constantly and has no movement in it, it turns into a blocker after 7 business days16:25
ogra_jdstrand, did you notice the new failures in the security testing ?16:26
sil2100Wellark: so it's actually on the list for a reason :)16:26
sil2100Wellark: this rule is irritating, but it's our way of making sure such small things like crashers or some misc autopilot failures do not get lost somewhere in development16:27
sil2100Wellark: there is pressure on pushing further, we know, but we also have to consider quality and make sure we're crasher-free16:27
jdstrandogra_: where? mako seems to be ok16:27
sil2100(even in cases when it's not visible to the user)16:27
ogra_jdstrand, on mako :)16:28
ogra_http://ci.ubuntu.com/smokeng/utopic/touch/mako/207:20140825:20140811.1/9930/security/16:28
sil2100Wellark: otherwise we could end up with an end-version of our images that have like 10 crashes on every boot16:28
jdstrandCannot install /tmp/qrt_tests/assets/com.example.am-i-confined_0.1_armhf.click: Signature verification error: debsig: Origin Signature check failed.16:29
jdstrandThis deb might not be signed.16:29
jdstrandok, I'll need to fix that16:29
jdstrandogra_: thanks16:29
ogra_jdstrand, i guess thats fallout of the other click issue16:29
ogra_jdstrand, the SDK broke too with the gpg signing stuff16:29
jdstrandyeah-- I use pkcon to install packages16:29
ogra_jdstrand, right16:30
ogra_jdstrand, like all other tools everywhere16:30
jdstrandwell, that is what we were instructed to do :)16:30
ogra_right16:30
ogra_but with the gpg signed clicks that doesnt work anymore16:30
jdstrandyeah16:31
ogra_we will most likely have to revert this landing16:31
ogra_(but it will come back)16:31
jdstrandI guess should either just warn (which is what I thought we discussed) or gain an option like click did to allow installing them16:31
jdstrand(pkcon install-local that is)16:32
ogra_right16:32
* ogra_ votes for the latter16:32
sergiusensjdstrand: ogra_ like android, allow unsigned clicks (an option in developer mode)16:34
jdstrandthe spec says not to expose it in the gui16:35
ogra_uh, why would you expose it at all :)16:35
jdstrandright16:35
jdstranda command line option seems totally reasonable16:35
ogra_yep16:36
ogra_sadly mvo is gone this week :/16:37
Wellarksil2100: I kindly request that you whitelist the issue. As is stands, it's a Low priority bug and it will not get any attention (as there simply is not enough people to work on Low stuff) until this list is cleared: https://bugs.launchpad.net/~kaijanmaki/+assignedbugs16:43
Wellarkjust having the issue on an email will not bump it's priority16:44
WellarkI'm sorry, but this is the reality.16:44
sil2100Wellark: it's not just 'having it on an e-mail'... when it's on the e-mail in the Blocker field it means it's a blocker, and if a blocker is not fixed no promotions will happen16:45
sil2100Wellark: so the priority will have to be bumped or things will get halted16:45
sil2100Wellark: managers will need to make sure that either this gets the right attention, or actions will have to be performed to get this whitelisted16:46
sil2100Wellark: if you and your manager do not want to fix this blocker, you can reach out to QA and convince them, which is when we will whitelist it then16:47
sil2100Wellark: but the rules that we have now have been set up with QA's blessing to make sure the quality of images does not degrade too much16:49
brendandsil2100, now i'm curious which issue :)16:49
bfillerrobru: has citrain device-upgrade been updated to point at RTM silos yet?16:49
sil2100brendand: it's the indicator-network crasher we're seeing in autopilot - it passed the 7 day period16:49
sil2100brendand: so it became a blocker now as there has been no attention on it16:49
brendandsil2100, is it one found by the soak tests?16:49
sil2100brendand: the crashers in smoketesting, not affecting users (so counter counted days)16:50
ogra_brendand, yes, the ever re-occuring one in dialer and messaging tests16:50
Wellarksil2100: so what has to happen to get this whitelisted?16:51
ogra_Wellark, why not fix it16:51
ogra_we can not whitelist something eternally16:51
Wellarkogra_: because there are more important things to implement. nobody asks it to be whitelisted eternally. This is not the time to artificially bump up priorities on Low bugs16:52
ogra_(we should definitely not whitelist anything more than twice at all imho)16:52
Wellarkso I ask again, what needs to happen to get it whitelisted?16:52
ogra_sil2100 told you above16:53
ogra_QA needs to approve it16:53
sil2100Wellark: go contact QA (brendand or jfunk) and try to convince them to get it whitelisted16:53
sil2100They need to give a +1 on a recurring crasher happening at least since 10 days reproducible every time16:53
Wellarkthank you for the names.16:53
Wellarkbrendand, jfunk: --^16:53
brendandsil2100, i'd say it can be whitelisted if it does not impact on one of our readiness criteria16:53
brendandsil2100, which i have a feeling it might16:54
Wellarkbrendand, jfunk: please see the backlog for 50 minutes16:54
Wellarkstarting from16:54
* ogra_ thinks we need ageneral rule about how oiften we can whitelist at all ... so things dont slip to long16:54
Wellark19:16 < Wellark> sil2100: bug #1355130 seems to have slipped to the (promition) blocker list on  Friday by accident16:54
ubot5`bug 1355130 in indicator-network (Ubuntu) "indicator-network crashing during dialer-app and default tests on smoketesting" [Low,New] https://launchpad.net/bugs/135513016:54
ogra_whitelisting two promotions curretly means the devleoper had two extra weeks to fix it for example16:55
=== Ursinha is now known as Ursinha-afk
ogra_so that you have to have a fix in the third week16:55
sil2100I can understand there are more important things to work on indeed, this is why we're not blocking on it instantly but give it a grace period16:55
sil2100Maybe we should increase the grace period then, hm16:56
robrubfiller: the trick is that the ppa:foo/bar URL scheme has no way to specify what distro you're using, so if your phone is non-rtm, then 'citrain device-upgrade' will get you the non-rtm silo, but if your phone has an RTM image flashed then you'll get the RTM silo.16:56
Wellarkc'mon, we don't have divisions or multi-developer teams working on single components16:56
Wellarkit's usually just _one_ developer who is working on multiple components16:56
bfillerrobru: that works, boiko ^^^^^^^16:57
sil2100robru: remember you can use ppa:foo/ubuntu-rt/bar16:57
=== ubot5` is now known as ubot5
sil2100robru: and ppa:foo/ubuntu/bar16:57
robrusil2100: can you? nobody ever told me that16:57
sil2100*ppa:foo/ubuntu-rtm/bar16:57
Wellarkforcing the priorities of that one developer will mean that other Critical stuff is not getting done16:58
robrusil2100: in fact I raised this with colin and he just said "yeah, you get the RTM silo on RTM, or the ubuntu silo on ubuntu"16:58
sil2100robru: yes! That's an old change cjwatson made, ppa:foo/bar is now an implicit alias to the correct ppa:foo/ubuntu/bar form16:58
ogra_Wellark, slipping forever means that it is never fixed :)16:58
sil2100robru: although you need to modify the dput.cf a bit16:58
sil2100(if you want to push to it)16:58
robrusil2100: so this means you can install a rtm silo on a non-rtm phone image?16:58
ogra_sil2100, someone should write a mail about the dput changes you need ;)16:58
ogra_*nudge* *nudge* *wink* *wink*16:59
brendandWellark, probably best to email jfunk16:59
sil2100robru: it should be possible! I never tried though ;)16:59
brendandWellark, you can cc me as well16:59
Wellarksure, I can fix the bug. That means: Critical RTM feature of the system-settings gets even more delayed, indicator-network Critical missing features get delayed, critical bugs fixes for real production system get delayed16:59
Wellarkthe bugs will not get lost17:00
Wellarkthey are in launchpad17:00
Wellarkthey are on the plate17:00
ogra_Wellark, well, friday is the final day for features ... so get it whitelsted one more time and you have next week to work on it ...17:00
bfillerrobru: didn't work actually, I have an rtm image (at least I think I do) and ran device upgrade and it gave me an ubuntu silo not the ubuntu-rtm silo17:00
sil2100Wellark: it's a very old rule that we have and we do understand your concerns, we just want to make sure in some way that this bug does not get forgotten and left without action for a very long time17:00
Wellarkbut forcing priorities to go up on Low bugs "just because" is not acceptable17:00
Wellarkagain: it's in LP17:00
Wellarkit will not get abandoned17:00
ogra_it isnt "just because" ... not every dev looks at the landing mails or smoke tests regulary17:01
sil2100Wellark: we have many many bugs that are laying around unfixed for months and they're on LP17:01
Wellarkogra_: no, I'm not asking for one additional week. I'm saying it will not get fixed before all of the more important bugs are dealt with17:01
ogra_and not everyone is a good enough citizen to not forget about these bugs17:01
robrubfiller: erk, really?17:01
robrubfiller: I never tried it either but colin told me it should work ;-)17:01
Wellarkif it takes more than a week then that is what it is17:01
sil2100Wellark: there's always something more important to work on, we can't let some bugs get starved because of that, we need to work like schedulers making sure everything gets attention17:02
bfillerrobru: if you look at this ppa: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu-rtm/landing-00317:02
bfillerrobru: seems it's references this ppa:ci-train-ppa-service/landing-00317:02
=== plars changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | CI Train support: trainguards | Vanguard (general help): cihelp | CI Train Status: #203 promoted | CI Train Dashboard: http://bit.ly/1mDv1FS | Known issues: citrain struggles with source packages, don't WATCH_ONLY until *after* the source is built in PPA. http://youtu.be/-Rnw0D2AdYU.
bfillerrobru: which is incorrect, but when you expand the sources.list entry on that page it looks correct17:03
ogra_bfiller, you *must* adjust your dput.cf17:03
bfillerogra_: I'm not dputting.. just fetching17:03
ogra_ah, yeah, fetching seems to have other issues17:03
ogra_(lsb-release giving the wrong distro name would be one)17:03
Wellarksil2100: low priority bug is a low priority bug. they have low priority for a reason and forcing them to be fixed on expence of higher priority tasks will gain us nothing17:04
robrubfiller: yes exactly. if you look at the code (`cat $(which citrain)`), it runs the line "phablet-config writable-image --ppa $PPA/$SILO" and the value of $PPA/$SILO works out to "ppa:ci-train-ppa-service/landing-003" which is supposed to just Do The Right Thing because it doesn't specify which distro to use.17:04
robruogra_: so are you saying that installing from RTM silos is broken because ubuntu-rtm doesn't know that itself is not ubuntu? and my script is fine?17:05
robrubfiller: anyway, try editing the line that says 'PPA="ppa:ci-train-ppa-service"' to instead say 'PPA="ppa:ci-train-ppa-service/ubuntu-rtm"' and let me know if that works for you17:05
Wellarkbut if you say that I would get "one additional" week of whitelisting only to "fix it next week", then there is no point continuing this discussion. I will reply to the ML and if you decide to go to TRAINCON-0 in the end for Low priority bug then it's your decision17:05
ogra_robru, i havent seen your script, sorry if you have any way that makes silos work i guess thats fine for now :)17:06
robruogra_: http://bazaar.launchpad.net/+branch/phablet-tools/view/head:/citrain if you're curious. As far as I can tell it should work without modification in RTM but I guess RTM is just broken17:07
bfillerrobru: trying17:07
sil2100Wellark: as I said - I do understand your concerns and I will certainly bring those up with QA, but you have to understand our rationale as well: experience shows that bugs like these get left alone without attention for a very long time17:07
sil2100Wellark: I'm sure QA can be convinced for a whitelist17:07
ogra_sil2100, btw, see my conversation with mvo in -touch17:08
sil2100Wellark: since if we don't do this, I'm afraid that we might become overwhelmed by seemingly low-priority bugs which, after pilling up, will not make our product top-quality17:09
ogra_Wellark, dont forget that OEMs might use the test results as a base for signoff (not sure they do, but they surely could)17:09
sil2100As a crasher is NEVER a good thing and no one can say that it's supposed to be acceptable17:09
sil2100And if we are supposed to one day move away from manual testing and get everything automated, well, a crash during automated testing will certainly be a problem17:09
bdmurrayplars: what determines what artifcats are included in a failing test case?17:10
bfillerrobru: didn't exactly work, sources.list.d ended up with this deb http://ppa.launchpad.net/ci-train-ppa-service/ubuntu-rtm/ubuntu utopic main and should have been this: deb http://ppa.launchpad.net/ci-train-ppa-service/landing-003/ubuntu-rtm 14.09 main17:10
jfunkWellark: which crasher are we talking about?17:10
sil2100Wellark: but that being said, I'm not QA so I'm not the one to have a final word on how quality is measured17:11
robrubfiller: bah. sounds like a bug in phablet-config then. sergiusens !17:11
plarsbdmurray: pretty much anything the test drops, plus other things that we collect (such as all of /var/crash, certain system logs, etc) What are you missing?17:11
sil2100jfunk: there's this bug: LP: #135513017:11
ubot5Launchpad bug 1355130 in indicator-network (Ubuntu) "indicator-network crashing during dialer-app and default tests on smoketesting" [Low,New] https://launchpad.net/bugs/135513017:11
Wellarkogra_: when they do, then it becomes Critical. unless proven otherwise, it stays Low17:11
ogra_robru, i dont think that script  will work atm since  add-apt-repository will use ubuntu instead of ubuntu-.rtm or 14.0917:11
sil2100jfunk: it's 100% reproducible on smoketesting and didn't have any attention for 10 days now17:11
bdmurrayplars: given that OOPSIDS are now logged in /var/log/upstart/whoopsie.log instead of /var/log/syslog it seems to be we should gather /var/log/upstart/whoopsie.log.17:11
sil2100jfunk: but yeah, it's only appearing in smoketesting and does not affect users, so it might be whitelisted17:12
plarsbdmurray: I'll add it17:12
bdmurrayplars: cool, thanks!17:12
robruogra_: but i don't call add-apt-repository, unless phablet-config does (checking)17:12
robruogra_: oh, yeah, it does.17:12
robruogra_: so it's a bug in add-apt-repository then?17:13
jfunkrobru: the above defect mentioned by Wellark, does it show up as a top crasher?17:14
jfunkin the LRT17:14
ogra_robru, most likely ... mvo just showed up ... i was planning to ask him about it and then i'll fix it ... most likely lsb-release just returns the wrong distro name17:14
robrujfunk: I dunno, I haven't been tracking that one. sil2100 might know?17:15
jfunkrobru: nm, sorry for some reason I got you mixed with robotfuel17:15
jfunkrobotfuel: see above17:15
robruno worries17:15
mvoogra_: fix the --allow-unauthenticated?17:16
ogra_mvo, no, being able to add ubuntu-rtm PPAs to the ubuntu-rtm images17:17
ogra_mvo, does add-apt-repository use lsb-release info for that ?17:17
ogra_or wheer does it get the distro name17:17
sil2100ogra_: so... does it mean the revert is no longer needed?17:18
=== gatox_lunch is now known as gatox
sil2100ogra_: (it's in http://people.canonical.com/~lzemczak/packaging/ if anything)17:18
ogra_the ubuntu-rtm concept is an actual derivative distro ... not just a flavour17:18
ogra_sil2100, see #ubuntu-touch ... but it looks liek a simple pkcon fix would work17:18
sil2100ogra_: in case it's needed, it's there17:20
* sil2100 goes of to prepare the e-mail17:20
ogra_sil2100, right, thanks, but i think mvo has ideas for a proper fix instead (for click)17:21
* sil2100 seems to turn into a bad-guy after the recent happenings17:23
ogra_sil2100, well, let me land the developer mode and everyone will be distracted by things not working anymore ... and hate me instead17:24
sil2100Can't wait!17:24
ogra_its a temporary thing :)17:24
sil2100ogra_: work faster!17:24
ogra_haha17:24
ogra_yeah17:24
ogra_yay17:29
Wellarkjfunk: it does not matter if it's the most frequent crasher or not (no, it's not). The point is that it's a Low priority bug and there are major Critical ones (including the most frequent crasher) that need to be fixed first. So my question is, can I get a whitelisting for undetermined time (until the higher priority bugs are dealt with) or not.17:33
ogra_Wellark, on what base do you say its not ?17:34
ogra_indicator-network is by far the most crashy things in testing over the whole cycle ... we only had very few images where we didnt have any crashes from it17:36
ogra_*thing17:36
jfunkogra_: it so happens that the #1 crasher is in the indicator-network17:36
jfunkand is not this defect17:36
jfunkWellark: who is pressuring you to fix this? is it landing?17:37
ogra_(i understand that the former crashes were induced by ofono phonesim etc, but still its by far the most crashy piece in testing)17:37
ogra_jfunk, it was added to the blocker list17:37
ogra_since it did hit the timer17:37
jfunkogra_: how was it on the timer list at all as a low defect?17:37
Wellarkjfunk: the fact that it's been marked as "Blocker" which will lead to TRAINCON-0 if not fixed17:38
ogra_jfunk, its is a constantly re-occuring crasher17:38
Wellarkogra_: as I've explained to QA, features first, crasher after that. The most frequent crasher is already Critical.17:38
ogra_jfunk, after a certain period these get on the timer ... we dont judge prioriity in tegh landing team, just occurence17:38
bfillerogra_: do we have a diff between ubuntu-touch/utopic-proposed and ubuntu-touch/ubuntu-rtm/14.09-proposed anywhere?17:38
ogra_jfunk, if y crash doesnt get worked on in a while it gets on the timer which then counts down17:39
bfillerweird I'm seeing an old version of ubuntu-keyboard on ubuntu-rtm17:39
bfillerwas released last weds17:39
ogra_bfiller, no, we have a -cahnges ML ... nothing landed or changed since friday17:39
ogra_bfiller, https://lists.ubuntu.com/archives/rtm-14.09-changes/2014-August/thread.html17:39
ogra_bfiller, last manual ubuntu -> rtm sync was on the 19th ...17:40
ogra_everything after this is expected to be landed via the new landing process17:40
bfillerogra_: ok17:40
jfunkogra_: Wellark: I am inclined to agre with Wellark on this, maybe if the crasher is so important, then the priority of the defect should be raised?  Am I missing somethign?  I don't feel that we should blindly obey a process.17:40
ogra_bfiller, we should perhaps ask cjwatson to do one final sync though17:40
bfillerogra_: yes, seeing that sil2100 email went out on Aug 21st and everything was branched then I would expect RTM to contain everything up until then17:42
ogra_jfunk, it isnt about bug importantce at all ... nobody looks at bugs in the landing team in that context17:42
Wellarkogra_: I would *love* to get the crashers fixed. could you get me a team of two or three developers? We have priorities on the bugs. When there are multiple Criticals it's up to the people responsible of the component to prioritize between them. Having a process that forces Low bugs over Criticals and blindly mandates artificial priorities on the developers and owners of any component is just unacceptable.17:42
ogra_jfunk, it is simply about stuff not being worked on at all17:42
ogra_jfunk, if we need a whitelisting, so be it ... by the default rules this crasher had to be put onot the blocker list after it has been around for a certain timeframe17:43
jfunkogra_: how did this get on the list in the first place, was it a regression?17:43
sil2100jfunk: remember our discussions in Malta?17:43
ogra_jfunk, we look at the test results twice a day ...17:43
ogra_jfunk, if something new shows up we contact the dev17:43
sil2100jfunk: we agreed that any non-impacting-users issue when it's reproducible we give a grace period of 7 days, and after it passes we set it to a blocker17:43
jfunkahh ok, I see now17:43
ogra_jfunk, after a defined timeframe if there has been no work on the issue the issue gets a countdown17:44
ogra_after this it turns into a blocker17:44
sil2100jfunk: it happens all the time so we're just acting according to the rules we set17:44
jfunkso this is a case of a test which was passing has started to fail17:44
ogra_everyone hits these all the times ...17:44
jfunkcorrect?17:44
Wellarkand if somebody migh think that "Wellark would have fixed this issue already if instead of bitching about it" rest assured. It's almost 9pm here (although I'm sure nobody here is counting). I'm having this conversation simply because I care of the end result and also about having a process that does not inflict additional pain on our most important assets; the people.17:44
sil2100jfunk: not a test, it's a crasher, a crash happening all the time - it wasn't there before and it started happening since some time17:45
ogra_jfunk, well, in this case it isnt a test failure but a crash ...17:45
sil2100jfunk: it's a crash visible on smoketesting17:45
ogra_jfunk, we cant do anything with these crashes ... we only see them happen, the dev who owns the componnent needs to retrace it, judge severity etc17:45
Wellarkjfunk: no tests are failing.17:45
Wellarkimpact 017:45
ogra_Wellark, can you prove that ?17:46
Wellarkogra_: yes. if it has an impact it will come through errors.ubuntu.com as crasher on a production system17:46
ogra_sorry to be an ass here ... but "impact 0" need to be claimed based on some data17:46
Wellarkand all of those have priority Critical17:46
Wellarkogra_: It's not me17:46
Wellarklook at the bug17:47
ogra_Wellark, and how do yu knwo it doesnt taint any testing results ?17:47
sil2100A crasher is never expected behavior17:47
ogra_right17:47
sil2100If crashers are expected behaviors, then why do we even care to list their count on smoketesting?17:47
ogra_Wellark, even through apport-collect running in the next test this could have impact on test results17:47
sil2100If crashers are normal, then I would remove the 'Crashes' field in the dashboard, then it will not be an issue17:47
ogra_because your CPU is hogged or whatnot17:47
Wellarkogra_: which AP test is failing because of this?17:48
ogra_Wellark, i dont know, but i also dont claim "impact 0"17:48
Wellarkwhich component is affected because of this?17:48
Wellarkwell, the current data we have says "impact 0"17:48
ogra_i'm just doubting you can reliably claim impact 017:48
ogra_the current data says we have a crasher every image17:48
jfunkogra_: sil2100: perhaps we need to revisit the value of the process surrounding crashers in the smoke test17:48
sil2100Sure17:49
jfunkatm, we have a "top crasher" list17:49
ogra_we had times where we didnt have a crasher of this componnent every image17:49
Wellarkthis is not the top crasher17:49
ogra_so somethig *is* broken now17:49
jfunkwhich to me, will get the most value from time spent solving crashes17:49
Wellarkor even if it is17:49
ogra_and we dont now if it has impact or not17:49
sil2100jfunk: ok, then the rules we set in Malta are no longer valid? Why did we discuss this and agree to this in the first place?17:49
sil2100Anyway, I'm all for changing the rules17:49
jfunksil2100: I have a feeling some things have evolved since Malta17:49
sil2100Right, then why no one informed us about this that the QA criteria have changed?17:50
jfunkI'm not saying we throw the "rules" away, but perhaps we look at the value we are getting from them, and the risks we run if we change them17:50
Wellarkas I said. get me a team of couple of people to crunch down the list of Criticals and High and then this will be fixed. unless that happens this Low bug will not be fixed until I myself will get to it on my list.17:51
jfunksil2100: no one informed you because the rules haven't changed17:51
jfunkthere's a discussion happening here17:51
ogra_jfunk, in trusty we had the rule to only release fully green images ... that brought us a reasonable quality ... but also caused immense pain for devs not being able to land stuff ... if you look at http://ci.ubuntu.com/smokeng/utopic/touch/ you will notice how much green there is when scrolling17:52
sil2100Well, I think I'm tired already and a bit agitated17:52
ogra_and as an enduser and dogfooder i can only say that very much reflects the quality difference between trusty and utopic today17:52
sil2100I'm fine for revisiting the rules, but it doesn't change the fact that we just do as per our rules that have been set, and it's not that we are doing this because we want to be irritating and slow people down17:53
ogra_jfunk, we were very lax in utopic wrt blocking stuff for promoting images ... but we need to find some middle ground to actually get one green image *once* the timer was one approach towards that17:53
WellarkI will send my reply to the ML. I don't think this is something that gets resolved here.17:54
jfunkWellark: +117:54
ogra_if we start ignoreing crashers we can as well stop testing altogether ... or doing traincon states etc17:54
ogra_at least fro the automated part17:55
jfunkogra_: sil2100: perhaps someone can spell out the rules which surround crashers found in the smoke tests17:55
jfunkogra_: sil2100: in response to Wellark's mail17:55
sil2100jfunk: there was an announcement regarding that like a month ago17:55
ogra_having crashers aroudn for months and nobody working on them wont get us any better quality17:55
jfunksil2100: great, do you have the subject handy17:55
jfunkit would be great to read17:55
* ogra_ guesses in one of the langing mails 17:55
sil2100I need to find the landing e-mail, as it was in the daily mail17:55
ogra_*landing17:55
jfunkogra_: I see the risk being that if we allow new crashes in the smoke tests then we will accumulate debt in the form of crashes17:56
sil2100The only thing that irritates me here is that it was some people from QA that had concerns about us ignoring crashers and AP test failures that have no impact on users17:56
ogra_jfunk, right, that is exactly what happens atm17:56
sil2100Since we were whitelisting them normally before17:56
ogra_and why we had the timer setup discussed in malta17:57
sil2100Then QA had concerns, so we started this17:57
sil2100And now we're being pointed out as the bad people for this17:57
ogra_so the dev has time to put an issue on low prio for a while but needs to fix it after some time17:57
Wellarkogra_: the reality is that we don't have enough people to work on _all_ of the crashers17:57
ogra_this was the alternative to blocking completely to retain some basic quality17:57
sil2100Wellark: that's true17:58
Wellarkwe need to prioritice them17:58
ogra_we dont want to go back17:58
ogra_but seemingly the loosening of the rules just makes us being shouted it17:58
sil2100Wellark: right... in a perfect world though, I would like those low priority ones to also get some attention17:58
Wellarksil2100: me too17:58
ogra_*at17:58
ogra_well, in the old ubuntu world you would have asked a community enthusiast to look at it while doing the important stuff17:59
ogra_but these guys get rare17:59
jfunkWellark: so what I am starting to see from this discussion is that the crasher they are mentioning is important because we need to incrementally improve our quality, and if we let it slide, we are basically opening the gate to incrememntally accumulating new debt17:59
sil2100Wellark: anyway, sorry for being a bit rough, in any case if jfunk or some QA representative thinks it's fine to whitelist, then we can do that without any problems17:59
sil2100Since we did that in the past and it didn't kill us ;)17:59
Wellarksil2100: it's ok.17:59
* sil2100 gets back to his e-mail since he's grumpy18:00
Wellarkit's me who is constantly second guessing myself with this18:00
sil2100;p18:00
WellarkI don't want to argue18:00
Wellarkbut the reality me and a lot of devs face with the pressures we are having I just can't leave this be18:00
Wellarknothing personal18:01
Wellarksil2100, ogra_: -^18:01
sil2100True18:03
sil2100There's not that much time and much work18:03
* ogra_ hugs Wellark 18:07
ralsinasil2100: silo 1 is tested on utopic, can we get the srccopy to the rtm silo?18:09
=== Ursinha-afk is now known as Ursinha
Wellarkogra_: <318:11
Wellarksil2100: <318:11
Wellarkjfunk: <318:11
* sil2100 hugs Wellark too18:11
sil2100;)18:11
sil2100Anyway, all sides expressed their concerns, the arguments have been set so I think, if QA (jfunk!) gives a blessing we can whitelist it18:12
ogra_oh fun18:13
ogra_whats going on there ?18:13
robruurk18:13
robruogra_: I ran 'archive landed requests' and now the bot is freaking out18:13
ogra_looks like a complete spreadsheet refresh18:13
robruogra_: no idea why... queuebot is supposed to index based on landing descriptions, not line numbers.18:13
* jfunk is in PM negotiations right now, will have word shortly18:14
ogra_heh, tell stgraber :)18:14
robruogra_: it occurs to me that the original "index on landing description" code was leaky, since it would always add new descriptions to the dict and never clear them... maybe stgraber "fixed" it...18:14
robrubfiller: you got silos 15 and 1618:15
bfillerrobru: thanks18:16
robruralsina: request on line 36 conflicts with silo rtm-518:16
robrubfiller: you're welcome18:16
ralsinasergiusens: ^18:16
ralsinarobru: you mean silo 5 of rtm ?18:18
robruralsina: yes that's what I said. although I'm confused, it looks like the thing in silo 5 is missing from the spreadsheet, I wonder if those are the same landings but just got disconnected somehow18:20
ralsinacould be! We asked sil2100 for a rtm silo earlier, maybe that's 5?18:20
sil2100ralsina: yes! rtm silo 5 is yours :)18:21
sil2100(didn't do the binary copy yet)18:21
ralsinaok, so no conflict. I don't need a new silo, I just need the things copied whhen you can :-)18:21
sergiusensrobru: is there a dashboard for that?18:21
sergiusenswhere to I click "build"18:21
robrusergiusens: http://people.canonical.com/~platform/citrain_dashboard/#?distro=ubuntu-rtm&q= same dashboard grew an rtm page18:21
sergiusenssil2100: robru since this is all go and staticky; ralsina and myself are highy confident it would just work18:22
ralsinatrue18:22
sergiusensa bin copy would work even, but we can tackle that another day18:22
robrusergiusens: thank go for saving us from ourselves18:22
sergiusens:)18:23
ogra_sergiusens, please no bin copies ...18:23
sergiusensnice pun18:23
ogra_unless cjwatson asks you to :P18:23
sergiusensogra_: now you are being cautios ;-)18:23
ogra_heh18:23
sergiusensogra_: doesn't matter; the package takes time to build due to the packag install; actual build takes 2 seconds ;-18:24
sergiusens;-)18:24
ogra_sergiusens, not cautious ... didnt you notice, i just passed the bucket to someone not here ;)18:24
sergiusensogra_: :-P18:24
ogra_thats not cautious, thats *clever*18:24
* ogra_ grins 18:24
sergiusensogra_: well cjwatson was fine with a bin copy taking the necessary precautions; we did test the utopic target on an rtm channel fwiw ;-)18:25
ogra_yeah, the general rule shouldnt be bin copies though18:25
sil2100Bin copies can work most of the time as well!18:25
ogra_i agree it is unlikely to cause issues18:25
sil2100Buuuuuut18:25
sil2100Not sure if we have the tools for that right now18:25
sil2100LP doesn't cope well with copying anything between ubuntu and ubuntu-rtm PPAs right now18:25
ogra_launchpad-lib should have everything you need18:26
ogra_at least fro copying around stuff between PPAs18:26
=== robru changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | CI Train support: trainguards | Vanguard (general help): cihelp | CI Train Status: #203 promoted | CI Train Dashboard: http://bit.ly/1mDv1FS | Known issues: There are vastly more silo requests than silos available. They are being assigned on a first-come, first-serve basis. Please ping trainguards if you want a silo and are here to use it right now.
jfunksil2100: ogra_: I am ok to whitelist the defect until after FF, can you please begin the countdown anew the day after FF is complete?18:29
jfunkWellark: ^18:29
ogra_sil2100, see jfunk18:30
ogra_jfunk, sill maintins the counters18:30
ogra_*maintains18:30
sergiusensrobru: sil2100 that packages build above is a false; https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu-rtm/landing-00518:30
sergiusensit's empty18:30
ogra_did you put on the right glasses ?18:31
sergiusensdo I have to mention the packages?18:31
robrusergiusens: sounds like somebody ran a WATCH_ONLY, it sets the status stupidly to 'packages built.'18:31
Wellarkjfunk: thanks18:31
ogra_yeah18:31
sergiusensrobru: well the job finished as well18:31
ogra_you need to add the package names to the spreadhseet18:31
sergiusensrobru: https://ci-train.ubuntu.com/job/ubuntu-rtm-landing-005-1-build/1/console18:31
ogra_and do a watch only build after they have built18:31
sergiusensdidn't do anything18:31
sil2100We didn't push the packages yet, one moment18:32
robrusergiusens: yeah, because there's nothing to do. the packages need to get dput in before the build job is run. the build job is just to acknowledge the packages were uploaded.18:32
sergiusensogra_: they show up in the dashboard :-)18:32
ogra_oh18:32
sergiusenssil2100: robru ah, I thought build would do that for us now :-)18:32
robrusergiusens: yeah the dashboard reflects what was copied into it from the spreadsheet.18:32
sil2100sergiusens: it will be done tomorrow!18:32
sil2100I mean... that's the ETA for what at least ;/18:32
=== robru is now known as robru_breakfast
sergiusenssil2100: oh, that's what will be done, no worries18:33
robru_breakfastbrb18:33
ogra_yeah18:33
ogra_we were a bit premature with all the rtm switch18:33
sil2100sergiusens: actually it will be working in such a way that you can fill in a landing that will be marked as 'target: dual' and you will get 2 silos assigned at once, and pressing Build on the ubuntu silo will cause source packages of your uploads be pushed to the ubuntu-rtm silo as well18:34
sil2100sergiusens: ...more or less18:34
sergiusenssil2100: that is the dream :-)18:34
sil2100sergiusens: the end result might be a bit different from the dream design but yeah ;p18:35
=== slangase` is now known as slangasek
ogra_but close :)18:35
sil2100Indeed!18:38
sil2100sergiusens: I just uploaded the modified source packages, they should appear in the PPA soon - pretty soon you can do a WATCH_ONLY build18:41
sil2100Ok, I really need to go and write that e-mail18:41
sil2100ogra_: revert click anyway I suppose18:47
ogra_yeah, just got a PM about thet too :)18:49
sil2100I guess we should really be more willing to revert! ;)18:50
ogra_well ...18:51
ogra_sil2100, i was just saying to slangasek, reverting immediately might make us miss issues the breakage causes ... which then hit us the next time it lands18:52
ogra_so i think a short period of breakage is actually a good thing18:52
sil2100Right, but theoretically if the breakage has been noticed, well, we anyway have it in an image18:54
ogra_indeed18:54
sil2100So we can test it by installing this particular image in case we want to take a good look at it18:54
sil2100At least...18:54
slangasekogra_: people can continue to test with the existing image18:54
* slangasek nods18:54
sil2100But true, instant revert = no, quick revert = yes ;) i.e. no longer waiting for someone to create a fix for half a day18:55
ogra_right18:55
ogra_i think thats what we can agree on :)18:56
sil2100See you tomorrow everyone o/19:02
ogra_sil2100, wait19:02
ogra_sil2100, where was that soource package ?19:02
sil2100uh, what's up? Did I revert something wrong?19:02
sil2100Ah19:02
sil2100http://people.canonical.com/~lzemczak/packaging/19:02
ogra_ah, got it, enjoy your evening19:02
sil2100(there's some old revert there as well, ignore that)19:02
sil2100Thanks o/19:03
ogra_yup19:03
ogra_i think i uploaded the older one before :)19:03
sil2100Right, you're my main-reverter ;)19:03
cjwatsonogra_,slangasek: I disagree with reverting all of click when we can revert just one revision instead - see mail19:20
ogra_cjwatson, to late ...19:20
cjwatsonGRRRRR19:20
ogra_already uploaded ...19:20
cjwatsontempted to immediately upload with the better revert19:21
ogra_feel free19:21
cjwatsonnow incomprehensible version19:22
cjwatsonOh I can use 0.4.31.2.is.0.4.3019:24
cjwatsonEr, 0.4.31.219:24
cjwatsonSIGH, 0.4.31.319:24
thostr_what happened to silo 12? was that taken away on purpose?19:27
thostr_ci-train dashboard still lists it for my MPs...19:27
cjwatsonogra_,slangasek: ok, more surgical revert uploaded now, https://launchpad.net/ubuntu/+source/click/0.4.31.319:27
cjwatsonhttp://paste.ubuntu.com/8143369/19:27
ogra_cjwatson, great, thanks19:27
ogra_heh19:27
ogra_thats simply19:28
ogra_*simple19:28
cjwatsonthe full revert deleted a facility that the error tracker is either now using or is about to rely on19:28
cjwatsonso it was inappropriate IMO19:28
ogra_well, i wouldnt have expected it to be reverted for long ... but yeah19:28
ogra_cjwatson, do you have any idea what we need to do for add-apt-repository to work with rtm ? seems it defaults to add ubuntu and not ubuntu-rtm19:29
cjwatsonogra_: wgrant had a to-do item for that I think19:29
ogra_oh, cool19:29
cjwatsonogra_: I think it probably just needs to be taught about something like new-style archive references19:29
ogra_asac had asked me to check that19:29
cjwatsonwith a fallback to the old default19:30
cjwatsonso you'd do add-apt-repository ppa:ci-train-ppa-service/ubuntu-rtm/landing-00119:30
ogra_it doesnt read from lsb-release, right (someone claimed that)19:30
ogra_?19:30
cjwatsonlsb-release is irrelevant19:30
ogra_right19:30
ogra_well, we should change it nontheless ... but not urgent19:30
cjwatsonnew-style archive references are the way forward19:30
cjwatsonthis has all been canonicalised in Launchpad since we started the derived distros resurrection19:31
cjwatsonwe just have a few more things to go round and update19:31
ogra_right, QA was a bit desparate though19:31
thostr_cjwatson: how to we get the silos back into ci sheet?19:42
robru_breakfastthostr_: in case of discrepancy, the dashboard is always more correct than the spreadsheet. I can fix that soon19:42
thostr_robru_breakfast: right. BUT, I cannot trigger a rebuild since it cannot find any references in ci sheet any more19:43
ogra_thostr_, you mean http://bit.ly/1mDv1FS ? or the spreadsheet ?19:43
robru_breakfastthostr_: build should be fine. Just reconfigure will fail. One sec19:43
thostr_ogra_: yes, that is fine... I tried a recon and that failed then...19:44
ogra_ah19:44
cjwatsonthostr_: err I have no idea, not my field19:44
robru_breakfastthostr_: the build failure is unrelated19:45
thostr_robru_breakfast: it was a recon failure19:46
=== Ursinha is now known as Ursinha-afk
=== renato is now known as Guest55515
robru_breakfastthostr_: it quite clearly said build failure, anyway I fixed the spreadsheet for you19:53
robru_breakfastthostr_: hmm, looking at the jenkins build timestamps, it does look like the recon was run since the most recent build. i guess there's a new bug that recon failures aren't updating the status. yay.19:54
thostr_robru_breakfast: yes, it was a build failure which we fixed by now. that's why I wanted to recon/rebuild19:54
robru_breakfastthostr_: ok, sorry about the mixup, spreadsheet just loses status sometimes. haven't been able to track it down. it's ready to recon if you need it.19:55
thostr_robru_breakfast: if it's the very same MP I don't need to recon, do I?19:55
sergiusensrobru_breakfast: ogra_ seems  bunch of packages are missing from the rtm sync; how do I make sure teams do the right thing?20:01
sergiusensrobru_breakfast: ogra_ I need https://launchpad.net/ubuntu/+source/powerd and  https://launchpad.net/ubuntu/+source/indicator-datetime in rtm silo 520:02
sergiusenscan we dput that as well?20:03
sergiusensralsina: ^^20:03
robru_breakfastthostr_: no, recon is only for new mps.20:04
thostr_robru_breakfast: ok20:04
robru_breakfastsergiusens: i thought sil was handling that? I can get to it shortly if not20:04
sergiusensrobru_breakfast: I thought he took of for the day20:05
sergiusensrobru_breakfast: in any case needs silo reconfig and those to sources added20:05
sergiusensno need to rebuild what already there thanks to dbus20:05
robru_breakfastsergiusens: ack will do soon20:05
=== robru_breakfast is now known as robru
robrusergiusens: ok sorry, what should I put in rtm-5? is there an ubuntu silo to copy from or do you just want the copy from ubuntu archive?20:15
sergiusensrobru: from the archive as these two packages are from last week20:16
robrusergiusens: ok no worries20:16
sergiusensrobru: I guess this will happen a bit until things normalize and the things that missed the sync get synced20:16
ralsinasergiusens: ack20:18
robruralsina: sergiusens ^ don't build yet20:23
robrufor some reason it's taking me an absurdly long time to download those source packages.20:24
=== Ursinha-afk is now known as Ursinha
robrusergiusens: ok sorry that took so long, it's my first time doing one of these super-manual RTM syncs. just did the upload, hopefully it was accepted. will keep an eye for that20:36
robrusergiusens: ok it looks as though the upload has been accepted, source packages are just rebuilding right now.20:37
robrusergiusens: unfortunately I have no clue how to track what packages are in sync and which ones are out of sync between RTM vs ubuntu, so I guess we'll just need to discover those on a case by case basis.20:38
=== robru changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | CI Train support: trainguards | Vanguard (general help): cihelp | CI Train Status: #203 promoted | CI Train Dashboard: http://bit.ly/1mDv1FS | Known issues: There are vastly more silo requests than silos available. They are being assigned on a first-ping, first-serve basis. Please ping trainguards if you want a silo and are here to use it right now.
sergiusensralsina: I'm testing the packages now to see what gives20:59
sergiusensmeh, will have to wait after aikido21:03
sergiusensneed to updated powerd from recovery...21:03
ToyKeeperWeird.  Slow day for silos.21:19
robruToyKeeper: everything ground to a halt with the RTM confusion. nobody knows where to land or what to do.21:24
robruToyKeeper: I'm sure I can find something for you to test. how about rtm-5?21:24
ToyKeeperDarn, I was hoping that would be smoothed out by today.21:24
ToyKeeperI have no shortage of other things to do though.  :)21:24
robruToyKeeper: http://people.canonical.com/~platform/citrain_dashboard/#?distro=ubuntu-rtm&q= btw not sure if you saw, but here's the RTM dashboard. the link to switch from ubuntu to rtm is in the top left of the page.21:24
ToyKeeperrobru: Thanks, I had not seen that yet...  and it looks like there's one waiting on QA now.21:25
ToyKeeperI just wasn't checking in the right view.21:25
ToyKeeper... and a few script mods later, I think I can probably get rtm silos installed.21:29
robruToyKeeper: yeah I initially made those as separate pages simply because it was the easier thing to do, and then I justified it by thinking "well, it would be overwhelming to display 40 silos on the same page." maybe I need to rethink that, the rtm sub-page is a little bit ghetto-ized...21:29
robruToyKeeper: yeah citrain script fails as well, apparently the bug is in add-apt-repository, I heard colin was working on a fix for that21:30
ToyKeeperI think it would probably be easier to number them 1 .. 40, and arbitrarily decide that 20-40 are for RTM.21:31
ToyKeeperUnless, of course, we intend to scale further for more branches.21:31
robruToyKeeper: yeah, that was exactly the reason that it was decided to duplicate the names. so we don't paint ourselves into a corner with the numbering. although I've wanted to increase the number of silos by at least 50% on several occaisions and it never happened. I'm afraid that the ability to add extra silos has bitrotted away21:34
ToyKeeperWell, with apt-add-repository out of the loop, the silo-add script runs faster.  :)21:38
ToyKeeperOh, oops.  I need to mod my flash script too, to pull rtm images instead of utopic.21:39
cjwatsonrobru: I didn't say I was, no - at DebConf so a bit tricky.  I thought wgrant had a work item for it22:30
oSoMoNrobru, hey, can silo 10 be published?22:33
wgrantOh, citrain actually uses add-apt-repository?22:33
oSoMoNlooks like the auto-updated status in the CI train spreadsheet is messed up, btw22:34
robruwgrant: only indirectly. `citrain` tool (a script for end users to install silos on their devices, not used internally in the train) shells out to phablet-config, which in turn shells out to add-apt-repository.22:34
robruwgrant: cjwatson it would be nice if 'add-apt-repository ppa:foo/bar' did the right thing, eg installed the ppa with the distro that matches the system.22:35
robruoSoMoN: looking22:35
robruoSoMoN: ugh, yeah it seems the spreadsheet has lost track of many silos. you claim silo 10 is tested?22:36
oSoMoNrobru, it is indeed, tested, re-tested, and then tested again22:36
robruoSoMoN: ok, I'm gonna need you to test it one more time, then rebuild, then test again, then file your request in triplicate, and you'll need to include your gradeschool report cards from grades 1, 2, 3, not 4, 5, and not 6.22:37
oSoMoNrobru, mmm, this CI train thingy is getting a little bureaucratic lately :)22:38
oSoMoNbut ok, let me get back to you with a recommendation letter for my silo from my previous employer in a bit22:39
robruoSoMoN: oh it's not that bad, but we are going to need those reports notarized, signed in blood, and blessed by the pope. good luck.22:39
robruoSoMoN: ok, seems everything is in order here ;-) https://ci-train.ubuntu.com/job/ubuntu-landing-010-2-publish/11/console22:40
oSoMoNrobru, thanks :)22:40
* oSoMoN heads to bed22:40
robruoSoMoN: sweet drams!22:40
oSoMoNcheers22:40
robruwow22:44
robruwhat's the status of silo 1? anybody?22:44
robrustgraber wtf ^22:45
robru... is it safe? wow23:00
robruugh23:00
stgraberrestarted queuebot, let's see if it gets confused again23:04
robrustgraber: hey, is lp:queuebot in sync with the code that's actually in production? it's been doing some odd stuff lately that doesn't seem to correspond to the code I wrote23:05
stgraberrobru: just checking and yeah, it's in sync with the LP branch23:06
stgraber*checked23:06
robrustgraber: like, I ran 'archive landed requests' in the spreadsheet (which causes the rows to get moved upwards) and then queuebot re-reported a bunch of statuses. it didn't make sense, I thought we indexed the statuses by column A, not by row number.23:06
stgraberok, so it's still confused, turning off that plugin23:06
stgraberdone, it'll now only run the silo plugin and not the landing one23:07
robrustgraber: ok, I guess I'll tinker with the code and see if I can't get it working and then submit a branch.23:07
robrustgraber: thanks for stopping the flood23:07
robruralsina: sergiusens: so you have this superceded MP in your silo: https://code.launchpad.net/~sergiusens/account-polld/qtcontact/+merge/228969 not sure if you want to replace that one with the one that replaces it, or what your intentions are there...23:18
sergiusensrobru: right.... forgot to update23:18
=== robru is now known as robru_brb
sergiusensrobru: from my last status I didn't test this yet (well I did test utopic) but not rtm as powerd required me to go to recorery'23:19
sergiusensrobru_brb: let me get back to you; today is going to be a looooong night23:19
robru_brbsergiusens: no worries, I'll be available for at least 2 more hours officially, and then I'll be out for dinner but reachable with some delay.23:47
=== robru_brb is now known as robru
=== slangase` is now known as slangasek
sergiusensrobru: installing powerd is just a tad complicated; requires chrooting into recovery as there are overlays from krillin23:59

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!