[01:08] <robru> asac, you're welcome!
[01:31] <robru> i'm heading out, but will bring laptop with me. brb
[02:04] <imgbot> [03:08] <bzoltan> Mirv: almost all tests fail for the gallery app on a stock 187 image
[03:39] <imgbot> [03:39] <imgbot> [03:48] <Mirv> bzoltan: gallery was "fixed" in #187..
[03:49] <bzoltan> Mirv: "fixed" like "fixed"?
[03:55] <bzoltan> Mirv: on the stock image  28-31 of 41 tests fail consistently
[03:55] <Mirv> bzoltan: fixed for certain tsrget, but seems a lot of failures on mako in dashboard
[03:56] <bzoltan> Mirv:  All right, it means I can not test the UITK release candidate against the gallery autopilot tests... all other tests are OK
[03:57] <bzoltan> Mirv:  I am still struggling with the -gles package to disable the component tests ... after it is done the SIlo7 will be good to go
[03:58] <Mirv> ok. we can assume another attempt to make gallery APs work everywhere correctly is being made
[04:34] <rsalveti> cool, can't launch any app with latest
[04:34] <robru> rsalveti, pf, apps are for chumps
[04:35] <rsalveti> and I'm now forced to add a passcode
[04:36] <robru> rsalveti, indeed libubuntu-app-launch2 is in the new image, can you try downgrading that (or flashing a lower image and then upgrading it if that's easier for you...)
[04:36] <rsalveti> yeah
[04:36] <rsalveti> got a few denieds
[04:36] <rsalveti> http://paste.ubuntu.com/8042111/
[04:37] <rsalveti> nah, will just go to bed, that's fun for the european folks
[04:37] <rsalveti> ogra_: ^^
[04:37] <robru> Mirv, ^^ ;-)
[04:55] <Mirv> doh :)
[06:57] <Mirv> sil2100: I didn't publish UITK yet, as zoltan thought a QA signoff might be required (I'm not sure if that's normal for UITK, or was it just because of the traincon earlier)
[07:08] <bzoltan> sil2100: Mirv: I am confident that the UITK will not cause regression, but I am happy with an extra QA signoff if you want.
[07:28] <sil2100> bzoltan: I appreciate additional safety anytime!
[07:28] <sil2100> bzoltan: let's wait for davmor2 to appear for a quick dogfooding run
[07:28] <bzoltan> sil2100: OK
[07:46] <sil2100> rsalveti, ogra_, davmor2: I got a note from robru that applications don't start on #188
[07:46] <sil2100> Is that true?
[07:46]  * sil2100 reflashes
[07:48] <jibel> sil2100, any in particular or all the applications? on mako it works fine
[07:48] <Mirv> sil2100: they do start for me, everything seems normal
[07:48] <sil2100> The e-mail mentioned 'any apps', so I would guess all are broken... probably rsalveti knows more details about those
[07:49] <Mirv> I've been using browser, gallery, terminal...
[07:49] <sil2100> Mirv: ACK
[08:08] <ogra_> sil2100, the smoke tests disagree somehow
[08:10] <ogra_> doesnt look liks anything changed for the camera app
[08:10] <ogra_> :/
[08:14] <brendand> ogra_, haven't got that fix in yet. was hoping for tvoss 's branch to land first
[08:14] <tvoss> brendand, testing the silo right now
[08:14] <brendand> ogra_, but i'll submit it today either way
[08:16] <ogra_> hmm
[08:16] <ogra_> mesa pulled in llvm3.4
[08:16] <ogra_> that forced a downgrade of llvm
[08:16] <ogra_> hmpf ... and llvm3.5 was a dep of mir it seems
[08:18] <ogra_> brendand, ah, i thought the upstart change was the only thing needed (apart from setting the property)
[08:19] <sil2100> uh
[08:20] <ogra_> and i cant find where the original 3.5 dep comes from, it came in with the mir landing
[08:21] <brendand> ogra_, camera-app has to be modified to set the property and restart the service (and then reset it when done)
[08:21] <brendand> well not the app, the tests
[08:21] <ogra_> i thought we'd do that in the infrastructure scripts when setting up the test
[08:22] <brendand> ogra_, hmm. if we want to disable the location prompts for all suites, yes we could
[08:22] <brendand> ogra_, is that what you were thinking? i was just focused on fixing camera_app
[08:23] <ogra_> yeah, i guess trust store will get its own functional test anyway ...
[08:23] <ogra_> so we could skip it for apps
[08:24]  * sil2100 thinks 'fudge location prompts'
[08:24]  * sil2100 thinks a bit too loudly
[08:24]  * ogra_ grumbles ... 
[08:24] <Laney> 'the fudge is under your bed'
[08:24] <ogra_> i cant find out why libllvm3.5 came in with image 186
[08:24] <Laney> mesa
[08:25] <ogra_> Laney, oh, was it deliberately downgraded to 3.4 tonight ?
[08:25] <Laney> yes
[08:25] <Laney> 3.5 broke stuff
[08:25] <ogra_> phe, ok
[08:25] <ogra_> thanks !
[08:26]  * ogra_ gets meeting coffee
[08:31] <sil2100> popey, davmor2: meeting happy times
[08:32] <popey> yeah one mo
[08:32] <psivaa> hangout says 'wait'
[08:33] <popey> same
[08:33]  * popey restarts chrome
[08:34] <psivaa> ff worked here
[08:37] <davmor2> Chipaca: No notification on a new image again
[08:37] <Chipaca> davmor2: which new image is that?
[08:38] <Chipaca> my phone just rebooted from the updated image
[08:38] <Chipaca> 188 here
[08:38] <davmor2> Chipaca: this mornings image
[08:38] <tvoss> ogra_, llvm 3.5 caused issues with llvmpipe in mesa, which impacts our ci as some build time tests of apps run under xvfb
[08:38] <Chipaca> davmor2: number?
[08:38] <davmor2> 188, let me double check though
[08:38] <tvoss> ogra_, popey is it known that apps don't start with latest image?
[08:38] <popey> no
[08:38] <popey> they start here
[08:39] <popey> if by latest you mean proposed?
[08:39] <ogra_> tvoss, yes, we know it is your fault :)
[08:39] <ogra_> popey, other arch
[08:39] <tvoss> ogra_, ?
[08:39] <popey> oh
[08:39] <Chipaca> davmor2: i got: 2014/08/14 04:05:06.049686 DEBUG broadcast chan:0 app: topLevel:1949 payloads:[{"ubuntu-touch/devel-proposed/flo":[188,"ubuntu-touch/utopic-proposed"],"ubuntu-touch/devel-proposed/generic":[18
[08:39] <Chipaca> davmor2: in the logs
[08:39] <ogra_> tvoss, adding cgroups support to u-a-l
[08:39] <Chipaca> davmor2: could you zgrep broadcast ~/.cache/upstart/ubuntu-push-client*
[08:40] <ogra_> tvoss, that kind of only works if the kernel actually supports cgroups
[08:40] <tvoss> ogra_, that wasn't me :) and davmor tested it
[08:40] <ogra_> oh, someone mentioned it was you and tde
[08:40] <ogra_> *ted
[08:40] <davmor2> tvoss: tested on mako though
[08:40] <ogra_> (in the landing meeting)
[08:40] <tvoss> davmor2, okay
[08:43] <popey> tvoss: fyi i dont have that device, only nexus 4
[08:44] <tvoss> popey, ack
[08:55] <brendand> tvoss, do you have a silo with the modification of the upstart job for ubuntu-location-service?
[08:56] <tvoss> brendand, not yet, let me get to it after having tested my trust-store silo
[08:56] <brendand> psivaa, first we need this branch to land: https://code.launchpad.net/~thomas-voss/location-service/make-sure-trust-store-restarts-on-location-service-restart/+merge/230599
[08:57] <sil2100> Mirv: I see you're working in public today!
[08:58] <psivaa> brendand: ok, could you let me know when it's landed and then may be we could test your workaround
[08:58] <psivaa> ?
[08:58] <brendand> psivaa, the command you will then need to run is 'sudo setprop custom.location.testing "true"'
[08:58] <brendand> psivaa, followed by 'sudo restart ubuntu-location-service'
[08:59] <brendand> psivaa, and then in the clean up step do the same again but unset the property (or set it to false)
[08:59] <tvoss> brendand, just added the mp to silo 15
[08:59] <psivaa> brendand: ok, but that's after the above branch lands, right?
[09:00] <brendand> psivaa, there's a way to make it work without that branch, but we may as well wait to test the actual solution, just to make sure
[09:00] <Mirv> sil2100: yeah :) and pretty nerdy looking in that with mako attached to one usb port, charger in use, wired headphones, and a USB extra battery charging from the other USB port
[09:01] <psivaa> brendand: ack, do you have vpn setup to the ci lab btw?
[09:01] <popey> davmor2: Chipaca notifications work for me.
[09:01] <brendand> Mirv, you're in Finland though so i think it's not that unusual to be fully wired up :)
[09:02] <Mirv> brendand: hehe :)
[09:02] <brendand> sil2100, well at least the gallery-app issue is reproducible
[09:03] <brendand> sil2100, that's not really good news i know, but...
[09:03] <sil2100> Mirv: hah
[09:03] <sil2100> brendand: yeah, not sure if I should be happy or not!
[09:03] <sil2100> (I guess I should!)
[09:04] <thostr_> can i get a silo for line 28 please?
[09:04] <davmor2> Chipaca: my battery ran out of go go juice in the night but it is on 187 and I got not ping for 188, however I did on my tablets so I'm assuming that it was just the battery but then is it not meant to check once it is powered back on?
[09:04] <davmor2> s/not/no
[09:04] <Mirv> sil2100: FYI post-188 I published Unity8 in the morning. it seems Saviq did a late-nighter and was able to even test it during the night.
[09:05] <Mirv> or robert tried to publish it but I got the packaging ack and really published it
[09:05] <Chipaca> davmor2: it is, yes. Did you do that grep?
[09:06] <Mirv> brendand: like I mentioned, the latest fix in gallery removed some blockingness from loading the main screen, so I'm wondering whether the tests now think that the gallery-app as loaded too early, and executes tests too early too
[09:06] <davmor2> Chipaca: no sorry I'll do that now
[09:08] <Mirv> weird though that some tests pass. but for me everything I try in gallery just seems to work.
[09:08] <davmor2> Chipaca: http://paste.ubuntu.com/8043645/
[09:09] <Chipaca> davmor2: ok. Could you add -A10 to that last zgrep and repeat?
[09:09] <Chipaca> davmor2: the first of those notifications was posted, at least; we'll see if it was presented (and if not, why not)
[09:10] <Chipaca> also, dude, adb shell sucks; phablet-shell ftw
[09:10] <davmor2> Chipaca: http://paste.ubuntu.com/8043656/
[09:10] <davmor2> Chipaca: :P
[09:11] <Chipaca> davmor2: it says there that it presented the notification for 188
[09:11] <Chipaca> davmor2: at 4am
[09:11] <davmor2> Chipaca: too used to adb shell  I should use phablet-shell I just forget about it :)
[09:12] <davmor2> Chipaca: right and then the battery died so when I powered on no notification
[09:12] <Chipaca> davmor2: surely every time you've got to tset for it to get the geometry right is enough to remind you?
[09:12] <brendand> Mirv, if it was something that we worked around before then maybe
[09:12] <Chipaca> davmor2: ah! well, no, notifications don't survive reboots.
[09:14] <davmor2> Chipaca: which kinda sucks hard as you don't get the 1 on the system-settings app or the notification area so unless you open system settings ever day to check for updates you'd never know there was one :(
[09:15] <davmor2> Chipaca: is there a bug for that do you know?
[09:15] <Chipaca> davmor2: I'm not following
[09:16] <Chipaca> davmor2: it's not a bug imho (but then, i didn't understand your issue yet)
[09:17] <davmor2> Chipaca: there should be a way to either store the notifications so that when your device reboots you still know what is going on or a recheck after a reboot I think, other wise you can miss the fact that there are updates on your system, most people don't check the setting app daily to see if there are updates
[09:17] <brendand> Mirv, looks like there is something racy there
[09:18] <Chipaca> davmor2: well... most people don't reboot their device daily either
[09:18] <Chipaca> davmor2: however, for the special case of system updates, the current situation will change and i believe what you want to happen will still happen
[09:18] <Chipaca> system and package updates, realy
[09:18] <Chipaca> -lly*
[09:18] <davmor2> Chipaca: no but they will turn them off for plane, battery dying, important meetings etc
[09:19] <davmor2> woohoo!
[09:20] <Chipaca> davmor2: the designed behaviour is that the update message triggers the download (if you have that enabled), and you only get presented the notification when the download completes
[09:20] <Chipaca> you'd only get the notification directly if you have disabled automatic downloading
[09:21] <Chipaca> davmor2: if we really want notifications to survive reboots, we probably need to fix it in too many places to start now
[09:21] <davmor2> indeed :(
[09:22] <Chipaca> davmor2: and I don't think we do, anyway; at most, we want to provide a hook for apps to check for notifications on boot or somesuch
[09:23] <Chipaca> but there are so many races in that that it needs to be done very carefully by all parts
[09:23] <Chipaca> hands up if you think "done very carefully" is something you can trust appdevs with
[09:28] <Saviq> cihelp hey, seems the slave here http://s-jenkins.ubuntu-ci:8080/computer/ps-utopic-server-amd64-2/? broke again :|
[09:29] <psivaa> Saviq: let me take a look
[09:29] <Saviq> psivaa, thanks
[09:46] <bzoltan> sil2100: Mirv: is there any news about the UITK landing from the SIlo7?
[09:47] <sil2100> bzoltan: I asked davmor2 to perform QA sign-off on silo 007 during the morning meeting
[09:47] <sil2100> davmor2: how's it going?
[09:47] <psivaa> Saviq: that node is now online
[09:48] <Mirv> bzoltan: not yet
[09:49] <ricmm> sil2100: hi, think I could get a silo for line 32 ?
[09:51] <davmor2> bzoltan: installing now give me about an hour
[09:52] <sil2100> ricmm: sure o/
[09:53] <ricmm> thanks
[09:58] <brendand> Mirv, i found an issue
[09:58] <brendand> Mirv, it's not *really* a functional issue, but the user can notice it
[09:58] <brendand> Mirv, launch the app and wait how long it takes for the tab names to appear
[09:59] <brendand> i guess it can be worked around in the test, but i think that's worth a bug
[10:00] <sil2100> Mirv: oh, so  that would explain the 'tab object not found' errors during smoketesting indeed
[10:00] <sil2100> I mean, brendand
[10:01] <sil2100> brendand: is that the cause of all the failures?
[10:01] <brendand> sil2100, not sure
[10:09] <brendand> sil2100, probably all the ones with that error. i'll scan them and see which ones might have a different reason
[10:10] <brendand> sil2100, no there are some with a different reason
[10:14] <bzoltan> davmor2: sweet, thanks
[10:26] <Mirv> brendand: right.. good stuff
[10:26] <Mirv> sil2100: yep
[10:35] <davmor2> popey: can you confirm this one please asked you about it ages ago and just filed it https://bugs.launchpad.net/ubuntu/+source/messaging-app/+bug/1356811  unless I filed it before and just couldn't find it :)
[10:35] <davmor2> all the bugs merge into one
[10:36] <bzoltan> sil2100: I have flashed the image 188 and set a password... now I see the password input, but the OSK does not come up.
[10:36] <bzoltan> Mirv: ^
[10:36] <popey> davmor2: you filed that before
[10:36] <davmor2> popey: I was pretty sure I had I just couldn't find it
[10:37] <popey> nvm, confirmed
[10:37] <davmor2> popey: it can get merged if I have
[10:39] <bzoltan> davmor2: sil2100: I see people are changing stuff what effects the application development story. Would you guys pay attention that the SDK features do not suffer?
[10:41] <sil2100> bzoltan: hm, it works fine here
[10:41] <sil2100> bzoltan: I just set a passphrase and my keyboard appeared during unlocking
[10:42] <bzoltan> sil2100: good for you :) I am flashing again ... does this feature has test coverage?
[10:43] <sil2100> bzoltan: I remember seeing it in unity8 tests, but I'm not entirely sure... but I know it's not using the OSK for keyboard input anyway
[10:43] <sil2100> As we have no coverage for the keyboard AT ALL
[10:43] <bzoltan> sil2100:  is there any other easter egg I need to be worried about?
[10:43] <bzoltan> sil2100: uhh...
[10:43] <sil2100> bzoltan: hah! No idea! ;) Better ask Saviq ;p
[10:44] <Saviq> http://www.bettercallsaul.com/
[10:44] <bzoltan> sil2100: Saviq: for us it is super important, like for _REAL_ that we do not land anything what breaks the app development features.
[10:44] <Saviq> bzoltan, and what did I do?
[10:45] <bzoltan> Saviq: nothing yet :D
[10:46] <Saviq> bzoltan, *if* it was that you can't type your password in, I'd say that's more than a development feature that's broken :P
[10:46] <bzoltan> Saviq: I am reflashing the 188 and see if I got lucky this time and I can type in my password
[10:48] <tvoss> bzoltan, is there a manual test plan that covers all app-development specific features?
[10:48] <tvoss> bzoltan, obviously, an automatic one would surely be appreciated, too :)
[10:50] <davmor2> bzoltan, sil2100: Silo 007 looks good here on mako
[10:50] <psivaa> sil2100: the testing on 188, is now complete with uitk rerun having 12 failures( it had 16 in the first run)
[10:50] <sil2100> psivaa: ouch :o
[10:51] <bzoltan> davmor2:  thank you
[10:51] <sil2100> davmor2: thanks!
[10:51] <sil2100> bzoltan: ok, so I'll publish UITK in a minute then ;)
[10:51] <bzoltan> tvoss: we have
[10:51] <sil2100> bzoltan: btw. we might later poke you about some UITK test failures, probably caused by some other landing... ;/
[10:52] <sil2100> (or we just got very unlucky)
[10:52] <bzoltan> sil2100:  whut! tell me now:)
[10:53] <bzoltan> tvoss: this is the manual tests I run before releases -> https://docs.google.com/a/canonical.com/document/d/1D7J8TgxqDBpuilE8z1EGtUF4OK_kGQm39DodzVbbOKY/edit#heading=h.p8k2gcui1js8
[10:54] <Mirv> bzoltan: do you mean the PIN code screen lock? it works for me.
[10:54] <Mirv> there's the own keypad for it
[10:54] <bzoltan> tvoss: and we have autopilot tests to cover the most critical (chroot and emulator creation) parts.. I will make app deployment tests too soon.
[10:55] <bzoltan> Mirv:  it was a one time problem... i had password lock and the keyboard did not show up
[10:55] <Mirv> right
[10:58] <bzoltan> tvoss: but just simple open the Ubunu SDK, plug in the device, create a simple app, select the device as target and hit the Run... if the app shows up then it is fine. Should not take longer than a minute... given that you have set up armhf chroot (the SDK does it for you on the first run)
[11:00] <bzoltan> Saviq: why the lock security setting is hidden in "About this phone" -> "Developer mode" -> Lock security?
[11:00] <Saviq> bzoltan, it's not, it's in Security and Privacy, too
[11:00] <sil2100> bzoltan: sooo! We noticed some tests failing on 188 - this is a re-run already but we still had 12 failures
[11:00] <bzoltan> Saviq:  ohh redundancy :) what a great invention ...: D
[11:00] <sil2100> bzoltan: http://ci.ubuntu.com/smokeng/utopic/touch/mako/188:20140814:20140811.1/9648/ubuntuuitoolkit/
[11:00] <Saviq> bzoltan, it's there in Developer mode because we only allow enabling dev mode *if* you have a pass set
[11:01] <sil2100> bzoltan: it had 16 failures on the first run ;)
[11:02] <bzoltan> sil2100:  so who broke the SDK in the meantime? I have tested on 187 and it was all OK
[11:03] <Saviq> bzoltan, http://people.canonical.com/~ogra/touch-image-stats/188.changes
[11:03] <sil2100> bzoltan: no idea yet! In the past we saw such big number of failures already, but it was happening once per like 20-30 runs
[11:03] <sil2100> bzoltan: this time it happened again after a re-run, so we're like hmm, not sure what it's about
[11:04] <sil2100> bzoltan: might be the same here, as I said we might just have been very unlucky and hit it twice
[11:04] <bzoltan> sil2100:  RuntimeError: Application Launch Failed: Application failed to start.
[11:13] <popey> hmm, in #188 the OSK keeps coming up sideways for me
[11:18] <Saviq> psivaa, it looks like we have a publishing problem https://jenkins.qa.ubuntu.com/job/unity-phablet-qmluitests-utopic/900/console
[11:18] <Saviq> psivaa, or something...
[11:18] <Saviq> right, not a publishing one of course, because that's published...
[11:19] <bzoltan> sil2100:  I am running and rerunning the failing tests and they are OK sometimes...all failures are "RuntimeError: Application Launch Failed: Application failed to start"
[11:24] <psivaa> Saviq: that issue is i think because of the addition of archiving results, that we did yesterday.. not sure why but that's where jenkins is throwing the exception
[11:24] <sil2100> bzoltan: thanks! We'll also try looking into the reasons for those ourselves
[11:24] <Saviq> psivaa, ;( seems to have worked a few times http://s-jenkins.ubuntu-ci:8080/job/unity-phablet-qmluitests-utopic/895/ :|
[11:25] <Saviq> until this morning
[11:26] <psivaa> Saviq: yea, this time it could be transient. i'm rebuilding one without archiving and if it succeeds will rebuild with it again
[11:27] <Saviq> psivaa, ok there's one about to complete here too http://s-jenkins.ubuntu-ci:8080/job/unity-phablet-qmluitests-utopic/901/console
[11:28] <davmor2> ogra_: who is responsible for the security lock pages?
[11:28] <psivaa> Saviq: yep, watching that too
[11:28] <ogra_> davmor2, mterry
[11:28] <davmor2> ogra_: thanks
[11:29] <davmor2> ogra_: there is a small bug,  the set button is clickable before the the confirmation code/password is input.
[11:36] <popey> davmor2: app store is showing me [install] button for apps i already have installed, you getting that?
[11:36] <davmor2> popey: I had that ages ago and you couldn't reproduce it ;) let me check
[11:40] <popey> davmor2: bug 1356837 for you if you reproduce
[11:40] <davmor2> popey: right I think this is a refresh issue.  If I search for 2048Native, install it hit the back button to the search results and then click on it again I see the install button.  If I keep hitting back till I get to the store and it shows as installed then click on it I then get unistall open as options
[11:40] <davmor2> popey: I think it is that the original search is cached so is still showing the app as uninstalled
[11:41] <popey> right
[11:41] <popey> which is wrong.
[11:41] <popey> As aq says, the two hardest things in computing. 1) Naming things. 2) caching.
[11:44] <popey> Ok, anything else?
[11:44] <davmor2> popey: anything I have installed prior to opening the scope shows as unistall/open though
[11:44] <popey> ^ wrong channel
[11:45] <psivaa> Saviq: 901 did not see that exception
[11:45] <Saviq> psivaa, ok, maybe I was too fast, thanks
[11:46] <psivaa> Saviq: np
[11:51] <tvoss> trainguards, could someone hit publish for 14?
[12:06] <Mirv> tvoss: sure
[12:06] <tvoss> Mirv, thanks
[12:07] <Mirv> tvoss: not approved...
[12:08] <Mirv> this process should maybe warn about it at least verbally at build time already :)
[12:08] <tvoss> Mirv, hmmm, weird .. let me see
[12:09] <Mirv> tvoss: https://code.launchpad.net/~thomas-voss/trust-store/add-reporting-for-cached-agent/+merge/230470 + https://code.launchpad.net/~thomas-voss/trust-store/add-preseed-support-executable/+merge/230496
[12:10] <tvoss> Mirv, yup
[12:19] <nik90> sil2100, bzoltan: It seems silo-007 has been given QA-sign off. Are we waiting on something else before publishing it?
[12:19] <Mirv> nik90: we're waiting for "davmor2 sign-off"
[12:19] <nik90> Mirv: he already did sign off
[12:19] <Mirv> unless of course that's already there, I just thought I'd see it on IRC too
[12:19] <Mirv> nik90: right..
[12:20] <Mirv> oh, now I see, it was just in the middle of these other discussions
[12:20] <davmor2> Mirv: see spreadsheet it is signed off about an hour ago :P
[12:20] <nik90> np, might check with bzoltan to be sure. It is his landing in case he is holding it for a reason
[12:20] <Mirv> I think sil2100 just forgot about it, since he said he'll publish it "in a minute" :)
[12:20] <Mirv> it's not anymore in zoltan's hands as such
[12:20] <nik90> ah ok
[12:21] <nik90> then go go go :D
[12:21]  * Mirv goes goes goes
[12:38] <Saviq> psivaa, is it expected that the two nodes are launching for a few minutes now http://s-jenkins.ubuntu-ci:8080/label/utopic&&amd64/? ?
[12:47] <Ursinha> Saviq: I don't think so. Last week same thing happened, I believe one vm had to be restored and the other one tweaked
[12:47] <Ursinha> I can have a look
[12:50] <Saviq> thanks
[12:57] <sil2100> So!
[12:58] <sil2100> I went to lunch before publishing, since we want to build an image after landing UITK
[12:58] <sil2100> And we want one more fix in
[13:06] <psivaa> Saviq: sorry went afk for lunch
[13:07] <psivaa> Ursinha: thanks for looking :)
[13:07] <Saviq> psivaa, nw, you're not on vanguard :)
[13:07] <Saviq> psivaa, I just hold on to you once I caught you ;)(
[13:07] <psivaa> :), that's alright
[13:18] <ogra_> sil2100, did you plan an image during the day ?
[13:18] <sil2100> ogra_: yes
[13:18] <ogra_> can you wait til dbus-property-service is in ?
[13:19] <sil2100> ogra_: waiting for UITK to migrate first
[13:19] <ogra_> cool
[13:19] <sil2100> ogra_: sure, UITK will take a while anyway :)
[13:19]  * ogra_ bets he is faster :) 
[13:19] <ogra_> yeah
[13:32] <popey> ogra_: when's next image being built? do we have one planned?
[13:33] <ogra_> popey, see above :)
[13:33] <ogra_> soon, but waiting fo UITK
[13:33] <popey> haha
[13:33] <popey> ok
[13:33]  * popey quickly approves 10 clicks into the store
[13:33] <ogra_> :)
[13:34] <sil2100> Noooo

[13:34] <popey> Mwuhahahaha
[13:35] <thostr_> can I get a silo for line 33?
[13:36] <sil2100> thostr_: o/
[13:36] <thostr_> sil2100: thanks
[13:36] <sil2100> hmmm, actually...
[13:36] <sil2100> Where's queuebot?!
[13:36] <sil2100> :O
[13:37]  * sil2100 ressurrects CI Train bot then
[13:37] <sil2100> No wonder I didn't see any pings today
[13:37] <sil2100> Two bots and both down
[13:54] <ogra_> must be a bank holiday in botland then :)
[13:59] <davmor2> sil2100: can you add this to the known issues list please https://bugs.launchpad.net/webapps-core/+bug/1356417
[14:00] <davmor2> sil2100: it's made more serious by not being able to recover facebook, which of itself is possibly a good thing see what people post on it, but from an end user point of view not so good ;)
[14:01] <sil2100> ;p
[14:01] <sil2100> davmor2: hm, is that enough serious to be a blocker?
[14:01] <sil2100> Or just a visible issue?
[14:02] <davmor2> sil2100: visible for now we can always uprate it latter.
[14:03] <davmor2> sil2100: I blame oSoMoN for doing such a good job on the browser memory :)
[14:04] <rsalveti> ogra_: sil2100: sorry, I should have said which device I tested latest image with :-)
[14:04] <ogra_> heh, well, all solved now
[14:04] <rsalveti> that was a mystery for you guys to solve :P
[14:05] <rsalveti> but I got a bunch of denied when testing kr yesterday
[14:05] <ogra_> yeah
[14:13] <fginther> Saviq, problem with the utopic VMs has been resolved, they are all working again
[14:19] <sil2100> \o/
[14:19] <sil2100> ogra_: building a new image, UITK in the archive
[14:19] <ogra_> +1
[14:19] <sil2100> ogra_: are you ready with your things as well?
[14:20] <ogra_> yep
[14:23] <Saviq> fginther, thanks, great
[14:27] <ralsina> ogra_: can I get a publish in silo 2? Did-rocks needs to push the button after it gets to the NEW queue and he's in a hurry :-)
[14:28] <sil2100> ralsina: I guess ogra_ has no power over CI Train ;)
[14:28] <sil2100> ralsina: let me do that
[14:28] <ogra_> :)
[14:28] <ralsina> right, I got the wrong nicj
[14:28] <sil2100> Sorry for that, without the bots around we're a bit blind, as we got used to being poked on IRC
[14:28] <ralsina> I keep confusing ogra and rob-ru in my memory :-)
[14:29] <ralsina> sil2100: no problem at all, thanks!
[14:29] <imgbot> [14:30] <popey> woop woop
[14:30] <davmor2> and popey turns into zoidberg again
[14:31] <nik90> sil2100: is the uitk in image 189?
[14:31] <nik90> sil2100: nvr mind..just read backlog
[14:31] <nik90> :)
[14:31] <sil2100> nik90: yes ;)
[14:31] <popey> (\/) (°,,,°) (\/)
[14:31] <popey> woop woop woop woop
[14:31] <davmor2> popey: hahahaahahaha
[14:31] <sil2100> Crab people?!
[14:33] <ogra_> lol
[14:37] <dobey> is jenkins ok? seems exceptionally slower than normal lately
[14:41] <brendand> sil2100, https://bugs.launchpad.net/gallery-app/+bug/1356841. get this on your radar
[14:41] <sil2100> Uh oh!
[14:41] <sil2100> brendand: thanks!
[14:42] <brendand> sil2100, i'm still looking at the other failures - they seem less straightforward
[14:43] <thostr_> sil2100: could you reconfig silo 10
[14:43] <sil2100> thostr_: sure
[14:45] <ogra_> yay, the bot is back
[14:46] <sil2100> Yeah, poked stgraber about that
[14:46] <sil2100> It seems it died for unknown reasons
[14:47] <ogra_> it had a meeting with the other bots
[15:04] <brendand> tvoss, do you want me to help land silo15?
[15:08] <jgdx> cihelp: tried to abort http://s-jenkins.ubuntu-ci:8080/job/ubuntu-system-settings-ci/1231/ and it seems to be hanging
[15:08] <jgdx> thanks
[15:09] <Ursinha> jgdx: looking
[15:11] <jgdx> Ursinha, thank yuo
[15:16] <brendand> psivaa, did you try that location-service code?
[15:16] <psivaa> brendand: has that MP landed?
[15:17] <brendand> psivaa, no - but there's a way to do it before the mp lands
[15:17] <brendand> psivaa, you just have to 'restart ubuntu-location-trust-stored' as well
[15:17] <psivaa> brendand: ohh, i thought you were suggesting to wait till it lands
[15:18] <brendand> psivaa, is it a long process to change things in jenkins?
[15:18] <brendand> psivaa, or just a simple mp?
[15:23] <tvoss> brendand, in a few, checking something right now
[15:37] <elopio> ping Ursinha: I'm waiting on ubuntu-experience-tests to go into the archive. But it has been synching for a long time now: https://launchpad.net/ubuntu/utopic/+queue
[15:37] <elopio> Do you know if that's normal?
[15:37] <brendand> psivaa, here's the code you need: http://paste.ubuntu.com/8046270/
[15:37] <Ursinha> elopio: I'll have a look, trying to fix another stuck job and will get to it
[15:38] <elopio> thanks Ursinha.
[15:39] <nik90> ogra_: imgbot has quit...what did you say :P
[15:40] <ogra_> it missed the meeting of the other bots ... and now it is depressed
[15:40] <Ursinha> jgdx: so, it seems that it aborted successfully but because there is another build job of lower number yet to complete, that job has to wait to finish
[15:41] <davmor2> nik90: today is the national bot day say they are all on holiday
[15:42] <sil2100> ;p
[15:42] <sil2100> Oh!
[15:43] <sil2100> We have a national holiday tomorrow, so maybe it's related?
[15:43] <davmor2> sil2100bot
[15:45] <sil2100> MacSlow: hey! How's progress on LP: #1354406 ?
[15:46] <nik90> hehe
[15:46] <nik90> davmor2: feel free to fire the bot when it returns :P
[15:49] <psivaa> brendand: yea, flashing  a device to run the tests.
[15:54] <MacSlow> sil2100, got it figured out... working on fix and test
[15:55] <MacSlow> sil2100, should have an MP by tomorrow... will chase you for a review then :)
[15:55] <sil2100> MacSlow: thanks!
[15:55] <sil2100> :)
[15:59] <imgbot> [15:59] <imgbot> [16:00] <popey> \o/
[16:07] <jgdx> Ursinha, ack, thanks
[16:10] <oSoMoN> are there known issues with otto today?
[16:24] <bfiller> robru: need a silo for line 37 please
[16:26] <robru> bfiller, ok you got 2
[16:26] <bfiller> robru: thank you sir!
[16:27] <robru> bfiller, you're welcome!
[16:27] <sil2100> robru: hey! You ready for some RTM-breakage? :)
[16:27] <robru> sil2100, yes!
[16:27] <brendand> psivaa, i'll do the same in camera-app instead
[16:27] <brendand> psivaa, so don't worry about it now
[16:28] <psivaa> brendand: ok, thx
[16:48] <davmor2> nik90: so where is my updated clock already?
[16:48] <nik90> davmor2: we are discussing that at the meeting atm :)
[16:48]  * davmor2 switches the impatience game on nik90 :D
[16:53] <robru> sil2100, ^ you breaking stuff?
[16:54] <sil2100> Yeah ;)
[16:54] <sil2100> I moved a column and suddenly all built packages were 'tested: yes'
[16:54] <robru> sil2100, hmm that's strange, I guess queuebot read the ready column as the tested column.
[16:54] <bfiller> robru: just marked silo 4 as ready for publish, but the spreadsheet seems busted. it's not showing the silo and only showing "yes#" as choice in dropdown
[16:54] <robru> sil2100, that shouldn't have happened, my code doesn't hard-code column headers!
[16:54] <sil2100> Uh oh!
[16:55]  * sil2100 slaps queuebot around a bit with a large trout
[16:55] <elopio> sil2100, robru, plars: would it be useful to split the ubuntu-experience-tests into a sanity check to run before the 900 tests, and an integration suite to run after?
[16:55] <elopio> or is it better to just run everything in parallel at the same time?
[16:55] <robru> bfiller, what row is it?
[16:55] <bfiller> robru: 15
[16:57] <sil2100> bfiller: we're now changing stuff
[16:58] <sil2100> bfiller: I will announce the changes that have been made once they're made
[16:58] <sil2100> bfiller: now besides setting to Yes you'll also have to put in which image number you have tested it against (that's the #)
[16:58] <bfiller> sil2100: ok thanks
[16:59] <sil2100> Right now we're officially breaking the train
[16:59]  * sil2100 merges the RTM branch
[16:59] <bfiller> sil2100: can silo 4 land first?
[16:59] <popey> sil2100: got 5 mins to join a hangout?
[16:59] <sil2100> popey: sure
[16:59] <sil2100> bfiller: ok, robru can you help out with publishing it? ^
[16:59] <popey> sil2100: https://plus.google.com/hangouts/_/gwknhvhj3egtbnz4ir2fnssci4a
[16:59] <sil2100> We can wait for it to publish
[17:00] <robru> sil2100, ok it seems the dashboard is coping with the changes, just that one row lost it's requestid and silo name for whatever reason
[17:00] <robru> yeah
[17:00] <plars> elopio: which do you mean?
[17:00] <bfiller> thanks guys
[17:00] <elopio> plars: I will write one that launches all the installed apps.
[17:03] <robru> infinity, you around for a packaging ack? https://ci-train.ubuntu.com/job/landing-004-2-publish/69/
[17:08] <balloons> fginther, when you get a moment, let's talk about running qml tests for the community core apps during merges
[17:09] <Ursinha> elopio: what happens in that case is the package seems to not exist in ubuntu
[17:09] <Ursinha> elopio: it's waiting on NEW to be approved
[17:11] <Ursinha> elopio: the "sync" information there shows where the package came from, not the status
[17:11] <Ursinha> elopio: the status is NEW waiting for an archive admin to approve that
[17:15] <elopio> jdstrand or seb128, maybe you are around to approve ubuntu-experience-tests: https://launchpad.net/ubuntu/utopic/+queue
[17:20] <infinity> robru: Ish.  Looking.
[17:24] <robru> sil2100, hey what's the scoop? are we renaming those jenkins jobs or what?
[17:24] <robru> sil2100, also, why don't we get deploy citrain to do that? ;-)
[17:25] <infinity> robru: Looks reasonable.
[17:25] <robru> infinity, thanks!
[17:27] <tvoss> robru, mind publishing 14?
[17:28] <robru> tvoss, sure
[17:28] <robru> infinity, https://ci-train.ubuntu.com/job/landing-014-2-publish/lastSuccessfulBuild/artifact/packaging_changes_trust-store_1.0.0+14.10.20140814.1-0ubuntu1.diff ;-)
[17:29] <seb128> elopio, I can have a look tomorrow
[17:30] <elopio> seb128: thanks.
[17:30] <infinity> robru: Also looks sane.
[17:30] <robru> infinity, thanks!
[17:33] <sil2100> robru: yeah, we could, but deploy-citrain would leave the old jobs laying around
[17:33] <sil2100> robru: and temporarily hacking it would be laaame
[17:33] <sil2100> Real men do it by HAND
[17:33] <robru> sil2100, don't you think just deleting the old jobs by hand would be easier than renaming the old jobs by hand?
[17:34] <ogra_> sil2100, one handed even !
[17:34] <robru> sil2100, I mean we need to update deploy-citrain to not recreate the old names anyway, right?
[17:34] <sil2100> Once I switch the backend everything will be in place
[17:36] <robru> sil2100, ok, let me know what you want me to do...
[17:36] <robru> sil2100, oh, I see there already are ubuntu-landing-* jobs...
[17:37] <robru> sil2100, is it too early to switch the dashboard to use those?
[17:37] <sil2100> Right, those got created when I was creating the ubuntu-rtm ones! We could use those, but hmmm... we'll loose all history of the old jobs
[17:37] <robru> oh right, the history is nice
[17:37] <sil2100> robru: switch to the new jobs maybe
[17:37] <sil2100> Let's keep the old ones for a while to have the history
[17:38] <sil2100> (we'll have a bloated jenkins but only for the transitional period)
[17:38] <sil2100> In case we want to revert or something ;)
[17:39] <sil2100> Ooook so the backend is switched, the spreadsheet is switched, hmmm
[17:39] <sil2100> Ah
[17:39] <robru> ok, I'm just about ready to switch the dashboard
[17:40] <sil2100> Do the switch!
[17:40] <sil2100> I'll try to fight jenkins, since it got a permission error on redeploying the prepare-silo job
[17:40] <robru> sil2100, ok!
[17:43] <Saviq> robru, icanhassilo for line 38 please?
[17:43] <robru> Saviq, one sec
[17:43] <sil2100> Saviq: CI Train out of order right now!
[17:43] <Saviq> ohnoes!
[17:44] <sil2100> Saviq: check the topic!
[17:44] <Saviq> sil2100, it's too long, wraps ;P
[17:44] <sil2100> http://www.funnyjunk.com/funny_gifs/2473414/All
[17:44] <sil2100>  ;p
[17:44] <Saviq> sil2100, robru, whenever you're ready, I can be a guinea pig if needed
[17:45] <robru> sil2100, ok, I pushed a new dashboard, it has links for switching from ubuntu to ubuntu-rtm. the PPA and jenkins links work, but the only thing is it doesn't get the new statuses from the rtm json yet
[17:45] <sil2100> robru: ok, I'm working on the prepare jobs right now
[17:45] <robru> sil2100, are you going to make my dreams come true by creating http://people.canonical.com/~platform/citrain/ubuntu/ ?
[17:46] <sil2100> It should be created!
[17:46] <sil2100> Uh oh!
[17:46] <robru> i don't see!
[17:46] <sil2100> :<
[17:46]  * sil2100 checks what went wrongz
[17:48] <sil2100> Ah
[17:49] <sil2100> So it seems we first need to have some projects that use the new silo naming scheme
[17:49] <robru> sil2100, what?
[17:50] <sil2100> Give me a moment, need to think about it... I might have to just modify that directly from jenkins
[17:50] <robru> sil2100, if I dig in there and copy the files by hand, will they be created in the right place from now on?
[17:50] <sil2100> As all the silos are configured now for landing-xxx, not ubuntu/landing-xxx, we need to tweak the json configs
[17:50] <sil2100> Let me do that
[17:51] <robru> sil2100, i don't understand what you mean, what configured?
[17:51] <sil2100> http://people.canonical.com/~platform/citrain/landing-001
[17:51] <sil2100> This has "siloname": "landing-001"
[17:51] <sil2100> We need it to be "siloname": "ubuntu/landing-001" now
[17:51] <sil2100> So I need to tweak it on ci-train.ubuntu.com
[17:52] <robru> sil2100, ah right
[17:54] <robru> sil2100, ok, I've got a branch that can correctly load json from /~platform/citrain/ubuntu{,-rtm}/landing-XXX depending on what distro it's told to look at, but I won't push it yet because there are no RTM silos, and the /ubuntu/ ones just 404, so even though it's "workng", it never shows any status ;-)
[17:55] <robru> sil2100, http://people.canonical.com/~rbpark/citrain/index.html#?distro=ubuntu-rtm here's my staging if you want to play with it. check the web console, it tells what it's doing
[17:58] <robru> sil2100, is it safe to assign a silo?
[17:58] <sil2100> uuh
[17:58] <sil2100> Maybe not yet
[17:58] <sil2100> The worst thing is that we have no direct access to this machine...
[18:05] <robru> sil2100, what machine? I can ssh into people and move around the json files, all you have to do is change the script so that the future files are created in the right place
[18:05] <sil2100> It's not on people, it's on ci-train.ubuntu.com which is on prodstack :|
[18:06] <sil2100> So we have no access, only through jenkins or IS
[18:06] <robru> sil2100, but I want http://people.canonical.com/~platform/citrain/landing-* moved, I can move those
[18:06] <sil2100> Don't, they're invalid right now anyway
[18:07] <sil2100> They should be correctly synced up automatically once I fix it on the backend
[18:08] <sil2100> It takes time as it's like really irritating, like blind work...
[18:10] <cyphermox> eh, what?
[18:10] <robru> cyphermox, we're overhauling citrain live for RTM stuff
[18:11] <robru> sil2100, uh you broke the spreadsheet
[18:11] <robru> sil2100, how did all the request IDs disappear?
[18:13] <sil2100> robru: will fix that in a moment
[18:14] <robru> if only we could disable queuebot
[18:14]  * sil2100 curses prodstack
[18:15] <robru> sil2100, oh, the directory looks nice now
[18:15] <sil2100> Not yet
[18:15] <sil2100> Still working on it ;p
[18:15] <robru> sil2100, i put my dashboard in production, it's working on my end ;-)
[18:17] <robru> sil2100, do you want me to copy the requestids back into the spreadsheet? or will they just be cleared again?
[18:17] <sil2100> They'll be cleared again for now
[18:20] <robru> sil2100, ok well ping me if you want me to help with anything
[18:20] <sil2100> Sure :8 Kill prodstack for meeeh!
[18:20]  * robru kills prodstack. paf paf!
[18:24] <sil2100> robru: ok, could you help with re-entering the silo-names and UIDs to the spreadsheet?
[18:24] <robru> sil2100, yes
[18:24] <sil2100> robru: remember to include the silo name from the silo config, i.e. ubuntu/landing-001
[18:24] <sil2100> WIth the distro prefix like that
[18:25] <robru> ahhh
[18:25] <sil2100> robru: or maybe wait one moment!
[18:26] <robru> yeah my dashboard broke. heh ;-)
[18:30] <robru> sil2100, ok dashboard is fixed. ready for me?
[18:30] <sergiusens> robru: sil2100 that new landing mention I just had isn't real, right?
[18:30] <sil2100> Not yet!
[18:30]  * sergiusens hasn't touched the spreadsheet
[18:30] <robru> sergiusens, nothing is real! nothing is real!
[18:31] <sergiusens> good :-)
[18:31] <sil2100> sergiusens:  no no, we're breaking the spreadsheet and *everything*
[18:31] <sergiusens> not my imagination :-)
[18:42] <sil2100> Ok
[18:42] <sil2100> robru: could you now re-enter teh numberz?
[18:42] <robru> sil2100, ok
[18:42] <sil2100> I don't guarantee it'll stay ;p
[18:45] <robru> sil2100, yep, nope, it's deleting stuff.
[18:45] <sil2100> HOW DARE YOU
[18:46] <sil2100> Ok, let me debug the script :|
[18:52] <Saviq> fginther, hey, we seem to be getting failures like http://s-jenkins.ubuntu-ci:8080/job/unity-phablet-qmluitests-utopic/911/console from time to time, seems to be related to uploading the artifacts, any idea?
[18:55] <fginther> Saviq, it's a jenkins bug, archiving of artifacts can get stuck when the slave node is connected via ssh (which we have to use for the VMs).
[18:55] <robru> sil2100, any ETA on a fix? it's about lunchtime for me, but I don't want to leave if you'll be done in under 20min...
[18:55] <fginther> Saviq, the solution is to not store the artifacts, but it's possible to harvest the test results w/o storing the artifaccs
[18:56] <Saviq> fginther, ok yeah, let's not store them then (and let's move out of VMs asap)
[18:56] <sil2100> robru: no idea! The spreadsheet seems to hate me, trying to find out where it's actually unassigning the silo
[18:56] <sil2100> Debugging right now
[18:57] <fginther> Saviq, I've removed the saving of the artifacts, test collection is still there though. Lets revisit this after a test run to see if it broke worse
[18:57] <Saviq> fginther, thanks
[18:57] <fginther> me grammar bad
[18:58] <robru> sil2100, yeah ok, i'll be back after a short lunch then ;-)
[18:59] <sil2100> Ok, found where it's removed, but this part of code is actually the most 'strange'
[18:59] <sil2100> Never really understood what Didier had in mind there ;)
[19:05] <ToyKeeper> I don't suppose there's any chance of making queuebot speak in normal channel text instead of channel notices?  Ironically, I can't get my irc client to notify me on notices.
[19:07] <davmor2> ToyKeeper: just add queuebot/#ubuntu-ci-eng- as a nick you care about
[19:07] <ToyKeeper> Then it'd ping me constantly, no?
[19:08] <davmor2> ToyKeeper: no pleasing some people ;)
[19:10] <sil2100> robru: yay it works! (I think)
[19:11] <rsalveti> hm, just noticed the spreadsheet looks broken
[19:11] <rsalveti> guess that is the known issue described by the channel topic
[19:13] <rsalveti> sil2100: does that mean we cannot allocate silos until this is fixed?
[19:13] <sil2100> rsalveti: yeah, well... we're doing a switch right now
[19:13] <rsalveti> sil2100: switch? :-)
[19:14] <sil2100> rsalveti: switching to RTM-enabled CI Train ;) Expect everything to be b0rken anyway!
[19:15] <sil2100> (there will be an annoucement later when we're done)
[19:15] <rsalveti> sil2100: oh, right, but we'll only switch to rtm next week, right?
[19:15] <rsalveti> officially
[19:15] <rsalveti> guess we're first enabling the train to be rtm compatible
[19:15] <sil2100> Yeah, but the train will be now able to drive it on production
[19:15] <ogra_> yeah, on the 19th is ETA iirc
[19:15] <rsalveti> cool
[19:15] <sil2100> Since we did test drives on preprod, but now we want all to be ready
[19:15] <rsalveti> sil2100: when are we going to be able to requests silos again?
[19:16] <sil2100> rsalveti: I think we should be done in like 30 minutes (if no additional problems appear)
[19:16] <rsalveti> oh, cool
[19:17] <sil2100> rsalveti: which silo you wanted to have assigned?
[19:18] <sil2100> I mean, for which landing?
[19:18] <sil2100> I want to test silo assignment
[19:19] <rsalveti> sil2100: mine is not yet ready, but you can start with AlbertA's one
[19:19] <rsalveti> sil2100: line 39
[19:19] <sil2100> Ok, no guarantee it will assign correctly though!
[19:19] <rsalveti> sil2100: no worries
[19:21] <boiko> robru: sil2100: I am trying to merge & clean silo 004, but it is failing: https://ci-train.ubuntu.com/job/landing-004-3-merge-clean/44/console
[19:21] <sil2100> boiko: hey, wait a minute
[19:21] <sil2100> The job links have changed
[19:22] <sil2100> boiko: for now maybe not use the train :)
[19:23] <sil2100> Ok, silo assignment *seems* to work
[19:26] <robru> sil2100, the computed status doesn't seem to work right. there's a long delay (like 30s) after the silo name is entered, it normally appears immediately
[19:27] <sil2100> hm, seems correct here?
[19:27] <sil2100> I mean, looking at the spreadsheet at least
[19:27] <robru> sil2100, yeah, it gets the correct value 30s after I expect it to
[19:28] <robru> sil2100, it used to be that the very second I entered a silo name, the computed status would spring into life. try filling out one of the silo statuses. huge delay before computed status updates
[19:28] <sil2100> The timing shouldn't have changed, but maybe it takes a bit longer now because we have two directories to poll?
[19:28] <robru> sil2100, no, computed status field shouldn't be polling anything, it just checks the assigned/tested state and fills in a cell.
[19:29] <sil2100> I didn't change that part
[19:29] <robru> sil2100, column N
[19:29] <robru> bah
[19:29] <sil2100> Nothing from the 'onSpreadsheetChanged' part has been changed
[19:29] <alesage> fginther tedg arriving with a question about symbol checking for this build https://jenkins.qa.ubuntu.com/job/url-dispatcher-utopic-amd64-ci/29/consoleFull
[19:30] <alesage> fginther I've seen this only once or twice, tedg suggesting we disable dpkg-gensymbol checking by setting the check level to zero, e.g.
[19:30] <alesage> fginther, do you recall our ever fiddling with that var before?
[19:30] <robru> sil2100, but you changed the formula in column N
[19:30] <sil2100> Ah! Right, it's probably slower now
[19:31] <robru> sil2100, yeah, ISNUMBER, is that really necessary? seems really slow
[19:32] <sil2100> robru: yeah... otherwise it will barf-out...
[19:33] <fginther> alesage, I have never seen that before, is there a proper fix for it? The gensymbols check has always been something we need to do, otherwise omissions slip through
[19:34] <alesage> fginther yes it's a pickle, wondering if it's possible to ignore just those symbols
[19:35] <fginther> alesage, why doesn't this happen for all builds?
[19:36] <alesage> fginther, I assume b/c tedg is insisting on strict checking in his debian/rules, check level 4
[19:36] <fginther> alesage, i wonder if the gensymbols level is set to 0 in the packaging itself.
[19:36] <fginther> ahh
[19:36]  * alesage goes to verify
[19:37] <alesage> fginther yep; I don't know what the diff levels mean
[19:39] <alesage> fginther, for the record this kind-of goes away in future system when 'coverage' build is separate, have already treated in this mangling project
[19:40] <sil2100> robru: btw. is the dashboard changed now? :)
[19:40] <sil2100> robru: does it point to the right jenkins jobs?
[19:41] <robru> sil2100, yes, it points at the right jenkins jobs and PPAs and even json data. just hooking up the spreadsheet now
[19:41] <sil2100> \o/
[19:41] <sil2100> robru: silo assignment succeeded
[19:42] <sil2100> robru: are you re-adding the UIDs and silo assignments now?
[19:43] <robru> sil2100, I will once I fix the dashboard so I can confirm I'm doing it correctly
[19:44] <fginther> alesage, it looks like we have an existing pbuilderjenkins hook for disabling gensymbols
[19:44] <fginther> alesage, the best suggestion I have is to try that
[19:45] <tvoss> sil2100, I'm confused: https://ci-train.ubuntu.com/job/landing-014-3-merge-clean/53/console
[19:45] <sil2100> tvoss: see topic
[19:45] <fginther> alesage, I'm trying a test here: http://s-jenkins.ubuntu-ci:8080/job/url-dispatcher-utopic-amd64-ci/30/
[19:45] <sil2100> tvoss: we should be done soon, the jenkins jobs will change
[19:45] <tvoss> sil2100, ack
[19:48] <alesage> fginther, this helps thanks--that was tedg's suggestion after all
[19:55] <robru> sil2100, can you check silo 4 (line 15)? it seems the json and the spreadsheet have inconsistent statuses, i guess the spreadsheet isn't getting updated from the backend anymore?
[19:55] <robru> or maybe it just needs a minute to sync...
[19:56] <robru> sil2100, nm, good now
[20:00] <kgunn_> robru, something funny going on for me.... row 39 says silo12 ready...but dashboard shows empty
[20:00] <robru> kgunn_, yep, we're working on it
[20:00] <kgunn_> ack
[20:00] <kgunn_> AlbertA2, ^
[20:00] <robru> kgunn_, well, reload the page
[20:03] <Ursinha> hello, trainguards (I presume), do I need to have special permissions to add a landing request to the spreadsheet?
[20:03] <sil2100> Ursinha: yes, we need to add you to the people that have edit rights
[20:03] <sil2100> Let me do that in a moment
[20:04] <Ursinha> sil2100: thanks :)
[20:05] <popey> #198 has some odd rendering issues
[20:05] <robru> sil2100, ok it looks like I've restored the spreadsheet, and everything looks good in the dashboard
[20:05] <popey> *189
[20:05] <sil2100> \o/
[20:05] <sil2100> robru: thanks! :)
[20:05] <robru> sil2100, you're welcome
[20:07] <robru> sil2100, OH MY GOD... what... CAN IT BE???
[20:07] <sil2100> WHAT?! :O
[20:07] <robru> sil2100, https://ci-train.ubuntu.com/job/ubuntu-landing-014-3-merge-clean/1/console it's.... IT'S WORKING
[20:07] <sil2100> NO
[20:07] <sil2100> IT CANNOT BE
[20:07]  * sil2100 panics
[20:07] <Ursinha> lol
[20:09] <robru> sil2100, ok where are we at? can I click publish on some things?
[20:09] <asac> haha
[20:09] <sil2100> robru: ok, let's try that, I would also like to see if the build jobs all work correctly with the new setup
[20:09] <fginther> alesage, it didn't work, the hook modifies debian/rules, but not sufficiently: paste.ubuntu.com/8048082/
[20:10] <robru> sil2100, ok I triggered some jobs, check the dashboard.
[20:10] <alesage> fginther, I see I see
[20:10] <sil2100> robru: I'm a bit worried that rebuilds of previously-assigned silos might have problems
[20:10] <robru> sil2100, I just had a crazy thought... can we just rip out the part of the spreadsheet that displays silo statuses? the dashboard is sooooo much nicer anyway.
[20:10] <alesage> fginther, I may be able to fix
[20:11] <sil2100> robru: that's a thought, let's think about it! For this week I would say we did enough chaos :)
[20:11] <robru> sil2100, yeah agreed. but I love making the spreadsheet simpler ;-)
[20:11] <fginther> alesage, that would be awesome, thanks
[20:11] <alesage> fginther, will work on now
[20:12] <robru> sil2100, so I guess stgraber never merged my queuebot fixes, so queuebot will not be pinging about any silo statuses until he gets back
[20:12] <robru> well, it can ping from the spreadsheet, but not the JSON. so you'll get testing pass and ready for assignment pings only
[20:12] <bfiller> robru: need a silo for line 40 when convenient, I know you guys are transitioning
[20:13] <robru> bfiller, we can try that ;-)
[20:13] <robru> bfiller, telephony-service conflicts in silo 4
[20:14] <robru> oh, but that one's just merging now
[20:14] <robru> no worries
[20:15] <Ursinha> sil2100: out of curiosity, when you guys need to put citrain into maintenance mode, how far in advance do you announce that? and how long is the maintenance window?
[20:15] <sil2100> Ursinha: that was the first time
[20:16] <camako> Requesting a silo for row 41 plz
[20:16] <camako> trainguards ^
[20:16] <robru> Ursinha, hahahahahahah
[20:16] <sil2100> Ursinha: usually it's a swift transition and usually doesn't take more than 15 minutes if we change something, but this time it was something bigger and blocked as for longer
[20:16] <sil2100> Ursinha: I informed about it yesterday
[20:16] <Ursinha> robru: that was a very descriptive answer
[20:16] <Ursinha> lol
[20:16] <robru> Ursinha, maintenance windows? we edit production live!
[20:16] <sil2100> YEA
[20:17] <Ursinha> hehe
[20:17] <Ursinha> it was just out of curiosity :)
[20:18] <robru> camako, ok you got silo 4
[20:19] <Ursinha> sil2100: can I haz spreadsheet perms? :)
[20:19] <sil2100> Suar, adding now!
[20:19] <camako> robru thanks!
[20:19] <robru> camako, you're welcome!
[20:20] <sil2100> Ursinha: you should be ready
[20:20] <robru> sil2100, things seem to be running. I got a couple publications into proposed, so publish and also check-migration are working
[20:20] <Ursinha> sil2100: it worked, thanks :D
[20:20] <sil2100> phew
[20:21] <robru> sil2100, build in silo 8 seems a bit goofy but it's sergiusens' fault, he pre-merged his MPs, so citrain was a bit confused, not a result of the recent changes I don't think. camako and bfiller have more normal builds that we can follow for now
[20:21] <sil2100> robru: oh, and seeing AlbertA2's landing I also see the build jobs seem working
[20:21] <robru> yeah that too ;-)
[20:21] <sil2100> Ok ;)
[20:23] <Ursinha> that's nice
[20:24] <Saviq> trainguards, is train back up? could I have a silo for line 38 please?
[20:24] <robru> Ursinha, you got 14
[20:24] <Ursinha> robru: sweet, thanks
[20:24] <robru> Ursinha, you're welcome
[20:24] <Saviq> robru, thanks!
[20:25] <robru> sil2100, another crazy idea: https://ci-train.ubuntu.com/job/prepare-silo/1392/console maybe the prepare job doesn't need to re-test unity8 conflicts every time. maybe it could just test once ;-)
[20:26] <robru> Saviq, you got 17. you're welcome!
[20:27] <ToyKeeper> I wonder why accounts -> back crashing is such a persistent bug.
[20:34] <Saviq> ToyKeeper, it's not crashing
[20:35] <robru> sil2100, http://people.canonical.com/~platform/citrain/ seems there's some extra files here, can you clean those?
[20:35] <Saviq> ToyKeeper, if you mean the 5s delay after exiting accounts, it's just exiting, the service stays on, waiting for another connection
[20:35] <robru> oh, my queuebot branch bitrotted, bah, gotta update that
[20:35] <Saviq> ToyKeeper, it basically is bug #1352251
[20:36] <Saviq> ToyKeeper, unless you really get crash reports of course :)
[20:40] <sil2100> robru: oh, right
[20:40] <sil2100> robru: I guess the sync happened in a bad moment, let me clean those
[20:42] <ToyKeeper> Saviq: Yeah, actual crash dumps.  And it seems they were even uploaded, woot!
[20:42] <robru> sil2100, thanks
[20:42] <ToyKeeper> Ooh, fun.  It looks like gdb also crashed while trying to prep the account app crash file.
[20:48] <bfiller> robru: could you reconfigure silo 20 please, added a new package
[20:53] <robru> Ubuntu CI Engineering Team | CI Train support: trainguards | Vanguard (general help): Ursinha | CI Train Status: #179 promoted | CI Train Dashboard: http://bit.ly/1mDv1FS | Known issues: queuebot isn't reporting silo statuses
[20:58] <robru> sil2100, hey, did you test this image number thing? it rejects any entry other than a literal 'Yes (#)'. You can't put a number there
[21:00] <sil2100> robru: yeah, but it seems that it got reverted back by some other changes I did - should be good now
[21:00] <robru> sil2100, hm ok
[21:00] <sil2100> robru: you'll still be getting warinings about it being invalid though, google spreadsheet doesn't support data validation with patterns or regex :<
[21:00] <robru> wow that's lame
[21:01] <sil2100> I was looking for that for long and nothing
[21:01] <robru> sil2100, and what, you didn't want to type out every possible number? what are you, lazy?
[21:01] <sil2100> hah ;) Yeah... :(
[21:05] <robru> bfiller, oh sorry, I did the reconfigure, forgot to ping you. it's ready to build
[21:05] <bfiller> robru: thanks
[21:05] <robru> bfiller, you're welcome
[21:11] <alesage> fginther, offering this having tested separately as shell script https://code.launchpad.net/~allanlesage/pbuilderjenkins/dpkg-gensymbols-fix-existing-level-setting/+merge/230892
[21:28] <Ursinha> robru: how long it takes for a package to be published after bot says it's ready to land?
[21:28] <fginther> alesage, thanks, that looks good. can you please add a changelog entry?
[21:28] <fginther> alesage, that will trigger a publish to our ppa
[21:28] <alesage> fginther, yessir
[21:31] <robru> Ursinha, first it takes until somebody notices it. then once it's published it has to go through -proposed, which (depending on the package) will run some autopkgtests, which can take up to 3 hours for big things like mir. it can also get stuck there depending on a number of conditions, until somebody fixes it.
[21:33] <robru> Ursinha, so I've just published it: https://ci-train.ubuntu.com/job/ubuntu-landing-014-2-publish/2/console it'll go into -proposed shortly
[21:33] <Ursinha> robru: got it
[21:34] <Ursinha> robru: I wanted to know more about the manual part (the citrain publishing part, or sending stuff to snakefruit)
[21:34] <Ursinha> I noticed the bot ping here and wanted to know if there was a policy or something to respond to that
[21:34] <Ursinha> robru: thanks :)
[21:35] <robru> Ursinha, yeah, right now I'm responsible for that, in EU times it's sil2100. I was afk for a couple minutes so when I got back I saw your ping and the bot ping
[21:35] <alesage> fginther, pushed changelog entry, pls review for format :)
[21:36] <Ursinha> robru: got it
[21:48] <sil2100> robru: ok, so I think everything seems working, right?
[21:49] <Ursinha> yes yes
[21:49] <robru> sil2100, yep, it looks good to me so far, some builds have been done and a few publishes
[21:50] <robru> sil2100, cleaning is also flawless
[21:50] <robru> Ursinha, ok you got silo 10
[21:50] <Ursinha> robru: thanks
[21:51] <robru> Ursinha, you're welcome!
[21:51] <Ursinha> :)
[21:52] <robru> ah, looks like stgraber merged my queuebot fixes! yay!
[21:53] <Ursinha> robru: now I have this package that is already signed by rsalveti and needs to be dput to the ppa: http://people.canonical.com/~rsalveti/hybris/ -- I'm doing this to get a grasp of the landing process
[21:54] <Ursinha> I understand only supercow powered people can do that
[21:54] <robru> Ursinha, right, so only certain people can dput into the ppas
[21:54] <Ursinha> robru: are you one of these people? :)
[21:54] <robru> Ursinha, rsalveti is one of those people. is he around to do the upload himself?
[21:54]  * rsalveti hides
[21:54] <robru> Ursinha, I am, but I'd have to re-sign the the thing. if rsalveti signed it, best if he uploads it
[21:55] <rsalveti> robru: you don't need to re-sing
[21:55] <rsalveti> sign
[21:55] <robru> rsalveti, oh right. I just had to resign mterry's becuse he's not special enough ;-)
[21:55] <rsalveti> remember dput still push stuff to a ftp server
[21:55] <rsalveti> robru: oh, indeed
[21:55] <robru> Ursinha, ok whatever. i can upload it. one sec
[21:55] <Ursinha> robru: I didn't ask rsalveti because the point of this is emulate how a person that's not special enough feels using citrain :)
[21:56] <rsalveti> then for a normal human being you'd need to resign the src package
[21:56] <robru> Ursinha, oh, are you evaluating citrain so that you can avoid making the same mistakes in ci airline?
[21:57] <Ursinha> robru: it's a way to put it :)
[21:57] <robru> Ursinha, oh ho ho let me tell you some things...
[21:57] <Ursinha> lol
[21:57] <rsalveti> robru: yeah, she's helping me with my landings today
[21:57] <rsalveti> so she can feel the pain herself
[21:58] <Ursinha> robru: for example... first step of the airline is to eliminate as many human interventions as possible
[21:58] <Ursinha> for source packages we're almost there
[21:58] <robru> rsalveti, ugh, whats your key id? I can't upload because I don't have your public key to verify
[21:59] <robru> Ursinha, yes, eliminate the monkey button pushing that consumes my day
[21:59] <rsalveti> robru: : http://keyserver.ubuntu.com:11371/pks/lookup?search=0xA9F32C8C77966D012FA4D25073606C99B16223A3&op=index
[21:59] <Ursinha> robru: yeah, we don't need that
[21:59] <Ursinha> for most things we don't
[21:59] <robru> ok, upload started, let's see if the PPA takes it!
[21:59] <Ursinha> robru: thanks :)
[22:00] <robru> Ursinha, you're welcome
[22:00] <robru> Ursinha, now I have to warn you, just yesterday citrain totally failed to be able to publish a source package, so I don't have much faith in this right now...
[22:00] <bfiller> robru: need a silo for line 37, it fixes on of the promotion blockers with gallery
[22:01] <robru> bfiller, you got silo 11...
[22:01] <Ursinha> robru: uh, what happened?
[22:01] <bfiller> robru: thanks
[22:01] <robru> bfiller, you're welcome
[22:02] <robru> Ursinha, well the silo was one MP + one source package. the build job built the MP but refused to even acknowledge the presence of the source package, even though it was already built in the PPA. so the publish job failed because it claimed the package wasn't built. but the build job wouldn't "build" it (it didn't even need building, just needed the build job to acknowledge it had already been built)
[22:02] <Ursinha> robru: do I have to "build watch-only" for source packages? or should I wait for the ppa to pick that up and update stuff?
[22:02] <fginther> alesage, thanks!
[22:02] <robru> Ursinha, https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-010/+packages looks like the package is there
[22:02] <robru> Ursinha, so what you do now is called a WATCH_ONLY build
[22:03] <Ursinha> robru: do I need to wait for the package to build or only for it to show up in the ppa?
[22:03] <robru> Ursinha, you have to trigger the build job, and check 'WATCH_ONLY' so that it just looks in the PPA to acknowledge the source package.
[22:03] <robru> Ursinha, only have to wait for it to show up before triggering the WATCH_ONLY
[22:03] <Ursinha> robru: got it
[22:03] <robru> Ursinha, eg, so do that now ;-)
[22:03]  * Ursinha builds WATCH_ONLY
[22:04] <robru> Ursinha, SUCCESS! it failed!
[22:04] <Ursinha> wat
[22:04] <Ursinha> lol
[22:04] <Ursinha> that means citrain now knows it needs to watch the ppa and then publish in the end?
[22:04] <robru> Ursinha, https://ci-train.ubuntu.com/job/ubuntu-landing-010-1-build/1/console finished successfully, but note that it does not make any reference to the source package whatsoever. which means yeah, it won't ack it, so it won't be able to publish it either.
[22:04] <Ursinha> oh
[22:05] <robru> sil2100, ^^ our source package handling has been busted for at least a couple days
[22:06] <robru> Ursinha, so basically, give up on this one, let rsalveti upload to distro the old fashioned way.
[22:06] <Ursinha> man
[22:06] <robru> Ursinha, yep
[22:06] <Ursinha> robru: so you're saying citrain is broken for source packages?
[22:06] <robru> this is exactly what happened to mterry yesterday
[22:06] <robru> Ursinha, yes.
[22:06] <Ursinha> this should be on the channel topic
[22:07] <Ursinha> robru: thanks :)
[22:07] <robru> I should dive into this
[22:07] <robru> the citrain code is so scary though...
[22:07] <rsalveti> robru: please don't free the silo just yet
[22:07] <robru> rsalveti, no worries
[22:11] <asac> robru: please dive; very important feature and i am sure folks want to land with source packages soon
[22:12] <asac> but guess now that we switched you are not that busy anymore on that part
[22:12] <asac> thanks
[22:12] <robru> asac, yeah looking at it. only 400 lines of undocumented spaghetti
[22:12] <asac> cool
[22:12] <asac> sorry :/
[22:12] <Ursinha> robru: I can kinda try to help a bit
[22:12] <robru> asac, sorry, no worries.
[22:12] <Ursinha> :P
[22:13] <asac> thanks
[22:13] <robru> Ursinha, nah, fortunately it's python.
[22:13] <Ursinha> :)
[22:14] <sil2100> robru: I wonder what got b0rken
[22:15] <sil2100> robru: since when you said it's broken?
[22:15] <robru> sil2100, it seems the build job just doesn't even bother to even try to look at source packages.
[22:15] <sil2100> uh
[22:15] <robru> sil2100, first time I noticed it not working was yesterday.
[22:15] <robru> sil2100, but honestly I don't remember the last time I saw it work. could be months
[22:15] <sil2100> robru: let me take a quick look at that, maybe I broke it with some of my earlier code
[22:15] <sil2100> But I remember some people doing source-only uploads
[22:16] <sil2100> Recently
[22:16] <robru> sil2100, my very first guess is that line 126 of build script seems to only iterate on MPs, not source packages. still reading though.
[22:18] <sil2100> robru: it has to, as the loop in 126 is preparing source packages and does uploads for those, same for changelog generation etc.
[22:18] <sil2100> While source packages are completely detached
[22:20] <sil2100> robru: the code looks fine on first look, the check is in lines 352
[22:22] <sil2100> robru: I didn't touch that part, but maybe there's something racy/buggy there
[22:22] <robru> sil2100, yeah I don't understand this code at all
[22:22] <sil2100> Me neither
[22:22] <sil2100> :D
[22:22] <robru> sil2100, oh but you've been steeped in it... molded by it...
[22:22] <Wellark> any debian packaging guru's around?
[22:22] <sil2100> I just type in random python code in it with hopes it'll work
[22:23] <sil2100> Don't tell anyone though
[22:23] <sil2100> Ooops...
[22:23] <robru> sil2100, maybe you're a little bit too molded by it.
[22:23] <sil2100> robru: hah, but in all seriousness, sadly some parts of the code are really hard to understand, some places are still unknown to me
[22:24] <sil2100> robru: I'm not always able to get what exactly Didier had in mind in some places
[22:24] <robru> sil2100, well fundamentally somewhere there's a for loop that isn't running because the list it's iterating is empty. fill it full of print statements until you can figure out what list doesn't have the expected value ;-)
[22:24] <alesage> Wellark, not me but what's your q?
[22:24] <robru> sil2100, ack, br
[22:24] <robru> brb
[22:25] <sil2100> Ok, I log out from IRC already, but in case something breaks mup me or mail me ;)
[22:25] <sil2100> I'll also try digging more in the code for the reasons of the source package problem
[22:25] <sil2100> o/
[22:25] <ToyKeeper> Back when I mostly wrote everything in C, I'd usually write some pseudocode in a comment before anything interesting, to work out the algorithm and explain what the following code was supposed to do.
[22:26] <ToyKeeper> Then I discovered the pseudocode I was writing was actually executable and had a name.  It was called Python.
[22:26] <Wellark> alesage: just see the backlog at #ubuntu-touch for the question I asked from cjwatson
[22:26] <ToyKeeper> It sounds like this isn't that kind of Python.  ;P
[22:26] <Ursinha> ToyKeeper: haha, yeah
[22:27] <alesage> Wellark, reading, wanting to volunteer robru if he has a moment :)
[22:28]  * alesage likes to 'facilitate'
[22:28] <robru> alesage, Wellark: sorry citrain is exploding, maybe later
[22:32] <robru> Ursinha, rsalveti https://ci-train.ubuntu.com/job/ubuntu-landing-010-1-build/3/console ok magically it's working now.
[22:32] <Ursinha> lol
[22:32] <robru> literally no idea what changed there. i didn't change any code
[22:32] <rsalveti> lol, now that I just published it manually
[22:33] <ToyKeeper> Also sounds like this would be a good time to know the train engineer who knows exactly the right place to perform a little percussive maintenance.
[22:33] <robru> of course!
[22:33] <rsalveti> https://launchpad.net/ubuntu/+source/libhybris/0.1.0+git20131207+e452e83-0ubuntu26
[22:33] <ToyKeeper> "Hitting the citrain with a hammer?  That's only $5.  The other $995 is for knowing where to hit it to make it work again."
[22:33] <rsalveti> should have another src package for tomorrow, we can test that again
[22:33] <robru> I put it in debug mode, maybe there's a race condition that debug mode causes it to win...
[22:33] <rsalveti> right
[22:34] <robru> although I didn't think citrain was threaded. not sure what the race might be
[22:35] <Ursinha> robru: lol
[22:35] <rsalveti> :-)
[22:36] <robru> ToyKeeper, http://bazaar.launchpad.net/~cupstream2distro-maintainers/cupstream2distro/trunk/view/head:/citrain/build#L280 you were saying something about hammers?
[22:37] <Ursinha> robru: the package in silo 14 says packages are on destination, probably means they're already published in release pocket (?)
[22:37] <Ursinha> do I have to do anything else now or that's just admin cleanup?
[22:37] <ToyKeeper> robru: *facepalm*
[22:37] <robru> Ursinha, yes! in theory it means that, however there is a race condition where citrain reports that prematurely
[22:38] <Ursinha> oh god
[22:38] <Ursinha> robru: and what one has to do to be sure?
[22:39] <robru> Ursinha, yeah, I don't fully understand the details (ask colin) but basically when the package gets copied from proposed to release, it takes 10 minutes or so. launchpad (and thus citrain) report that at the *beginning* of the process, rmadison reports it at the end. rmadison is the one that counts when you're doing things like image builds.
[22:39] <robru> Ursinha, well, it depends what you're doing. if you just want to clean the silo, that's safe to do after citrain reports the packages migrated.
[22:39] <robru> Ursinha, if you want to kick an image build, you have to double check with rmadison
[22:39] <robru> Ursinha, if you kick an image build the same minute citrain tells you that a package migrated, it won't be in your image build.
[22:40] <robru> that bit me several times when citrain was new
[22:40] <ToyKeeper> gahhh...  that source file has no logical organization at all.  Not even a single 'def'.  Reminds me of the input-dispatching loop of something I wrote in 1992.
[22:40] <Ursinha> robru: that kinda makes no sense
[22:41] <robru> Ursinha, for all of citrain's faults, this is actually launchpad's fault, not ours
[22:41] <robru> ToyKeeper, yep. it's spaghetti
[22:41] <Ursinha> robru: well, citrain, as a system, could perform the rmadison check before reporting the package had migrated...
[22:42] <robru> Ursinha, oh but then you'd have to like shell out or something, you don't get rmadison in a nice api like launchpad gives. ;-)
[22:43] <Ursinha> hahaha yeah, sure
[22:43] <Ursinha> why write a script that loops over a shell if you can get a person to do that manually, right? :P
[22:44] <robru> ToyKeeper, btw, that "hammer" with "-100" means "we don't know how many commits are new commits since the last release, so let's just put the 100 most recent commits into the changelog." ;-)
[22:44] <robru> Ursinha, yeah, I mean, you don't want to put me out of work, do you? ... DO YOU?!
[22:45] <robru> ;-)
[22:45] <Ursinha> lol