[01:50] <robru> any core devs around for a quick ack ^ ?
[01:50] <robru> infinity, cjwatson ^ ?
[01:57] <infinity> robru: I assume there are appropriate code changes to match the s/python-six/python-pil/ bit?
[01:58] <robru> infinity, I also assume that.
[01:58] <infinity> (I'm not sure I agree with the truncated debian-dir-only diff review, it's a blind review, matching packaging changes to upstream bits we don't see)
[02:00] <robru> infinity, yeah, just checked the upstream diff, they rip out six and introduce PIL
[02:00] <infinity> robru: So, the changes look fine, my only complaint is that the changelog doesn't mention python-pil, nor the XDG_RUNTIME_DIR fix.
[02:00] <robru> infinity, k, thanks
[02:05] <imgbot> [02:09] <veebers> infinity, robru: ah rats, I must have missed that change when putting together the change logs
[03:33] <Mirv> great that the autopilot got in!
[03:40] <imgbot> [03:40] <imgbot> [06:58] <oSoMoN> sil2100, good morning! can silo 15 be published?
[07:21] <oSoMoN> sil2100, you around?
[07:22] <sil2100> oSoMoN: hello! Sorry, was AFK for a moment
[07:22] <oSoMoN> sil2100, no worries! would you mind publishing silo 15 for me?
[07:23] <sil2100> oSoMoN: sure, one moment
[07:27] <tvoss> good morning
[07:28] <tvoss> davmor2, you around?
[07:30] <sil2100> hmmm
[07:30] <sil2100> oSoMoN: ok, we seem to have some problems, one moment
[07:31] <oSoMoN> sil2100, what sort of problems?
[07:32] <oSoMoN> woot, this error message doesn’t make sense
[07:32] <sil2100> oSoMoN: trying to dig into why CI Train doesn't see the oxide upload
[07:34] <sil2100> oSoMoN: this might take a minute...
[07:35] <oSoMoN> if it’s only a minute, I’m fine with it :)
[07:55] <sil2100> oSoMoN: ok, fixed... I *should* be able to publish your package in the nearest minutes ;)
[07:59] <sil2100> oSoMoN: done \o/
[08:03] <tvoss> Saviq, reading zoltan's update mail and wondering, if the UbuntuShape optimizations have landed in the image, yet?
[08:06] <Mirv> tvoss: since 173, yes
[08:07] <tvoss> Mirv, ah, just flashed devel-proposed and I was expecting some visible improvements in the app scope
[08:08] <cjwatson> sil2100: Which package(s) did you push to dogfood?
[08:09] <cjwatson> sil2100: I'm pretty worried that we aren't going to have a recent promotion before the point when it comes time to branch :-/
[08:10] <Mirv> tvoss: I guess the app scope perf problems are more complicated than just ubuntushape
[08:18] <sil2100> cjwatson: I pushed indicator-location till the end
[08:18] <sil2100> cjwatson: yeah... today I'll push more on various fixes to get a promotable image ASAP, but we're a bit worried about that as well
[08:21] <sil2100> ogra_: hey, do you remember if plars and jdstrand were able to find someone who can fix the bug causing the AP failures?
[08:22] <sil2100> grrr
[08:22] <sil2100> Even more failing tests
[08:22] <sil2100> I think we need to stop the line and only concentrate on fixes
[08:22] <Saviq> tvoss, don't think so
[08:23] <tvoss> Saviq, Mirv just told me that they landed in 173
[08:24] <ogra_> sil2100, in 174 i see the gallery-app test has passed fine ... so there is hope
[08:25] <sil2100> ogra_: yeah, but address-book-app has 13 failures, messaging and dialer suddenly got worse as well
[08:25] <sil2100> ogra_: while I have been told that the uploads for those were supposed to *fix*
[08:25] <Saviq> tvoss, ah correct http://people.canonical.com/~ogra/touch-image-stats/173.changes
[08:25] <oSoMoN> sil2100, thanks!
[08:25] <Mirv> the batching of ubuntu shapes was introduced here: https://launchpad.net/ubuntu/+source/ubuntu-ui-toolkit/1.1.1179+14.10.20140804-0ubuntu1
[08:25] <Mirv> hmm, it seems the changelog was broken
[08:25] <ogra_> sil2100, https://launchpad.net/ubuntu/utopic/+source/dbus-cpp/4.0.0+14.10.20140730-0ubuntu1 this landed in 163 ... i wonder if tearing down the dbus daemon could cause such apparmor issues
[08:26] <Mirv> in that none of the bugs were actually closed automatically
[08:27] <tvoss> ogra_, it tears down its private instance of the daemon, not the actual daemons
[08:28] <ogra_> tvoss, right, but does apparmor know (or care) and could this cause all the denials
[08:29] <tvoss> ogra_, apparmor does not know, the testing buses are used during build time tests only
[08:29] <tvoss> ogra_, and don't run on the image
[08:29] <ogra_> tvoss, the denials we see are for a specific autopilot introspection interface
[08:29] <ogra_> which could well be the one you have just torn down
[08:30] <tvoss> ogra_, nope, the testing buses are not run during image tests
[08:30] <tvoss> ogra_, they are solely run at package build time
[08:30] <sil2100> ogra_: anyway, the address-book-app failures do not seem apparmor related, need to find renato___ or boiko...
[08:30] <ogra_> tvoss, we have to call out TRAINCON0 since we have 100s of tests failing due to that ... i'm grabbing for straws here
[08:30] <sil2100> renato___: ping
[08:30] <tvoss> ogra_, do you have the denials in a pastebin somewhere?
[08:31] <ogra_> tvoss, we have them in every console log for every test run :) one sed
[08:31] <ogra_> *sec
[08:31] <ogra_> https://jenkins.qa.ubuntu.com/job/utopic-touch-mako-smoke-daily/666/artifact/clientlogs/gallery_app/syslog/*view*/
[08:31] <ogra_> either there
[08:31] <ogra_> http://jenkins.qa.ubuntu.com/job/utopic-touch-mako-smoke-daily/666/consoleFull or there
[08:31] <ogra_> just search for "DEN"
[08:32] <ogra_> Aug  6 05:11:51 ubuntu-phablet dbus[2210]: apparmor="DENIED" operation="dbus_method_call"  bus="session" path="/com/canonical/Autopilot/Introspection" interface="com.canonical.Autopilot.Introspection" member="GetVersion" name=":1.88" mask="receive" pid=3704 profile="com.ubuntu.gallery_gallery_2.9.1.1025" peer_pid=3689 peer_profile="unconfined"
[08:32] <ogra_> thats a typical one
[08:33] <Saviq> sil2100, hey, reconfigure silo 1 please, added unity-api there
[08:35] <tvoss> ogra_, interesting ... the peer calling into gallery is unconfined
[08:35] <tvoss> ogra_, also, apparmor wouldn't deny if the daemon wasn't running
[08:35] <tvoss> ogra_, did you verify that the peer pid actually *is* autopilot?
[08:36] <ogra_> tvoss, i dont know what was researched by plars and jdstrand, they spent quite some time on this already (apparently unsuccessfull)
[08:43] <jgdx> what is the generic-deb job for uss and why can't it find a package that all the other jobs find?
[08:44] <Saviq> trainguards, reconfigure of silo 1 please, added unity-api there
[08:44] <sil2100> Saviq: one moment, in a meeting :)
[08:44] <Saviq> sil2100, pfft! ;P
[08:45] <tvoss> Saviq, camako I'm about to set silo 10 to testing done, please make sure to rebuild your qtmir(-gles) silos
[08:45] <Saviq> tvoss, kk
[08:45] <camako> tvoss, ack.. thanks for the heads up!
[08:46] <mhr3> ogra_, is there a known issue with 175? it kinda doesn't boot
[08:46] <mhr3> stuck on google logo
[08:46] <ogra_> theerare plenty of issues, but booting isnt one
[08:46] <ogra_> http://ci.ubuntu.com/smokeng/utopic/touch/mako/175:20140806:20140805.2/9528/ tests run ...
[08:46] <tvoss> Saviq, also https://code.launchpad.net/~thomas-voss/qtmir/gles-sync-20140804/+merge/229425
[08:46] <mhr3> can adb in though
[08:47] <ogra_> check with top if apparmor still parses stuff
[08:47] <mhr3> ogra_, nope, not doing anything really
[08:47] <ogra_> werid
[08:47] <ogra_> and thats a normal readonly OTA upgrade ?
[08:48] <mhr3> ogra_, no, normal ubuntu-device-flash
[08:48] <mhr3> with non-wiped data
[08:48] <ogra_> with --bootstrap ?
[08:48] <mhr3> no
[08:48] <ogra_> that might be your issue, we got new android bits (OTA would have been cleverer here)
[08:49] <ogra_> (your kernel/initrd could be out of sync)
[08:49] <mhr3> i'll bootstrap then
[08:51] <tvoss> sil2100, could you check why silo10 never finishes building?
[08:51] <Saviq> tvoss, not published on powerpc/ppc64el yet
[08:52] <Saviq> tvoss, that's gonna be a problem
[08:52] <tvoss> Saviq, for which package?
[08:52] <Saviq> tvoss, location-service
[08:52] <Saviq> tvoss, it was available in all arches before https://launchpad.net/ubuntu/+source/location-service/2.0.1+14.10.20140731-0ubuntu1
[08:53] <tvoss> Saviq, ack ... deps on trust-store
[08:53] <tvoss> Saviq, which now deps on mir, which is not available on those architectures
[08:54] <Saviq> tvoss, yeah, that means some archive guru needs to have a look
[08:54] <tvoss> Saviq, I will just make the build dep optional
[08:54] <Saviq> tvoss, like we need to drop the unsupported arches' packages from archive
[08:54] <Saviq> tvoss, or yeah, if you can make it go for other arches still, sure
[08:58] <brendand> sil2100, do you have the details on how the apparmor denials were introduced?
[09:01] <cjwatson> sil2100: awesome, so that made it through proposed-migrationd
[09:01] <cjwatson> sil2100: maybe we should consider branching from not a promoted image ...
[09:15] <Mirv> Saviq: looking at timestamps you didn't get the reconfiguration yet, so doing that
[09:16] <Saviq> Mirv, nope, didn't, thanks
[09:16] <sil2100> cjwatson: \o/ We're announcing TRAINCON-0 today, as we want to resolve the image situation ASAP
[09:18] <Mirv> Saviq: sil2100: reconfigure done
[09:19] <Saviq> NOOOOOOOOO
[09:19] <Saviq> sil2100, is it now?
[09:19] <Mirv> Saviq: so sad about reconfigure? ;) (I know, I know)
[09:24] <sil2100> Saviq: we're announcing it in a moment!
[09:24] <Saviq> sil2100, can you wait until I publish silo 1? ;D
[09:24] <sil2100> ;p
[09:28] <camako> Saviq, I rebuilt qtmir and version changed to 0.4.0+14.10.20140806-0ubuntu1. I need to update the twin. If I use 'dch -v' then your old entry remains. Should I manually update your entry?
[09:29] <Saviq> camako, was the previous version released?
[09:29] <camako> Saviq, no
[09:29] <Saviq> camako, if it was UNRELEASED, then it should get updated, let me have a look
[09:31] <camako> Saviq, qtmir still building armhf though (not completely finished)
[09:31] <Saviq> camako, shouldn't be a problem
[09:31] <Saviq> camako, that's silo 7 is it?
[09:31] <camako> Saviq, yes
[09:32] <Saviq> camako, why did you rebuild btw?
[09:32] <Saviq> camako, if on tvoss's orders then too early ;)
[09:32] <tvoss> Saviq, getting there with trust-store
[09:32] <camako> Saviq, a lot of packaging changes broke other downstreams, then we changed mir to fix.. thought I'd rebuild qtmir as well
[09:33] <Saviq> camako, ah ok
[09:33] <camako> Saviq, yes... not on tvoss' orders
[09:34] <Saviq> camako, in any case, http://paste.ubuntu.com/7968849/
[09:34] <Saviq> camako, if I go `dch -v 0.4.0+14.10.20140806-0ubuntu1 ""`, only the version and time gets updated
[09:35] <Saviq> camako, is that not what's happening for you?
[09:35]  * camako looks
[09:35] <davmor2> sil2100: new blocker I'm just going to dig into when it landed.  The guide swipes away and leaves the phone in a position where none of the apps actually open
[09:37] <cjwatson> sil2100: ok
[09:38] <camako> Saviq, I get http://pastebin.ubuntu.com/7968870/
[09:38] <camako> I guess my identity needs updating
[09:38] <Saviq> camako, yeah, that, too
[09:39] <Saviq> camako, but yeah, *inside* a version entry clear it up to taste
[09:39] <camako> Saviq, ok thanks.
[09:39] <Saviq> camako, export DEBEMAIL="michal.sawicz@canonical.com" in .profile
[09:39] <Saviq> camako, and DEBNAME accordingly
[09:39] <camako> Saviq, ah I see
[09:40] <Saviq> well, export your own email of course ;)
[09:40] <camako> :-)
[09:40] <camako> I always wanted to have dreadlocks...
[09:53] <camako> Saviq, the twin needs to be rebuilt after the version update, right?
[09:53] <Saviq> camako, yup
[09:54] <sil2100> davmor2: yeah, I added that to the blocker list already
[09:54] <sil2100> davmor2: oh, or wait, no, that's something different I gues?
[09:57] <davmor2> sil2100: I'm just digging back to find when it landed, Saviq already thinks it is likely the Scope as an app landing but still going to confirm
[10:03] <popey> sil2100: you approved the ms2 branch, do you know when that will hit the archive? https://bugs.launchpad.net/music-app/+bug/1350529
[10:12]  * popey lols at sil2100's mail - "incontinences"
[10:21] <davmor2> popey, sil2100: inconvenience I bet is the word you were after
[10:22] <sil2100> Typooo!
[10:23] <sil2100> popey: anyway, hm, let me check ms2
[10:23] <sil2100> popey: ok, it's in the archive now
[10:24] <mhr3> ogra_, could you merge this pls? https://code.launchpad.net/~pete-woods/click-sync/add-youtube-scope/+merge/229065
[10:24] <mhr3> it's been sitting there for a week
[10:24] <mhr3> ish
[10:24] <slangasek> sil2100: how did the apparmor denials shake out overnight?
[10:24] <sil2100> slangasek: what do you mean?
[10:25] <slangasek> sil2100: did the fixes that plars/jdstrand landed fix things, or are we still broken?
[10:25] <sil2100> slangasek: they're still broken, I didn't know that jdstrand/plars had any fixes - I only heard they identified/triaged the problem, but as it's something on the system level they could not do much
[10:26] <slangasek> hmm
[10:26] <sil2100> slangasek: in any case, 174 is still broken
[10:26] <slangasek> not sure why it being on the system level prevents them from doing things
[10:26] <slangasek> but if they can't, who owns it now?
[10:26] <sil2100> slangasek: the problem seems that they don't know how exactly to proceed and in which component the problem lies, and this is something that needs to be done
[10:28] <slangasek> right, so, who owns this now?  Given how critical this is, it would be awesome if someone were working on this in European time
[10:28] <sil2100> slangasek: it seems that sometimes the system clock is not set correctly at the right time (if I understood it correctly?) and it causes all those apparmor problems with profiles
[10:28] <slangasek> yes, that's roughly what I understood but I don't have the details
[10:28] <sil2100> We have no idea, we were trying to think of someone during the meeting but we're clueless
[10:28] <brendand> sil2100, it looks like most of the failures are down to a uitk change
[10:28] <slangasek> hmm ;)
[10:28] <sil2100> The only info we have is from the backlogs
[10:29] <brendand> sil2100, since messaging, address-book and dialer all share the same cause
[10:29] <sil2100> We can provide those to the general public, but that's why I called out for people that could help out with this issue
[10:29] <sil2100> brendand: oh!
[10:29] <brendand> sil2100, StateNotFoundError: Object not found with name 'UbuntuShape' and properties {'objectName': 'bottomEdgeTip'}.
[10:29]  * sil2100 gives bzoltan the evil eye (again)
[10:29] <sil2100> ;D
[10:29] <bzoltan> sil2100:  what?
[10:30] <bzoltan> brendand: is that an autopilot test?
[10:30] <brendand> bzoltan, yes
[10:30] <sil2100> bzoltan: j/k with that evil eye! But yeah, brendand mentions that all our recent failures are related to the UbuntuShape bottomEdgeTip
[10:30] <sil2100> bzoltan: do you know anything regarding that?
[10:30] <sil2100> t1mp: hey!
[10:30] <bzoltan> sil2100: yes
[10:30] <brendand> bzoltan, and actually it seems to indicate a functional failure
[10:31] <bzoltan> sil2100:  that is the fix I sent for the short app...
[10:31] <bzoltan> brendand:  No it indicates that the APs should not use the UbunuShape as objecttype
[10:31] <sil2100> bzoltan: ok, so it seems that we'll need the same fixes for the other apps then
[10:32] <sil2100> zsombi: hello! Any progress on bug LP: #1351024 ?
[10:32] <bzoltan> sil2100:  how it did not pop out on the autopilot tests?
[10:32] <brendand> bzoltan, that's not a justification
[10:32] <bzoltan> brendand:  what is not a justification?
[10:32] <brendand> bzoltan, if the tests break, then they need to be fixed
[10:33] <bzoltan> brendand:  What I say that the tests did not break when I run them
[10:33] <sil2100> jdstrand, plars: btw. guys, did you have a bug for the autopilot apparmor denials caused by the clock?
[10:34] <sil2100> bzoltan, brendand: yeah, so hm, maybe the problem was in some recent address-book+dialer+messaging landing?
[10:35] <sil2100> bzoltan, brendand: since when UITK was landing, I remember 1-2 landings of those apps as well
[10:35] <bzoltan> sil2100:  I have like 3 OK logs for these apps with the UITK I landed
[10:36] <brendand> bzoltan, the animationn broke on all 3 apps too
[10:37] <brendand> bzoltan, i don't think they could all break in the same way simultaneously
[10:37] <brendand> bzoltan, i guess it's possible, but not likely
[10:37] <bzoltan> brendand:  could you please give me some more context
[10:37] <brendand> bzoltan, try it on 175
[10:37] <zsombi> sil2100: just got it, I have an other ugly bug to fix before that
[10:37] <brendand> bzoltan, it's a bit hard to explain. basically when you pull the bottom edge up, the motion is all strange
[10:38] <brendand> bzoltan, i'll post a video :)
[10:38] <bzoltan> brendand:  try what? I still do not understand whatthe problem is. I can help... but tell me what should I look for
[10:40] <brendand> bzoltan, best way to sum it up is the bottom edge tip is not drawn at the top of the page during the animation
[10:40] <brendand> bzoltan, but again, only a video, or trying it yourself really does it justice
[10:41] <t1mp> sil2100: what's up?
[10:41] <sil2100> t1mp: ah, sorry, unping ;) Since I remembered you assigned on the date-picker bug, but then noticed that it got reassigned to zsombi
[10:42] <sil2100> t1mp: sorry for the disturbance!
[10:44] <t1mp> sil2100: ok :)
[10:47] <bzoltan> sil2100:  if I understand well the new UITK landed on rev173 and these problems appeared on rev175. So I  think the UITK i snot the prime suspect
[10:49] <sil2100> bzoltan: right, as mentioned it might be one of the messaging/dialer/address-book landings, just wanted to make sure it's unrelated as brendand pointed out UITK as a possibility
[10:49] <sil2100> Anyway, I will poke boiko once he's up anyway
[10:49] <brendand> sil2100, i was pointing at uitk because it was common
[10:50] <brendand> sil2100, but since it broke when all those apps got updated then the finger points in a different direction now
[10:50] <brendand> sil2100, i'm really curious how people are actually testing stuff
[10:50] <sil2100> brendand, bzoltan: yeah, and thanks to this we know more or less where to poke further :)
[10:50] <brendand> sil2100, several really bad regressions here
[10:51] <sil2100> brendand: indeed... need to make sure boiko runs all his tests on a mako device
[10:51] <bzoltan> sil2100:  me and the whole SDK team is at your disposal if you need more investigation or bugfix.
[10:53] <brendand> sil2100, where can i get more details about those app landings?
[10:53] <brendand> sil2100, so i can try and find what might be the cause of the problem
[11:02] <davmor2> sil2100: ha that was nice timing I'd already sent out a heads up email for the traincon0 and then your email landed shortly after :)
[11:04] <sil2100> brendand: you can use http://people.canonical.com/~lzemczak/landing-team/ to check what landed when :)
[11:04] <sil2100> davmor2: hah ;)
[11:10] <jibel> brendand, that issue in dialer/messaging/address-book you're talking about isn't it caused by the 'visual updates' of these applications that landed yesterday?
[11:11] <brendand> jibel, well yes - that part we know
[11:11] <brendand> jibel, trying to figure out how they missed it
[11:12] <brendand> sil2100, anyway i'll get a bug filed for all three projects
[11:12] <sil2100> brendand: ok, thanks :) I'll poke boiko when he's up automatically anyway
[11:15] <brendand> sil2100, https://bugs.launchpad.net/address-book-app/+bug/1353420
[11:16] <davmor2> sil2100, ogra_, Saviq: so the guide works as expected in 173, trying 174 now
[11:16] <sil2100> davmor2: ok, so a recent regression then!
[11:16] <Saviq> davmor2, really, you can stop :)
[11:16] <sil2100> brendand: thanks! Adding to teh blockerz
[11:16] <Saviq> davmor2, it's dash-as-app's fault 100%
[11:16] <Saviq> davmor2, I'll be on it just after I eat something
[11:18] <davmor2> Saviq: I'm just being thorough it's what we do you know ;)  No worries enjoy your food :)
[11:19] <brendand> sil2100, video of the issue :) https://plus.google.com/u/1/110434705244077414661/posts/YVnQPrvmWJE?pid=6044400154069719906&oid=110434705244077414661
[11:25] <davmor2> Saviq, ogra_, sil2100: yeap so 174 is the broken landing, I'll write up a bug for it and completely blame Saviq ;)
[11:26] <Saviq> davmor2, there is one already
[11:26] <davmor2> Saviq: oh nice
[11:26] <Saviq> davmor2, bug #1353351
[11:27] <davmor2> ah brendand beat me too it :)
[11:30] <zsombi> brendand: sil2100: bottom edge swipe problems - seems the apps are using their own one, and seem recently in 174 one app copied the component from teh other...
[11:30] <zsombi> brendand: sil2100: so it aint came from UITK
[11:31] <brendand> zsombi, yeah - i filed the bug against the apps themselves
[11:31] <zsombi> brendand: sil2100: all UITK provides is a Panel component (invisible one) which provides bottom-edge swipe detection, all the rest comes from apps
[11:31] <zsombi> brendand: ok, thx!!
[11:54] <renato___> sil2100, hi
[12:14] <Saviq> sil2100, for your viewing pleasure: bug #v
[12:14] <Saviq> 1353451
[12:14] <Saviq> bug #1353451
[12:14] <Saviq> dammit
[12:15] <renato___> sil2100, the problem with address-book dialer messaging app is that the UbuntuShape has changed the type name, we need to update the autopilot tests for that
[12:45] <brendand> sil2100, so it appears that the app updates were never tested with 173, which had the version of UITK that impacted them
[12:45] <brendand> bzoltan, ^ - fyi
[12:45] <brendand> sil2100, we need to have a tighter test/release cycle
[12:46] <bzoltan> brendand: sil2100: what is the problem?
[12:46] <brendand> sil2100, silos must be sure to be tested with the image prior to the one they are to be published in
[12:46] <brendand> bzoltan, the apps were broken by the uitk update in 173
[12:46] <brendand> bzoltan, but they were never tested with 173
[12:46] <bzoltan> brendand: what apps?
[12:47] <brendand> bzoltan, so uitk did break them, but it's not strictly speaking your fault
[12:47] <bzoltan> brendand:  how can I test with an image what does not exist?
[12:47] <brendand> just a faulty process
[12:47] <brendand> bzoltan, you can't - i'm not placing any blame on you
[12:47] <bzoltan> brendand:  Than please do not :)
[12:48] <bzoltan> brendand:  The UITK did not break anything ... I have tested the UITK with the  latest and most fresh available image. UITK can not be blamed for failures integrated after it lands.
[12:48] <bzoltan> sil2100: ^
[12:49] <brendand> bzoltan, that's what i said - it's not the fault of uitk or it's release process, the fault is in the general landing process
[12:49] <bzoltan> brendand: I kind of expected that after I run 24 apps autopilot test suites and ~800 tests I will be asked to test against the development branches of the apps too :D
[12:49] <bzoltan> brendand:  what if the apps would test against the latest released UITK at the first place?
[12:50] <brendand> bzoltan, they should - and that's what went wrong
[12:50] <bzoltan> brendand:  I am bagging for a staged releasing system for ages... we need an image with the staging UITK and one for the staging Qt  to foresee the possible upcoming issues.
[12:51] <brendand> bzoltan - that certainly would be helpful given the present release process
[12:51] <bzoltan> brendand:  we should have channels where we can customize the development branches and possible set PPAs with development versions
[12:51] <bzoltan> brendand:  this is something I am asking for very long time.
[13:05] <olli> sil2100, are you guys doing a standup with ppl working on the promotion blockers
[13:05] <sil2100> renato___: could you guys take care of those tests? :)
[13:07] <sil2100> brendand: hm, so, in other words - the behavior changed in UITK but this did not break the tests, but when the new apps were released those were not tested against the new UITK and tried using 'old' features that got removed in UITK, right?
[13:07] <sil2100> olli: we're pushing people about the blockers all the time (except for lunch periods)
[13:08] <jdstrand> sil2100: I did not file a bug. I was thinking plars would after his investigations
[13:08] <sil2100> zsombi: give me a sign once you have any updates regarding the date-picker in calendar
[13:09] <olli> sil2100, are you getting the right attention
[13:09] <brendand> sil2100, kind of. i'll explain more in the landing meeting
[13:09] <sil2100> jdstrand: ok, since we would like to find someone who could actually work on fixing the root cause - the bug itself is a bit strange (or maybe I lack some understanding here), but do you have any idea who could be the best person or at least which component should be worked on to get this fixed?
[13:10] <sil2100> olli: so far yes, everyone's responding to pings when they're available, the only thing left is finding someone who could work on the apparmor-smoketesting problem
[13:10] <plars> jdstrand: sil2100: I don't have one yet, or know yet where to file it. We know "what" is happening, but still have no idea why
[13:11] <sil2100> plars: right... this is exactly what I was asking jdstrand now, and we were also thinking about that during the morning meeting
[13:11] <sil2100> As we don't really even know 'who' to poke about getting this fixed
[13:11] <sil2100> Any ideas?
[13:11] <jdstrand> sil2100: honestly, I have no idea. it could be a provisioning issue, it could be a hardware issue, it could be a bug in Ubuntu
[13:12] <ogra_> is that the clock issue ?
[13:12] <jdstrand> there is probably something else it could be that I am not thinking of
[13:12] <jdstrand> ogra_: yes
[13:13] <ogra_> so i was wondering ... if the hwclock doesnt get updated (i dont think we do that) ... your clock wil be off until ntpdate is called by /etc/if-up.d
[13:13] <jdstrand> it could be worked around in the CI infrastructure, but I agree with plars that it is unwise to paper over it in case it is a bug in Ubuntu since we don't want phones running around thinking it is 1970
[13:13] <ogra_> that will only be called once NM starts
[13:14] <plars> sil2100: I'd be happy to put a placeholder for it somewhere though, do we have a catch-all place for stuff like this on touch?
[13:14] <ogra_> if you reboot and the apparmor profiling gets re-run at that point, the clock will still be off
[13:14] <plars> ogra_: right, but why is it off by so much sometimes, and not others?
[13:14] <jdstrand> ogra_: there is a hwclock-save upstart job on the device that runs in runlevels 0 and 6, but aiui, adb reboot (what the CI tools do) doesn't run through those runlevels
[13:14] <ogra_> i wonder if making the apparmor upstart job "start on started network-manager" would help here
[13:14] <plars> ogra_: for some runs, when the device is installed and boots, it's off by maybe 1-8 seconds. For others, it's gone way back in time to January 12, 1970
[13:15] <ogra_> plars, because we never set it
[13:15] <jdstrand> ogra_: it isn't the apparmor upstart job
[13:15] <ogra_> plars, you always operate with the same hwclock setting until ntpdate kicks in
[13:15] <plars> ogra_: I think we'd still have the race even if we wait to ntpdate after nm starts if that's what you mean
[13:15] <plars> ogra_: it might narrow the window a bit, but that's all
[13:16] <ogra_> jdstrand, i thought it was the time discrepancy of the generated files
[13:16] <plars> ogra_: sure, but it's never set for any of them
[13:16] <plars> ogra_: yet some preserve a fairly close time, some are just way off
[13:16] <jdstrand> we have precompiled policy cache on the device with a date of 2014. when the clock is off, aa-clickhook is run with the clock still at 1970. it generates new policy with timestamps of 1970. it tries to load them into the kernel, but see the precompiled cache has  date of 2014
[13:16] <jdstrand> so it skips them
[13:17] <ogra_> jdstrand, right ... and making sure the clock is right first (by starting after NM) might be a (not very good, but usable)workaround
[13:17] <jdstrand> this isn't the apparmor upstart job
[13:17] <jdstrand> aa-clickhook is run by the CI tools
[13:17] <jdstrand> aa-clickhook -f --include=...
[13:17] <ogra_> hmm
[13:18] <sil2100> hmmm
[13:18] <jdstrand> that command ^ is run when the clock is at 1970
[13:18] <ogra_> we could just wrap that with: ntpdate ntp.ubuntu.com
[13:18] <jdstrand> the clock needs to be correct when that runs
[13:18] <jdstrand> sure, there are various workarounds
[13:18] <jdstrand> we could after running ntpdate do 'start hwclock-save'
[13:18] <tedg> traingaurds, can I please get a silo for line 26 ?
[13:19] <jdstrand> but that doesn't address why the clock is at 1970 in the first place
[13:19] <pmcgowan> jdstrand, right, it should only happen when the battery and the RTC backup are totally depleted
[13:19] <plars> ogra_: sure, we can force ntpdate before we do anything, we could even force hwclock as a provisioning step. But we don't want to potentially conceal a bug here that when a user installs and boots a phone, sometimes they could have a good date, other times they won't. Also, if they never shutdown the phone normally they may never run hwclock at all and the whole thing could start over
[13:19] <ogra_> we should actually make it fail if $clock < $installation date
[13:20] <plars> ogra_: my gut feeling is that most people would never go through a "normal" shutdown, but rather wait until the phone is stuck and hold the power button for a hard reboot
[13:20] <jdstrand> plars: pmcgowan had an interesting point. is there a way to detect that?
[13:20] <pmcgowan> but lab devices are all plugged in I assume
[13:20] <plars> jdstrand: sure, hwclock by default will read what it's set to
[13:20] <ogra_> plars, that would be sad for the people that implemented the new shutdown/reboot dialog :P
[13:21] <plars> ogra_: sure, but do you ever reboot your personal phone for fun? :)
[13:21] <jdstrand> (which btw, that dialog comes up every time I touch the power button it seems)
[13:21]  * jdstrand assumes that is a bug fixed in proposed...
[13:21] <ogra_> plars, nah, never indeed
[13:23] <tvoss> jdstrand, plars do you have the kernel logs/logcat available from the lab devices?
[13:23] <ogra_> well, hwclock -s/-w both work fine
[13:23] <ogra_> so we should simply make sure that we set it on boot
[13:23] <ogra_> or if ntpdate retrieved a new time or ... or ...
[13:24] <jdstrand> currently the upstart job that does that only runs in 0 and 6
[13:24] <plars> tvoss: yes, one moment and I'll point you at some
[13:24] <jdstrand> I was wondering why the hwclock wasn't set after ntpdate
[13:24] <tvoss> ogra_, it still would just hide the issue
[13:24] <jdstrand> (just as a matter of course)
[13:25] <plars> tvoss: basically anytime you see these apparmor "DENIED" results, it's typically this problem (for now at least): http://ci.ubuntu.com/smokeng/utopic/touch/mako/175:20140806:20140805.2/9528/ubuntu_calculator_app/1496111/
[13:25] <plars> tvoss: from there, you can link to syslog below for the full syslog
[13:25] <plars> tvoss: and in syslog, you'll find: Aug  6 05:06:31 ubuntu-phablet ntpdate[2787]: step time server 91.189.89.199 offset 1400004685.653897 sec
[13:26] <ogra_> tvoss, huh ?
[13:26] <jdstrand> plars: interesting, that has a date of Mar 26 instead of Jan 12
[13:26] <ogra_> tvoss, the issue is that whenever we set the time we dont set the hwclock
[13:26] <plars> jdstrand: that would at least keep it from recurring on later boots
[13:26] <jdstrand> s/whenever/has/
[13:26] <plars> jdstrand: oh, didn't notice - yeah the ones we looked at yesterday were all Jan 12
[13:26] <plars> we're moving forward in time at least :)
[13:26] <tvoss> ogra_, are we sure that that is the actual issue?
[13:26] <jdstrand> Mar 26 is enough behind to cause a problem though
[13:27] <plars> indeed
[13:27] <jdstrand> plars: we'll get there!
[13:27] <jdstrand> hehe
[13:27] <ogra_> tvoss, if the HW clock is permanently at 1970 i'm 99.5% sure, yes
[13:27] <plars> sloooowly
[13:27] <tvoss> ogra_, why is that only showing up now?
[13:27] <ogra_> tvoss, no idea ...
[13:27] <tvoss> ogra_, that's the point I'm trying to make
[13:27] <plars> well, it is a race
[13:27] <ogra_> probably apparmor didnt care about timestamps before ?
[13:27] <plars> but we are certainly hitting it a lot now, where we didn't seem to at all before
[13:27] <tvoss> ogra_, I'm pretty sure it did
[13:28] <tvoss> jdstrand, ^
[13:28] <ogra_> or it simply ran a nanosecond later
[13:28] <jdstrand> the issue is only showing up now I think because we only started precompiling apparmor policy
[13:28] <ogra_> which is the nanosecond that ntpdate runs delayed today
[13:28] <jdstrand> recently
[13:28] <ogra_> right
[13:28] <jdstrand> I don't know the exact date
[13:28] <ogra_> it was before
[13:28] <jdstrand> but within the last couple of weeks I think
[13:28] <ogra_> something else changed additionally that causes the race now
[13:29] <ogra_> it used to work for like a week
[13:29] <jdstrand> ah
[13:29] <ogra_> the change to pecompiled profiles must have been in 140-150 somewhere
[13:29] <ogra_> the issue only started showing in the 160s
[13:30]  * jdstrand isn't sure-- rsalveti enabled it server side
[13:30] <ogra_> yup
[13:30]  * ogra_ followed that 
[13:30] <jdstrand> but that seems like plausible timing
[13:30] <jdstrand> ok
[13:30] <tvoss> jdstrand, plars so looking at https://jenkins.qa.ubuntu.com/job/utopic-touch-mako-smoke-daily/666/artifact/clientlogs/ubuntu_calculator_app/syslog/*view*/
[13:30] <ogra_> anyway, we need to somehow set the hwclock and be happy :)
[13:32] <tedg> sil2100, can I please get a silo for line 26?
[13:33] <sil2100> tedg: sure thing, assigning
[13:35] <tedg> sil2100, thanks!
[13:35] <tvoss> jdstrand, plars seems like something is setting the hwclock
[13:35] <jdstrand> yeah, it sometimes is ok, sometimes isn't
[13:36] <plars> ogra_: we'd need to make sure that we don't run phablet-config until ntpdate is synced though
[13:36] <sil2100> popey: I see that the music-app test-fix branch from Victor had problems autolanding - do you have direct contact with Victor?
[13:36] <tvoss> jdstrand, well, it is forcefully set to that time. seems like whatever is setting it considers the wrong value
[13:37] <ogra_> plars, we could (ans an interim) hack around it in phablet-network
[13:37] <ogra_> plars, simply call "adb shell hwclock -w" after the network interface is known up
[13:39] <plars> ogra_: ntpdate ntp.ubuntu.com first, and make sure that succeeds
[13:39] <sil2100> Saviq: hi! Soooo... does silo 1 have anything else besides the fixes for those 2 issues? (the welcome tutorial and the icon colors)
[13:40] <ogra_> plars, http://paste.ubuntu.com/7970504/
[13:40] <Saviq> sil2100, noooo, why would you ask that?
[13:40] <ogra_> plars, right, perhaps prefix that
[13:40] <plars> ogra_: like we said, the workaround wouldn't be that hard. But we should ensure that we aren't hiding something that we aren't completely sure why it just started happening so frequently
[13:40]  * sil2100 suspects a trap
[13:40] <sil2100> ;)
[13:40] <Saviq> sil2100, it will require a QA sign off if that's what you're asking
[13:40] <Saviq> sil2100, but it will fix the two issues as well
[13:40] <sil2100> Ok, davmor2 will be our man then
[13:41] <plars> ogra_: yeah, we'd need to run ntpdate in a loop until we are sure it's passed first too
[13:41] <Saviq> sil2100, I just kicked a rebuild, should be good after that
[13:41] <plars> ogra_: in case the network thinks it's up, but still can't resolve yet (as we've seen before)
[13:41] <sil2100> davmor2: hey, how busy are you right now? :)
[13:41] <ogra_> plars, then http://paste.ubuntu.com/7970518/
[13:42] <davmor2> sil2100: I can cover that quite happily :)
[13:42] <ogra_> plars, if we are definitely sure network is up (which we are after the ping succeeds) this should work
[13:43] <ogra_> or probably stuff it into an else for eth return value check
[13:43] <ogra_> plars, we might be hiding something, sure, but for the mooment i really dont care as long as hacking around it brings us out of TRAINCON-0
[13:44] <ogra_> this is currently more important
[13:44] <zsombi> sil2100: I will give a sign once I know what's wrong there, since there hasn't been any changes on the toolkit recently on DatePickers...
[13:44] <ogra_> we know we have to re-visit the clock
[13:44] <sil2100> zsombi: thanks, will wait with anticipation then ;)
[13:44] <plars> ogra_: no, I think we'll exit before we get there if the network is up, right?
[13:46] <plars> sil2100: what are your thoughts? do we work around this for now?
[13:47] <plars> ogra_: we don't even necessarily have to do it in phablet-network, I can add some safety checks around it too, since it's not strictly related to bringing the network up
[13:49] <sil2100> plars, ogra_: just so that I understand it right - first of all, we still don't know what changed that it suddenly became an issue, right?
[13:50] <sil2100> plars, ogra_: second, from what I understand, it's not really something that affects normal users, right? As in a normal user workflow the time will get corrected automatically?
[13:50] <sil2100> I'm a bit worried about the case of no-network available
[13:51] <plars> sil2100: yeah, I think we can get around the no-network with some sensible amount of retrying
[13:51] <plars> sil2100: normal users are probably not running aa-clickhook
[13:52] <plars> sil2100: so it *could* affect other things for them though, since the time could be really odd at the beginning for them, and could even revert to that early time if they never go through the right shutdown steps
[13:53] <plars> sil2100: so I think there are a couple of things at least that should be done
[13:53] <plars> sil2100: 1. update hwclock as soon as the time is set as jdstrand suggested earlier I think
[13:53] <plars> that seems like a no-brainer
[13:54] <plars> it won't fix this issue, but it would make it less likely to keep occuring across boots
[13:54] <boiko> sil2100: hey, the UbuntuShape causing autopilot failures on dialer, messaging and address-book app, I have fixes for those already
[13:54] <plars> and it would also fix the issue where a normal user would keep seeing this if they never shut down their phone the "right way"
[13:55] <plars> sil2100: 2. we should probably open a bug somewhere about the fact that after an install we sometimes have a realistic time, and sometimes not
[13:56] <sil2100> boiko: \o/ excellent
[13:56] <plars> if we work around this, that's unfortunately probably going to go to the bottom of the priority list for whoever gets it, and be harder to track down
[13:56] <sil2100> plars: hmmm
[13:56] <plars> but it doesn't make sense to me why it's not at least predictable
[13:56] <sil2100> plars: let me think about that for a moment, but first I need to go to UE live
[13:57] <boiko> sil2100: I was going to propose it for landing yesterday, but CI is taking ages to run nowadays, and in the end almost all autopilot tests failed in the CI jobs for the three apps, something is very broken
[13:57] <plars> sil2100: then, at your discretion, we could work around this to get us out of traincon
[13:58] <plars> sil2100: that would basically involve just going through a loop a few times to make sure the network is up, make sure ntpdate has run successfully, and hwclock is set. If we really want to be clever, we could even diff it against the host time and make sure it's not off by too much
[13:59] <sil2100> plars: so, I think what we could do indeed is work around it until promotion and then remove the workaround to get back to a broken state, to make sure this issue gets the attention it deserves
[14:01] <plars> sil2100: any suggestion on where the bug could go? do we have a catch-all category for touch? I haven't seen one I don't think
[14:01] <sil2100> plars: hm, I don't think we have one... we can report it for 'Ubuntu' in overall, but not sure if that makes sense ;)
[14:03] <ogra_> sil2100, but overall ubuntu would be wrong
[14:03] <sil2100> Right, that's what I said, it might not make sense ;)
[14:03] <ogra_> sil2100, we are in that situation because we reset the device instead of shutting down
[14:04] <sil2100> ogra_: so maybe ci-services-itself or what was that?
[14:04] <ogra_> theoretically this isnt even a bug but mis-use of a tool ;)
[14:04]  * sil2100 remembers a project like that once
[14:04] <ogra_> "adb reboot" vs "adb shell reboot"
[14:04] <sil2100> ubuntu-ci-services-itself
[14:04] <sil2100> Maybe this migth be a good place for the bug
[14:04] <ogra_> yeah, something like that
[14:05] <ogra_> plars, hmm, in fact ... we dont need all this if your first reboot of the device becomes "adb shell reboot"
[14:05] <ogra_> i think
[14:05] <sil2100> hm, I remember one time when we were actually happy we're doing adb reboot instead of adb shell reboot, but can't remember what that was ;)
[14:06] <ogra_> sil2100, it is a lot faster
[14:06] <ogra_> like the reset button on your PC is faster than properly shutting down
[14:09] <sil2100> Right, but I somehow remember that we had a bug somewhere that was actually triggered and hanging devices in the lab (not rebooting properly) when adb shell reboot was used
[14:09] <sil2100> Or something like that..!
[14:10] <ogra_> if a proper upstart shutdown is hanging thats a serious bug
[14:10] <ogra_> and needs immediate fixing
[14:10] <ogra_> iirc we had shutdown issues a year ago ... but not since
[14:13] <sil2100> I hope so!
[14:14] <plars> ogra_: we actually used to do adb shell reboot quite a while back, and I don't remember what exactly the problem was, but sometimes ran into issues with that. iirc it was you that said we should really be using adb reboot instead
[14:15] <sil2100> plars: right, that was the same thing I remembered, but I can't remember the actual problem we had...
[14:15] <plars> sil2100: ubuntu-ci-services-itself is certainly not a good place for this bug
[14:15] <ogra_> plars, right, that is fine ... but we need to run adb shell reboot once for processing the shutdown bits ... we *used* to have an issue there but that was fixed long agoo
[14:15] <ogra_> -o
[14:16] <plars> ogra_: but even adb shell reboot would only help us if we're really sure that ntpdate has run successfully. If we're going to do that, we may as well just run hwclock -w
[14:17] <ogra_> even if not i think the sw clock wont be at 1970
[14:18] <ogra_> (we dont read the hwclock on booot i think)
[14:18]  * ogra_ wonders whats up with his "o" key
[14:37] <sil2100> ogra_, plars: anyway, let's just have a bug filled in 'anywhere' and _at least_ workaround it for now for one promotion
[14:38] <sil2100> As the actual impact on users right now is minimal, it's just critical for our smoketesting
[14:41] <sil2100> boiko: so, related to those bottom edge things in messaging etc... you still having problems on getting those passing on CI?
[14:42] <boiko> sil2100: yes, but renato___ has an idea on how to fix that. I didn't know jenkins doesn't run using all packages up-to-date, so I'll add versioning to the deps that are causing it to fail
[14:42] <sil2100> Oh :)
[14:53] <sil2100> popey: btw.! Can you use your power somehow to get the new reminders-app released to the store?
[14:53] <popey> sil2100: what bzr rev?
[14:54] <sil2100> popey: 211, at least this one is said to fix the no-notes-visible bug
[14:54] <sil2100> As per https://bugs.launchpad.net/reminders-app/+bug/1351041
[14:56] <popey> sil2100: i did.. dunno why the version in store is old
[14:57] <popey> balloons: morning, could you push http://s-jenkins.ubuntu-ci:8080/job/reminders-app-click/lastSuccessfulBuild/artifact/out/com.ubuntu.reminders_0.5.211_armhf.click only to the store ?
[14:57] <davmor2> sil2100: A promotion tomorrow are you mad ;)
[14:57] <balloons> certainly
[14:57] <popey> thanks
[14:58] <balloons> popey, there's changes waiting review already
[14:58] <popey> hmmm
[14:58] <balloons> I upped it again, but check the whole queue
[14:59] <popey> ok
[14:59] <popey> thats better, thanks
[14:59] <popey> sil2100: approved to store, thanks balloons
[15:00] <davmor2> Saviq: so silo1 is ready for testing now right?
[15:03] <Saviq> davmor2, not just yet, building unity8 right now
[15:04] <sil2100> popey, balloons: thanks guys, one blocker less!
[15:04] <davmor2> Saviq: no worries
[15:05] <Saviq> uh oh
[15:05] <Saviq> ci-train.ubuntu.com down?
[15:08] <jgdx> trainguards https://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-utopic/2018 fails in publishing like so https://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-utopic/2018 – might cause some confusion if it becomes an issue for other jobs as well?
[15:08] <jgdx> err, second link http://pastebin.ubuntu.com/7971078/
[15:10] <sil2100> jgdx: I think the otto CI infrastructure is seperate from the CI Train, so most probably something for cihelp ;)
[15:11] <tedg> Saviq, Not getting it either
[15:11] <sil2100> hmmm
[15:11] <Saviq> yeah, train dead :|
[15:11] <jgdx> sil2100, okay
[15:12] <sil2100> Great :| Jenkins is dead
[15:12] <sil2100> Let's get IS on it
[15:13] <sil2100> Saviq, tedg: as per topic on IS: 'Known Issues: multiple prodstack services down'
[15:17] <tvoss> sil2100, silo 10 is good to go
[15:17] <sil2100> tvoss: ok, well, nothing we can do about it now. Jenkins is dead
[15:17] <tvoss> wtf?
[15:20] <plars> sil2100: do you already have an RT with IS on that?
[15:23] <sil2100> plars: no, but they say that everyone from IS is working on that, it's some bigger outage
[15:23] <plars> sil2100: ah, ok
[15:28] <plars> jgdx: does that happen on every run? I'm looking at the build publisher on s-jenkins and it seems ok
[15:29] <plars> jgdx: hopefully it was a temporary issue, possibly caused by the wider problems mentioned above
[15:30] <plars> jgdx: other runs seem to be ok
[15:33] <jgdx> plars, it has happened a couple of times for that branch
[15:33] <robotfuel> ev: ping, errors.ubuntu.com is not working, it's blank.
[15:34] <ev> robotfuel: working here
[15:34] <ev> the graph appears broken
[15:34] <ev> is that what you mean/
[15:34] <robotfuel> ev: yes the graph is blank :D
[15:34] <kenvandine> ev it wasn't finding any issues for me a few minutes ago
[15:35] <ogra_> ev, is someone looking into the fact that it doesnt work at all on the phone btw ?
[15:35] <ev> ogra_: elaborate please? :)
[15:35] <robotfuel> ev: it looks like it's working now.
[15:35] <ogra_> ev, well opening your own errors.u.c page on the phone has never shown any reports
[15:35] <ogra_> ev, via the system-settings UI
[15:36] <ev> does it just go to a blank page, what happens?
[15:36] <plars> jgdx: does it happen every time you run it, or does it just seem to be an occasional failure?
[15:36] <ogra_> ev, the browser URL is errors.u.c/user/$whoopsie_id ... but that one is always empty
[15:36] <ogra_> ev, we talked about that in malta :)
[15:36] <sil2100> Ok, it seems we're back
[15:37] <plars> jgdx: I don't think there's anything specific about that branch that can cause jenkins to have a publishing problem, but I've certainly seen temporary issues with jenkins publishing  before
[15:37] <sil2100> Saviq: can you confirm ci-train is back for you?
[15:37] <kenvandine> ev, it's working now though
[15:37] <plars> sil2100: looks up to me
[15:37] <sil2100> Excellent
[15:37] <Saviq> sil2100, yes \o/
[15:38] <jgdx> plars, okay. That build is failing consistently, but AFAICS there's no way to figure out why. I don't know if the publish error is hiding something from me? What do you think?
[15:38] <ev> ogra_: do you have a bug for this? slangasek and I just spoke about it and we don't think it's possible for RTM. This may be a task more for the desktop team though, given that it's in system-settings.
[15:38] <plars> jgdx: ok, so it is every time
[15:39] <ogra_> ev, well the whoopsie_id used there is correct ... and there are .uploaded files, that must be a server side thing
[15:39] <Saviq> damn ^W
[15:40] <ogra_> ev, it pbviously goes to the right URL, system-settings is fine
[15:40] <pmcgowan> ev, ogra_ this is what you were helping me with yesterday, the whoopsie id the phone gave me was not the same as the one the report went under
[15:40] <ogra_> *obviously
[15:40] <ogra_> pmcgowan, !
[15:40] <jgdx> plars, well, not sure the publishing fails each time, but that job fails due to tests and timeouts.
[15:40] <ogra_> pmcgowan, so i assume there is a bug open already then ?
[15:40] <jgdx> each time
[15:41] <pmcgowan> ogra_, not yet I think, at least I did not file one
[15:41] <plars> jgdx: ok, I thought you were asking about the publishing failure
[15:41] <sergiusens> plars: hey there, can I get ci MP builds for lp:ciborium ?
[15:42] <jibel> ogra_, likely one of bug 1339916 or bug 1340063
[15:42] <pmcgowan> yeah I did not reboot
[15:42] <plars> jgdx: I'll try to take a look, or find someone more familiar with otto to see if they can determine why it fails. do you have a way to retry this run? I didn't see anything that was clearly an error until the build publishing glitch
[15:42] <plars> sergiusens: can you point at where I'd find that?
[15:43] <pmcgowan> jibel, that second bug is the one I saw
[15:43] <plars> sergiusens: oh, you are asking to add it to cupstream2distro-config?
[15:43] <jgdx> plars, thanks. This is above the publish failure http://pastebin.ubuntu.com/7971359/
[15:43] <jgdx> plars, you can retry it by going to http://s-jenkins.ubuntu-ci:8080/job/ubuntu-system-settings-ci/1159/rebuild
[15:44] <tvoss> trainguards, can I get a silo for line 32?
[15:44] <sil2100> Sure, one moment
[15:45] <sil2100> tvoss: make sure you rebuild once we land silo 10
[15:46] <tvoss> sil2100, obviously :) me and my alter ego are in contact usually ;)
[15:46] <sergiusens> plars: yeah :-)
[15:46] <ogra_> jibel, thanks
[15:47] <sergiusens> plars: what I tried to imply with MP (Merge Proposal) ci :-)
[15:47] <Saviq> ough, is this a known issue sil2100 https://launchpadlibrarian.net/181665851/buildlog_ubuntu-utopic-amd64.unity8_8.00%2B14.10.20140806-0ubuntu1_FAILEDTOBUILD.txt.gz ?
[15:47] <plars> sergiusens: yeah, just not a piece that I've done much with, so it's not the first thought I have
[15:47] <sil2100> huh
[15:47] <sil2100> Saviq: that's something new
[15:51] <plars> sergiusens: I can give it a shot, but would be good if you could review the cu2d change to make sure I did something sane. which stack would this go in?
[15:51] <sergiusens> plars: stacks are not really important since the daily release days ended
[15:52] <sergiusens> plars: core, system or a similar name
[15:56] <sil2100> RIP daily-release
[15:57] <plars> sergiusens: head/services.cfg maybe?
[15:58] <Saviq> sil2100, could you please try and restart unity8 builds in silo 1, it built fine locally :/
[15:58] <sergiusens> plars: sounds good
[15:58] <sil2100> Saviq: ok, let me try that, hmm
[15:58] <Saviq> thanks
[16:00] <tedg> I'm a bit confused, seems the PPA can't install dbus-test-runner. Anyone else having an issue there?  https://launchpadlibrarian.net/181666621/buildlog_ubuntu-utopic-amd64.pay-service_2.0.0%2B14.10.20140806.2-0ubuntu1_FAILEDTOBUILD.txt.gz
[16:01] <sil2100> Saviq: it seems to be a real issue somewhere...
[16:01] <sil2100> Let me try looking into that after the meeting
[16:02] <Saviq> sil2100, thanks :|
[16:02] <tedg> Ah, Saviq has the same issue :-)
[16:02] <Saviq> tedg, yeah, same for unity8
[16:02] <Saviq> tedg, weird part is it built just fine in sbuild here
[16:03] <tedg> Yeah, curious if it's something in proposed. I think the PPAs pull it in.
[16:03] <sil2100> plars: meeting!
[16:03] <Saviq> tedg, ah indeed
[16:04]  * Saviq enables in sbuild
[16:04] <plars> sil2100: argh, brt
[16:06] <Saviq> tedg, confirmed
[16:07] <tedg> Saviq, Can you see which package changed?
[16:07] <tedg> I don't see anything obvious
[16:08] <Saviq> tedg, http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html#systemd
[16:09] <Saviq> tedg, there's things that depend on new sysvinit, but it's nowhere to be found
[16:10] <Saviq> yeah, missing a sync from debian
[16:13] <popey> balloons: music app seems to be doing something odd, clicking off-screen in this test.. http://91.189.93.70:8080/job/generic-mediumtests-utopic/1314/? from this branch https://code.launchpad.net/~vthompson/music-app/fix-swipe-delete-test/+merge/229718
[16:13] <popey> any ideas balloons ?
[16:14]  * balloons looks
[16:15] <balloons> ohhh it's music.. I wonder if there's custom swipe code
[16:25] <balloons> popey, yea looks custom. should probably just migrate to the helper
[16:27] <balloons> ohh, nvm.. the failures aren't about wipig
[16:30] <popey> balloons: ☻
[16:33] <balloons> popey, easy to see it fail.. It's odd it finds the object
[16:34] <popey> balloons: can you leave a comment on the branch if you find anything interesting? I need to go afk
[16:35] <balloons> popey, yes, I'll leave a comment.
[16:35] <bzoltan> sil2100:  I have the fix for the broken UbunuShapes (not for the autopilot bug, but the wrong size) in the line35
[16:37] <Saviq> enough today :(
[16:37] <sil2100> Ok, let me try looking why PPA builds are b0rken :|
[16:37] <brendand> plars, when is the RTM branch opening?
[16:38] <plars> brendand: don't know, I think someone just said friday perhaps?
[16:39] <plars> brendand: probably a better question for cjwatson
[16:39] <Saviq> sil2100, missing sysvinit merge from debian
[16:39] <Saviq> sil2100, pitti's been poked
[16:39] <sil2100> Aw come ooon
[16:39] <Saviq> sil2100, http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html#systemd
[16:40] <Saviq> sil2100, indeed
[16:40] <sil2100> :|
[16:40] <sil2100> Ok, so that means no testing for me then
[16:41] <sil2100> davmor2: ok, so, do you give a QA sign-off for silo 10 ? :)
[16:41] <sil2100> davmor2: you sure it won't break anything? :)
[16:42] <sil2100> davmor2: ...triple sure?
[16:42] <davmor2> sil2100: it already is signed off :)
[16:42] <sil2100> Let me publish then, and let's build a new image once it's in
[16:42] <davmor2> sil2100: it just turns the current location stuff into a trusted helper
[16:43] <bzoltan> sil2100: can a Silo do a build and be reconfigured and start to build an other package? I just added the -gles branch to the Sheet.
[16:48] <sil2100> cjwatson, slangasek: hi! We would need a core-dev+archive-admin +1 packaging ACK on diffs for silo 10
[16:48] <sil2100> cjwatson, slangasek: the diffs: https://ci-train.ubuntu.com/job/landing-010-2-publish/lastSuccessfulBuild/artifact/packaging_changes_trust-store_1.0.0+14.10.20140806.7-0ubuntu1.diff https://ci-train.ubuntu.com/job/landing-010-2-publish/lastSuccessfulBuild/artifact/packaging_changes_location-service_2.0.1+14.10.20140806-0ubuntu1.diff
[16:50] <sil2100> cjwatson, slangasek: trust-store adds a new binary package
[17:14] <kenvandine> oh sigh... held packages
[17:15] <sil2100> kenvandine: yeah ;/
[17:17] <kenvandine> sil2100, so what's the breakage?  i don't see an upload of gvfs
[17:20] <sil2100> kenvandine: so, from what I heard, a sysvinit merge is missing in the archive it seems
[17:20] <sil2100> kenvandine: pitti is on it it seems
[17:20] <kenvandine> yikes
[17:22] <bzoltan> sil2100:  would you please reconfigure the silo15?
[17:24] <sil2100> bzoltan: sure! About that building, that could be done but remember that the builds will fail
[17:24] <sil2100> (like in, PPA builds)
[17:25] <bzoltan> sil2100: I remember that it failed before. I have no idea why.
[17:25] <kenvandine> the archive is broken
[17:25] <kenvandine> nothing will build
[17:25] <sil2100> stgraber: hello! If you're around, maybe you could provide some +1 packaging ACK on a main package - we're also adding a new binary package to another one:
[17:25] <sil2100> stgraber: https://ci-train.ubuntu.com/job/landing-010-2-publish/lastSuccessfulBuild/artifact/packaging_changes_trust-store_1.0.0+14.10.20140806.7-0ubuntu1.diff and https://ci-train.ubuntu.com/job/landing-010-2-publish/lastSuccessfulBuild/artifact/packaging_changes_location-service_2.0.1+14.10.20140806-0ubuntu1.diff
[17:26] <sil2100> stgraber: so we need a core-dev with archive admin powers to review this ;)
[17:43] <sil2100> infinity: hey! As you'll be doing trainguarding this month probably, could you maybe review some packaging changes for some main packages? There's one that also adds a binary package, so a +1 from an archive admin is required:
[17:43] <sil2100> infinity: https://ci-train.ubuntu.com/job/landing-010-2-publish/lastSuccessfulBuild/artifact/packaging_changes_trust-store_1.0.0+14.10.20140806.7-0ubuntu1.diff and https://ci-train.ubuntu.com/job/landing-010-2-publish/lastSuccessfulBuild/artifact/packaging_changes_location-service_2.0.1+14.10.20140806-0ubuntu1.diff
[17:43] <sil2100> robru: could you make sure this gets reviewed, landed and an image kicked after it reaches the archive?
[17:46] <infinity> sil2100: Looking.
[17:46] <sil2100> infinity: thanks o/
[17:51] <infinity> sil2100: The first diff looks fine, except for the part where the changelog really doesn't document half of it. :P
[17:52] <infinity> sil2100: And second one is fine.
[17:52] <sil2100> hah ;)
[17:53] <sil2100> infinity: yeah, we have problems enforcing upstream developers to include all the info that's required in the changelogs... ogra_ was trying to inform developers, but yeah, it didn't seem to work ;p
[17:53] <sil2100> infinity: anyway, thanks!
[17:54] <ogra_> infinity, i wrote an angry mail already that i wouldnt ACK landings that have incomplete changelogs anymore :)
[17:55] <ogra_> it simply makes forensic work (which we have to do at least once a week) nearly impossible
[17:55] <infinity> Indeed.
[17:55] <infinity> sil2100: The simple way to enforce it is to enforce it.
[17:55] <infinity> sil2100: Not being funny, but it's not your job to let everything in, it's your job to let things in when they're correct.
[17:56] <sil2100> Yeah, we tend to reject some landings because of that, but today we're a bit less strict due to traincon ;)
[17:56] <sil2100> Since time is of the essence
[17:59] <rsalveti> sil2100: mind reconfiguring silo 18?
[17:59] <sil2100> rsalveti: sure
[17:59] <rsalveti> sil2100: thanks
[18:00] <rsalveti> seems it's still building
[18:01] <sil2100> rsalveti: reconfigured
[18:01] <rsalveti> sil2100: thanks
[18:04] <sil2100> robru: ah, and remember to poke ToyKeeper to do QA signing-off, but try keeping the velocity of landings low
[18:04] <sil2100> robru: prioritize blocker fixes!
[18:06] <sil2100> robru: and remember to keep pushing people for fixes ;) Be as annoying as possible ;p
[18:06] <sil2100> robru: (j/k!)
[18:07] <sil2100> o/
[18:10] <jgdx> plars, I've just updated a dep which was (could be) wrong in https://code.launchpad.net/~jonas-drange/ubuntu-system-settings/expandable-sim-name-editor/+merge/229450
[18:10] <plars> jgdx: ok, cool
[18:11] <plars> jgdx: hopefully that will work
[18:11] <jgdx> plars, hopefully!
[18:12] <jgdx> plars, can you explain "candidate_revision" to me?
[18:12] <jgdx> for e.g. http://s-jenkins.ubuntu-ci:8080/job/ubuntu-system-settings-ci/1161/rebuild/?
[18:13] <davmor2> robru: did sil kick another image in the end?
[18:13] <plars> jgdx: if I had to guess, the candidate revision is the one it's attempting to land, but where are you seeing that?
[18:14] <rsalveti> are we kicking another image now?
[18:14] <ogra_> davmor2, silo16 wouldnt mind a QA signoff :)
[18:14] <jgdx> plars, ^above link
[18:14] <ogra_> bah, the bot was faster than me :P
[18:14] <plars> jgdx: I don't see that anywhere
[18:15] <davmor2> ogra_: I'm not sure I trust the dev that threw the code together, can you vouch for him ;)
[18:16] <ogra_> davmor2, not sure ... but the code change is one line and some includes ... so i think even that untrustworthy guy cant do much wrong with that :)
[18:16] <plars> jgdx: but yeah, https://wiki.canonical.com/UbuntuEngineering/CI/Playbook/UpstreamMerger?highlight=%28candidate_revision%29#Running_a_ci_.2BAC8_autolanding_job_manually - candidate_revision is the most recently pushed revision number on the mp
[18:16] <plars> jgdx: oh, I wasn't logged in to s-jenkins so the link you sent was redirecting me
[18:17] <jgdx> plars, aaa
[18:17] <plars> jgdx: so, yeah that ^
[18:17] <davmor2> ogra_: installing if this introduces bugs though I'm sending pitti round to sort you out ;)
[18:17] <ogra_> lol
[18:17] <jgdx> plars, it seems that value is wrong then.. Could explain everything.
[18:19] <tedg> Are things building again?
[18:19] <tedg> Ah, missed pitti's ping. Cool.
[18:23] <bzoltan> rsalveti: do you know th ereason for this failure https://ci-train.ubuntu.com/job/landing-015-1-build/148/console ?
[18:28] <davmor2> ogra_: found a bug
[18:28] <ogra_> oh ??
[18:28] <ogra_> in my code ?
[18:28] <davmor2> ogra_: I rang pitti he is on his way ;)
[18:28]  * ogra_ makes coffee
[18:29] <davmor2> ogra_: so currently developer mode is on an dthe system is locked as there is no password.  I would of thought it would only lock with the developer mode off
[18:30] <ogra_> davmor2, the toggle only reflects the actual state, we will only default to off once the UI bits are landed
[18:30] <ogra_> then this state cant be possible anymore
[18:30] <ogra_> (we force it to on underneath the UI currently)
[18:31] <davmor2> ogra_: ah right in that case I forgive you, don't do it again ;)
[18:31] <ogra_> blame sergiusens :P
[18:31]  * ogra_ hides somewhere 
[18:32] <sergiusens> ogra_: what did i do?
[18:32] <ogra_> sergiusens, setting the default for apt :)
[18:32] <sergiusens> apt?
[18:32] <ogra_> sergiusens, ignore me ;)
[18:32] <ogra_> adb
[18:32] <davmor2> sergiusens: does it matter?  it's your fault :D
[18:32] <ogra_> heh
[18:32] <sergiusens> lol
[18:32] <sergiusens> I was complaining during lunch with rsalveti about how unfair traincon is to me
[18:32] <sergiusens> really affects me as I don't do the staging branch dance
[18:33] <ogra_> sergiusens, its just a matter of bribing the QA guys the right way
[18:33] <sergiusens> doesn't matter, still unfair
[18:35] <Saviq> robru, hey, can you please hit rebuilds of unity8 and indicator-network in silo 1, proposed's been fixed already
[18:35] <rsalveti> bzoltan: didn't find the tarball
[18:35] <Saviq> or well, trainguards ↑↑
[18:36] <davmor2> ogra_: open the lock security, set a 4-digit pin, turn off dev mode then turn it back on again.  Now go back into lock mode and select Swipe (no security)
[18:36] <rsalveti> bzoltan: you need to change the changelog header to match the upstream version
[18:37] <davmor2> ogra_: It appears you can never go back, the 4 digit pin becomes your password and is recorded there instead
[18:37] <rsalveti> bzoltan: https://code.launchpad.net/~bzoltan/ubuntu-ui-toolkit/sync_landing_0608/+merge/229827
[18:37] <Saviq> bzoltan, you need `dch -v  1.1.1181+14.10.20140806-0ubuntu1 ""`
[18:37] <rsalveti> bzoltan: see you're trying to sync with 1.1.1181+14.10.20140806-0ubuntu1, but the version for this package is 1.1.1181+14.10.20140804-0ubuntu3
[18:37] <Saviq> bzoltan, the part before -0ubuntu1 has to be the same as the non-gles package
[18:37] <rsalveti> yeah
[18:39] <ogra_> davmor2, go back to what ?
[18:40] <davmor2> ogra_: nevermind, it was because I had turned the developer mode back on so it wouldn't allow me to unset and go back to swipe
[18:40] <ogra_> ah
[18:41] <ogra_> i was just wondering ... since i jumped back and forth between all possible combinations all day today
[18:46] <Saviq> ogra_, could you please hit rebuilds in unity8 and indicator-network in https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-001/+packages
[18:47]  * Saviq really needs to get in on the ci train drivers, hate having to ask people about things like this :p
[18:52] <davmor2> popey: are there autopilot tests for reminders?   should they be landed into the autotesting framework if it is to be a core app?
[18:53] <davmor2> ogra_: silo 16 looks good and doesn't interfere with anything else
[18:54]  * ogra_ hugs davmor2 
[18:54] <balloons> davmor2, yes and yes. As to why they aren't there, we're working on it. They require new tooling for CI
[18:54] <popey> ^
[18:54] <ogra_> Saviq, gimme a bit, just having breakfast (yes i know how late it is :) )
[18:54] <Saviq> lol
[18:55] <davmor2> popey:  wow you sounded just like balloons then :D
[18:55] <plars> davmor2: there are, we're working to get them into smoke, but that won't get them into autolanding. Not sure what needs to happen to make that work, but I'd bet fginther has plans for it already since he did a lot of the work to make autopkgtests work in smoke
[18:55] <balloons> mmm.. the full answer ^^
[18:55] <balloons> now popey is sounding like plars
[18:55] <plars> he's a ventriloquist
[18:56] <plars> which, I guess makes me a dummy
[18:56] <ogra_> yummy  ...
[18:56] <ogra_> smoked images
[18:56] <davmor2> popey: with a  talent like that you should do a show at the next sprint :D
[18:56] <popey> http://drool.popey.com/
[18:56] <balloons> and the kneeslapper of the day award goes to . . . plars
[18:57] <davmor2> plars: I'll leap to your defence, being as no one else has,  No your not a dummy :)
[18:57] <davmor2> s/your/you're
[19:06] <ogra_> Saviq, hmm, i just found out that i have no clue how to trigger a rebuild with the new train UI :/
[19:06] <Saviq> ogra_, oh no
[19:06] <Saviq> ogra_, just rebuild in PPA
[19:07] <ogra_> ah, seems i found it
[19:07] <Saviq> ogra_, in the failed builds in https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-001/+sourcepub/4336601/+listing-archive-extra and https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-001/+sourcepub/4336570/+listing-archive-extra
[19:07] <ogra_> you want the indicator and unity8. right ?
[19:07] <Saviq> ogra_, yes please
[19:07] <Saviq> ogra_, the unity8 ones that are dep wait you can leave out
[19:07] <Saviq> they'll dep-wait anyway
[19:07] <Saviq> but the others were failing because of a proposed issue that's been solved, so no need to reupload, rebuild is enough
[19:08] <ogra_> seems to be running
[19:08] <ogra_> (if a blinking giant dot means running at least :P )
[19:20] <tvoss> Saviq, ogra_ can I press merge & clean myself?
[19:20] <ogra_> tvoss, no idea
[19:22] <ogra_> tvoss, i guess if it lets you you can :P
[19:22] <Saviq> tvoss, sure, it's something you do yourself
[19:23] <ogra_> Saviq, silo 1 doesnt look so great now :(
[19:23]  * ogra_ hopes he didnt mess up something 
[19:23] <Saviq> ogra_, no, you ran a build job, that I can do myself
[19:24] <Saviq> ogra_, what I need you to do, assuming you actually have the rights to do it...
[19:24] <Saviq> ogra_, is to press rebuild on https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-001/+build/6247005
[19:24] <ogra_> oh. you wanted me to click on each and every package in the PPA and click rebuild ?
[19:24] <Saviq> ogra_, just two packages * arches, something like 9 in total :D
[19:24] <ogra_> yeah, i definitely can do that ... i thought you wanted some fancy CI stuff :P
[19:25] <Saviq> ogra_, thank you :)
[19:26] <ogra_> seems to have failed again (pretty quickly)
[19:26] <Saviq> uuuugh
[19:27] <ogra_> fun ... given that diff
[19:27] <Saviq> stoopid proposed :|
[19:27] <ogra_> The following packages have unmet dependencies:
[19:27] <ogra_>  libconnectivity-cpp-dev : Depends: libconnectivity-cpp0 (= 0.0.1+14.10.20140722-0ubuntu1) but it is not going to be installed
[19:27] <ogra_> E: Unable to correct problems, you have held broken packages.
[19:28] <Saviq> ogra_, yeah, trying to reproduce here
[19:28] <Saviq> huh, gvfs-backends should've been fixed by now!
[19:29]  * Saviq is gonna cry
[19:30] <Saviq> and I don't get it, it installs fine here by now
[20:03] <brendand> robru, is silo009 going to land today?
[20:07] <Wellark> just for the record I have not touch libconnectivity-cpp
[20:07] <Wellark> but I'm going to touch it this night
[20:07] <Wellark> oh, it's already night time..
[20:07] <Wellark> anyway
[20:10] <kenvandine> yay, systemd built... maybe the archive is *unbroke*
[20:22] <tvoss> trainguards, could you reconfigure silo 32, please?
[20:23] <pmcgowan> kenvandine, whats holding up jonas dual sim branch, ci run?
[20:23] <kenvandine> which dual sim?
[20:23] <kenvandine> there's one that has a prereq on the reset branch
[20:24] <pmcgowan> kenvandine, sim name editor?
[20:24] <kenvandine> ah, that's just the UI tweaks for the label editor
[20:24] <kenvandine> i'll check it out
[20:25] <kenvandine> pmcgowan, his other dual sim related branch requires the reset branch
[20:25] <pmcgowan> kenvandine, ah, ok, man cant wait for all that to go in
[20:25] <kenvandine> yeah
[20:25] <kenvandine> pmcgowan, ok, the sim name editor branch is still failing CI
[20:26] <kenvandine> which... we know everything for settings fails CI
[20:26] <kenvandine> but... the failures are in the cellular tests
[20:27] <kenvandine> so i'm ok with landing unrelated things knowing the failing tests aren't related to the branch, but not branches that are in the same place as the failing tests
[20:27] <pmcgowan> I thought john's layout change went in yesterday but dont see it
[20:27] <kenvandine> it did
[20:27] <kenvandine> not in an image yet
[20:27] <pmcgowan> ok
[20:27] <rsalveti> ogra_: robru: mind if a trigger a new image?
[20:29] <rsalveti> will do once rmadison tells me ubuntu-system-settings is finally in
[20:36] <kenvandine> pmcgowan, the broken archive is not going to help us land any of this :/
[20:36] <pmcgowan> kenvandine, still broke?
[20:36] <kenvandine> yeah
[20:37] <pmcgowan> kenvandine, remind me why we build against proposed?
[20:37] <kenvandine> the systemd/sysvinit fix is in... but still busted
[20:37] <kenvandine> because other depends may be in proposed :)
[20:37] <rsalveti> crap
[20:49] <sergiusens> robru: cjwatson is there a process to unsupport an architecture? I'm switching from gccgo to golang-go (at least temporarily)
[20:58] <pmcgowan> kenvandine, what is the archive breakage preventing right now, the builds themselves?
[21:03] <kenvandine> yeah
[21:03] <kenvandine> builds fail in the PPA
[21:04] <pmcgowan> kenvandine,what exactly, the build or making a build environment?
[21:04] <kenvandine> build env
[21:04] <pmcgowan> yeha
[21:04] <kenvandine> it can't install the build depends
[21:04] <pmcgowan> same as a local build chroot, where the rule is dont use proposed
[21:04] <kenvandine> dependency failures, uninstallable packages
[21:04] <kenvandine> yup
[21:05] <pmcgowan> why dont we have a canned build env from known state
[21:06] <kenvandine> that wouldn't be good
[21:06] <kenvandine> it would need to be updated every time a package in it gets updated
[21:06] <kenvandine> at least libs and stuff
[21:06] <kenvandine> at least for archive builds
[21:07] <kenvandine> it would be nice for app developers doing builds though
[21:07] <pmcgowan> its the same thing we do with local builds
[21:07] <kenvandine> building for the archive we need to be sure linking is down against what's in the archive
[21:08] <kenvandine> s/down/done
[21:08] <pmcgowan> I may not have the answer but this is intolerable
[21:09] <kenvandine> pmcgowan, extremely frustrating
[21:09] <kenvandine> i tried bisecting based on what's hit proposed today
[21:09] <kenvandine> systemd has to be the culprit
[21:09] <kenvandine> but pitti uploaded a fix for that
[21:09] <kenvandine> and... Saviq has tried just adding the packages it complains explicitly and it all magicly works
[21:09] <kenvandine> makes me think there is some apt dep resolution bug
[21:10] <pmcgowan> I have seen that before yeah
[21:10] <kenvandine> it feels like half a day here wasted, which is frustrating
[21:53] <kgunn> robru: hey, i gotta twin i need to add to silo 2, can you help with the reconfig ?
[21:59] <robru> kgunn, heya
[21:59] <robru> can do
[22:00] <boiko> robru: silo 9 has some autopilot fixes and some packaging deps only, it is all tested and working
[22:01] <sergiusens> robru: did you see my question above?
[22:01] <robru> sergiusens, yeah, just reading scrollback now. is the problem that golang-go supports fewer arches than gccgo?
[22:02] <sergiusens> robru: yes
[22:02] <robru> rsalveti, image is fine with me. did you do it yet?
[22:02] <sergiusens> robru: I lose the weird ones basically
[22:02] <sergiusens> robru: so no arm64 or power*
[22:02] <robru> sergiusens, right, so that's fine by me, but it's not up to me ;-) you need somebody like cjwatson or infinity to force that through.
[22:03] <sergiusens> robru: won't jenkins fail the silo saying some arches failed to build though?
[22:03] <robru> and by "force that through" I mean they need to delete the existing binaries from the archive so that the archive won't block on that arch regression.
[22:03] <sergiusens> robru: or maybe it's buggy, as it says it's building still even though it finished hours ago: https://ci-train.ubuntu.com/job/landing-020-1-build/82/console
[22:03] <robru> sergiusens, not if an archive admin fixes it first
[22:04] <sergiusens> robru: ack
[22:04] <sergiusens> robru: so good thing I also ping cjwatson about it :-)
[22:04] <robru> sergiusens, yeah
[22:04] <rsalveti> robru: no, not sure if archive is in a proper state
[22:05] <robru> oh right
[22:05] <sergiusens> robru: based on doko's reply to my email on canonical tech, I say it should be unblocked easily
[22:15] <robru> davmor2, no an image wasn't kicked
[22:15] <robru> Saviq, did you get your rebuild of silo 1? i just got back
[22:16] <Saviq> robru, old news, silos are broken anyway due to systemd being stuck in proposed, 'causing systemd to be stuck in proposed, causing systemd to be stuck in proposed, causing systemd to be stuck in proposed... and so on
[22:17] <robru> Saviq, that's a shame, I hope it doesn't cause systemd to get stuck in proposed...
[22:17] <Saviq> I'm afraid it might
[22:18] <sergiusens> Saviq: we should integrate unity8 into systemd
[22:18] <Saviq> sergiusens, to get stuck in proposed?
[22:18] <sergiusens> Saviq: to make the archive just one package
[22:18] <Saviq> sergiusens, that's stuck in proposed? :D
[22:19] <sergiusens> Saviq: well once it's one package, it will be easier :-P
[22:26] <robru> racarr, ping about that stuff. how's it going?
[22:38] <slangasek> jdstrand, plars, robru: so sil2100's mail seems to have a dangling reference to this bug filed for the apparmor denials.  Does someone have the link?
[22:39] <plars> slangasek: sure, one sec
[22:39] <robru> slangasek, hm, i think plars has it ;-)
[22:40] <plars> slangasek: robru: https://bugs.launchpad.net/ubuntu/+bug/1353591
[22:40] <plars> I'll cc it to the email, didn't see that it didn't get pasted
[22:41] <plars> slangasek: oh, no it's in the email. I see it
[22:42] <plars> slangasek: he just put it in an odd place :)
[22:42] <slangasek> oh?  I found a [3] footnote but it was a different bug
[22:42] <cjwatson> kenvandine: it's perfectly possible to choose to build against release rather than proposed on a silo-by-silo basis, but I would certainly maintain that it is not generally appropriate
[22:42] <plars> slangasek: yeah, it's further down... confusing
[22:42] <cjwatson> sergiusens: It's possible for us to remove binaries from the release pocket, which would permit that, but I'd need to look into the specifics to see whether there are any consequential problems
[22:43] <cjwatson> sergiusens: what silo is this?
[22:43] <cjwatson> so has the stack of build failures from scrollback been unblocked?
[22:44] <robru> cjwatson, as far as I can tell, no, archive is broken due to systemd and builds are failing
[22:44] <cjwatson> robru: do you have an example?  I saw an upload of systemd that purported to fix it
[22:45] <robru> cjwatson, well I'm looking at proposed-excuses and it says systemd not considered due to a regression in udisks2
[22:45] <cjwatson> And it's only currently blocked in -proposed by some failing autopkgtest or other that isn't hugely relevant for this purpose
[22:45] <cjwatson> That doesn't affect builds
[22:45] <cjwatson> Do you have a current example of a failed PPA build?
[22:45] <robru> cjwatson, sorry I'm not up to speed on all the builds because I've been at the doctor most of the day
[22:45] <Wellark> are the builds working again and if not is there any ETA?
[22:46] <robru> cjwatson, but silo 17 and 1 seem to be failing
[22:46] <Wellark> it's almost 2am and I'm thinking if I should to to bed or not
[22:46] <robru> cjwatson, also the silo sergiusens was asking about is 20
[22:46] <cjwatson> Wellark: they should be working now but I'm just about to check
[22:46] <Wellark> cjwatson: \o/
[22:46] <cjwatson> ok, 17 is old, will retry in the PPA
[22:46] <Wellark> how is silo 1 looking? (haven't checked..)
[22:47] <cjwatson> Wellark: I'll check that in a moment
[22:47] <cjwatson> but that failure's old too
[22:49] <cjwatson> hm, still maybe something wrong, let me see
[22:51] <cjwatson> huh, what
[22:51] <cjwatson> ubuntu-archive@snakefruit:~$ chdist apt-get utopic-proposed-ppc64el install gvfs-backends gvfs gvfs-daemons udisks2 udev parted libpam-systemd procps systemd initscripts
[22:51] <cjwatson> fails
[22:51] <cjwatson> ubuntu-archive@snakefruit:~$ chdist apt-get utopic-proposed-ppc64el install gvfs-backends gvfs gvfs-daemons udisks2 udev parted libpam-systemd procps systemd initscripts upstart
[22:51] <cjwatson> succeeds
[22:51] <cjwatson> the first failure was "initscripts : Depends: upstart but it is not going to be installed"
[22:52] <robru> kenvandine, ^^ can you fill cjwatson in on the details?
[22:52] <cjwatson> robru: no need
[22:52] <cjwatson> kenvandine: ^-
[22:53] <cjwatson> I saw kenvandine's comments earlier and I seem to be retracing the same steps
[22:54] <cjwatson> It *might* just need a chroot upgrade to get past this
[22:56] <cjwatson> infinity: ^- you still around?
[22:58] <cjwatson> http://paste.ubuntu.com/7974417/ is the apt debug output from the first command above
[22:58] <ToyKeeper> For traincon, there has been a surprising lack of silos to test.  (I'm not complaining though!)
[22:59] <cjwatson> Looks like it's picking systemd-sysv as a preferred alternative to systemd-shim and then failing to resolve it, but not totally clear on why
[23:00] <cjwatson> so, I can unblock silos 1 and 17 by temporarily switching them to -proposed, but that isn't a long-term solution
[23:00] <infinity> cjwatson: I am, ish.
[23:01] <infinity> cjwatson: Erm, aren't all silos supposed to have -proposed enabled anyway?
[23:01] <cjwatson> infinity: Any ideas on the above mess?  I'm not sure if a chroot upgrade will actually help given the debug output there
[23:01] <cjwatson> infinity: They do.  In this case that's why they're failing.
[23:01] <cjwatson> (Though in general I think it's a good thing, otherwise transitions involving anything in a silo would be super-painful.)
[23:02] <infinity> cjwatson: It was "so, I can unblock silos 1 and 17 by temporarily switching them to -proposed" I was reponding to..
[23:02] <cjwatson> Er, sorry, braino
[23:02] <cjwatson> By temporarily switching them to release
[23:03]  * infinity digs a bit.
[23:03] <cjwatson> 1 and 17 switched, builds retrying
[23:03] <infinity> If this mess is in proposed, manual upgrading won't help, unless I upgrade the chroots to proposed, which I don't usually.
[23:03] <cjwatson> (You can always copy what's in 17 to another PPA with -proposed enabled if you want to debug)
[23:04] <cjwatson> infinity: chdist might not be totally reliable for this, as it's starting from a blank slate rather than from a system with -proposed enabled.
[23:04] <infinity> Right, I'm trying a chroot.
[23:04] <infinity> Is this failing on upgrade, or build-dep install?
[23:04] <infinity> Log somewhere?
[23:05] <cjwatson> infinity: But a good test is whether "apt-get build-dep content-hub" works with -proposed enabled after upgrading the chroot to current release.
[23:05] <infinity> cjwatson: Upgrading to proposed, actually.
[23:05] <cjwatson> I've probably just killed the logs, but it amounts to apt-get build-dep content-hub (or indicator-network, or unity8)
[23:05] <cjwatson> infinity: Is there anything to upgrade in release first?
[23:06] <infinity> cjwatson: I mean, the reproducer would be to upgrade to proposed, since that's what lp-buildd does.
[23:06] <cjwatson> Oh true.
[23:06] <infinity> cjwatson: There are certainly things to upgrade in release first, which might help work around it, but I'd rather understand the bug first.
[23:06] <cjwatson> I agree, it's just something to experiment with
[23:06] <infinity> If it's another apt issue, fine.  If it's something wrong with the packaging that's going to break upgrades from 14.04, we should fix it, not force it.
[23:07] <cjwatson> I couldn't see anything obvious with the packaging, but that's not definitive
[23:07] <robru> ToyKeeper, yeah, only things that aren't critical bugfixes require qa signoff, so I guess we had a well-behaved traincon, only silos with fixes were considered so far ;-)
[23:08] <cjwatson> Bumped versioned dep systemd-shim, but the target version exists in utopic
[23:08] <infinity> cjwatson: Comes down to libpam-systemd forcing curious changes to the install list.
[23:09] <cjwatson> It does?  That's just bumped systemd version to match the source, added dbus, bumped systemd-shim version (to one available in utopic)
[23:10] <cjwatson> At least on amd64
[23:10] <infinity> http://paste.ubuntu.com/7974475/
[23:11] <cjwatson> infinity: Maybe removing the no-longer-essential init from the chroots would help?
[23:11] <infinity> cjwatson: Might do.
[23:12] <infinity> Nope.
[23:12] <infinity> Same result.
[23:13] <cjwatson> infinity: Adding systemd-shim to your list (without libpam-systemd) also fixes it
[23:13] <cjwatson> Which matches the apt debug output, which starts going wrong where it tries to install systemd-sysv
[23:13] <infinity> http://paste.ubuntu.com/7974485/
[23:14] <cjwatson> Oh
[23:14] <infinity> Yeah.  Can be narrowed down, at least, to just trying to install udisks2, which is a lot fewer interacting packages than gvfs.
[23:14] <cjwatson> The problem is that systemd-sysv now exists in -proposed
[23:14] <cjwatson> It didn't before, so dependencies didn't pick it
[23:15] <infinity> Oh, we might need something high enough up the stack to just prefer systemd-shim?
[23:15] <cjwatson> Switching libpam-systemd's alternative order would probably bodge it for now
[23:15] <infinity> To hint apt in the right direction?
[23:15] <cjwatson> Obviously not correct, but I'm pretty sure it would do for the moment
[23:15] <infinity> I'm not sure if there is a correct here.
[23:16] <infinity> I think the assumption should be that an installed system comes with one alternative or the other already working.
[23:16] <infinity> So, this is only a switching problem.
[23:16] <infinity> But the upgrade is irksome.
[23:17] <cjwatson> infinity: You're probably right, but it needs more of an expert than I am to get the architecture right.  In the meantime can you review http://paste.ubuntu.com/7974519/ ?
[23:17] <infinity> But yeah, twiddling the order on libpam-systemd would probably do it for now.
[23:18] <infinity> cjwatson: lgtm.
[23:18] <infinity> And now to run off to that late lunch that kept getting later...
[23:19] <cjwatson> Ah, good, robru did a watch-only build on those, thanks
[23:19] <cjwatson> infinity: Cool, thanks
[23:19] <cjwatson> Nose held, uploaded
[23:19] <robru> cjwatson, you're welcome
[23:20] <cjwatson> And I've flipped those PPAs back to their previous configuration before we forget
[23:24] <racarr> robru: Pong...sorry wasnt watching this channel
[23:24] <racarr> you mean the bottom edge swipe stuff?
[23:25] <robru> racarr, yeah I was told to ping you about that, we want to get that in asap ;-)
[23:25] <racarr> It's difficult
[23:25] <racarr> I think i've located it but no chance of a fix today
[23:25] <racarr> as far as I can tell (and im not 100% sure yet)
[23:25] <racarr> the emulator just doesn't support resizing egl surfaces
[23:25] <racarr> in it's egl impl
[23:25] <racarr> and we now resize the surfaces on startup as part of a weird dance
[23:25] <racarr> to solve a bunch of other bugs
[23:26] <robru> racarr, hmmm, ok.
[23:26] <cjwatson> sergiusens: removed now-unbuildable ciborium binaries for you; hopefully that will publish before citrain's current build finishes and it will notice, otherwise abort the build if necessary and do a watch-only build
[23:26] <cjwatson> (once rmadison is up to date)
[23:27] <racarr> robru: Should have something more definitive by tomorrow or later today if I get lucky but I think I am unlikely to make much more progress until I can poke someone who knows a little more about the android drivers
[23:27] <robru> racarr, before you EOD, can you reply to the landing team email with your findings?
[23:28] <racarr> robru: Is that better than commenting on the bug?
[23:28] <robru> racarr, hm, I guess commenting on the bug is better.
[23:28] <racarr> :)
[23:28] <racarr> I just made one basic comment...hopefully I should find out a little more before EOD
[23:32] <cjwatson> ^- that's just a build1 upload which we'd normally discard; I'll force