[05:03] <pitti> Good morning
[06:36] <dholbach> good morning
[06:45] <dpm> seb128, wgrant, morning. Eventually, the manual .po file uploads for evolution got imported: https://translations.launchpad.net/ubuntu/wily/+source/evolution/+imports
[06:46] <seb128> dpm, hey, good to know
[06:46] <dpm> there is a caveat, though
[06:47] <dpm> after long waiting and seeing that the .po files did not get imported automatically after the template had, I had to explicitly choose the template they belonged to
[06:47] <dpm> then they got imported
[06:47] <dpm> perhaps it was me being impatient and they would have been auto-imported, but after a few hours waiting, I wasn't sure
[08:26] <TJ-> Has anyone seen this package-hold for aptdaemon. I'm not making sense of it since there's nothing in /var/lib/dpkg/status that shows the version dependency mentioned: "python3-aptdaemon : Depends: gir1.2-packagekitglib-1.0 (< 0.9) but 0.9.5-0ubuntu1 is to be installed"
[08:40] <TJ-> Ahhh bug 1496292
[08:45] <seb128> TJ-, you used wily-proposed and got a buggy version that got deleted later, you might want to sudo apt-get install python3-aptdaemon/wily
[08:50] <TJ-> Yeah, I figured it out once I found the older changelog entry
[08:58] <tjaalton> is there a way to narrow down the updates that hit wily around Sep 10th, because something triggers X crashes since then
[09:01] <TJ-> tjaalton: Other that /var/log/apt/history* ?
[09:01] <seb128> tjaalton, wily-changes list?
[09:01] <tjaalton> TJ-: not mine, trying to figure out distro-wide what happened when
[09:01] <seb128> tjaalton, which ones? do you have a bt?
[09:02] <tjaalton> seb128: well kinda, gives some sort of an idea
[09:02] <tjaalton> at least I can rule out some X stuff..
[09:02] <tjaalton> dupes of https://bugs.launchpad.net/ubuntu/+source/xorg-server/+bug/1237904
[09:03] <tjaalton> 1495049 and up
[09:03] <tjaalton> which was Sep 14th actually
[09:04] <seb128> tjaalton, the first issue on e.u.c?
[09:04] <seb128> that seems to be gdm trying to start a xserver when there is already one active
[09:04] <seb128> they all have gdm options in their command line
[09:05] <tjaalton> ah, that would be a good hint
[09:05] <seb128> I pinged darkxst several time about it
[09:05] <seb128> installing gdm on my test laptop also bricks it
[09:05] <seb128> e.g no dm is starting
[09:06] <seb128> so I think it's gdm screwed up
[09:07] <seb128> tjaalton, the xorg logs all have
[09:07] <seb128> [   451.402] _XSERVTransSocketUNIXCreateListener: ...SocketCreateListener() failed
[09:07] <seb128> [   451.402] _XSERVTransMakeAllCOTSServerListeners: server already running
[09:07] <seb128> [   451.403]
[09:07] <tjaalton> good
[09:07] <seb128> btw the xorg hook seems grumpy
[09:07] <seb128> https://launchpadlibrarian.net/221676859/HookError_source_xorg_server.txt
[09:07] <seb128>   File "/usr/lib/python3/dist-packages/problem_report.py", line 627, in __setitem__
[09:07] <seb128>     assert k.replace('.', '').replace('-', '').replace('_', '').isalnum()
[09:07] <seb128> AssertionError
[09:08] <seb128> tseliot, ^
[09:16] <tseliot> seb128: yay, something else failed. I'll look into that
[09:16] <seb128> tseliot, thanks
[09:20] <tjaalton> gdm didn't change since Aug 7th
[09:20] <tjaalton> but it's some other change that triggers the bug then
[09:33] <TJ-> tjaalton: looking at the DpkgLog attached to 1495049 shows the dates/packages recently updated; nothing entirely obvious, but libc-bin, dbus, systemd might be suspect
[09:37] <darkxst> seb128, I still havent been able to reproduce on either of my systems
[09:37] <seb128> darkxst, seeing the number of report you must have some users who can?
[09:43] <darkxst> seb128, I will ask around
[09:44] <seb128> thanks
[09:45] <darkxst> seb128, do you have the e.u.c link handy?
[09:47] <seb128> darkxst, https://errors.ubuntu.com/problem/27941445d596fb71be692b9008c0847d2ce6c428
[09:50] <darkxst> emailed to QA team
[09:53] <tjaalton> thanks
[09:54] <TJ-> The common issue from the Xorg logs is the first session starts on VT7, fails, and the replacement starts on VT2
[09:56] <seb128> darkxst, tjaalton, ^
[09:56] <darkxst> TJ-, gdm should start on vt7
[09:56] <darkxst> the user session should start on vt2
[09:59] <TJ-> darkxst: but should the process on VT7 remain running?
[10:01] <darkxst> TJ-, yes
[10:04] <darkxst> and actually gdm should be running on wayland
[10:04] <darkxst> for the FOSS drivers atleast
[10:04] <darkxst> maybe it isnt since we don't seed xwayland yet
[10:46] <TJ-> I'd suspect the first X server on VT7 owns/uses "/tmp/.X11-unix/" sockets and the 2nd server on VT2 tries to use the same socket
[10:54] <cjwatson> should be a different name under /tmp/.X11-unix/ for each server - X0, X1, etc.
[10:59] <TJ-> cjwatson: that's what I'd expect; unfortunately that info isn't available in the bug report logs. "_XSERVTransMakeAllCOTSServerListeners: server already running" certainly suggests something has the socket the user session wants
[11:14] <seb128> TJ-, would the stacktrace help? if you tag the bug apport-request-retrace you can get a next one from the next duplicate (rather than having it cleaned by the retracer)
[11:15] <TJ-> seb128: I was just wondering how we could do that; I'm not convinced the stacktrace in the master report is the same. I've been picking dupes at random and those after 1495049 all look different, most obviously ProcCmdline
[11:15] <seb128> no, the master is not the same
[11:15] <seb128> e.u.c launchpad batch OsAbort() errors it seems
[11:16] <seb128> the master was a VT_ACTIVATE fail
[11:16] <TJ-> added the tag
[11:16] <seb128> but the recent ones are all gdm ones
[11:16] <seb128> so I expect a new retrace is going to give us a new bt
[11:16] <seb128> showing the gdm issue
[11:18] <TJ-> I use KDE so I'm not familiar with the way gdm is running 2 X servers for (presumably the greeter) and the user session, but the ProcCmdline entries of all the dupes I've looked at are referring to the gdm owned X on VT7
[12:38] <psychic_> quit
[13:16] <ogra_> grrr ... why do pipes not work in the initrd when i use libc tools but work when using the busybox commands
[13:18]  * ogra_ wonders if there is a trick 
[13:28] <infinity> ogra_: One tool printing to stderr and the other to stdout?
[13:28] <ogra_> hmm
[13:28] <infinity> (Just a guess)
[13:31] <ogra_> infinity, hah, no ... but seems - means something else
[13:32] <ogra_> "/bin/wget -O - http://192.168.2.74:1500/ubuntu-15.04-snappy-armhf-rpi2.img.xz | /usr/bin/xzcat | dd of=/dev/mmcblk0" works fine
[13:32] <ogra_> "/bin/wget -O - http://192.168.2.74:1500/ubuntu-15.04-snappy-armhf-rpi2.img.xz | /usr/bin/xzcat - | dd of=/dev/mmcblk0" doesnt
[13:32] <pitti> ah, busybox wget not accepting - ?
[13:32] <pitti> uh, odd
[13:32] <ogra_> no, xzacat not accepting -
[13:32] <ogra_> *xzcat
[13:33] <ogra_> using the libc dd in that line gets even more weird :)
[13:33] <ogra_> but yay ... my initrd installer constriction works
[13:33] <davmor2> ogra_: lets just stick with because it hates you ;)
[13:33]  * ogra_ wraps more duct tape around it 
[13:33] <infinity> An initrd that grabs an unathenticated http URL and blats it to disk seems super secure.
[13:34] <ogra_> yeah, awesome, aint it ? :)
[13:34] <infinity> Also, that should be armhf+rpi2 ...
[13:34] <davmor2> infinity: shhhhh you're not meant to notice that
[13:34] <ogra_> it is the same machine the serial cable is attached to ... for image tests
[13:34] <ogra_> infinity, it so should ...
[13:35] <ogra_> well, effectively armhf+raspi2
[13:35] <infinity> Or that, yes.
[13:35] <ogra_> but that name was already used when i antered the snappy team :/
[13:36] <ogra_> like the BBB image is called armhf-beagle or some such weird thing
[13:37]  * ogra_ is curious how long that dd will actually take ... 
[13:38] <ogra_> (really bad that busybox dd doesnt know the blocksize arg)
[13:43] <seb128> wgrant, why did you reject https://code.launchpad.net/~cjohnston/launchpad/1260760-unset-source/+merge/229462? (I guess it was wrong bug would have been nice to comment about what was the issue)
[13:43] <seb128> or asked differently, what launchpad dev to I need to pay a beer to to get https://bugs.launchpad.net/launchpad/+bug/1260760 fixed?
[13:59] <ogra_> wheee !
[13:59] <ogra_> it boots (and only took ~20min for dd)
[14:57] <dholbach> barry, do the two python sessions need a blueprint in LP for tracking work items and stuff?
[14:57] <dholbach> (just noticed them in summit)
[14:59] <barry> dholbach: maybe couldn't hurt although we all know what needs to be done.  mostly i want to make sure everyone's aligned with the plan
[15:00] <dholbach> ok, wfm
[15:00] <barry> brb
[16:17] <infinity> rbasak: I have a weird juju testsuite regression in wily that's about to hold up the release, who do I talk to to debug this ASAP?
[16:17] <rbasak> infinity: sinzui. I've asked him to join.
[16:19] <rbasak> infinity: BTW, I'm quite bothered by https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1490110. It breaks upgrades for anyone with lxc and juju-local installed.
[16:19] <rbasak> Which is Every Charm Developer I think.
[16:20] <cjwatson> juju bootstrap totally works here on current wily, notwithstanding what the test suite is saying, but then I don't understand why it's failing
[16:20] <sinzui> hi infinity can I help with a juju issue
[16:20] <infinity> sinzui: Hey.  What do you make of https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-wily/wily/amd64/j/juju-core/20151021_155639@/log.gz ?
[16:20] <rbasak> I can tell you what future-{local,manual}-provider do.
[16:21] <rbasak> They fake a future release to make sure that the development release isn't broken as soon as it opens.
[16:21] <rbasak> panic: osVersion reported an error: Could not determine series
[16:21] <rbasak> That may be related.
[16:21] <rbasak> The update to distro-info-data may be related.
[16:21] <cjwatson> oh, I didn't notice that that was future-
[16:21] <infinity> So, it's faking a future release incorrectly? :P
[16:22] <rbasak> Hmm, maybe.
[16:22] <rbasak> Do you happen to have a shell at the point the test breaks? Otherwise I can do it.
[16:22] <cjwatson> I suspect that it is faking something insufficiently far in the future
[16:22] <cjwatson> or maybe is confused by the last line having LTS in it?
[16:23] <rbasak> It seems likely to me that this is a bug in my faking-the-future code.
[16:24] <sinzui> Juju has a very stupid policy to hard coding known releases and treating them as everything that will ever exist. The reading of ubuntu.csv by juju ti to get a more current list.
[16:24]  * rbasak is busy reproducing
[16:25] <sinzui> cjwatson: infinity rbasak: this is the bug I think you are seeing https://bugs.launchpad.net/juju-core/+bug/1465317
[16:25] <infinity> rbasak: Okay, so I think this bears investigation, as the test is clearly doing an oops of some sort, but now that I understand the scope, I'm also going to ignore the failure.  Thanks for pointing out the future bit.
[16:26] <cjwatson> the hacked up file seems to have one more field than it should
[16:27] <cjwatson> but I think it's always been that way, so WTF
[16:27] <cjwatson> echo "${version},${codename},${series},${created},${release},${eol},${eol}" >> /usr/share/distro-info/ubuntu.csv
[16:27] <cjwatson> looks like that's one more ,${eol} than there should be
[16:28] <rbasak> I think that might have been intentional
[16:28] <sinzui> infinity: cjwatson  what has changed between this builds? http://autopkgtest.ubuntu.com/packages/j/juju-core/wily/amd64/ to get the failure? If nothing changed, then I think a race condition is in the code where the wrong table of versions was read
[16:28] <rbasak> version,codename,series,created,release,eol,eol-server
[16:28] <rbasak> cjwatson: so two eol fields
[16:28] <cjwatson> sinzui: new distro-info-data
[16:28] <cjwatson> rbasak: oh, also, it's emitting dates that aren't dates
[16:28] <cjwatson> rbasak: (ah, I forgot eol-server.  right)
[16:28] <cjwatson> 2015-26-19,2015-26-20,2015-26-22,2015-26-22
[16:29] <sinzui> ah. bugger. I really hate jujus design to fail with the unknonwn
[16:29] <cjwatson> haha, I see the date problem
[16:29] <cjwatson> created=`date -d '-2 days' +%Y-%M-%d`
[16:29] <cjwatson> release=`date -d '-1 days' +%Y-%M-%d`
[16:29] <cjwatson> eol=`date -d '+1 days' +%Y-%M-%d`
[16:29] <cjwatson> %M should be %m
[16:29] <cjwatson> don't know if that's the problem here
[16:30] <cjwatson> it's probably not, but should be fixed!
[16:31] <rbasak> Oops. Sorry. Thanks!
[16:32] <rbasak> I might as well try that fix first and see if the error goes away.
[16:33] <cjwatson> we think that's not the problem because the historical passes are at times that ought to have guaranteed an invalid month
[16:36] <sinzui> rbasak: should I plan to rebase 1.24.7 on 1.24.6-0ubuntu3 packaging :)
[16:36] <rbasak> sinzui: :)
[16:36] <rbasak> sinzui: yes in theory, but a fix to the test should be trivial to merge.
[16:37] <sinzui> rbasak: I have automated more of the certifification for Juju. It get less painful each time
[16:38] <Odd_Bloke> Laney: Do we have a bug where we're tracking getting the UI needed to handle bug 1463072?
[16:42] <rbasak> It's not /usr/share/distro-info/ubuntu.csv I don't think. Either /etc/lsb-release or /etc/os-release.
[16:44] <rbasak> I think it's os-release.
[16:49] <Laney> Odd_Bloke: I don't know if there is a bug, but I have asking upstream on my to do list for the beginning ish of the cycle (no promises given the URL case which is the reason most people want this has a different way to achieve the same thing)
[16:50]  * ogra_ wonders if thats related to always having the quotes of a quoted string selected too on double click 
[16:50] <infinity> ogra_: That was fixed.
[16:50] <ogra_> (it used to only select the bits between quotes in earlier versions)
[16:50] <ogra_> infinity, oh, in wily ?
[16:50] <infinity> ogra_: Yeah.
[16:50] <ogra_> i guess i found my first reason to upgarde then :)
[16:51] <infinity> ogra_: It wasn't that the behaviour changed, it was that some tools (like wget) switched from using ' and " to using unicode pretty quotes.
[16:51] <ogra_> yeah
[16:51] <infinity> ogra_: And VTE/gnome-terminal didn't recognize those as boundaries.
[16:51] <infinity> ogra_: But, yeah, that's fixed.
[16:52] <ogra_> dpkg-buildpackage too ... that is where it annoys me most
[16:52] <Odd_Bloke> Laney: I'm probably not going to chase you any more, as I've just switched to Terminator which does what I want. :p
[16:53] <rbasak> sinzui: I think the future faking is working correctly, apart from the date month/minute bug that cjwatson pointed out that doesn't seem to be the problem here.
[16:54] <rbasak> sinzui: it's something to do with how Juju is parsing /etc/os-release I think. If I revert os-release (but not the others, causing everything to be mismatched incidentally) then Juju works.
[16:54] <rbasak> sinzui: however the faking is just making it look like there's a "Y" release and the machine is on it. One difference is that it claims that Y is LTS.
[16:56] <sinzui> rbasak: understood. I think the issue is the bug I posted earlier. Juju specifies exact version instead of minimum versions. My bug is about stopping the hard coding so that all of us can upgrade.
[16:56] <rbasak> sinzui: I think it's over to upstream to look further now please. I would also note that release dates faked aren't coherent (Y released before X) so if that's the problem then please let me know and we can make the faking smarter.
[16:56] <rbasak> sinzui: I'm not convinced it's the same bug but I can see how it could be.
[16:57] <rbasak> Though it seems unlikely that Juju is parsing dates as it didn't barf on impossible dates :)
[16:59] <Laney> rbasak: Is is that you put "LTS" in VERSION_ID?
[16:59] <Laney> At least deleting that seems to make juju version work here
[17:00] <rbasak> Hmm. Good point.
[17:00]  * rbasak tries
[17:00] <rbasak> Yes, that's it. Thanks Laney. Sorry sinzui.
[17:01] <infinity> Ahh, yes, VERSION_ID is always a number, LTS doesn't belong there.
[17:02] <rbasak> Yeah I have two variables, $version which includes a potential LTS and $version_ that doesn't. VERSION_ID should be using the latter.
[17:02]  * rbasak 's code pre-dates os-version, but never mind. He'll fix it.
[17:03] <rbasak> infinity: do you want an upload if this fixes the issue?
[17:03] <rbasak> I can fix the date thing as well.
[17:04] <infinity> rbasak: It's not on any images, so sure.
[17:04] <infinity> rbasak: It's always nice to have tests passing.
[17:05] <rbasak> OK. Just testing my fix from scratch now.
[17:05] <Laney> What if they test that you cry when slapped in the face?
[17:11] <rbasak> Juju fix uploaded.
[17:11]  * rbasak EODS
[17:12] <infinity> rbasak: Ta.
[17:21] <hallyn_> so bug 1490110 is looking weirder and weirder.  I guess actually do-release-upgrade is crashing, then apport crashes while trying to report on it bc do-release-upgrade's python version was upgraded (so /proc/self/exe is deleted)
[17:22]  * hallyn_ retries the whole shebang again with hopes of gdb attaching
[18:33] <slangasek> pitti: so there are some duplicates of LP: #1504897 coming in... e.g. LP: #1508533
[19:02] <hallyn_> ok so when lxc is built for wily, the pkg builder adds a versioned dependency on dbus, a version which is > what's in vivid.   so do-release-upgrade from vivid fails bc dbus is too old.
[19:02] <hallyn_> should lxc add a pre-depends on dbus for this, or should this be being handled by whatever is adding the depends?
[19:02] <hallyn_> (this is still bug 1490110)
[19:06] <cjwatson> a pre-depends will only make any difference at all if you're expecting the new dbus to be unpacked and configured *when your preinst runs*
[19:15] <hallyn_> cjwatson: i dunno.  when i pre-install the newer version of dbus, it still fails with
[19:15] <hallyn_> invoke-rc.d: unknown initscript, /etc/init.d/lxc not found.
[19:15] <hallyn_> which makes no sense since there is no /etc/init.d/lxc in the old pkg either
[19:38] <rbasak> hallyn_: http://paste.ubuntu.com/12887946/ line 131 maybe?
[19:40] <hallyn_> rbasak: that looks fine though
[19:41] <hallyn_> i mean it should be updated, but at the moment it's fine bc the upstart job is shipped
[19:41] <hallyn_> question is why is invoke-rc.d insisting on using the sysvinit job?
[19:41] <hallyn_> rbasak: also you're supposed to be eod :)
[19:41] <rbasak> So it calls invoke-rc.d even though there is no init.d script because it finds the upstart script
[19:42] <rbasak> But upstart isn't present, so invoke-rc.d will do whatever it does when upstart isn't present.
[19:42] <rbasak> That logic makes no sense.
[19:42] <rbasak> I don't really understand how it is supposed to interact with systemd though.
[19:43] <hallyn_> rbasak: 'invoke-rc.d lxc start' talks to systemd just fine
[19:43] <hallyn_> at least, when i do it by hand
[19:43] <hallyn_> obviously not during this upgrade :)
[19:43] <hallyn_> but yeah, maybe this is just a kick in the pants to change this
[19:43] <rbasak> hallyn_: could it be related to dh_systemd_start handling happening a little bit later in the generated postinst?
[19:44] <rbasak> I can't see how though
[19:44] <hallyn_> hm.
[19:44] <hallyn_> btw i was thinking that mustve come from lxc.postinst but it does not, so that's just the autogenerated postinst code.... nothing lxc can do about it afaics
[19:45] <rbasak> My hypothesis is that it's either a bug in dh_installinit/dh_systemd* or the way you're calling it in the package. Maybe if you drop the upstart job (rm_conffile etc) the problem will go away.
[19:46] <hallyn_> can't do that, there are still upstart users
[19:46] <hallyn_> like phone
[19:46] <rbasak> Ah, good point.
[19:46] <hallyn_> mbiebl_: pitti: hey, one of you good fellows around?
[19:47] <hallyn_> we're looking at bug 1490110 .   the lxc package ships upstart and systemd jobs, not sysvinit.  upgrade from vivid to wily fails on invoke-rc.d lxc start insisting on there being a /etc/init.d/lxc, and we can't tell why.
[19:49] <TJ-> hallyn_: I saw something very similar with a user recently, using mongodb, and tracked it down to the postinst script incorrectly trying to use update-rc.d when there's an /etc/init.d/.legacy-bootordering
[19:50] <TJ-> hallyn_: Hazy recall now, but I *think* the control flow was going via invoke-rc.d or similar
[19:53] <hallyn_> TJ-: what should postinst be doing instea dof update-rc.d?
[20:01] <TJ-> hallyn_: just gone back through my logs - terrible memory! It was actually showing up via (manually reproduced doing) "dpkg --configure  libpam-systemd"
[20:01] <TJ-> hallyn_: "the root cause is a call to "update-rc.d systemd-logind defaults" "
[20:08] <TJ-> hallyn_: if you're seeing the same/similar issue, lines 15-19 of the 14.04 libpam-systemd.postinst were the crux: http://paste.ubuntu.com/12739482/
[20:09] <hallyn_> TJ-: the 'defaults' line, or also line 19?
[20:10] <TJ-> hallyn_: I'm trying hard to recall! I *think* I originally though line 16, but then we did a "set -x" trace and it showed it was line 19 (invoke-rc.d)
[20:12] <hallyn_> lxc is using invoke-rc.d to start some other things (mainly apparmor and dnsmasq)
[20:12] <hallyn_> what did you replace that with?
[20:13] <TJ-> hallyn_: Just found the log (the user changed nicknames): "dpkg-divert --local --rename --add /usr/sbin/invoke-rc.d" and "ln -s /bin/true /usr/sbin/invoke-rc.d"  and then undid it afterwards.
[20:13] <rbasak> Wow!
[20:14] <rbasak> That seems a bit...extreme :)
[20:14] <hallyn_> TJ-: will try that, thanks much :)
[20:14] <TJ-> rbasak: I was trying to get the user out of a broken "apt-get -f install" situation
[20:14] <hallyn_> although, hm, i don't want invoke-rc.d t onot run
[20:15] <sarnold> heh, I'd recommend asking one of the greybeards before going that far :)
[20:15] <TJ-> hallyn_: right, I think it depends on which services require it, but as I said that was the root cause for the user I was helping
[20:15] <TJ-> I seem to recall I found the work-around recommended on the -devel mailing list from some time back, or else in a launchpad bug comment
[20:17] <TJ-> Not sure how sane this was, but I found myself saying: "...the reason that invoke-rc.d is failing
[20:17] <TJ-> since it can't find the 'upstart' version of initctl, it tries to use sysv-init, and there is no sysv-init script for systemd-logind
[20:17] <hallyn_> should this count as a bug against sysv-rc?
[20:30] <doko> kees, is harderning-wrapper still maintained, or is it obsolete now?
[20:31] <achiang> mdeslaur: hey, have there been any 14.04 security updates that might break LXC? trying to start a container and getting: lxc-start: conf.c: setup_pts: 1772 Permission denied - mount failed '/dev/pts/ptmx'->'/dev/ptmx
[20:32]  * achiang decides to try and boot an older kernel to see if that is related
[20:33] <pitti> slangasek: right, and bug 1502536 is essentially the same; I haven't yet managed to reproduce it
[20:34] <pitti> slangasek: but vila attached the apt-clone tarball yesterday; I guess this can somehow be used to reconstruct the original system
[20:34] <rbasak> pitti: that looks like the same as https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1490110 that hallyn is working on
[20:34] <pitti> hallyn_: hello; this was meant to be fixed in vivid's invoke-rc.d and service already, hmm
[20:35] <pitti> hallyn_: actually sounds like the same issue I was just telling slangasek
[20:35] <pitti> rbasak: right, these are by and large all the same
[20:35] <rbasak> pitti: we currently have a reproducer
[20:35] <pitti> rbasak: oh, you do? I tried umpteen scenarios for ofono and nfs-common
[20:35] <rbasak> pitti: https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1490110/comments/20
[20:36] <rbasak> On Vivid install juju-local + lxc then do-release-upgrade to Wily.
[20:37] <pitti> rbasak: note that for fixing upgrades it's enough to have the fix in -updates, so we don't need to respin for that
[20:37] <pitti> just a 0-day SRU
[20:37] <rbasak> ack
[20:37] <pitti> rbasak: odd; from a vivid cloud image I tried installing nfs-common, ofono, then dist-upgrade and what not; but I"ll try with this
[20:37] <rbasak> I used uvt-kvm
[20:37] <sarnold> achiang: looks a bit like https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1504781
[20:37] <rbasak> I have the cloud image I used. I haven't tried multiple times so I don't know if there's something non-deterministic involved.
[20:38] <achiang> mdeslaur: hm, yes. booting with 3.13.0-65-generic #106-Ubuntu allows my vagrant-lxc container to boot again
[20:38] <achiang> and with 3.13.0-66-generic it is broken
[20:38] <sarnold> achiang: see also https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1507959 -- it may be worth unduping those two?
[20:40] <achiang> sarnold: reading through the first one 1504781 certainly describes my symptoms
[20:41]  * rbasak goes to bed
[20:43] <achiang> running apt-get upgrade to get lxc 1.0.7-0ubuntu0.9 and will try with kernel -66 again
[20:43] <achiang> brb
[20:44] <achiang> Yeah, that fixed it. Thanks sarnold
[21:00] <hallyn_> pitti: hi, so you have a handle on that bug?
[21:00] <pitti> hallyn_: I duplicated/summarized it, but I don't have a working reproducer yet
[21:01] <pitti> obviously it's some mis-handling in invoke-rc.d
[21:01] <pitti> in vivid we added some handling for packages which don't have init.d scripts, but apparently there's some case where this doesn't work
[21:01] <hallyn_> hm.  as rbasak said uvt-kvm makes it very easy, or you could do it with lxc-create -t
[21:01] <hallyn_> uh, lxc-create -t ubuntu-cloud
[21:01] <pitti> hallyn_: I'm running dist-upgrades in qemu
[21:02] <pitti> so far I installed lxc and  juju-local, and now dist-upgrading
[21:02] <pitti> which still works fine
[21:02] <hallyn_> so it's something about the clou dimages then
[21:02] <pitti> but so far indeed I didn't use do-release-upgrade, I'll try that next
[21:02] <hallyn_> (which is what uvt-kvm uses)
[21:02] <hallyn_> oh!
[21:02] <hallyn_> ok
[21:02] <pitti> hallyn_: no, I also got reports from desktops
[21:02] <pitti> like vila's
[21:02] <hallyn_> ok
[21:03] <pitti> I can't imagine how apt full-upgrade vs. do-release-upgrade would make any difference, but who knows
[21:05] <pitti> hallyn_: do you have a way to reproduce it?
[21:06] <pitti> so far every reporter couldn't reproduce it after the dist-upgrade, any further dpkg-reconfigure, apt-get install -f, apt-get install --reinstall, sh -x invoke-rc.d etc. went fine
[21:09] <pitti> ah! do-release-upgrade indeed!
[21:21] <hallyn_> indeed :)
[21:21] <hallyn_> does it set some weird configuration?
[21:22] <pitti> imported/invoke-rc.d
[21:22] <pitti> SRSLY?
[21:22] <pitti> our dist upgrade tarballs have their own version?

[21:22] <pitti> and lo and behold, diff -u /usr/sbin/invoke-rc.d imported/invoke-rc.d
[21:23] <pitti> all the systemd support is missing
[21:23] <Laney> umm?
[21:23] <pitti> mvoooooooooooooooooooooooo
[21:23] <hallyn_> phew
[21:24] <Laney> because it wants to apply this diff to it?
[21:25] <pitti> Laney: no, I ran diff between the version in wily.tar.gz (the release upgrader tarball) and the real invoke-rc.d from sysv-rc
[21:25] <pitti> I updated the bug
[21:25] <pitti> now, I don't have the slightest idea how this upgrade tarball is constructed
[21:26] <pitti> bdmurray: ^ do you happen to know? or is that still all mvo's domain?
[21:27] <Laney> pitti: I mean it has this copy because it has DistUpgrader/imported/invoke-rc.diff applied or what?
[21:27] <Laney> (in the u-r-u source package)
[21:27] <pitti> Laney: perhaps, yes
[21:27] <Laney> does it just come right from here to the tarball?
[21:28] <pitti> Laney: yes, I just unpacked it and looked at imported/invoke-rc.d
[21:28] <pitti> do-r-u leaves it behind in /tmp/
[21:29] <bdmurray> pitti: DistUpgrade/build-tarball.sh creates it
[21:30] <cjwatson> and it's spat out as part of the ubuntu-release-upgrader package build and gets into LP via a raw-upgrader (or some such) dpkg-distaddfile
[21:30] <cjwatson> raw-dist-upgrader, that's the one
[21:30] <pitti> cjwatson: ah, so uploading u-r-u should do it?
[21:30] <Laney> So yes, can fix the one in the package
[21:30] <cjwatson> pitti: right
[21:31] <cjwatson> (disclaimer: have not been following the rest of the discussion at all)
[21:31] <pitti> cjwatson: do you know, would that also work when SRUing this? i. e. upload to wily-updates?
[21:31] <pitti> I don't think we want to rebuild images for this
[21:31] <pitti> (or maybe it's necessary, but checking options here)
[21:31] <cjwatson> I believe so but would have to check
[21:31] <bdmurray> pitti: yes, it would we just need to change the meta-release file url to point to -updates
[21:31] <pitti> cjwatson: rest of the discussion is not that important; the essence is, the current wily.tar.gz upgrade tarball is buggy
[21:32] <bdmurray> pitti: I already have an SRU pending for bug 1508539
[21:32] <cjwatson> oh that's it
[21:32] <cjwatson> it gets the URL from http://changelogs.ubuntu.com/meta-release*
[21:32] <cjwatson> (the various versions of that, there's an index at the top level)
[21:32] <cjwatson> I think that's updated by hand but it can be made to point to -updates
[21:33] <cjwatson> and indeed does for some series
[21:33] <bdmurray> cjwatson: that's what I was saying
[21:33] <pitti> bdmurray: ah good; mind if I reject your upload and add the invoke-rc.d update and reupload?
[21:33] <cjwatson> bdmurray: right, indeed, just wanted to fill in the details
[21:33] <bdmurray> pitti: no, that sounds find. What bug is this fixing?
[21:33] <pitti> bdmurray: seems you didn't yet push the release tag to bzr anyway :)
[21:33] <cjwatson> since I think this is poorly understood by a lot of devs
[21:33] <pitti> bdmurray: bug 1504897
[21:34] <pitti> bdmurray: i. e. various packages failing to upgrade with "no such init.d script" and exit status 100
[21:35] <bdmurray> pitti: so bug 1508247 then?
[21:35] <pitti> bdmurray: yep, duplicated; similar to bug 1490110 (also already duped)
[21:36] <bdmurray> pitti: okay, I'll keep an eye out for those
[21:37] <Laney> can we just take this diff into the normal invoke-rc.d in future?
[21:37] <Laney> feel like this trap might come up again
[21:38] <pitti> seems reasonably fine, yes
[21:38] <pitti> for X we should really do that
[21:38]  * Laney writes a note for friday
[21:42]  * pitti taps foot for this looong pre-build.sh to finsih
[21:47] <pitti> ok, uploaded
[21:48] <pitti> can we test this somehow while it's in -proposed?
[21:48] <Laney> I guess re-target http://changelogs.ubuntu.com/meta-release-development ?
[21:49] <pitti> like temporarily pointing ^ to -proposed ?
[21:49] <Laney> bdmurray: ^?
[21:49] <pitti> I wonder if there's some secret option/env var to use a local file
[21:50] <pitti> arges: ^ as you currently seem to do SRUs, would you mind looking at ubuntu-release-upgrader in wily? (I uploaded it, so don't want to self-accept)
[21:50] <bdmurray> usually meta-release-proposed is used to point at proposed
[21:50] <pitti> arges: this should probably become a 0-day SRU tomorrow
[21:51] <pitti> arges: (sorry for the shameless abuse!)
[21:51] <cjwatson> I think you can pass it a local configuration file too
[21:51] <Laney> It's ok, I'm reviewing it
[21:51] <TJ-> pitti: I did something with it some years ago; edited "/etc/update-manager/meta-release"
[21:51] <cjwatson> stash it in /etc/update-manager/meta-release IIRC
[21:52] <cjwatson> snap
[21:52] <Laney> I've released 0-day SRUs before
[21:52] <pitti> TJ-, cjwatson: awesome, thanks
[21:52] <Laney> so don't feel too naughty accepting it with thise intention :)
[21:52] <Laney> cjwatson: ah, feel free
[21:52] <pitti> Laney: ack, thanks; you have more context now, too
[21:52] <cjwatson> I'm not reviewing it
[21:52] <Laney> oh
[21:52] <cjwatson> my snap was to TJ-
[21:52] <Laney> fair
[21:52] <pitti> so once it's accepted/built, we can test the upgrade
[21:53] <arges> pitti: will look
[21:53] <pitti> arges: seems Laney is on it, so nevermind; thanks anyway!
[21:53] <arges> pitti: oh I don't usually look at dev releases
[21:53] <arges> pitti: ok ok
[21:53] <pitti> arges: it's that gray area between dev and stable now :)
[21:53] <lifeless> dable? stev?
[21:54] <pitti> lifeless: almost-solid?
[21:55] <lifeless> hmm, what do you call jellies that are nearly set ?
[21:55] <pitti> cjwatson: while we can soak in your wisdom: how does http://changelogs.ubuntu.com/meta-release{,-development} get updated once this gets released? is that "vi" by someone with perms, or does that live in some bzr?
[21:55] <TJ-> lifeless: Ready to eat
[21:56] <pitti> (or "emacs" of course -- this ain't no flamebait)
[21:57] <bdmurray> pitti: it lives in bzr but someone needs to put on the server
[21:57] <bdmurray> pitti: ~ubuntu-core-dev/meta-release/ubuntu/
[21:59] <pitti> bdmurray: ah, is that an RT?
[21:59] <bdmurray> pitti: no, I have access
[21:59] <pitti> bdmurray: ah, ok
[22:00] <pitti> hallyn_, rbasak, Laney, bdmurray, cjwatson: thanks for your help -- nice teamwork!
[22:00] <cjwatson> as do I, Michael, and Adam
[22:01] <Laney> pitti: accepted, thanks
[22:01] <Laney> modulo hotel wifi (done now)
[22:02] <pitti> Laney: ok,thanks; it's getting late, so unless someone beats me to it I'll test it first thing in the morning
[22:03] <Laney> I'm guessing you might beat us to being awake/online :)
[22:03] <bdmurray> pitti: I can test it what should I have installed?
[22:03] <Laney> (but if not, will do)
[22:03] <pitti> bdmurray: I added a test case to bug 1504897 description
[22:04] <bdmurray> pitti: cool, thanks. Its also possible to just manually download the dist-upgrader tarball, extract it and run ./wily
[22:05] <pitti> bdmurray: that'd be great, thanks; you also have sru-release and ~ubuntu-core-dev/meta-release/ubuntu/ powers, so we can land this by tomorrow
[22:05] <pitti> bdmurray: ah, ./wily instead of do-release-upgrade -d?
[22:06] <pitti> I don't see the spethial tarball on https://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:15.10.14/+build/8156705
[22:06] <bdmurray> pitti: do-release-upgrade -d does the checking of the metarelease file and stuff. I'm saying you can http://archive.ubuntu.com/ubuntu/dists/wily-proposed/main/dist-upgrader-all/current/ download wily.tar.gz and unpack it and then run it
[22:06] <pitti> so that doesn't work through dpkg-distaddfile, but some other side channel I suppose
[22:07] <pitti> bdmurray: ah, cool; so that's essentially what do-r-u does
[22:07] <bdmurray> pitti: yes
[22:08]  * pitti waves good night then, cu tomorrow
[22:08] <Laney> night pitti
[22:08] <pitti> "today" already :)
[22:08] <sarnold> night pitti, see you in two or three hours? :)
[22:08] <cjwatson> pitti: it really is dpkg-distaddfile, you just don't see it in the UI
[22:09] <cjwatson> pitti: but follow the link to the build log and you'll see it being added there
[22:09] <hallyn_> pitti: thank  you!
[22:10] <cjwatson> (the modelling of custom uploads in LP's database is ... inadequate, which is why the web UI doesn't display them)
[22:11]  * Laney showers, hopefully this is published shortly and I can check it
[22:42] <bdmurray> Laney: I'm testing it already fwiw
[22:45] <Laney> bdmurray: me too, but your internets are probably more internet than my internet
[22:46] <sarnold> hehehe
[22:53] <israeldahl> Hi all, can anyone point me to any documentation on building a custom PowerPC ISO.
[22:53] <israeldahl> I have already built for x86
[22:54] <israeldahl> 32 bit and 64bit are not hard to do (for me) using deboostrap and chroot... but PowerPC is harder does anyone know the secret?
[23:02] <Laney> bdmurray: OK, finished(!) and I don't see those messages now
[23:04] <bdmurray> Laney: neither do I
[23:05] <israeldahl> Does anyone know where documentation to build a Power PC chroot is?
[23:06] <israeldahl> I have PPC and x86.  I would like to do it on x86 (if possible) but would be willing to use PPC if I have to
[23:06] <israeldahl> Does debootstrap work with chroot on PPC?
[23:06] <sarnold> israeldahl: wouldn't debootstrap on the ppc work?
[23:06] <sarnold> heh
[23:07] <cjwatson> Native debootstrap always works.  Our own build processes rely on it.
[23:07] <israeldahl> thanks cjwatson
[23:07] <Laney> bdmurray: unless you want to take care of it I'll sru-release in the morning
[23:07]  * Laney sleeps now, night
[23:07] <israeldahl> cjwatson is it possible to use qemu or something to do it on x86?
[23:07] <cjwatson> If you were doing it on x86 then in practice you would need to grab an existing powerpc image and use qemu-system-powerpc.  It would be slow.
[23:08] <cjwatson> qemu-system-ppc, sorry
[23:08] <cjwatson> In theory you can probably start the process with on-the-fly per-process emulation using qemu-user-static, but in practice you would run into trouble somewhere along the line with that approach.
[23:08] <israeldahl> cjwatson you mean I would need to loop mount the iso and chroot in?
[23:09] <cjwatson> If you have a powerpc system then the path of least resistance is definitely to use that.
[23:09] <cjwatson> qemu-system-ppc != chroot.  Anyway, go native if you can.
[23:10] <israeldahl> cjwatson thanks I will try it.  I have scripts for building on x86, so everything should be pretty much the same (except kernel names) but the process functions the same, correct?
[23:10] <cjwatson> Then you don't need qemu and everything's much simpler and quite probably faster as well, unless your powerpc system is exceptionally slow.
[23:10] <cjwatson> Right.
[23:10] <cjwatson> Well, and the top-level ISO bits are a bit different.
[23:11] <israeldahl> cjwatson does isolinux (this is precise based) work the same for PPC?
[23:11] <cjwatson> It's not in a shape that you can run directly, but the stuff in "bzr branch lp:~ubuntu-cdimage/debian-cd/ubuntu debian-cd" (directory tools/boot/trusty/ etc.) at least shows you the options we use
[23:11] <cjwatson> isolinux is x86-specific, and does not work in any way, shape, or form for powerpc
[23:11] <israeldahl> cool thank you I will totally check it out!
[23:11] <israeldahl> ok so I need yaboot and all that
[23:12] <cjwatson> right, or in theory grub but our current images are yaboot-based
[23:13] <israeldahl> Great cjwatson you have been very helpful!  I may pop back in after a few days if I need more insight
[23:13] <israeldahl> you rock :)
[23:14] <cjwatson> np