[00:44] <RAOF> You know what's totally awesome? When packages get released from -proposed without a comment and then someone marks the bug as verification-done, also without comment.
[00:55] <infinity> RAOF: ?
[01:00] <RAOF> infinity: https://bugs.launchpad.net/bugs/1186273
[01:00] <ubot2`> Launchpad bug 1186273 in network-manager (Ubuntu Quantal) "DUN via bluetooth with a password fails" [High,Fix committed]
[01:01] <cyphermox_> moo?
[01:02] <RAOF> Baaaa
[01:02] <RAOF> cyphermox_: Oh, you're highlighted on network-manager? :P
[01:02] <cyphermox_> ssshh
[01:03] <cyphermox_> RAOF: I don't think it has been released, but I'll talk to jpds tomorrow
[01:03] <RAOF> cyphermox_: Check the bug, comment #22 - it's been released to raring.
[01:04] <cyphermox_> oh right
[01:04] <cyphermox_> and I was parsing that as longer ago than it really is
[01:04] <cyphermox_> I doubt the part about verification done on precise though
[01:05] <cyphermox_> I'll double-check with the responsible parties tomorrow, in the meantime I've reverted the tag, if that's ok with you
[01:06] <RAOF> cyphermox_: Yeah, that's fine.
[01:08] <RAOF> Yes! A successful linux-firmware verification!
[01:19] <infinity> RAOF: raring... The same raring that's EOL?
[01:20] <RAOF> infinity: Yeah, that raring. Did it's EOL status trigger acceptance of everything in raring-proposed?
[01:20] <infinity> No.
[01:20] <infinity> bdmurray released it.
[01:20] <infinity> But EOL should trigger empyting the -proposed pocket, so I'll go delete anything else in there.
[01:21] <infinity> ... if there is anything else.
[01:21] <infinity> Although, I'm not sure if we've ever done that in the past.
[01:21] <infinity> Not like it hurts to archive it with the pending proposed bits frozen in time.
[01:22] <infinity> And, indeed, previous releases were EOLed with stuff still in proposed, so meh.
=== Ursinha is now known as Ursinha-afk
[05:34] <infinity> Comment: NBS
[05:34] <infinity> 354 packages successfully removed.
[05:34] <infinity> Why is that so satisfying every time?
[06:41] <cjwatson> Who accepted cloud-init/precise-proposed?
[06:43] <cjwatson> It's not on CD images as far as I can see, so maybe it's OK - but please check with me before accepting anything into precise-proposed until 12.04.4 is out, as mentioned on -devel-announce
[06:44] <cjwatson> (Or copying anything into precise-updates)
[06:45] <cjwatson> chrisccoulson: re firefox, I'm not going to wait for it; if I have to respin and include it, so be it, otherwise people will pick it up on upgrades
[06:47] <RAOF> cjwatson: 'twas me, sorry.
[07:14] <cjwatson> RAOF: ok, no problem in this case :)
[09:11] <xnox> cjwatson: cloud-init is on the cloud-images however - http://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-i386.manifest and takes 6h+ to respin all of those.
[09:12] <cjwatson> I guess that's up to utlemming_.  it's not in -updates yet
=== lool- is now known as lool
=== doko_ is now known as doko
=== yofel_ is now known as yofel
[11:48]  * apw notes compiz in trusty appears to be borked
[11:52] <xnox> apw: yeah, there was protobuf abi break which is progress to be fixed.
[11:54] <ogra_> it is fixed, ou could pull the deb from -proposed
[11:54] <apw> xnox, how did that not get noticed by versioning and get stuck by britney etc
[11:55] <ogra_> (might be good to actually get feedback if compiz is fine with it again)
[11:55] <apw> ogra_, ack
[11:55] <ogra_> apw, non exported symbol change
[11:55] <ogra_> (i.e. ABI change without bump)
[11:55] <apw> if the symbol isn't exported then noone could be using it :)
[11:56] <ogra_> a symbol was changed and not properly exported (as i understood)
[11:57] <ogra_> anyway, try the libprotobuf from proposed and see if it helps
[11:57]  * ogra_ only tested on phone 
[12:00] <apw> ogra_, confirming that upgrading to that fixes my machine
[12:03]  * apw notes that these binaries are blocked in proposed
[12:05] <ogra_> apw, well, i think cjwatson can unleash it
[12:06] <ogra_> seems like a manual block
[12:10] <apw> yeah waiting on testing on the phone, we are doing some more testing here as well; though it seems to have fixed my compiz and mumble
[12:17] <cjwatson> apw: protobuf doesn't have .symbols files, so no fine-grained checking of exported symbols
[12:17] <cjwatson> that should be fixed, but I'm only driving-by this ...
=== Ursinha-afk is now known as Ursinha
[13:14] <apw> cjwatson, makes sensew
=== medberry is now known as med_
[16:25] <stgraber> LTSP on Edubuntu hangs at boot time somehow, trying to figure out what's causing this...
[16:26] <infinity> stgraber: Poorly-written software would be my guess.
[16:26] <infinity> *duck*
[16:27] <cjwatson> stgraber: precise?
[16:27] <stgraber> infinity: :) well, looking at the bit of code that seems to be hanging, my first guess would be "the kernel", so I very much hope I'm wrong ;)
[16:27] <stgraber> cjwatson: yep, precise
[16:28] <infinity> stgraber: I think the kernel would fall into my description of the problem.
[16:28] <infinity> But yes, here's hoping it's not that.
[16:29] <infinity> We're somewhere between "way too late" and "hahaha" for timing on pushing another kernel to 12.04.4
[16:29] <mdeslaur> lol
[16:29] <apw> stgraber, got a pointer to the code which is hanging
[16:32] <stgraber> apw: oh actually, there's one line at the end of that initrd script which I didn't see which seems a whole more likely to cause the problem I'm seeing than the rest of the script.
[16:32] <stgraber> apw: the rest of the script just does tmpfs, bind-mounts and move mounts, none of which are terribly likely to be broken in interesting ways that we didn't notice already
[16:33] <stgraber> apw: the one line I didn't notice before on the other end is chrooting into the system and running a command, so my current guess is that something's going wrong in whatever it's running
[17:21] <stgraber> apw: so it looks like the hang comes from "chroot $rootmnt test -w /" over a nbd mounted squashfs + overlayfs on top
[17:22] <stgraber> apw: apparently I can list the whole fs just fine (find $rootmnt), though reading data from it causes the hang, digging some more
[17:23] <stgraber> though, that's with a 3.2 kernel (Edubuntu uses the 3.2 kernel for LTSP due to a lot of our users having non-pae hardware), so now I'm really confused as to how this regressed
[17:29] <apw> stgraber, well its not changing a whole heap thats for sure being so very old
[17:41] <apw> stgraber, what kernel would the last tested known good kernel have been in this form
[17:42] <stgraber> apw: it sure used to work when we released 12.04.3, so whatever the current 3.2 was at that time. I'm testing a few downgrades now to see if it's that simple (in which case, we're pretty much screwed for 12.04.4 but at least I don't need to do emergency ltsp updates)
[17:42]  * apw wonders how to figure out the .3 2.6.32 kernel
[17:48] <stgraber> apw: edubuntu 12.04.3 lts had 3.2.0-52.78, I'll start by downgrading to that one and see if things work as expected, then try some random ones in between
[17:48] <xnox> stgraber: could be related to "fixed nfsroot" with hwe kernels as part of https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/1217041
[17:48] <ubot2`> Launchpad bug 1217041 in initramfs-tools (Ubuntu Saucy) "initramfs-tools: please include separated nfs modules" [Undecided,Fix released]
[17:49] <xnox> stgraber: which in turn regressed 12.04.0 kernel?
[17:49] <apw> stgraber, the manifest for the edubuntu 12.04.3 seems to contains the lts-raring kernel, so i assume its somewhere else ?
[17:50] <stgraber> apw: yeah, we only ship the -pae 3.2 kernel for LTSP which is in a ltsp.img squashfs on the iso image and doesn't appear in the manifest
[17:51] <stgraber> apw: I figured out the version of the 3.2 kernel based on that of linux-libc-dev in the 12.04.3 manifest which does match whatever 3.2 kernel was current at the time
[17:51] <apw> stgraber, ok no changes in squashfs or overlayfs for the range you mention, a small delta in nfs
[17:56] <apw> stgraber, what filesytem type is the overlay writable layer
[17:57] <ogra_> in LTSP ?
[17:57] <ogra_> thats tmpfs
[17:57] <apw> yes
[17:57] <apw> nothing there either then
[17:58] <ogra_> any nbd changes perhaps ?
[17:58] <ogra_> the squashfs gets provided via nbd
[17:58] <ogra_> (or could the nfs changes have influenced nbd)
[17:58] <stgraber> apw: ro layer is nbd mounted squashfs, rw layer is tmpfs, though the hang happens even before I mount the overlay
[17:59] <apw> there is some hcange to nbD
[17:59] <ogra_> there we go :)
[17:59] <stgraber> gah, just reproduced the same hang with 3.2.0-52-generic-pae, so now I'm really really confused
[18:00] <stgraber> alright, screw those VMs, I want to see the same thing on actual hardware before I dig any deeper into that mess...
[18:01] <apw> ogra_, the change there is only at disconnect time, which would be on unmount i assume i this case; and we anr't doing that normal startup and use is unchanged
[18:01] <apw> stgraber, so "it never worked" is where we are, erp
[18:02] <ogra_> heh
[18:03] <stgraber> apw: well, Edubuntu 12.04.3 certainly worked or we wouldn't have released it, now I just need to figure out what's causing the change in behaviour... the squashfs is clearly valid, the nbd server seems to work (as I can mount and read the image without any trouble from another box), so I'm left with either a very very weird bug or kvm playin tricks on me
[18:03] <stgraber> I'm currently very much hoping on the latter
[18:16] <stgraber> apw: works fine on actual hardware... so I have no idea what's going on with nbd over kvm's network/bridges here but whatever, it works on metal and that's all we care about
[18:17] <ogra_> \m/
[18:17] <ogra_> metal ...
[19:47] <apw> stgraber, nnng, sounds good in the sense it is releaseable, sounds bad in the sense it implies that kvm networking is bored on your host or something
[19:53] <stgraber> apw: doh, just found what's wrong, not sure how it go that way though
[19:54] <stgraber>           UP BROADCAST MULTICAST  MTU:1442  Metric:1
[19:56] <apw> stgraber, mtu ?  that looks familiar
[19:56] <apw> like an ipv6 tunnel size or something
[19:57] <stgraber> yeah, I don't know why that bridge's mtu went down to 1442 especially as it's only used for internal VM networking, it's not connected to any external device which may have caused the mtu change...
[19:57] <stgraber> anyway, setting it back to 1500 fixes all my issues, so that's good.
[19:57] <apw> it is slightly worrying that nbd is so sensitive to mtu
[19:58] <stgraber> well, my guess is that: VM1 has mtu 1500, bridge on host has mtu 1442, VM2 has mtu 1500
[19:58] <stgraber> so any packet larger than 1442 will magically disappear
[19:59] <stgraber> because the host only runs the bridge, it's not routing or anything so won't fragment properly, as far as both VMs are concerned, they have a mtu 1500 link so have no reason to fragment either
[20:03] <apw> stgraber, that sounds rather plausible yes
[22:22] <michagogo|cloud> When does Ubuntu 12.04.4 get released?
[22:25] <infinity> michagogo|cloud: Some time tomorrow.
[22:26] <michagogo|cloud> There isn't a specific set time?
[22:33] <infinity> michagogo|cloud: Nope.  There never is.  "When it's ready".
[22:33] <michagogo|cloud> What defines "Ready"?
[22:34] <knome> michagogo|cloud, when the release team thinks it's okay to release. there is no exact answer for your question, so please, stop asking
[22:35] <michagogo|cloud> Okay. What's the best way to find out that it's happened?
[22:37] <knome> follow the ubuntu-announce mailing list
[22:39] <cjwatson> *Probably* some time tomorrow.  Point releases have a bit more permissible slippage (though I'd prefer not to have to slip the date).
[22:45] <elfy> cjwatson: I marked those two ^^ tracker won't let me mark the alternates for some reason - but they are
[22:45] <michagogo|cloud> What does that mean?
[22:45] <knome> cjwatson, thanks for all the help.
[22:45] <elfy> yep +1 to that :)
[22:46] <knome> michagogo|cloud, the xubuntu team considers the xubuntu desktop images ready to be published.
[22:46] <michagogo|cloud> Ahh