=== sturmflut_ is now known as sturmflut
[07:43] <apw> SturmFlut, i am sure someone was somewhere in the world
[07:44] <sturmflut> Haha
[07:51] <sturmflut> I've debugged most of the problem and filed a bug report in the meantime, looks like the original problem is in ModemManager, but might affect the kernel as well in the end.
[07:53] <ogra_> sturmflut, but not a kernel anyone in here maintains :) 
[07:55] <sturmflut> ogra_: Shouldn't the cdc_acm module of my desktop kernel blacklist that specific USB ID, because it will never work anyways and just confuse the russians?
[07:55] <ogra_> dont underestimate the cleverness of the russians :) 
[07:56] <ogra_> and i'm not sure its a kernel thing ... did you check that modemmanager doesnt ship udev rules that cause the loading ? 
[07:57] <sturmflut> Hmm, now how does this udev stuff work with systemd...
[07:58] <ogra_> heh, te same as with other init systems :) 
[07:58] <ogra_> (i dont think systemd comes into play here ... but i guess modemmanager ships udev rules that react to uevents related to serial) 
[08:08] <sturmflut> ModemManager ships some rules, not to load the module for this device, but to tell itself to blacklist a lot of similar cases. So the simple solution would be to at least add the device there, then cdc_acm gets still loaded and fails, but ModemManager does no longer claim the device.
[08:08] <sturmflut> Not pretty, but I can live with that.
[12:41] <rtg> apw, ogasawara_ upstream discussion on CONFIG_VM86 http://thread.gmane.org/gmane.linux.kernel/1991020
[12:42] <rtg> kees suggests we turn it off
[12:48] <apw> rtg, yeah someone claims we have it off already, but it looks to be on to me
[12:48] <rtg> yup, their claim was erroneous
[12:48] <apw> rtg, it sounds like a security disaster in the making, so ripping it in wily and seeing if anything explodes seems good
[12:48] <ogasawara_> rtg: ack, do you already have a patch to push to wily to turn it off?
[12:48] <rtg> ogasawara_, I was just disabling it in unstable
[12:49] <rtg> ogasawara_, I can do wily master-next as well
[12:49] <ogasawara_> rtg: ack
[12:50] <infinity> Didn't we just turn that on in an SRU a few months back or something?
[12:50] <infinity> It seems awfully familiar.
[12:53] <infinity> Maybe I'm thinking of something else with a similar-sounding name.
[12:55] <rtg> infinity, not finding any CONFIG_VM86 changes in trusty
[12:55] <infinity> rtg: Ahh, yeah, I was thinking of CONFIG_X86_16BIT, nevermind.
[12:56] <infinity> So, VM86 seems scary and evil as all hell, I'm inclined to agree, but we need to make sure we don't need it for any old skool X drivers (vmware, vbox, etc)
[12:58] <infinity> Oh, but if it doesn't even work on x86_64, my carefactor just went down.
[12:58]  * infinity catches up on the rest of the thread.
[12:59] <rtg> infinity, it was 32 bit only in our configs
[12:59] <infinity> rtg: Yeah.  I'd be inclined to say we should disable it across all stable kernels too, then, as a pre-emptive security feature.
[13:00] <infinity> rtg: If no one was hunting for holes in it before, you can bet that thread will make grey hats start looking.
[13:01] <infinity> rtg: There are one or two X drivers that are i386-only.  If we can verify that they don't need VM86, I'd say the impact is minimal-to-none and worth the risk.
[13:01] <rtg> infinity, agreed, though the attack surface is much smaller given that it is only applicable to 32 bit machines. 
[13:02] <rtg> infinity, I'll run this by jjohansen and tyhicks as well
[13:03] <rtg> perhaps 32 bit cloud VMs are vulnerable
[13:04] <infinity> rtg: 32-bit still has an astoundingly large install base.
[13:05] <infinity> rtg: And certainly true going back to precise, where it was still our recommended image download.
[13:05] <rtg> yeah, I'd be willing to bet there are a lot of 32 bit instances in the cloud
[13:06] <infinity> rtg: Probably fewer than there should be (ie: I'd argue that most "tiny" and "small" instances should be i386, even if backed by amd64 hardware), but yeah, there are certainly some.
[13:07] <rtg> infinity, working on an SRU
[13:08] <infinity> rtg: I think we need a userspace impact analysis before we SRU it, but my hope is the impact is "none".
[13:09] <infinity> rtg: My only concern is ancient X drivers that haven't seen significant work in the last decade, and I think we only ship a couple of those.
[13:10] <rtg> infinity, it seems our choice is to orphan some vanishingly rare drivers, or expose possible vulnerabilities to every 32 bit user.
[13:11] <rtg> orphan -> regress
[13:11] <infinity> rtg: Hrm.  And vbetool.  That might need a looking at as well.
[13:11] <rtg> I'll make that part of the SRU discussion
[13:11] <infinity> rtg: But yes, we're on the same page.  I think the goal should be disabling VM86, we should just sort out the impact and see if we can minimize damage.
[13:14] <infinity> rtg: http://codesearch.debian.net/results/sys%2Fvm86.h
[13:15] <infinity> rtg: and http://codesearch.debian.net/results/asm%2Fvm86.h
[13:16] <infinity> rtg: Some might be false positives that we don't enable, others are probably build-time detected and would be broken if we regress the kernel support.
[13:18] <rtg> infinity, I guess Linus will howl if Lutomirski starts breaking user space
[13:20] <infinity> rtg: Well, it's never been a required kernel option, so it doesn't break userspace to disable it by default, by Linus's definition of break.
[13:21] <infinity> rtg: But distributions need to be aware of impact on already-shipped software if they disable features they used to enable.  That's where it gets sticky.
[13:21] <rtg> infinity, is userspace supposed to fall back into an emulation mode ?
[13:22] <infinity> rtg: I'd have to look at the glibc syscall wrapper.  It's possible that people using the abstracted sys/vm86.h are safe.
[13:22] <infinity> rtg: People using asm/vm86.h directly, though, are definitely broken if they don't have their own fallbacks in place.
[13:24] <infinity> rtg: I highly doubt we have an 8086 emulator in glibc though (in fact, I'm sure we don't), so in either case, if people are relying on the syscall without a fallback option, they'd explode regardless, I think.
[13:25] <rtg> infinity, would they explode at runtime or build time, or both ?
[13:25] <infinity> rtg: I'd assume just runtime, if the headers are still exposed when the option is off.
[13:26] <infinity> rtg: If the headers go away completely, then build time, but given the number of things that include the headers, I suspect that's not intended to happen.
[13:26] <rtg> thank you LP for blowing up and _not_ accepting my bug report.
[13:27] <infinity> rtg: LP is suffering some major infra mojo right now.
[13:27] <rtg> infinity, bug #1473447
[13:27] <ubot5> bug 1473447 in linux (Ubuntu) "linux: VM86 should be disabled" [Undecided,New] https://launchpad.net/bugs/1473447
[13:28] <rtg> infinity, we've got it disabled in Wily (4.1+), so we can experiment with the fallout
[13:29] <infinity> rtg: Yeahp, that seems like the safe approach for now.  We can abuse wily userspace a bit, cross-referencing with codesearch (which should have 100% coverage for what we care about, I'd be VERY shocked if we ship anything Ubuntu-only that uses vm86), and then sort out conclusions.
[13:31] <infinity> rtg: codesearch's only failure here is that it's only search unstable, so doesn't give us a clear view of the possibility that precise might be more broken than trusty or wily.
[13:31] <infinity> rtg: But it's a start anyway.
[13:32] <infinity> I really think we need to set up an Ubuntu codesearch instance that covers all supported series'... It's crazy useful.
[13:32] <rtg> infinity, seems like we should certainly have it disabled by 16.04, so experimenting on 15.10 makes sense
[13:33] <infinity> rtg: Yeahp, and in 15.10/16.04, I have zero issues with the idea that if some bits of userspace break without it, we can just deprecate/remove them if they're not fixable.
[13:33] <infinity> rtg: But that's a harder pill to swallow for precise and trusty.
[13:33] <rtg> agreed
[13:48]  * cking wonders why the watchdogs are going nuts: https://pastebin.canonical.com/135040/
[14:02] <infinity> cking: That's... Impressive.
[14:02] <infinity> cking: I've never seen a kernel try so hard to starve userspace.
[14:04] <cking> infinity, it did have some repercussions: https://pastebin.canonical.com/135045/
[14:10] <pointertonullval> Hey guys I'm building my own embedded device and it will connect over usb to a computer. How I can start writing a linux driver? Is a kernel level problem or more high level? Thanks
[14:11] <apw> usb is interesting because there is a userspace library interface iirc, so you can write things out of the kernel
[14:12] <TJ-> pointertonullval: Look at the USB 'gadget' interface
[14:15] <pointertonullval> apw, TJ- thanks! something like this? http://www.linux-usb.org/gadget/
[14:34] <unixabg> apw: it appears that casper does not like the multiple squashfs files on boot. But i am fairly 
[14:35] <unixabg> confident that is sees them since it does make the mount points and then fails (I think on 
[14:35] <unixabg> getting the overlay right).
[14:39] <apw> it is possible that casper is trying to lay down multiple layers of overlayfs and not using the multiple layer support
[14:39] <apw> for which there likely is no way to figure out if the functionality exists in the kernel ... hmm
[14:41] <infinity> apw: Does casper have support for multiple layers at all?  That would seem odd, if it's a new kernel feature.
[14:43] <apw> infinity, it has some support in the sense they are layerable, ie mount the bottom two together, then mount the results with the next layer up
[14:43] <apw> though you soon hit the stack depth limit in the kernel, so i think we bust on overlayfs at two mount layers, ie 3 layers
[14:43] <apw> unixabg, how many layers are there ?
[14:45] <infinity> apw: Anyhow, sorting out if functionality exists should be a non-issue, casper isn't the sort of tool one uses "blindly".  It should just do what you ask it to do, and if you're using a kernel that can't support that, you get to keep both pieces.
[14:46] <apw> thoug in a sense we have some way to do it two ways with differnet limits in each case
[14:47] <apw> and the second way only if you have a newer kernel, which doesn't tell you it can do it, though i guess for casper at least we kinda know what its built with
[14:48] <infinity> apw: If it's a question of mount options changing to support way 1 and way 2, and there's a distinct upstream version cutoff to go from 1 to 2, you could case uname -r.
[14:49] <infinity> Or, not case, but convert-and-compare, probably.
[14:49] <unixabg> apw: on my test with wily it fails by just adding a second layer. It may be helpful if I outline
[14:49] <unixabg> how I am testing, I basically am doing a quick remaster and I am generating my partial squashfs
[14:50] <apw> infinity, ugg you are not putting me in my happy place
[14:50] <unixabg> files with live-partial-squashfs-updates:
[14:50] <unixabg> http://live.debian.net/gitweb/?p=live-tools.git;a=blob_plain;f=bin/live-partial-squashfs-updates;hb=refs/heads/debian-next
[14:51] <infinity> apw: Do I ever?
[14:51] <unixabg> once I generate a partial-squashfs-update (psu) against the filesystem.squashfs then I button
[14:51] <apw> infinity, those days you offer to take me to the pub, yes
[14:51] <unixabg> the image back together and boot it. But it is not working. I did test one layer was working on 
[14:52] <unixabg> linuxmin17.1 I think, mom to check that,
[14:53] <unixabg> yes that is correct, but any more than filesystem.squashfs and psu1.squashfs caused failure.
[14:54] <unixabg> Let me run one more quick test, since late yesterday and I had many tests going I do believe that now
[14:54] <unixabg> I can not even stack on one psu.squashfs. Be back in a short bit...
[14:58] <unixabg> apw: ok wily failed when I tried to add only one psu.squashfs along with filesystem.squashfs, next testing
[14:58] <unixabg> linuxmint17.2
[15:07] <apw> unixabg, can't look at it right now, but it would be good to get a dmesg if you can to see if it is throwing a depth limit in there somewhere
[15:07] <apw> report any testing here and i'll see if i can figure it out
[15:13] <unixabg> apw: humble appreciation. Ok now I just booted linuxmint17.2 + 1psu.squashfs and it is ok.
[15:14] <unixabg> s/and/and remastered and booted live,/g
[15:16] <apw> unixabg, whihc kernel does that have in it
[15:18] <unixabg> apw: uname -r produces 3.19.0-20-generic
[15:20] <unixabg> apw: dmesg in an image: http://fyeox.com/downloads/20150710-loop1fail-wily.png
[15:22] <unixabg> It could be that it does not have the correct kernel version or something. Anyhow I am willing to
[15:22] <unixabg> assist however I can. I want the multiple squashfs to work well.
[15:44] <brainwash> cking: hey, I noticed bug 1470845 too. should this be forwarded somewhere upstream?
[15:44] <ubot5> bug 1470845 in systemd (Ubuntu) "systemd on wily desktop generating short lived threads every second in a quiet system" [Undecided,New] https://launchpad.net/bugs/1470845
[15:45] <brainwash> installed forkstat some minutes ago
[15:45] <cking> brainwash, I was hoping it would get some triaging and attention on the ubuntu side of things, it maybe the way it's configured at the moment and is a temporary issue
[15:48] <brainwash> cking: it? what could be misconfigured?
[15:48] <cking> brainwash, i don't know, that's why I filed the bug
[15:49] <brainwash> ok :) sadly, I only have access to ubuntu wily right now
[15:49] <brainwash> and I've never used forkstat before
[15:49] <infinity> cking: Did that happen before 222?
[15:49] <cking> infinity, I don't know either
[15:50] <brainwash> 222 was pushed just recently
[15:50] <cking> it's the "oh, I've noticed something that looks suspect and I don't know why or when it happened" kinda bug
[15:50] <brainwash> yeah, I don't understand this, but it made me wonder
[18:25] <kees> (rtg offline?) apw: in the past, usplash needed VM86, but it shouldn't be hard to deal with if that still an issue
[20:26] <infinity> kees: usplash was many, many years ago. :P
[20:36] <mjg59> Yeah sorry about that
[20:38] <ogra_> lol
[20:40] <jdstrand> ogasawara: hey
[20:40] <jdstrand> $ cat /proc/version_signature 
[20:40] <jdstrand> Ubuntu 4.1.0-1.1~dogfoodv1-generic 4.1.0
[20:40] <jdstrand> ogasawara: mic does not work
[20:41] <jdstrand> ogasawara: for a data point, 4.0 that is in wily with the two commits I mentioned yesterday does work
[20:42] <smoser> hey. question on https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1473527
[20:42] <ubot5> Launchpad bug 1473527 in cloud-init (Ubuntu) "module ssh-authkey-fingerprints fails Input/output error: /dev/console" [Undecided,New]
[20:43] <smoser> is there any reason that that should happen ? (IO error on /dev/console)
[20:43] <smoser> where cmdline: console=ttyS1,115200
[20:43] <smoser> and the ttyS1 is seemingly fine (watching it from an ipmi)
[20:44] <smoser> anyone think of some reason that such a thing would occur 
[20:44] <ogasawara> jdstrand: care to file me a bug for tracking?  I think that sforshee on my team (possibly even ppisati) have your same system.  I'll see if they can reproduce.  If not, I'll have jsalisbury walk through a bisect with you when he's back on Mon.
[20:47] <jdstrand> ogasawara: ok
[20:47] <jdstrand> ogasawara: ubuntu-bug linux?
[20:47] <ogasawara> jdstrand: yes please
[20:48] <jdstrand> ogasawara: is it worth updating linux-firmware to what is in wily too?
[20:48] <ogasawara> jdstrand: I don't believe so, has it even been updated for wily?
[20:49] <jdstrand> yes
[20:53] <jdstrand> it doesn't look like there is anything in there that would help this, but there are some other things. I'll update, reboot, retry and file
[21:01] <jdstrand> ogasawara: hrmm, it said it wasn't an official Ubuntu package. how do people normally report against these ppa kernels?
[21:02] <ogasawara> jdstrand: ah shoot, we don't typically have high volume of bug reports against the kernels we push to that ppa
[21:03] <ogasawara> jdstrand: I'd say run the 4.0 kernel which was working to run ubuntu-bug linux (just to gather the system details)
[21:03] <ogasawara> jdstrand: and in the description note the regression with 4.1
[21:03] <jdstrand> actually, I see there are some files in /etc/apport for firefox. let me try a few things
[21:04] <jdstrand> well, my 4.0 would fail too I think (I recompiled it with those two patches)
[21:04] <jdstrand> but yeah, I'll figure out how to report this thing :)
[21:04] <ogasawara> oh right
[21:05] <ogasawara> jdstrand: if all else fails, manually file and I'll still make sure we get eyes on it
[21:06] <jdstrand> ok, thanks
[21:11] <jdstrand> ogasawara: heh, after reading the apport source: APPORT_DISABLE_DISTRO_CHECK=1 ubuntu-bug linux
[21:12] <jdstrand> ogasawara: it still throws up a message about how it is unofficial, but it opens the browser to create the report
[21:13] <ogasawara> jdstrand: oh nice, I'll have to remember that.  I assume it does indeed attach the standard logs to the bug?
[21:14] <jdstrand> it says it will. I'll know in a moment once I finish the description
[21:16] <jdstrand> ogasawara: looks like it: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1473560
[21:16] <ubot5> Launchpad bug 1473560 in linux (Ubuntu) "microphone regression on 4.1" [Undecided,Confirmed]
[21:17] <ogasawara> jdstrand: great, thanks.  I'll take it from here.
[21:18] <jdstrand> ogasawara: thanks!
[21:39] <kees> infinity: yeah, totally, but just in case there were other sneaky things like that still floating around
[21:40] <infinity> kees: As noted in the bug, codesearch.debian.net certainly shows a fair few references to vm86.h, the question is how many have fallback modes or are ifdeffed to oblivion, etc.
[21:40] <infinity> kees: So, it needs a bit of investigation.
[21:48] <kees> infinity: yeah. fwiw, dosemu is fine. ;)
[21:55] <lamont> http://paste.ubuntu.com/11858310/ <-- lazy brain question... is that trace consistent with a "DO NOT BLOCK" kmalloc call?
[21:57] <lamont> trusty kernel
[22:34]  * lamont stands answered