| === sturmflut_ is now known as sturmflut | ||
| apw | SturmFlut, i am sure someone was somewhere in the world | 07:43 |
|---|---|---|
| sturmflut | Haha | 07:44 |
| sturmflut | I've debugged most of the problem and filed a bug report in the meantime, looks like the original problem is in ModemManager, but might affect the kernel as well in the end. | 07:51 |
| ogra_ | sturmflut, but not a kernel anyone in here maintains :) | 07:53 |
| sturmflut | ogra_: Shouldn't the cdc_acm module of my desktop kernel blacklist that specific USB ID, because it will never work anyways and just confuse the russians? | 07:55 |
| ogra_ | dont underestimate the cleverness of the russians :) | 07:55 |
| ogra_ | and i'm not sure its a kernel thing ... did you check that modemmanager doesnt ship udev rules that cause the loading ? | 07:56 |
| sturmflut | Hmm, now how does this udev stuff work with systemd... | 07:57 |
| ogra_ | heh, te same as with other init systems :) | 07:58 |
| ogra_ | (i dont think systemd comes into play here ... but i guess modemmanager ships udev rules that react to uevents related to serial) | 07:58 |
| sturmflut | ModemManager ships some rules, not to load the module for this device, but to tell itself to blacklist a lot of similar cases. So the simple solution would be to at least add the device there, then cdc_acm gets still loaded and fails, but ModemManager does no longer claim the device. | 08:08 |
| sturmflut | Not pretty, but I can live with that. | 08:08 |
| rtg | apw, ogasawara_ upstream discussion on CONFIG_VM86 http://thread.gmane.org/gmane.linux.kernel/1991020 | 12:41 |
| rtg | kees suggests we turn it off | 12:42 |
| apw | rtg, yeah someone claims we have it off already, but it looks to be on to me | 12:48 |
| rtg | yup, their claim was erroneous | 12:48 |
| apw | rtg, it sounds like a security disaster in the making, so ripping it in wily and seeing if anything explodes seems good | 12:48 |
| ogasawara_ | rtg: ack, do you already have a patch to push to wily to turn it off? | 12:48 |
| rtg | ogasawara_, I was just disabling it in unstable | 12:48 |
| rtg | ogasawara_, I can do wily master-next as well | 12:49 |
| ogasawara_ | rtg: ack | 12:49 |
| infinity | Didn't we just turn that on in an SRU a few months back or something? | 12:50 |
| infinity | It seems awfully familiar. | 12:50 |
| infinity | Maybe I'm thinking of something else with a similar-sounding name. | 12:53 |
| rtg | infinity, not finding any CONFIG_VM86 changes in trusty | 12:55 |
| infinity | rtg: Ahh, yeah, I was thinking of CONFIG_X86_16BIT, nevermind. | 12:55 |
| infinity | So, VM86 seems scary and evil as all hell, I'm inclined to agree, but we need to make sure we don't need it for any old skool X drivers (vmware, vbox, etc) | 12:56 |
| infinity | Oh, but if it doesn't even work on x86_64, my carefactor just went down. | 12:58 |
| * infinity catches up on the rest of the thread. | 12:58 | |
| rtg | infinity, it was 32 bit only in our configs | 12:59 |
| infinity | rtg: Yeah. I'd be inclined to say we should disable it across all stable kernels too, then, as a pre-emptive security feature. | 12:59 |
| infinity | rtg: If no one was hunting for holes in it before, you can bet that thread will make grey hats start looking. | 13:00 |
| infinity | rtg: There are one or two X drivers that are i386-only. If we can verify that they don't need VM86, I'd say the impact is minimal-to-none and worth the risk. | 13:01 |
| rtg | infinity, agreed, though the attack surface is much smaller given that it is only applicable to 32 bit machines. | 13:01 |
| rtg | infinity, I'll run this by jjohansen and tyhicks as well | 13:02 |
| rtg | perhaps 32 bit cloud VMs are vulnerable | 13:03 |
| infinity | rtg: 32-bit still has an astoundingly large install base. | 13:04 |
| infinity | rtg: And certainly true going back to precise, where it was still our recommended image download. | 13:05 |
| rtg | yeah, I'd be willing to bet there are a lot of 32 bit instances in the cloud | 13:05 |
| infinity | rtg: Probably fewer than there should be (ie: I'd argue that most "tiny" and "small" instances should be i386, even if backed by amd64 hardware), but yeah, there are certainly some. | 13:06 |
| rtg | infinity, working on an SRU | 13:07 |
| infinity | rtg: I think we need a userspace impact analysis before we SRU it, but my hope is the impact is "none". | 13:08 |
| infinity | rtg: My only concern is ancient X drivers that haven't seen significant work in the last decade, and I think we only ship a couple of those. | 13:09 |
| rtg | infinity, it seems our choice is to orphan some vanishingly rare drivers, or expose possible vulnerabilities to every 32 bit user. | 13:10 |
| rtg | orphan -> regress | 13:11 |
| infinity | rtg: Hrm. And vbetool. That might need a looking at as well. | 13:11 |
| rtg | I'll make that part of the SRU discussion | 13:11 |
| infinity | rtg: But yes, we're on the same page. I think the goal should be disabling VM86, we should just sort out the impact and see if we can minimize damage. | 13:11 |
| infinity | rtg: http://codesearch.debian.net/results/sys%2Fvm86.h | 13:14 |
| infinity | rtg: and http://codesearch.debian.net/results/asm%2Fvm86.h | 13:15 |
| infinity | rtg: Some might be false positives that we don't enable, others are probably build-time detected and would be broken if we regress the kernel support. | 13:16 |
| rtg | infinity, I guess Linus will howl if Lutomirski starts breaking user space | 13:18 |
| infinity | rtg: Well, it's never been a required kernel option, so it doesn't break userspace to disable it by default, by Linus's definition of break. | 13:20 |
| infinity | rtg: But distributions need to be aware of impact on already-shipped software if they disable features they used to enable. That's where it gets sticky. | 13:21 |
| rtg | infinity, is userspace supposed to fall back into an emulation mode ? | 13:21 |
| infinity | rtg: I'd have to look at the glibc syscall wrapper. It's possible that people using the abstracted sys/vm86.h are safe. | 13:22 |
| infinity | rtg: People using asm/vm86.h directly, though, are definitely broken if they don't have their own fallbacks in place. | 13:22 |
| infinity | rtg: I highly doubt we have an 8086 emulator in glibc though (in fact, I'm sure we don't), so in either case, if people are relying on the syscall without a fallback option, they'd explode regardless, I think. | 13:24 |
| rtg | infinity, would they explode at runtime or build time, or both ? | 13:25 |
| infinity | rtg: I'd assume just runtime, if the headers are still exposed when the option is off. | 13:25 |
| infinity | rtg: If the headers go away completely, then build time, but given the number of things that include the headers, I suspect that's not intended to happen. | 13:26 |
| rtg | thank you LP for blowing up and _not_ accepting my bug report. | 13:26 |
| infinity | rtg: LP is suffering some major infra mojo right now. | 13:27 |
| rtg | infinity, bug #1473447 | 13:27 |
| ubot5 | bug 1473447 in linux (Ubuntu) "linux: VM86 should be disabled" [Undecided,New] https://launchpad.net/bugs/1473447 | 13:27 |
| rtg | infinity, we've got it disabled in Wily (4.1+), so we can experiment with the fallout | 13:28 |
| infinity | rtg: Yeahp, that seems like the safe approach for now. We can abuse wily userspace a bit, cross-referencing with codesearch (which should have 100% coverage for what we care about, I'd be VERY shocked if we ship anything Ubuntu-only that uses vm86), and then sort out conclusions. | 13:29 |
| infinity | rtg: codesearch's only failure here is that it's only search unstable, so doesn't give us a clear view of the possibility that precise might be more broken than trusty or wily. | 13:31 |
| infinity | rtg: But it's a start anyway. | 13:31 |
| infinity | I really think we need to set up an Ubuntu codesearch instance that covers all supported series'... It's crazy useful. | 13:32 |
| rtg | infinity, seems like we should certainly have it disabled by 16.04, so experimenting on 15.10 makes sense | 13:32 |
| infinity | rtg: Yeahp, and in 15.10/16.04, I have zero issues with the idea that if some bits of userspace break without it, we can just deprecate/remove them if they're not fixable. | 13:33 |
| infinity | rtg: But that's a harder pill to swallow for precise and trusty. | 13:33 |
| rtg | agreed | 13:33 |
| * cking wonders why the watchdogs are going nuts: https://pastebin.canonical.com/135040/ | 13:48 | |
| infinity | cking: That's... Impressive. | 14:02 |
| infinity | cking: I've never seen a kernel try so hard to starve userspace. | 14:02 |
| cking | infinity, it did have some repercussions: https://pastebin.canonical.com/135045/ | 14:04 |
| pointertonullval | Hey guys I'm building my own embedded device and it will connect over usb to a computer. How I can start writing a linux driver? Is a kernel level problem or more high level? Thanks | 14:10 |
| apw | usb is interesting because there is a userspace library interface iirc, so you can write things out of the kernel | 14:11 |
| TJ- | pointertonullval: Look at the USB 'gadget' interface | 14:12 |
| pointertonullval | apw, TJ- thanks! something like this? http://www.linux-usb.org/gadget/ | 14:15 |
| unixabg | apw: it appears that casper does not like the multiple squashfs files on boot. But i am fairly | 14:34 |
| unixabg | confident that is sees them since it does make the mount points and then fails (I think on | 14:35 |
| unixabg | getting the overlay right). | 14:35 |
| apw | it is possible that casper is trying to lay down multiple layers of overlayfs and not using the multiple layer support | 14:39 |
| apw | for which there likely is no way to figure out if the functionality exists in the kernel ... hmm | 14:39 |
| infinity | apw: Does casper have support for multiple layers at all? That would seem odd, if it's a new kernel feature. | 14:41 |
| apw | infinity, it has some support in the sense they are layerable, ie mount the bottom two together, then mount the results with the next layer up | 14:43 |
| apw | though you soon hit the stack depth limit in the kernel, so i think we bust on overlayfs at two mount layers, ie 3 layers | 14:43 |
| apw | unixabg, how many layers are there ? | 14:43 |
| infinity | apw: Anyhow, sorting out if functionality exists should be a non-issue, casper isn't the sort of tool one uses "blindly". It should just do what you ask it to do, and if you're using a kernel that can't support that, you get to keep both pieces. | 14:45 |
| apw | thoug in a sense we have some way to do it two ways with differnet limits in each case | 14:46 |
| apw | and the second way only if you have a newer kernel, which doesn't tell you it can do it, though i guess for casper at least we kinda know what its built with | 14:47 |
| infinity | apw: If it's a question of mount options changing to support way 1 and way 2, and there's a distinct upstream version cutoff to go from 1 to 2, you could case uname -r. | 14:48 |
| infinity | Or, not case, but convert-and-compare, probably. | 14:49 |
| unixabg | apw: on my test with wily it fails by just adding a second layer. It may be helpful if I outline | 14:49 |
| unixabg | how I am testing, I basically am doing a quick remaster and I am generating my partial squashfs | 14:49 |
| apw | infinity, ugg you are not putting me in my happy place | 14:50 |
| unixabg | files with live-partial-squashfs-updates: | 14:50 |
| unixabg | http://live.debian.net/gitweb/?p=live-tools.git;a=blob_plain;f=bin/live-partial-squashfs-updates;hb=refs/heads/debian-next | 14:50 |
| infinity | apw: Do I ever? | 14:51 |
| unixabg | once I generate a partial-squashfs-update (psu) against the filesystem.squashfs then I button | 14:51 |
| apw | infinity, those days you offer to take me to the pub, yes | 14:51 |
| unixabg | the image back together and boot it. But it is not working. I did test one layer was working on | 14:51 |
| unixabg | linuxmin17.1 I think, mom to check that, | 14:52 |
| unixabg | yes that is correct, but any more than filesystem.squashfs and psu1.squashfs caused failure. | 14:53 |
| unixabg | Let me run one more quick test, since late yesterday and I had many tests going I do believe that now | 14:54 |
| unixabg | I can not even stack on one psu.squashfs. Be back in a short bit... | 14:54 |
| unixabg | apw: ok wily failed when I tried to add only one psu.squashfs along with filesystem.squashfs, next testing | 14:58 |
| unixabg | linuxmint17.2 | 14:58 |
| apw | unixabg, can't look at it right now, but it would be good to get a dmesg if you can to see if it is throwing a depth limit in there somewhere | 15:07 |
| apw | report any testing here and i'll see if i can figure it out | 15:07 |
| unixabg | apw: humble appreciation. Ok now I just booted linuxmint17.2 + 1psu.squashfs and it is ok. | 15:13 |
| unixabg | s/and/and remastered and booted live,/g | 15:14 |
| apw | unixabg, whihc kernel does that have in it | 15:16 |
| unixabg | apw: uname -r produces 3.19.0-20-generic | 15:18 |
| unixabg | apw: dmesg in an image: http://fyeox.com/downloads/20150710-loop1fail-wily.png | 15:20 |
| unixabg | It could be that it does not have the correct kernel version or something. Anyhow I am willing to | 15:22 |
| unixabg | assist however I can. I want the multiple squashfs to work well. | 15:22 |
| brainwash | cking: hey, I noticed bug 1470845 too. should this be forwarded somewhere upstream? | 15:44 |
| ubot5 | bug 1470845 in systemd (Ubuntu) "systemd on wily desktop generating short lived threads every second in a quiet system" [Undecided,New] https://launchpad.net/bugs/1470845 | 15:44 |
| brainwash | installed forkstat some minutes ago | 15:45 |
| cking | brainwash, I was hoping it would get some triaging and attention on the ubuntu side of things, it maybe the way it's configured at the moment and is a temporary issue | 15:45 |
| brainwash | cking: it? what could be misconfigured? | 15:48 |
| cking | brainwash, i don't know, that's why I filed the bug | 15:48 |
| brainwash | ok :) sadly, I only have access to ubuntu wily right now | 15:49 |
| brainwash | and I've never used forkstat before | 15:49 |
| infinity | cking: Did that happen before 222? | 15:49 |
| cking | infinity, I don't know either | 15:49 |
| brainwash | 222 was pushed just recently | 15:50 |
| cking | it's the "oh, I've noticed something that looks suspect and I don't know why or when it happened" kinda bug | 15:50 |
| brainwash | yeah, I don't understand this, but it made me wonder | 15:50 |
| kees | (rtg offline?) apw: in the past, usplash needed VM86, but it shouldn't be hard to deal with if that still an issue | 18:25 |
| infinity | kees: usplash was many, many years ago. :P | 20:26 |
| mjg59 | Yeah sorry about that | 20:36 |
| ogra_ | lol | 20:38 |
| jdstrand | ogasawara: hey | 20:40 |
| jdstrand | $ cat /proc/version_signature | 20:40 |
| jdstrand | Ubuntu 4.1.0-1.1~dogfoodv1-generic 4.1.0 | 20:40 |
| jdstrand | ogasawara: mic does not work | 20:40 |
| jdstrand | ogasawara: for a data point, 4.0 that is in wily with the two commits I mentioned yesterday does work | 20:41 |
| smoser | hey. question on https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1473527 | 20:42 |
| ubot5 | Launchpad bug 1473527 in cloud-init (Ubuntu) "module ssh-authkey-fingerprints fails Input/output error: /dev/console" [Undecided,New] | 20:42 |
| smoser | is there any reason that that should happen ? (IO error on /dev/console) | 20:43 |
| smoser | where cmdline: console=ttyS1,115200 | 20:43 |
| smoser | and the ttyS1 is seemingly fine (watching it from an ipmi) | 20:43 |
| smoser | anyone think of some reason that such a thing would occur | 20:44 |
| ogasawara | jdstrand: care to file me a bug for tracking? I think that sforshee on my team (possibly even ppisati) have your same system. I'll see if they can reproduce. If not, I'll have jsalisbury walk through a bisect with you when he's back on Mon. | 20:44 |
| jdstrand | ogasawara: ok | 20:47 |
| jdstrand | ogasawara: ubuntu-bug linux? | 20:47 |
| ogasawara | jdstrand: yes please | 20:47 |
| jdstrand | ogasawara: is it worth updating linux-firmware to what is in wily too? | 20:48 |
| ogasawara | jdstrand: I don't believe so, has it even been updated for wily? | 20:48 |
| jdstrand | yes | 20:49 |
| jdstrand | it doesn't look like there is anything in there that would help this, but there are some other things. I'll update, reboot, retry and file | 20:53 |
| jdstrand | ogasawara: hrmm, it said it wasn't an official Ubuntu package. how do people normally report against these ppa kernels? | 21:01 |
| ogasawara | jdstrand: ah shoot, we don't typically have high volume of bug reports against the kernels we push to that ppa | 21:02 |
| ogasawara | jdstrand: I'd say run the 4.0 kernel which was working to run ubuntu-bug linux (just to gather the system details) | 21:03 |
| ogasawara | jdstrand: and in the description note the regression with 4.1 | 21:03 |
| jdstrand | actually, I see there are some files in /etc/apport for firefox. let me try a few things | 21:03 |
| jdstrand | well, my 4.0 would fail too I think (I recompiled it with those two patches) | 21:04 |
| jdstrand | but yeah, I'll figure out how to report this thing :) | 21:04 |
| ogasawara | oh right | 21:04 |
| ogasawara | jdstrand: if all else fails, manually file and I'll still make sure we get eyes on it | 21:05 |
| jdstrand | ok, thanks | 21:06 |
| jdstrand | ogasawara: heh, after reading the apport source: APPORT_DISABLE_DISTRO_CHECK=1 ubuntu-bug linux | 21:11 |
| jdstrand | ogasawara: it still throws up a message about how it is unofficial, but it opens the browser to create the report | 21:12 |
| ogasawara | jdstrand: oh nice, I'll have to remember that. I assume it does indeed attach the standard logs to the bug? | 21:13 |
| jdstrand | it says it will. I'll know in a moment once I finish the description | 21:14 |
| jdstrand | ogasawara: looks like it: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1473560 | 21:16 |
| ubot5 | Launchpad bug 1473560 in linux (Ubuntu) "microphone regression on 4.1" [Undecided,Confirmed] | 21:16 |
| ogasawara | jdstrand: great, thanks. I'll take it from here. | 21:17 |
| jdstrand | ogasawara: thanks! | 21:18 |
| kees | infinity: yeah, totally, but just in case there were other sneaky things like that still floating around | 21:39 |
| infinity | kees: As noted in the bug, codesearch.debian.net certainly shows a fair few references to vm86.h, the question is how many have fallback modes or are ifdeffed to oblivion, etc. | 21:40 |
| infinity | kees: So, it needs a bit of investigation. | 21:40 |
| kees | infinity: yeah. fwiw, dosemu is fine. ;) | 21:48 |
| lamont | http://paste.ubuntu.com/11858310/ <-- lazy brain question... is that trace consistent with a "DO NOT BLOCK" kmalloc call? | 21:55 |
| lamont | trusty kernel | 21:57 |
| * lamont stands answered | 22:34 | |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
