=== lamont` is now known as lamont [00:43] I forget, is Ubuntu using aufs2 on unionfs for live CDs now? [00:44] s/on/or/ [00:54] Womble2: yes aufs2 === bjf is now known as bjf-afk [07:16] Hi guys, I am still having trouble packaging this kernel. I am trying to work out how to get it to run update-initramfs after installation [07:16] I would appreciate if someone could point me in the right direction ? [08:47] mase_wk, when you install the binary .deb the initramfs should get rebuilt automatically [10:33] i had a question [10:33] when you look at any call trace in aoo [10:33] s/aoo/a/ [10:33] so.. when you look at any call trace in a kernel oops message [10:33] you see something like this : [10:34] function_name+num0/num1 [10:34] what is the significance of num0 and num1 ? [10:35] csurbhi: offset into function, total size of function [10:35] ok..thanks jk! [10:35] csurbhi: no problem :) === doko__ is now known as doko === lexical is now known as lexical_afk [11:43] should I file a bug for booting performance I have 1:27 boot time ? [12:10] aboSamoor, with which kernel version? [12:40] apw: 2.6.31-11-generic http://imgur.com/Etuo2.png [12:41] aboSamoor, cat /proc/version_signature [12:47] apw: Ubuntu 2.6.31-11.38-generic [12:47] aboSamoor, doesn't that boot show an fsck running ? an ext4 fsck ? [12:49] apw: that was the last boot and no fsck was done [12:49] there is an fsck.ext4 process running for most of the period? [12:49] apw: nothing was on the screen like a message or progress bar [12:50] now that i can believe ... [12:50] Keybuk, do we expect fsck to trigger usplash correctly currently? [12:50] as in the progress %ages and stuff? [12:51] though if not i would have expected you to see the fsck running on the console in text [12:52] apw: and that is the boot time all the times I turn on my computer. if you mean usplash like the one in 9.04 I can confirm that I did not see it any time in the boot process [12:52] it owuld be worth doing a second bootchart to see if the fsck is also there [12:53] sreadahead had read everything it needed by 20s in [12:53] so in theory you could be at gdb shortly after that [12:53] woot, we're replacing gdm with gdb? :D [12:53] heheh now that would be an interesting interface [12:54] apw: I will restart and return back in 4 minutes [12:59] apw: right [13:00] you don't see anything when fsck is runing atm [13:00] ahhh [13:00] so now to see if tis running fsck every time them [13:00] then [13:01] he has gone to do another bootchart, so if that shows fsck as well we may have an 'always fsck's issue [13:03] apw: this is worse 1:35 http://imgur.com/aXK3K.png [13:04] the previous one showed fsck.ext4, this one shows a quick fsck.ext4 then an fsck.ext3 [13:04] aboSamoor, while you were off it was discussed that fsck does not trigger usplash so you can't tell its going on, known issue [13:04] what's in your fstab? [13:04] yeah ... something odd going on here [13:05] it's not waiting for those fsck to complete [13:05] that means usually that you have several /mnt/* filesystems [13:05] apw: the first chart was for 04/10 this is the correct previous one http://imgur.com/O2VBD.png [13:05] maybe they just hit maximum mount count on subsequent boots? [13:06] yeah bad luck syndrome [13:07] Keybuk: this is the fstab http://paste.ubuntu.com/286930/ [13:07] right, you do have a lot of /media things [13:08] it was most likely those being checked [13:08] bah ... so aboSamoor perhaps one more ... sigh [13:09] the first one would it have been just after an update? [13:10] apw: Keybuk: just for clarification, the boot chart before I restarted was old. The previous boot is http://imgur.com/O2VBD.png and the current boot is http://imgur.com/aXK3K.png. [13:11] apw: do you want one more boot ? [13:11] can't hurt to have more info, need to be sure its not from the first boot after an update [13:13] apw: the current boot was after an update, the coming one won't be. [13:13] aboSamoor: there's nothing unusual in those charts [13:14] Keybuk: 90 seconds is the unusual thing, and it is not my machine because once I installed alpha it was booting in 20 seconds to get firefox, that was like heaven. [13:15] alpha3 [13:15] ok, restarting ... [13:15] aboSamoor: it's an HDD based machine? [13:15] I'd say quite a slow laptop-based HDD too [13:21] apw: Keybuk: back, now something crazy happened. it seems that my machine is upset for three boots in one day. it booted faster, no png generated, there is only .tgz file and no compiz is not working ! [13:21] hmmm [13:21] check dmesg [13:22] and see if drm loaded before intel-agp [13:26] apw: I can not open gedit, the machine is really slow, although the CPU is 0% [13:27] probabally grpahics accelration is not working [13:29] what does it imply if a module for a regulator isnt properly hooked into the kernel regulator code ? [13:30] apw, btw, i attached a few changes to bug 438680 to quiten down the niosy booting on imx51 [13:30] Malone bug 438680 in linux-fsl-imx51 "please quieten down bootmessages" [Low,Triaged] https://launchpad.net/bugs/438680 [13:31] i guess bjf is the one to tickle for this ? [13:31] yeah if he is about [13:31] right, likely to early for him [13:31] i'm a bit worried about all the regulator noise in this list of stuff [13:32] what sort of regulator is it? [13:32] MC13892 i think [13:32] it comes with the FSL patches [13:33] is that a cpu frequency regulator, or a power regulator [13:33] and i suspect its not properly integtared with what the kernel provides [13:33] apw: there was a an error message that drm failed to initialize agpart [13:33] i think MC13892 does both [13:33] aboSamoor, ok then thats likely the init ordering issue we have been seeing very recently [13:33] it has a cpufreq option as well as a power option in the config [13:34] where i915 initialised before agp apature ... that one is known mostly its a reboot job [13:34] apw: :'( 90 seconds is really awesome compared to this. how can I fix it now ?! [13:35] its random on a boot, so the next boot likely won't experience it [13:35] apw: i rebooted three times since the first occurrence [13:36] and its occuring every time? [13:37] apw: before the drm problem do you know any bug report for the 90 second problem that I can subscribe to and report logs. or this drm problem solved the previous one ? [13:37] apw: all the three times [13:37] the slowness overall was all three times [13:37] the latest slowness is graphics slowness, and a separate known issue from the possible boot slowness [13:37] which we've not configmed yet, as i see it from here [13:38] as you were post an install first time, got fsck fun the second time, and this drm issue the third time [13:38] yeah, after you asked me to look at the dmesg I rebooted two times without change [13:38] hrm [13:39] bug #441325 [13:39] * apw slaps ubot3 [13:39] apw: Error: Could not parse data returned by Malone: The read operation timed out [13:39] bug #441325 [13:40] https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/441325 [13:40] is the bug about the drm issue, and has a possible work around [13:40] apw: is shutting down different from rebooting ? [13:40] not normally significantly different [13:40] but yes, different [13:47] apw: I will try this https://bugs.launchpad.net/ubuntu/+source/linux/+bug/430694/comments/32 [13:48] Malone bug 430694 in linux "agpgart-intel not loaded before drm sometimes, causes KMS to fail" [Medium,Triaged] [13:55] apw: the work around but I don't feel the system snappy as it was [13:56] apw: the current boot chart http://imgur.com/NK6vb.png [13:56] if the work around works then that issue is not the cause of anything else, can you report your success in the bug with that workaround please [13:57] apw: you mean to report the the success of the workaround in the drm bug ? [13:57] yeah in the drm agp bug [13:58] Keybuk, are we going with that work-around short term for this bug? [13:59] not my call ;) [13:59] ok will discuss it and get back to you [14:00] apw, I tested a symbol link patch for this, it appears to do the right thing [14:00] a symbolic module dependancy in the kernel i assume? [14:00] frankly, we could use the same method for a number of modules [14:00] yes [14:00] is it a soft or hard dep ? [14:01] hard dep in this case [14:01] in this case a hard dep is fine, but i suspect we are going to find soft deps in there [14:01] is there any support for such a thing did you find? [14:01] * apw suspects there isn't such a thing [14:01] apw, probably, buit it'll fix our short term problem [14:01] yeah sounding good. got a kernle with it in i could test? i suspect my 10v will hit this issue once this update is done [14:02] and i can use it as a test [14:02] apw, I couldn't find anything, so I'm gonna do a MODULE_DEP macro [14:02] gimme a bit, I've got some other issue on my plate. [14:02] apw: I have to go, is the last boot chart helped to investigate the slowness problem ? [14:02] sounds like a good plan to me [14:03] not sure if i see anything unsual in this final one [14:04] apw: the problem that i feel that filing a bug report that my boot is slow is stupid. there could be tens of reasons, so if you know any related bug please point it to me so I follow the issue there :). [14:04] Keybuk, How is/where can I check how the start of X is serialized with the rest of the system? I got the feeling there is a race when using the nvidia binary driver between that and things like the acpid start [14:05] aboSamoor, i don't know of any specific one [14:06] apw: boot performance is targeting any package ? [14:06] its not obvious which package to target, as it depends on the cause [14:06] is boot slower now than it was in jaunty? [14:07] apw: it seems that I have to consider installing karmic again ! [14:08] apw: Keybuk: thanks very much for the support :) [14:13] smb, it looks like X generally talks to acpid as i get this in my Xorg.0.log [14:13] (II) Open ACPI successful (/var/run/acpid.socket) [14:14] so it sounds like there is a race generally available if you are getting failures there? [14:14] I had on nvidia and i915 netbook [14:14] (WW) Open ACPI failed (/var/run/acpid.socket) (No such file or directory) [14:14] yeah my netbook has a failure there [14:14] what is the downside? x eating lots of cpu? [14:14] why does X depend on acpid? [14:14] I did not see as bad things as on the nvidia [14:15] interestingly everything appears to be ok with that failure reported [14:15] Keybuk, I would think that it can catch vt switches [14:15] or could (befre kms) [14:15] i think it gets those 'direct' without KMS [14:16] asking on #u-x seems to say that its not used for much and disabled in fedora [14:17] It seems to be a generic issue that X might be starting with some of the files/sockets it tries to open not yet being present. But it seems worse on the nvidia binary driver [14:17] I see tons of ioctl(5, TCFLSH, 0x2) = -1 EIO (Input/output error) [14:17] with 5 being vt7 [14:17] ok yeah the X folks are saying it 'does' connect but its stupid sas we don't get anything from it [14:18] or rather /dev/tty7 [14:18] smb, i wonder if those are unrelated to acpi then [14:18] apw, It well might be. I just noticed this failing while comparing things [14:18] ok so X concensus is we can ignore any acpi failure, so there is no need for any new deps for that [14:18] but I could not change vt's in that case [14:18] on nvidia that is [14:19] yes, but that might not be related to acpid [14:19] there may be something else which comes up too slow that matters, like the initial vt switch perhaps? [14:19] apw, I would suspect that to be related [14:19] Without kms, acpi generates an output switch event [14:21] Just with i915/kms which is in the kernel that has no need anymore to get that from the acpid socket [14:23] apw, Ok, at least for my case this is not true. You would only get any events for internal devices [14:24] i thought that old school VT switching is done directly from the kernel via the FD for the VT, and it was vile, but i am not 100% sure [14:25] well there is DOS, which controls what the bios does on the switch [14:25] it can be doing something or only generating events or some other setting i forgot [14:26] apw, You can see the setting in /proc/acpi/video/*/DOS [14:26] what does it mean? i have 7 [14:26] oh thats vt 7? [14:27] no just coincidence [14:27] it means, bios should not automatically toggel output, but generate events and brightness should also not get automatically changed on ac-dc [14:30] ok #u-x confirm vt switching is not triggered by acpid when non-kms, no idea how it is ... but hey [14:31] apw, Thinking of it, I really confused output switching with console switching mechanics. doh [14:32] So switching consoles is just a normal keybord combo [14:32] the thing that DOS handles is the lcd to crt switch [14:34] That I am unable to switch consoles in the nvidia case probably is more related to the io error on tty7... [14:35] yep === tjaalton_ is now known as tjaalton === soren_ is now known as soren [16:52] ** [16:52] ** Ubuntu Kernel Team Meeting - Today @ 17:00 UTC - #ubuntu-meeting [16:52] ** [17:07] I check for just delaying too. Generically, although we seem to fail to see the reason, X tried to connect to the acpid socket on startup. Giving somtimes warnings in the x log which would be nice to avoid as well [17:08] X peeps tended to say it was stupid we were now connecting there, and implied we should be taking some patch to turn it off [17:08] even if it has no use. The only downside might be delaying the start of X for a litle. would need to bootchart that [17:10] we and now? isn't that them and maybe longer? [17:10] X does nothing useful whatsoever with that information [17:11] It seems to be a generic X thing to me as it happens on i915 and nvidia based startup [17:11] Horribly broken design that was used in the i810 driver, except even that got it wrong [17:11] Ok, so they could just not do it [17:11] Yeah, should really be ripped out entirely [17:12] I initially though of DOS handling... [17:13] Nope === fddfoo is now known as fdd === cking_ is now known as cking [18:04] rtg: "CarlFK, right. should be in the next upload, Monday perhaps?" didn't seem to happen. any chance of it happening this week? or am I not looking in the right place... [18:05] where it and place is http://archive.ubuntu.com/ubuntu/dists/karmic/main/installer-amd64/current/images/netboot/ubuntu-installer/amd64/linux [18:14] CarlFK, Update should have happened (only recently). Not sure whether meta update went out too [18:14] smb: so I should wait a few hours for things to propagate out? [18:17] CarlFK, Have not really seen the meta uploaded but maybe I just missed that. Lemme check [18:17] smb, meta is up [18:18] So, CarlFK should see a -12 by now [18:18] soon [18:18] soon is what I was expecting - thanks [19:09] hello all, I would like to raise a particular kernel problem with the RTL8187B wifi chipset in the past 3 releases of Ubuntu, (8.10,9.04,9.10). https://help.ubuntu.com/community/WifiDocs/Device/RealtekRTL8187b bucky, I have described the problem here its with the RTL8187B chip is my wiki on the chipset, It works out of box starting with 8.10, but its terribly inefficient and has been to today. can anyone shed light on this? point me to the pr [19:09] oject development? [19:16] tj83_, what pci id's or usb id's is it using [19:17] Bus 001 Device 002: ID 0bda:8197 Realtek Semiconductor Corp. [19:18] apw please if you will review the above ubuntu wiki URL [19:18] that page says little other than 'it seems to be supported post 2.6.27, and performance sucks' [19:18] thats what we got to work with :P sorry, i am not a kernel hacker or programmer [19:20] i do know that the auto rate adjustment is not happening correctly, for instance at an more than reasonable distance if i cannot get traffic flow iwconfig wlan0 rate 1M gets it crawling at minimum. [19:20] it does seem to be being updated and has changed in 2.6.31, so you may find it is better in the karmic kernel [19:20] I am using it currently apw [19:20] and its all the same [19:21] one assumes that realtek is not releasing the information required to improve the driver, and thats a tricky issue to get round [19:21] only difference i can see is that in dmesg the took out the warning about frying your hardware [19:22] apw, i was under the impression this was a total re-development [19:22] indeed they don't seem to think its fooking your hardware any more [19:23] bucky in #ubuntu+1 shared this with me just now: http://www.linux-archive.org/ubuntu-kernel-team/374927-fwd-realtek-rtl8187b-driver-karmic.html [19:24] so it looks like the missed the freeze kernel boat :( [19:26] apw bucky said it might be possible to build a dsc file... what is this and how difficult? i'ma noob [19:26] people are reporting from linux mint that its fixed for them [19:26] i will test to confirm. would be nice to be able to bring back into karmic [19:27] well the first thing to do would be to do as pgraner suggested and send the patch updating the driver to the kernel-team list, so it can be reviewed to see just how big a change it is [19:27] if its fixing 'bugs' we may be able to take it depending how invasive the fixes are, and how they affect other cards which use the same driver [19:28] also if those fixes are upstream, we may have them in wireless-testing which leads into LBM [19:28] sorry to chime in late. maybe it really helps to try out the recent rt2x patches. Seems those go all over rt drivers [19:29] smb, nah, appreciate it, i mean i was starting to think this was a lost cause.... 3 ubuntu versions later [19:29] rt2x patches? [19:29] http://patchwork.ozlabs.org/patch/35112/ [19:29] http://patchwork.ozlabs.org/patch/35113/ [19:29] its only a lost cause if we aren't getting info from the manufactuer, it sounds like they are coming round to helping [19:30] smb, think the driver in question is not 2x:drivers/net/wireless/rtl818x/ [19:30] Ah damn, then its just another one [19:31] But generically the same problem. One steps in the dark as long as there is no info about which bit to poke correctly [19:32] http://www.linux-archive.org/ubuntu-kernel-team/374927-fwd-realtek-rtl8187b-driver-karmic.html <--- i dont even know what i am looking at here.. but seems like ton of settings [19:32] this guy claims "fixed" [19:33] yeah but he didn't supply the new driver. [19:33] just says that he has it and it works for him [19:33] which isn't much use to us [19:34] well, then, i guess i should hunt this dude down. [19:34] if you can't find the driver posted to the kernel-team list, its worth poking him and asking him to do so [19:34] as even if it misses karmic, we want to try and do something for lucid [19:34] "extends wireless range from 30' to ~120'." i mean c'mon 30ft? and this is exactly the behavior i observe [19:34] right on... i am stoked about the next LTS [19:35] apw, The initial mail had a tarball, but it came quite late to just replace drivers [19:36] yeah agree its very late, but it would be nice to see the diff so we can tell if its major or nothing or ... [19:36] smb, does this mean you have the implied "fixed" drivers? [19:36] I have not looked into the tar yet, but there is something [19:37] smb we (many many many of us) are at your mercy [19:37] if you look at my wiki you will see a number of reported laptops and that is just a drop int he bucket of the number of instances of this hardware out there. its everywhere and we all have the same issue. [19:37] We (apw and me) need to find a quiet moment to look at it. [19:37] its a busy busy time for us, the release is coming [19:38] atm quiet moments a a bit scarce [19:38] * apw looks at the cover of his hitch-hikers guide and sighs [19:38] apw, we can try not to panic :) [19:38] :) [19:39] tj83_, Maybe there is a posibility to get it into bacports modules after release [19:39] i meant to ask is the new driver not upstream yet? if so LBM may get it magically without help [19:40] That might be a second posibility, yeah [19:40] one would hope realtek isn't just sending out updated drivers to people in the community and not to mainline [19:41] If we see it there it comes via compat-wireless, but that again needs a bit of time to verify [19:41] doesn't appear to be in 32-rc3 [19:41] could be in wireless testing of course [19:41] smb hitch hikers guide to the galaxy? i got the original BBS on my FTP :) share if you like [19:41] apw, just uploaded this am [19:42] rtg lbm ? [19:42] yep [19:42] omg, this is great... some real activity happening here! thank you all [19:42] tj83_, I got the big book and the original BBC series. :) thanks [19:42] tj83_, heh we are pretty active all the time, just not as obvious as one might ope [19:43] never doubted you at all... just thought that it was "forgotten" [19:43] hope ... there is a lot of blood, sweat, and no end of tears in every upload [19:43] forgotten no, burried in 100's of other issues likely yes [19:44] pgraner, have you retested your i915 suspend/power issue? [19:44] so, being a n00b like me... what can i do to help this along? [19:44] you can try and find the launchpad bug for this issue that would help us out [19:45] apw, i have actually searched.... but i will search again sure... will likely do this from class tonight, as its about time for me to head out of work, and onto school. [19:46] if you can't find one then one needs to be filed ... and the tarball of the driver attached, and if we can find out where it actually came frmo that would be helpful [19:46] apw i will do what i can, appreciate your and smb and rtg diligence on the matter. [19:46] the wireless-testing driver looks identicle to the mainline one, which is hardly modded from the one in karmic, so not sure its going to be any different [19:46] :( [19:47] so then its getting the information all together in the bug, so that when someone does have time they don't have to go searching all over for it [19:47] tj83_, that really would be a time saver for someone, and make it more likely to get looked at sooner [19:48] Is Karmic going to be released with 2.6.31 or with a later kernel? 2.6.30 introduced a regression wrt prism wifi cards that was fixed in 2.6.32 which is why I ask. [19:48] apw consider it done first opportunity I can, I'll follow up with you guys here too [19:48] http://bugzilla.kernel.org/show_bug.cgi?id=14000 [19:48] bugzilla.kernel.org bug 14000 in network-wireless "prism 2.5 broke in 2.6.30.x" [Normal,Closed: code_fix] [19:48] Laibsch, karmic will ahve a 2.6.31.2 based kernel [19:49] in all likelyhood, cirtainly a 2.6.31 base [19:49] OK, then we should look into trying to backport a patch [19:49] do we know the fix for it? if so then we want to get a LP bug filed with that upstream bug linked in [19:50] apw there is also another angle to look at this... if Ubuntu can fix it first... other users of different distributions may migrate to ubuntu and well thats the whole point [19:50] if it worked in jaunty then we need to get it tagged regression-potential [19:50] apw: I'll create a new ticket in a minute [19:50] tj83_, yep we have loads of things like that though, many of them already in, we literrally cannot fix every bug there just tooo many [19:51] and bugs which have some of the work done are more appealing than those without as they take less dev time to get fixed [19:51] i understand... just saying been many turned off of ubuntu because of this. [19:55] many thanks again. off to class. [19:56] tj83_, yep and we're not un-sympathetic ... we litterally can't do everything [19:56] apw, do we have a known victim that suffers from the agp/i915 race? [19:57] nobody can... we are all human. [19:57] rtg yeah there were a couple of people who seemed to be affected every boot on the bug [19:57] bug #430694 i think it was [19:57] Malone bug 430694 in linux "agpgart-intel not loaded before drm sometimes, causes KMS to fail" [Medium,Triaged] https://launchpad.net/bugs/430694 [19:58] apw, I wanna sic my module_depends patch on 'em. I've tested here on a mini-9 [19:58] there was a second on here, who i asked to report in on the same bug [19:58] apw: bug 444801 it is [19:58] Malone bug 444801 in linux "prism 2.5 broke in 2.6.30.x (fixed upstream in 2.6.32)" [Undecided,New] https://launchpad.net/bugs/444801 [20:06] thanks, apw [20:06] later guys [20:23] Can anyone here please point me to documentation on how to install either LKCD or KDUMP? I need to grab a kernel core to analyze for a high load issue on 9.04 64-bit. [20:23] Or other kernel coredump method? [20:26] superm1, ping .. do you have any notes on how to package src for dkms ? [20:27] r0cketman, https://wiki.ubuntu.com/KernelTeam/CrashdumpRecipe [20:27] Perfect, rtg. Thanks tons! [20:28] bjf, did you just do a dist-upgrade to get your pegatron up to Karmic? [20:30] rtg, no [20:30] rtg, 1) I don't have a pegatron [20:30] rtg, I have a babbage 2.5 board [20:31] bjf, you don't have an imx51 based platform? [20:31] rtg, yes, that is a babbage 2.5 reference board [20:31] rtg, a pegatron board is also imx51 based but it is different [20:31] bjf, well, they must install the same, right? [20:32] hi rtg [20:33] rtg, they are similar, yes, but probably need a different redboot image [20:33] Kano, ? [20:34] did you add the missing firmwarefile [20:34] Kanaren't you subscribed to the LP report? [20:35] Kano, ^^ [20:35] no, just looked at it [20:35] i have got 2 other patches for gspca you might be interested [20:35] one is tested by a kanotix user already [20:36] http://linuxtv.org/hg/~eandren/v4l-dvb/rev/3cf50d4a5aab [20:36] thats tested and working [20:36] Kano, licensing has been a real issue with a lot of the video firmware. [20:37] in the same hg you find 2 other similar patches [20:37] so i would add all 3, all affect the same file [20:37] they flip the picture in laptops [20:40] as it is hard to hold it upside down all the time ;) [20:40] Kano, file a bug report with clear directions about where to retrieve the firmware files from, as well as their licensing disposition. [20:40] for the same reason i would update libv4l to latest version too, as for uvcvideo the flipping is done in the userspace [20:41] rtg: you added the firmware it seems, thats nice, just those very simple patches would be nice to have too [20:41] i added em on my kernel variant,but thats suboptimimal [20:42] Kano, nothing is going to happen without a bug report. [21:12] manjo, follow http://wiki.centos.org/HowTos/BuildingKernelModules#head-d313bd351f90d4f25a2143b7bbcff73f927731f0, and then you can use dkms mkdeb or dkms mkdsc on the module [21:12] the packages produced should be in good order, lintian compliant etc [21:12] superm1, cool thanks for the pointer [21:14] manjo: what module do you need [21:14] Kano, omnibook [23:27] so... if I need to run a 2.6.28-11 kernel with karmic, does that work? [23:29] sure [23:29] but current .28 is -15 [23:29] good. though it will make me cry [23:30] Kano: yeah - -15 is b0rken [23:30] -11 is love. [23:30] whats the problem with 15 for you [23:30] -12 thru -14 are untested [23:30] kvm booting a windoze XP Pro SP3 image (or SP2, iirc) [23:31] ah [23:32] yeah - I somehow fear I'll be doing the git-bisect to find it one of these days [23:32] though there was a monsterous kvm/* change between -11 and -15, which seems likely [23:51] well in theory you could do a gib bisect, using ccache with a huge cache [23:51] did you consider doing this