[02:53] ogasawara: I have the powerpc-e500mc flavor done and ready to pull, I'm just waiting for the full powerpc build to finish to make sure the entire powerpc arch builds [02:53] ogasawara: when do you expect the next upload to quantal with the rebase in it? [06:35] I am trying to convert a sample application into kernel module, any equivalent of Posix message queues (mq_send, mq_recv) ? [06:43] I am trying to convert a sample application into kernel module, any equivalent of Posix message queues (mq_send, mq_recv) ? [07:13] moin [07:17] morning [08:56] apw, https://bugs.launchpad.net/bugs/657901 [08:56] Launchpad bug 657901 in linux "linux-virtual depends on wireless-crda and contains wireless modules" [Wishlist,Triaged] [08:56] Wondering whether now the main package could not depend on crda and the extra modules could [08:57] cirtainly can if that is where they are [08:57] smb, ^^ [08:57] apw, yeah, need to check though not clear it is [08:58] Would it have effect on the installer? Though I guess the way it gets modules is still udebs and those would be independent in packaging [08:59] right, and the live installer installs both packages, so we could split them out any way we wish [08:59] indeed the splitting system has been fixed up so it can have as many splits as we require [09:01] apw, That would be for quantal. For precise maybe similar, though then for the real virtual package. Not sure this is not already that way (need to download some pakcages to look) [09:03] smb, that bug is confusigly old and new at the same time, we may have fixed it for a few releases then reintroduced it [09:03] apw, I am not sure we ever really fixed it completely [09:04] I assume the dependency to crda was always with the linux-image package, even with the wireless modules possibly already being in -extra [09:05] that is entirly possible indeed [09:05] if you have the inclination to investigate then great, otherwise perhaps make a WI on flavours blueprint to investigate for a2 [09:30] remote root access to mysql: ouch [10:32] brb [10:44] smb and apw: for https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1008400 [10:44] Launchpad bug 1008400 in linux "Ubuntu server uses CFQ scheduler instead of deadline" [Medium,Confirmed] [10:45] i don't think we need to change on our kernel side, it'd better to ping server team to add a kernel parameters in server image [10:46] smb, i am starting to wonder if cfq has any place ... i wonder if we should be using deadline in the desktop too [10:46] cooloney, That sounds reasonable to me. I only saw you discuss this on irc in my backlog and had not remembered the bug reference. But I assume what they refer to is process stuck messages. [10:47] cking, i wonder if there is some way we can quantify whether moving to deadline by default for desktop would be any impact [10:47] cking, as likely it is the trigger for responsivness going to crap when writing to USB sticks sort of thing [10:47] apw, Is that not used to prevent one big produces to hog all the processing power [10:48] apw, depends on the use-cases too [10:48] smb, which used for that [10:48] * smb tries to remember whether he did see some upstream discussion about it [10:48] cking, indeed, musing as to whether given we know cfq is a bit crap at times for desktop as well, whether deadline is a more sensible default [10:49] apw, it depends on HDD or SSD too, but I can rig up some tests to explore this if you so desire [10:49] cooloney, i am unsure if it even has to be a kernel paramter, this may be something that can be sysctld [10:49] apw, the boot parameter [10:49] cking, i'd be interested if you can think of any valid ways to test desktop side [10:49] apw, After that I believe it is per queue [10:49] smb, ? [10:50] apw, You can change the io scheduler on the kernel command line for all queues [10:50] apw: yeah, there are several method to change that, but boot parameter is the easiest why i think [10:51] apw: but, if we can change desktop to deadline as well as server, that's a good fix. but need some testing. [10:51] normally I just tweak this per drive rather than globally [10:51] apw, elevator=x [10:51] cking: exactly, we can change that via /sys [10:52] they never like changing things on the kernel command line, its never a preferred solution is all i am saying [10:53] * ppisati -> goes looking for carbo [10:53] apw, the choice of CFQ vs Deadline may also depend on the file system choice too - just to make life more complex [10:53] cking, so when you are tweaking what are you tweaking for [10:53] apw, I wonder whether in some situations cfq just shows the problem more heavily. Like what the usb-creator does. Pukes out a fs copy and then queues up writing the boot sector, which I believe tries to unmount and that is sensibly stuck until the fs copy is done. [10:54] Still you get a lot of those "process x stuck for more than 120s" [10:54] smb, yes thats fine, but the effect on other users is what i care about [10:54] i don't care if its the umount which trips that error [10:54] smb, which highlights the question: what kind of scenarios are we worried about [10:54] the issue as reported here is other vms lose their disks due to virtual IO timeouts [10:54] i dont' care about the producer making himself sick [10:55] i care about the producer making the other consumers sick [10:55] which for desktop is 'my shit goes grey and unresponsive' and for server is 'omg where did my VM root disk go' [10:55] apw, Right but that just may be the same thing (or similar). Lacking all the info from the bug I don't know whether their images is files backed [10:55] cking, now in both cases the unrelated producer/consumer latency is probabally the concern [10:56] smb, i am sure it is file based, and here they are hammering the filesystem from the host for a backup [10:56] which really shouldn't kill the VM [10:56] but neither should my other windoers grey out [10:56] (and they do) [10:58] Yeah, I guess in that case deadline makes sure a write is not done later that a certain time. One would think cfq should do similar but apparently not :/ [10:58] it cirtainly has an odd definition of fair [11:00] me thinks the Andrew Morton Interactive Workload (AMIW) is a good test. http://kerneltrap.org/node/431 [11:02] Agreed, you would think fair should mean every process gets at least some slot to do writes. The problem likely is as well that the virtual disks maybe have no way of saying "yes I got your request but I am kind of under water right now, just be patient for a bit" [11:02] apw, You probably can get into quite a bit of trouble if you have your disks on something like iscsi or so [11:04] cking: yeah, i also saw this AMIW testing. [11:05] cking, In general what you need is multiple tasks that do independent heavy IO [11:05] smb, yep [11:07] * cking can't find any serious analysis of CFQ vs Deadline [11:10] cking: me either, but for Redhat server and SuSe server, looks like CFQ is still the default IO scheduler [11:10] yep [11:10] cking: how about this one http://www.redhat.com/magazine/008jun05/features/schedulers/ [11:14] cking: http://www.phoronix.com/scan.php?page=article&item=linux_iosched_2012&num=1 [11:14] cking: [11:14] ups, sorry [12:56] apw, cking So I am currently running a kvm-vm with 2G memory doing a loop creating an erasing a 4G file on its virtual disk(ext4) which is backed by a file on ext4 to which the host rsyncs 17G of files and I see no stall messages up to now (guest at about 50M/s, host at about 80M/s). [12:57] smb, and which elevator is the host using [12:57] apw cfq [12:57] smb, well i guess that is 'good' [12:58] though it makes reproing the bug hard [12:59] perhaps mixing a bunch of other I/O activity may force it [12:59] apw, Yeah, we should ask the bug reporter for a more detailed description of what is setup how and what is done [13:00] smb, i would suggest creating a lot of small files in the guest, to get it to push those 'gatey things' down [13:00] * apw struggles for the word today [13:00] Hm, the request size? [13:01] s/gatey things/barrier/ [13:01] make it use barrier writes for its journal [13:01] which should make it block itself waiting [13:01] for the IO to complete [13:02] Depends which parts do actually support barrier requests. The fs is on a lv on a vg which is on a dm-raid5 [13:03] i would expect barriers to be supported mostly [13:06] apw, dm used to be by large just a request mapper which makes doing quarantees on barrier requests hard if requests can end up on multiple disks downwards the stack [13:09] smb, all true [13:10] but the VM doesn't know that [13:10] so it will think the right way, blocking itself [13:11] apw, Would be what the virtio driver claims to support. So I see no barriers disabled messages (neiter host nor guest) which would mean they somehow support them or upstream got rid of the message... [13:17] smb, yeah i know dm can when it has simple mappings, so perhaps it can [13:22] apw, For simple ones (linear) it is no problem. I am wondering about the dmraid(dm-raid45) one. Theoretically and barrier would need to get converted into a write+flushes on all disks in the array. Now there could be either support, or ignorance and incorrectly ignoring that bit, or the message being hidden now... [13:42] apw: did you rebase overlayfs.v13 against v3.5-rc2? [13:42] dileks, i'd be waiting on leann doing the base rebase [13:42] apw: just fyi, I pushed the rebase this weekend [13:43] ogasawara, ahh cool. did overlayfs build ? [13:43] there is a 3.5-rc2-quantal kernel. did someone test the union-fs part? [13:43] apw: it did [13:43] apw: I think there was 1 minor fixup for it during the rebase [13:43] dileks, probabally not yet, its very early to dare running it :) [13:43] 1st patch with path stuff [13:43] dileks, but i'll get it building [13:44] (as in get a build going so i can test it) [13:44] I was hoping you offer a GIT branch in your overlayfs kernel URL [13:44] especially for 3.5-rcX [13:44] apw: I've got it running here on 2 systems and seems to be holding up to very light use [13:45] ogasawara, cool ... [13:45] dileks, i may well do that [13:45] dileks, given the normal delays [13:46] IIRC there is some atomic vfs stuff pending changing the do_dentry stuff, not sure if this is 3.5 material from al viro [13:46] apw, I'm spinning the HDD on a dev box here for Deadline vs CFQ across a bunch of file systems - will take a few hours to finish [13:46] cking, thanks [13:46] cking, you did 'testing thursday' on the alpha i think [13:47] yep I did [13:47] did it seem to take a real long time after finishing the install saying "installing system" before completing ? [13:47] apw: BTW, can you activate codel/fq_codel for 3.5? doing some initial bufferbloat testing with kamal [13:47] dileks, whats the config for that [13:47] apw, not from what I observed on a X220i [13:47] cking, perhaps i have just become impatient [13:47] http://kernel.ubuntu.com/git?p=kamal/ubuntu-quantal.git;a=commitdiff;h=d684fa3d61223a3d8ad4622c91497f520dc1c43b [13:48] and I did 3 installs, so I would have noticed [13:48] CONFIG_NET_SCH_CODEL=m [13:48] CONFIG_NET_SCH_FQ_CODEL=m [13:48] apw: ^^ [13:48] ogasawara, ^^ :) [13:48] * ogasawara looks [13:49] apw, i'll repeat the tests with a SSD later tonight once the thorough testing is completed on the HDD [13:51] dileks: both look to be =m already [13:51] dileks: /ubuntu-quantal/debian.master/config$ grep -rn "NET_SCH_CODEL" * [13:51] config.common.ubuntu:3638:CONFIG_NET_SCH_CODEL=m [13:51] apw: ping [13:51] ubuntu-quantal/debian.master/config$ grep -rn "NET_SCH_FQ_CODEL" * [13:51] config.common.ubuntu:3642:CONFIG_NET_SCH_FQ_CODEL=m [13:52] ogasawara: ah, I see. good to know [13:52] caribou, pong [13:53] BTW, the quantal-3.5-rc2 kernel panics here [13:53] howdy, can I come & bug you for a minute ? [13:53] missing some important iwlwifi patches from wireless.git#master [13:53] caribou, ask away [13:53] apw: remember the ddeb's you built for Natty's 2.6.38-8 kernel a while ago ? [13:53] yep [13:54] apw: how hard would it be to rebuild them with the exact same name as the original ? [13:54] apw: I'm getting "ERROR: Build-id mismatch: "kernel" vs. "vmlinux-2.6.38-8-server" with SystemTap. [13:54] apw: I think it's coming from the build name of your ddeb [13:54] caribou, not hard i don't think ... [13:55] caribou, one would need care to not mix them up with the real ones [13:55] apw: well, the real ones are gone now anyway (the ddebs) [13:56] well indeed, but to not let peopel think they are the originals [13:56] apw: ah, ok [13:56] though i assume it uses the unpacked stuff and we can hand mod the .ddeb filenames [13:56] apw: yep, I guess [13:58] apw: hold on a minute, I might be onto something. biab [14:00] apw: nervermind, I thought I had found a bug but it was for an old version [14:00] caribou, i find it odd it would call my version 'kernel' i'd expect it to be complaining about a longer name similar to the missmatch version with dates on the end [14:01] apw: same here. [14:01] apw: well, if you have a minute to build them, please do so so I can rule out this, I'll continue to hack at it in the meantime [14:02] caribou, yep, i'll get the builder on the case [14:08] apw: looks like a known issue : https://bugs.launchpad.net/ubuntu/+source/systemtap/+bug/996364 [14:08] Launchpad bug 996364 in systemtap "systemtap reports error 'Build-id mismatch'" [Undecided,New] [14:10] caribou, quality [14:11] apw: in the bug's case, it seems like it was indeed an version mismatch. I just ran the script on my box and it does indeed report the version with the long suffix [14:11] ok so you care about it, and the error is printing the wrong text ?? [14:12] apw: yes, I'll need the new ddebs as the script report the version with your suffix [14:26] apw: fyi, I must step out. back later [14:28] caribou, they take a while to make cause they are so big ... so i'll leave it building 'em [14:41] ogasawara: 3.5: FB_VESA is now bool, setting =m results is =n and FB_BOOT_VESA_SUPPORT=n [14:43] $ grep VESA /boot/config-3.5.0-030500rc2-generic [14:43] # CONFIG_FB_BOOT_VESA_SUPPORT is not set [14:43] CONFIG_FB_UVESA=m [14:43] # CONFIG_FB_VESA is not set [14:43] dunno if its intended [14:45] dileks: we carry a SAUCE patch to make that modular, and actually have a check in our config enforcer for it [14:46] thats overriding kbuild-system? [14:46] dileks: so that likely explains the discrepancy between the mainline build [14:48] * ogasawara back in 20 [14:54] apw: no worry, I won't use them before end of day. I'll ping you tomorrow. thanks a lot [15:34] * ppisati -> gym [15:45] * jsalisbury is seeing allot of Launchpad timeouts today :-( [16:22] infinity, I'm not sure we have a ppc chroot setup on tangerine for bug 1005699 [16:22] Launchpad bug 1005699 in linux "tg3: reports "eth0: DMA Status error. Resetting chip.", fails to work" [High,Confirmed] https://launchpad.net/bugs/1005699 [16:22] ogasawara, do you happen to know ^^^ [16:24] Tangering has no PPC cross compiler, no. [16:24] You could build it on davis. [16:24] hello, is this a kernel or compiz error ? ther's here a good man to explain me ? Jun 11 18:04:21 quantic kernel: [ 806.840646] unity-panel-ser[1996] general protection ip:4063bd sp:7fff05820a60 error:0 in unity-panel-service[400000+f000] [16:24] infinity, ok, cool. let me see if I can access davis [16:25] jsalisbury: if you don't have an account on davis, I do and can build the test kernel. [16:26] Jun 11 18:02:44 quantic kernel: [ 710.344448] compiz[1907] general protection ip:7fd099f3869e sp:7fffc6914b70 error:0 in libunityshell.so[7fd099cf8000+32e000] [16:26] ogasawara, infinity, looks like I can login :-) [16:26] infinity, I'll build a ppc test kernel and post the link to the bug [16:26] Danke. [16:26] I'll try to push to get it tested ASAP. [16:27] I assume it's the same bug on the Xserves and the G1s, but one can never tell for sure. [16:27] yeah [16:27] Best to know now, than to wait another SRU cycle or two. === smb` is now known as smb [16:39] infinity, do you happen to know if there is a precise or mainline src tree on davis? It will just make it a little quicker since I can --reference it versus cloning a whole new tree. [16:39] infinity, if not thats fine. I have the clone already going :-) [16:43] jsalisbury: No idea. If there is, it's not mine. [16:44] infinity, cool, thanks. [16:44] jsalisbury: "locate debian.master" finds lots of copies. :) [16:45] infinity, ahh, right. My clone just finished too :-D [16:45] jsalisbury: /home/ogasawara/quantal-powerpc/ubuntu-2.6/ looks promising. [16:45] Oh, nevermind then. :P [16:45] heh, I'm always in such a rush [16:45] Or precise-powerpc, rather. [16:45] But whatever. [16:45] It's not like a clone on that network takes long. [16:46] yeah, it was pretty fast [16:52] infinity, ogasawara, hmm. the ppc build failed pretty early on davis: [16:52] dpkg-deb: building package `linux-headers-3.2.0-25' in `../linux-headers-3.2.0-25_3.2.0-25.40_all.deb'. [16:52] make: *** No rule to make target `binary-generic'. Stop. [16:53] There is no generic kernel on PPC. [16:53] So that would make some sense. [16:53] infinity, ahh ok. I better fix up my fakeroot alias then :-) [16:53] Was that a "dpkg-buildpackage", or are you running debian/rules targets by hand? [16:54] If you're just hitting specific targets by hand, the one you'd want to test on the buildds would be powerpc64-smp [16:54] infinity, Just copied my bashrc stuff from tangerine. I have an alias to shorten the fakeroot command, and it was wrong. [16:54] infinity, cool, thanks [16:57] Is this build failure in precise a recent problem? [16:57] BenC, it was my mistake ;-) [16:57] Ah [16:58] If you need any help with it, let me know, I've got local ppc systems here [16:58] BenC, awesome, thanks [17:07] infinity, so the debian/rules target should be powerpc64-smp ? [17:08] jsalisbury: I'd assuming binary-powerpc64-smp, if it's congruous with binary-generic. I dunno. I just build the whole source package. :P [17:08] (ie: I just run "dpkg-buildpackage -rfakeroot -uc -us -B") [17:08] infinity, cool, thanks! [17:09] infinity, yeah, binary-powerpc64-smp is the proper target. I was leaving off the binary prefix [17:10] balloons: have you heard any feedback from your 12.10 kernel testing in 12.04 announcement? [17:11] ogasawara, yes I have [17:11] balloons: I was looking at our daily bug reports, but haven't seen a lot of bugs being reported [17:11] balloons: but that's not to say people aren't testing [17:11] i'm causing down some folks who've had issues but didn't report they tested to the tool, nor filed a bug [17:11] you can see the "results" in realtime but going to the page http://packages.qa.dev.stgraber.org/qatracker/milestones/223/builds/16265/testcases/1301/results [17:12] as you see, there's only 3 listed there. but I've got emails or random contacts with several other people who tried it and succeeded or didn't [17:12] * stgraber needs to change his hilight regexp not to match .stgraber.org ;) [17:12] * balloons waves [17:13] wow -- causing should be chasing.. I'm chasing down folks who did the work, but didn't report === yofel_ is now known as yofel [17:14] * balloons wishes we could get a number on how many people (unique ips or something) installed from a ppa [17:15] anyways ogasawara the point behind the qatracker is to capture everyone who's doing the install and making sure they report it in a sane place.. Any bugs they find should also be linked, making it all easy to see [17:16] balloons: indeed and agreed. I really want to have this help provide us a high confidence level for the 12.10 kernel in 12.04. [17:18] ogasawara, yep, we're on the same page. So I'm herding cats more or less.. it's something new, and this is our first pass at it, so I'm doing my best to followup and ensure we get the results recorded === smoser` is now known as smoser [18:38] * cking --> EOD [18:47] * smb follows [19:49] any reason why linux-backports-modules-3.2.0 is x86/amd64 only? [20:15] infinity, fyi, the kernel build is still happening on davis. There is some puppet process hogging all the cpu. Anyway, I'll post a link to the kernel in the bug when it's done. [20:20] jsalisbury: Alrighty. [20:21] janimo: I've wondered that myself, and never really looked into it. [20:23] infinity, ok. Btw I cannot yet claim the cookies, had no time to earn them [20:52] ogasawara: FYI, I have cleaner patches for the AACRAID/big-endian fixes. So if you hold off, I'll rebase and request-pull with that instead of the huge lumpy/krufty aacraid patch I have in there now [20:52] These patches are also going upstream at the same time [20:53] BenC: ack, I can wait to look. I haven't had a chance yet today anyways. === matti_ is now known as matti === cmagina-lunch is now known as cmagina === BenC_ is now known as BenC [23:45] ogasawara: can you handle my sub request for kernel-team@ please?