| fairuz | Hi guys | 07:07 |
|---|---|---|
| fairuz | If I compile a kernel with make uImage, how to flash the new kernel image to the boot partition? What do I need? I see that I must copy vmlinux and generate initrd? How to generate initrd. Thanks | 07:09 |
| === smb` is now known as smb | ||
| ppisati | fairuz: you can't flash it | 07:56 |
| ppisati | fairuz: you have to do eveyrhing by hand | 07:56 |
| ppisati | fairuz: and besides, you need a couple of specific options to make your vanilla kernel work with an ubuntu userland | 07:56 |
| ppisati | fairuz: hold on | 07:56 |
| fairuz | ppisati: my mistake, What I mean by flashing is to copy the vmlinux to /boot partition | 07:56 |
| ppisati | fairuz: https://wiki.ubuntu.com/KernelTeam/ARMKernelCrossCompile | 07:57 |
| ppisati | fairuz: FAQ point 6 | 07:57 |
| ppisati | fairuz: How do i compile a vanilla/upstream arm kernel that works with an Ubuntu userspace? | 07:57 |
| ppisati | fairuz: should help you | 07:57 |
| fairuz | Oh | 07:57 |
| fairuz | I already take a look at that | 07:57 |
| fairuz | So how about initrd and config? We can use the old one? | 07:58 |
| ppisati | fairuz: yes, old one | 07:58 |
| ppisati | fairuz: actually you don't actually need initrd | 07:58 |
| ppisati | fairuz: you don't need to use it | 07:58 |
| fairuz | But I think the one in the /boot is vmlinux, and in the wiki it says to copy uImage. Is that Ok? | 07:58 |
| ppisati | fairuz: but since i overwrite uImage with my own, it'll use the ubuntu one | 07:58 |
| fairuz | So I just re-point the vmlinux to the new uImage? | 07:58 |
| ppisati | fairuz: no no | 07:59 |
| ppisati | fairuz: it doesn't mention /boot | 07:59 |
| ppisati | fairuz: it says | 07:59 |
| ppisati | fairuz: "copy arch/arm/boot/uImage to the sd card first partition overwriting ubuntu stock uImage" | 07:59 |
| ppisati | fairuz: on arm you really don't care what you have in /boot during boot | 07:59 |
| fairuz | Oh. I thought /boot is the boot partition | 07:59 |
| ppisati | fairuz: well, at leat on our omap boards | 07:59 |
| fairuz | Which normally means the first partition | 07:59 |
| ppisati | *least | 08:00 |
| ppisati | fairuz: which board are you using? | 08:00 |
| fairuz | OMAP :) | 08:00 |
| * ppisati thinks i should better describe the boot process | 08:00 | |
| ppisati | fairuz: beagle? panda? | 08:00 |
| fairuz | panda | 08:00 |
| ppisati | fairuz: ok, then copy uImage to the first partition of your sd card and you are done | 08:01 |
| ppisati | fairuz: actually, if you do a lot of development | 08:01 |
| ppisati | fairuz: try a lot of different kernels etcetc | 08:01 |
| ppisati | fairuz: you should better modify boot.scr or preEnv.txt to boot an alternate uImage | 08:01 |
| ppisati | fairuz: when the board reboot stop at the boot loader prompt | 08:02 |
| ppisati | fairuz: and tell it to use tour alternate boot.scr/preEnv.txt | 08:02 |
| ppisati | fairuz: this way if your new kernel is borked | 08:02 |
| fairuz | Oh ok. Sounds like what I do with u-boot + Android kernel | 08:02 |
| ppisati | fairuz: you just boot with the original boot.scr/preEnv.txt and original uImage | 08:02 |
| ppisati | fairuz: but this is left as an exxrcise to the reader :) | 08:02 |
| fairuz | So just wondering if Ubuntu have additional work for new kernels | 08:02 |
| * ppisati will probably describe that too next time | 08:03 | |
| fairuz | So it is easy! | 08:03 |
| fairuz | :) | 08:03 |
| fairuz | Thanks ppisati! | 08:03 |
| fairuz | My problem is that I thought /boot is the first partition. | 08:04 |
| ppisati | fairuz: nope, it's not | 08:04 |
| ppisati | fairuz: w just install the kernel there | 08:04 |
| ppisati | fairuz: and flash-kernel pick it up from there to create the uImage | 08:04 |
| ppisati | fairuz: and copy it to the vfat partition at the beginning of the sd card | 08:05 |
| fairuz | Ohh | 08:05 |
| ppisati | fairuz: but after that, /boot is totally useless during the real boot | 08:05 |
| fairuz | it make sense now | 08:05 |
| fairuz | since uImage is a stripped version of vmlinux right? | 08:05 |
| ppisati | not stripped, mangled | 08:05 |
| ppisati | actually it has only 64bytes more than a normal vmlinuz IIRC | 08:06 |
| fairuz | What does mangled means? | 08:06 |
| ppisati | it has an header at the beginning | 08:06 |
| fairuz | It's bigger? | 08:06 |
| fairuz | ok | 08:07 |
| === henrix_ is now known as henrix | ||
| === henrix is now known as henrix_ | ||
| === henrix_ is now known as henrix | ||
| === doko_ is now known as doko | ||
| henrix | brendand: hi! do you have any news on bug #1063856? | 09:34 |
| ubot2 | Launchpad bug 1063856 in linux "[Dell Studio XPS 13] X in failsafe mode after boot" [Undecided,Confirmed] https://launchpad.net/bugs/1063856 | 09:34 |
| brendand | henrix, no. the lab is in montreal | 09:35 |
| henrix | brendand: yeah, but i thought that today you would be able to reboot the box with currently released kernel | 09:36 |
| brendand | henrix - the problem is i only have ssh access. i tried to reboot the system but it has not come back up. i'll need to get in touch with the engineer in montreal | 09:37 |
| henrix | brendand: ah, right. :) | 09:38 |
| henrix | brendand: please let me know when you're able to do that. | 09:38 |
| ppisati | crap, they just cancelled my return flight from barcrelona | 10:45 |
| ppisati | ... | 10:45 |
| apw | ppisati, heh was that with lufthansa or something? | 11:40 |
| apw | ppisati, this omap3 set for Q, what is the symptoms if we don't apply it for trying to do an install; it is sounding like boot failures? | 11:40 |
| apw | bjf, henrix, this is special -- linux-lts-backport-oneiric has released to -updates before its linux in oneiric, and the latter is where we do all the testing no ? | 11:48 |
| henrix | apw: i believe they are both tested | 11:49 |
| henrix | apw: at least both tracking bugs have the testing tasks | 11:49 |
| apw | henrix, ok that good then | 11:50 |
| ppisati | apw: they sais AZ something, now i've two options: either i'm rerouted around or i take a direct at 20:00 in the evening | 12:00 |
| ppisati | ufff | 12:00 |
| apw | direct always sounds better, you may as well sit in a coffee shop beforehand | 12:01 |
| ppisati | yeah | 12:02 |
| ppisati | apw: the bug is not a fatal for boot, but the board is a bit useless later on | 12:02 |
| ppisati | apw: since without usb you loose nic, and every i/o options except for the serial console | 12:03 |
| apw | ppisati, suspected as much, ok so its pretty important for release | 12:04 |
| ppisati | apw: well, no one really used that hw since at least July so if we defer it i'm not against it | 12:05 |
| apw | ppisati, gotcha | 12:07 |
| * henrix -> food | 12:11 | |
| rtg | apw, you gonna fix the bugs in '[PATCH 1/4] efi: Add support for a UEFI variable filesystem' pointed out by Tetsuo ? | 12:16 |
| ppisati | brb | 12:17 |
| apw | rtg, looking at them indeed, what a piece of crap code | 12:30 |
| rtg | apw, :) | 12:32 |
| apw | rtg, those seems to be the tip of the iceburg too | 12:40 |
| * apw hates efi | 12:40 | |
| * rtg hates the enforcer syntax. | 12:41 | |
| apw | rtg, does not supprise me, it does hard things, so it is hard to understand | 12:41 |
| rtg | apw, ppisati's patch is wrong, but I can't figure out the right syntax. I want something like 'flavour omap & !exists CONFIG_USB_EHCI_HCD_PLATFORM', don't I ? | 12:42 |
| apw | ok so this is only on armh and omap | 12:43 |
| rtg | apw, omap is built for both armel and armhf | 12:45 |
| apw | its mean to say just arm there | 12:45 |
| rtg | apw, But I don't want this rule enforced for all arm flavours, only omap | 12:45 |
| apw | indeed , and you only want it if you have the other variabled | 12:46 |
| apw | ppisati, which variables are the ones which exist on omap only | 12:46 |
| apw | rtg, as in yes you are right it is too liberal | 12:48 |
| rtg | apw, I'm just gonna NAK the enforcer patch and make him figure it out. | 12:49 |
| apw | ack | 12:49 |
| apw | rtg, waht i want to say for that enforcer rule is: ((flavour omap &/ value CONFIG_USB_EHCI_HCD_PLATFORM n & value CONFIG_USB_OHCI_HCD_PLATFORM n) | true) | 12:54 |
| apw | rtg, but we don't have a true | 12:54 |
| rtg | back in a bit | 12:56 |
| apw | rtg, ppisati, i have replied with a tested snippet | 13:03 |
| ppisati | apw: i'll take a look | 13:03 |
| rtg | apw, that is a frustratingly obtuse syntax | 13:04 |
| apw | rtg, i challenge you to maintain the flexibility and complexity of the rules, and simplify the syntax | 13:05 |
| rtg | yeah, right. I'll get right on that. | 13:05 |
| apw | rtg, i cannot deny you need to be stone cold sober, and have your glasses on, to have a hope of understanding it | 13:05 |
| rtg | or be well coffeed | 13:06 |
| apw | that can nonly help | 13:06 |
| apw | i generate mine by testing using genconfigs, its the only way to be sure | 13:06 |
| rtg | apw, what does '&/' mean ? | 13:07 |
| apw | that is & /, and means and primarily, but means if you fail after this, fail everything | 13:08 |
| apw | this allows us to say | 13:08 |
| apw | (arch armel &/ value FOO 10) | value FOO 20 | 13:08 |
| apw | and it mean what you think it means | 13:08 |
| apw | ie it does not allow FOO == 20 on armel | 13:08 |
| apw | it literally means if you succeed arch armel, then wahtver follows is the result don't try any other paths | 13:09 |
| apw | it is known as the cut operator, cutting the search short | 13:09 |
| apw | printk("VFS: Busy inodes after unmount of %s. " | 13:22 |
| apw | "Self-destruct in 5 seconds. Have a nice day...\n", | 13:22 |
| apw | quality error messages | 13:22 |
| === henrix_ is now known as henrix | ||
| pgraner | rtg, apw, have you guys seen probs with intel wifi not coming back after suspend? This only happens to me when I let the battery run down and it puts itself to sleep. Upon resume no wifi, lspci shows it, but the commandline tools nor NM will see it | 13:27 |
| pgraner | Intel Corporation Centrino Advanced-N 6205 (rev 34) | 13:27 |
| rtg | pgraner, I've had more problems with DNS resolution not working, but wifi is definitely connected. | 13:28 |
| pgraner | rtg, happens everytime without fail | 13:30 |
| rtg | pgraner, so normal suspend when the battery is good works OK ? | 13:30 |
| pgraner | rtg, yep, if I suspend from either the indication or by using pm-suspend, wait a few hit the pwer button it works. | 13:31 |
| rtg | pgraner, I wonder if you lose the EC when it suspends for lack of battery | 13:31 |
| rtg | pgraner, doesn't user space do a hibernate when the battery dies ? | 13:32 |
| pgraner | rtg, If I let is just sit and have the battery run down it goes to sleep and when I go to wake it up... no wifi, just a blank signal indicator | 13:32 |
| pgraner | rtg, in the "power" prefs, I have "when critically low do" and the drop down is blank | 13:33 |
| pgraner | so I'm not sure what its doing | 13:33 |
| rtg | neither am I | 13:33 |
| rtg | pgraner, I assume you are resuming _after_ plugging in the power ? Is there anything in dmesg that indicates wifi hates you ? | 13:39 |
| pgraner | rtg, not that I can see | 13:40 |
| apw | pgraner, have you tried rfkill on off cycle | 13:40 |
| apw | and have you looked to see if rfkill the command says anything is off ? | 13:40 |
| rtg | yeah, this feels kind of BIOS like... | 13:40 |
| pgraner | apw, yes and no joy | 13:41 |
| pgraner | apw, no I haven't queried rfkill but I will let me get the box into that state | 13:43 |
| apw | ok | 13:45 |
| diwic | I guess r will use kernel 3.8, right? Seems to line up well with the release? | 13:47 |
| ogasawara | diwic: I suspect so, although haven't given it a whole lot of thought yet | 13:51 |
| ogasawara | diwic: it does seem to line up nicely, ie likely to come out beginning of march | 13:51 |
| diwic | ogasawara, thanks | 13:52 |
| apw | smb, rtg, ok that linux-meta update got rejected, depending on an transitional package; i will fix | 13:52 |
| === ayan is now known as ayn | ||
| smb | apw, Hm ok... do you then have to depend on both amd64 and the i386 version? | 13:53 |
| apw | yeah | 13:54 |
| smb | Oh well... :/ Not sure why those need the architecture in the name while grub-pc does not | 13:55 |
| === ayn is now known as ayan | ||
| jgriffith | smb: Ping | 15:02 |
| smb | jgriffith, yep? | 15:03 |
| jgriffith | smb: Hey ya.. just saw your note | 15:03 |
| jgriffith | smb: Yeah, I'd be happy to go through devstack setup with you | 15:03 |
| jgriffith | smb: It's pretty straightforward, using virtualbox and a clean 12.04 server install | 15:04 |
| smb | jdstrand, I guess what I meant is, is it easy enough to write down in the bug report. Then I also have a place to go back whenever I need my memory refreshed. ;) | 15:04 |
| smb | err jgriffith ^ | 15:04 |
| jgriffith | smb: Sure, I'll just reference the devstack page probably | 15:05 |
| jgriffith | smb: Then mabye a modified script you can loop through | 15:05 |
| jgriffith | smb: I'll update it on the bug report | 15:05 |
| smb | jgriffith, Sure, sounds good. Anything that saves me the time to dig out things on my own is highly appreciated. :) | 15:05 |
| jgriffith | smb: Cool, I'll get it update this morning. Thanks!!! | 15:06 |
| smb | jgriffith, Thank you. I hope this will help into getting it reproduced in a way that allows to research it better | 15:07 |
| jgriffith | smb: That makes two of us :) | 15:07 |
| jgriffith | smb: I really appreciate the effort on this, I know it's a pain to try and repro | 15:08 |
| smb | jgriffith, It makes it hard to find out what is wrong when it fails to be wrong on testing. And neither it is very verbose on failing. | 15:09 |
| jgriffith | smb: Yeah, even worse when I finally got in the state where it failed consistently it locked the system up so I couldn't get ANY info :( | 15:10 |
| smb | Right | 15:10 |
| smb | caribou, Are you aware that the crash version in Quantal does not work on a Quantal crashdump? (cannot resolve xtime) Or is that an OPP? | 15:17 |
| caribou | smb: I thought I had tested it, lemme check | 15:18 |
| smb | caribou, I sort of just did (with an updated 64bit machine). So at least on the good news side the crashdump side does work. ;) | 15:19 |
| caribou | smb: which version of crash did you use ? Quantal has 5.1.6, Debian/Sid has 6.0.6 and upstream is now at 6.1.0 | 15:19 |
| caribou | smb: I never use our own package, it fails most of the time. I install Sid's version | 15:20 |
| smb | caribou, 5.1.6 of course (which is what we ship) | 15:20 |
| smb | caribou, You know that dog food thing, do you? :) | 15:20 |
| caribou | smb: sure does, which is why I'm starting to wonder how I could participate in having a more recent version for us too | 15:21 |
| smb | caribou, I guess in the same manner I would have to. Find out who sponsored it last time and coordinate a merge from debian at least... It is sometimes a bit annoying because you have everything prepared and then review takes ages and by that time Debian is on the next version... | 15:23 |
| smb | caribou, I mainly asked you right now because I was assuming you might have some relations there already as you were with kdump and that seems all going into the same direction | 15:24 |
| caribou | smb: I thought there was an 'automagic' import from Sid. | 15:25 |
| caribou | smb: at least this is what happened for makedumpfile in Quantal that has the version that I packaged (1.4.4) | 15:25 |
| caribou | smb: but maybe this is because crash is in Universe | 15:26 |
| smb | caribou, It seems only if it is really simple. Probably we find it on the list of "those are hard) | 15:26 |
| caribou | smb: I'll look into this. that would facilitate my work | 15:26 |
| caribou | smb: I'll create a bug for this | 15:27 |
| smb | caribou, I use it a lot too, but I am not sure I would have the time. Thanks, could you subscribe me to it? Then I keep track | 15:28 |
| caribou | smb: should I assign it to myself ? | 15:29 |
| smb | caribou, If you say you want to do work on it, I would say yes | 15:30 |
| smb | caribou, btw, there is a section on the merge-o-matic https://merges.ubuntu.com/main.html | 15:32 |
| caribou | smb: well, everytime I open a bug, it seems to fade into oblivion so I'm having doubts | 15:32 |
| smb | caribou, Yeah, that seems to be the fate for the request I made for it (bug 1054200) | 15:34 |
| ubot2 | Launchpad bug 1054200 in crash "Pick some upstream patches for Xen" [Wishlist,New] https://launchpad.net/bugs/1054200 | 15:34 |
| caribou | smb: there is current discussion about Xen/crash on the ML currently | 15:35 |
| smb | caribou, Daniel Kieper had submitted some updates which sounded like going into 6.0.9 upstream. At that time I was not aware of the quantal problem | 15:36 |
| caribou | smb: yeah, they're the ones I'm talking about | 15:36 |
| caribou | smb: gotta step out for a few minutes, biab | 15:37 |
| smb | caribou, Yeah, I just picked them and applied them to our current version. They are not needed if moving to the upstream version, but we normally only go for what is in Debian | 15:37 |
| smb | So when only merging for Debian they would be still useful. | 15:38 |
| jgriffith | smb: Haha... testing a script I wrote for you to run on this and it hit the problem and hung my system | 15:43 |
| smb | jgriffith, Sounds quite successful then :D | 15:45 |
| jgriffith | smb: We can only hope :) | 15:45 |
| * ppisati goes out for a bit | 15:46 | |
| jgriffith | smb: Ok, updated https://bugs.launchpad.net/cinder/+bug/1023755 with the info | 15:50 |
| ubot2 | Ubuntu bug 1023755 in linux "Precise kernel locks up while dd to /dev/mapper files > 1Gb (was: Unable to delete volume)" [Undecided,Confirmed] | 15:50 |
| jgriffith | smb: Let me know if you run into any glitches (other than the one we want | 15:50 |
| smb | jgriffith, Sure. Thanks. I will give it a try later (or at least tomorrow morning) | 15:51 |
| jgriffith | smb: cool | 15:52 |
| alexbligh | I, we are having a problem where we can accidentally DoS a Precise kernel. zcatting a large file (a precise image is good) into /tmp is enough to completely block any mysql or postgres writes for several minutes. We see (e.g.) "INFO: task mysqld:1358 blocked for more than 120 seconds." in the kernel logs. We have even got write DMA error messages and system hangs. This is reproducible on 2 different type of hardware.This is on kernel | 15:55 |
| alexbligh | 3.2.9-29-generic. Any ideas what this might be before I go and stick it in lanchpad etc.? I suspect it's a local DoS opportunity at the very least. | 15:55 |
| alexbligh | we think zcat to something on disk kills it, but zcat to /dev/null doesn't kill it. Looks like writes starving reads horribly or something. | 15:57 |
| alexbligh | And you can do this as an unprivilged user (essentially hang any daemon on the system). A single mysql 'UPDATE' call will hang for the duration of the zcat. Postgres is the same. | 15:58 |
| alexbligh | This did not happen on (e.g.) Natty-backports kernel under Lucid. | 15:59 |
| apw | alexbligh, what is your ioscheduler on that system? we have seem similar issues on servers using cfq on quantal, which led us to change the elevator there | 16:02 |
| alexbligh | apw, it's whatever the default is. Let me check. | 16:03 |
| apw | i would think for your use case deadline would be more approprate regardless | 16:04 |
| alexbligh | CONFIG_IOSCHED_NOOP=y | 16:05 |
| alexbligh | CONFIG_IOSCHED_DEADLINE=y | 16:05 |
| alexbligh | CONFIG_IOSCHED_CFQ=y | 16:05 |
| alexbligh | CONFIG_CFQ_GROUP_IOSCHED=y | 16:05 |
| apw | alexbligh, yeah ask the system which one it is using | 16:05 |
| alexbligh | (remind me how I tell what one is actualy running) | 16:05 |
| smb | cat /sys/block/sda/queue/scheduler | 16:05 |
| alexbligh | root@extility-qa-test:~# cat /sys/block/sda/queue/scheduler | 16:05 |
| alexbligh | noop deadline [cfq] | 16:05 |
| * apw comes back with the same paste | 16:06 | |
| alexbligh | :-) | 16:06 |
| apw | alexbligh, do try switching it to deadline, and see if it is reproducible any more | 16:06 |
| apw | you can echo into that same file | 16:06 |
| alexbligh | just testing now. | 16:07 |
| apw | alexbligh, when you say you are getting dma errors, is that from VMs hosted on the machine with cfq as its filesystem? | 16:07 |
| apw | as its elevator? | 16:08 |
| alexbligh | apw, scheduler change makes things much better. | 16:08 |
| alexbligh | apw, no this is nothing to do with VMs | 16:08 |
| alexbligh | no VMs at all | 16:08 |
| alexbligh | control plane machine, running just database, and the occasional zcat | 16:08 |
| apw | DMA errors do not seem to be something that a scheduler change could cause or not cause | 16:09 |
| alexbligh | apw, that's what I thought, which is why they went to change the hard disk, and then replicated on other systems. | 16:09 |
| alexbligh | Let me find you the exact error | 16:09 |
| apw | but cirtianly we have seem some cases of writes starving reads with cfq | 16:09 |
| alexbligh | Oct 9 09:10:26 extility-qa-test kernel: [69443.840122] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen | 16:10 |
| alexbligh | Oct 9 09:10:26 extility-qa-test kernel: [69443.840243] ata3.00: failed command: WRITE DMA EXT | 16:10 |
| alexbligh | Oct 9 09:10:26 extility-qa-test kernel: [69443.840337] ata3.00: cmd 35/00:00:00:20:d4/00:04:0b:00:00/e0 tag 0 dma 524288 out | 16:10 |
| apw | which is contrary to its name | 16:10 |
| alexbligh | And then the next time, kernel panic with: | 16:10 |
| alexbligh | sd 2: 0: 0 : 0 [sda] Unhandled error code | 16:10 |
| alexbligh | Result Host byte=DID_BAD_TARGET driverbyte=DRIVE_OK | 16:10 |
| apw | so t | 16:10 |
| apw | so those, don't seem likely to be elevator indeed | 16:10 |
| alexbligh | apw, actually we've just got it to do the same on the deadline scheduler. Seems to be less bad. | 16:12 |
| apw | alexbligh, i would expect deadline to eliminate the 120s delay hangs, but the other stuff i'd expect to be real | 16:12 |
| alexbligh | (the same is completely starving mysql so a single update takes an infinite amount of time) | 16:12 |
| apw | well if you have lost an IO like that, i'd assume all bets are off on it ever continuing | 16:13 |
| alexbligh | it's been rebooted and reinstalled from scratch since then! | 16:13 |
| alexbligh | apparently only one of the hardware platforms is producing the write dma errors (multiple times, multiple installs from CD). | 16:14 |
| alexbligh | both are showing the 120 second hangs | 16:14 |
| apw | ok, then i'd be suspicious of real h/w issues with the DMA error | 16:14 |
| alexbligh | The h/w platform that does NOT produce any write DMA errors is the one I'm playing with | 16:14 |
| alexbligh | that produces hangs on both deadline and cfq. | 16:14 |
| apw | the 120s hangs, did we say deadline eliminated, or only helped with those | 16:15 |
| alexbligh | I agree I am sceptical about the platform reporting write errors. | 16:15 |
| alexbligh | apw, helps, but does not eliminate. | 16:15 |
| alexbligh | local DoS still very possible. | 16:15 |
| apw | how much memory has this node, and have you tuned your dirty memory on it? | 16:15 |
| alexbligh | apw, 8G RAM. We've done no tuning bar whatever Precise does on a default install | 16:16 |
| apw | alexbligh, i would be suspicious your dirty ratio is too high, be worth checking | 16:17 |
| apw | as that job is emitting dirty memory at a huge rate and presumably hammering the disk into the ground | 16:17 |
| alexbligh | you mean /proc/sys/vm/dirty_ratio | 16:17 |
| apw | that or its brother yes | 16:17 |
| alexbligh | root@extility-qa-test:~# cat /proc/sys/vm/dirty_ratio | 16:18 |
| alexbligh | 20 | 16:18 |
| alexbligh | with deadline scheduler running any mysql command takes about 11 seconds, rather than hanging indefinitely. | 16:18 |
| alexbligh | root@extility-qa-test:~# cat /proc/sys/vm/dirty_background_ratio | 16:18 |
| alexbligh | 10 | 16:18 |
| apw | which is pretty much what i would expect with such a huge write load on the disks | 16:19 |
| apw | if i am remembering right this is 20% of RAM, so 20% of 8G which essentially can be fulled and then dumped as a group when the next sync hits | 16:20 |
| apw | onto the disk queue all at once, probabally that is rather high | 16:20 |
| alexbligh | Well the image(s) we pull down are 20G or more. | 16:20 |
| alexbligh | So it's always going to fill x% of RAM for any value of X. | 16:20 |
| apw | right, so your machine will let you fill 20% of ram before it'll really start doing anything about it, then it will drop the lot on the disk and say "hey deal with this" | 16:21 |
| apw | alexbligh, yes, but the point is the cuplprit doesn't get penalised until it hits the 20% of ram mark | 16:21 |
| apw | it won't stop till then, after then it becomes synchronous wrt its own backlog | 16:22 |
| apw | and cannot make things any worse, but things may be tooo bad by then | 16:22 |
| apw | it is normal to tune dirty ratio to your workload, amongst other things | 16:22 |
| alexbligh | apw, so I should make them lower? That doesn't make a huge amount of sense as even when zcat has been running for many minutes (and presumably passed this threshold) it's still causing problems. | 16:22 |
| apw | no it is only synchronous when there is more than 20% outstanding in your scenario, which is what | 16:23 |
| alexbligh | My confusion is that we don't hit this issue on Lucid AT ALL. EVER. with exactly the same values. | 16:23 |
| alexbligh | using cfq! | 16:24 |
| apw | so it is either broken, or possibly, more efficient such as we can get into these scenarios | 16:24 |
| alexbligh | (hardware guys report since they've changed the hard disks out they can't get the DMA error thing so I think that's a bogon). | 16:24 |
| apw | but i would also check the dirty ratio on the lucid box, to see if the default is the same ther | 16:24 |
| alexbligh | Have done. It is. | 16:25 |
| apw | ok so with this kind of write load, what is the average mqsql latency doing the same requests | 16:25 |
| alexbligh | It's 3.0.0-15-server #26~lucid1-Ubuntu on the Lucid box, which is I think natty-backports. | 16:26 |
| apw | thats and oneiric kernel isn't it? | 16:26 |
| alexbligh | With cfq, over 120 seconds for any query. With deadline, between 0 and 11. | 16:26 |
| alexbligh | yeah it might be oneiric-backports, sorry. | 16:27 |
| apw | ok, and on lucid | 16:27 |
| apw | which is likely deadline isn't it ? | 16:27 |
| apw | the default used to differ for server kernels | 16:27 |
| alexbligh | Let me start again. Everything is CFQ, 20 and 10 in the vm dirty_ things | 16:27 |
| apw | so lucid is not a server kernel then | 16:28 |
| alexbligh | With lucid OS running Oneiric backports kernel, we have only tested CFQ and we get less than a second latency | 16:28 |
| apw | oh i see, its L + O kernel, against P with P kernel | 16:28 |
| alexbligh | With Precise OS running the default kernel, we havve tested CFQ and get >120 seconds, and Deadline and get between 0 and 11 seconds. | 16:28 |
| alexbligh | Yeah - we moved to O-backports for better h/w compatibility. | 16:28 |
| rtg | arges, are you on gomeisa ? | 16:29 |
| arges | rtg, yes | 16:29 |
| alexbligh | Oneiric kernel (working one) is linux-image-3.0.0-15-server 3.0.0-15.26~lucid1 (I suspect -26 works fine and just no one has rebooted the box I'm looking at) | 16:30 |
| apw | alexbligh, that sounds bust indeed. i guess we need a bug, and someti | 16:31 |
| apw | something we can run to repro it | 16:31 |
| alexbligh | Precise is: | 16:31 |
| alexbligh | ii linux-image-3.2.0-29-generic 3.2.0-29.46 Linux kernel image for version 3.2.0 on 64 bit x86 SMP | 16:31 |
| alexbligh | though I am now wondering why that's not -server (we are just cutting over to Precise). | 16:31 |
| alexbligh | OK will put a bug in LP | 16:32 |
| alexbligh | Is -server much different scheduler wise? | 16:32 |
| apw | -server is an alias for -generic in P | 16:33 |
| alexbligh | apw, well it's not that then! | 16:33 |
| brendand | henrix, i got to reboot with the older kernel and the xorg failsafe log wasn't there | 16:34 |
| brendand | henrix, then i reinstalled the -proposed kernel and it's also not created | 16:34 |
| brendand | henrix, trying another test run to reproduce it | 16:34 |
| henrix | brendand: hmm... so it was some sort of transient thing | 16:35 |
| henrix | brendand: thanks for the update | 16:35 |
| brendand | henrix - maybe. let's see if the next test run is clean | 16:35 |
| henrix | brendand: maybe you can take a look at the /var/log/apt logs, to see if some error occurred during installation. | 16:36 |
| * henrix is not sure if these logs are wiped during testing | 16:36 | |
| jsalisbury | ** | 16:37 |
| jsalisbury | ** Ubuntu Kernel Team Meeting - Today @ 17:00 UTC - #ubuntu-meeting | 16:37 |
| jsalisbury | ** | 16:37 |
| alexbligh | apw, https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1064521 | 16:45 |
| ubot2 | alexbligh: Error: <Bugtracker.plugin.Launchpad instance at 0x91e6fac> bug 1064521 not found | 16:45 |
| alexbligh | I marked it as a security vulnerability because TBH it is I think. | 16:45 |
| alexbligh | Which means ubot2 can't find it. | 16:45 |
| alexbligh | Was that the right thing to do? | 16:45 |
| apw | someone will get it to us when they have reviewed it | 16:49 |
| alexbligh | apw, thamks | 16:50 |
| * ppisati -> EOD | 17:06 | |
| mdeslaur | I've made bug 1064521 public, since it got described here anyway | 17:25 |
| ubot2 | Launchpad bug 1064521 in linux "Kernel I/O scheduling writes starving reads, local DoS" [Undecided,New] https://launchpad.net/bugs/1064521 | 17:25 |
| ogasawara | rtg: I probably missed the memo, are the arm chroots for gomeisa being rebuilt? | 17:26 |
| === jsalisbury changed the topic of #ubuntu-kernel to: Home: https://wiki.ubuntu.com/Kernel/ || Ubuntu Kernel Team Meeting - Tues October 16th, 2012 - 17:00 UTC || If you have a question just ask, and do wait around for an answer! | ||
| rtg_ | ogasawara, hmm, lemme check. | 17:41 |
| rtg_ | ogasawara, they do appear to be missing. perhaps I wrecked them when I was experimenting with the android chroot | 17:43 |
| alexbligh1 | apw, https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1064521 | 17:45 |
| ubot2 | Ubuntu bug 1064521 in linux "Kernel I/O scheduling writes starving reads, local DoS" [Undecided,New] | 17:45 |
| rtg_ | ogasawara, rebuilding chroots on gomeisa. you're happier abusing tangerine anyways. | 17:52 |
| * rtg_ -> lunch | 17:52 | |
| * smb -> EOD | 18:00 | |
| kkaaj | Does the Ubuntu Kernel PPA also contain source packages? I can't find the linux-source package here http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.3.7-precise/. Is it somewhere else? | 18:36 |
| rtg | kkaaj, its built from one of the branches in git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git | 18:36 |
| kkaaj | rtg, and only the three patches from http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.3.7-precise/ are applied? | 18:38 |
| rtg | kkaaj, yep | 18:38 |
| kkaaj | Oh, 0001-base-packaging.patch is 7.6M big. :-) | 18:39 |
| rtg | kkaaj, its all debian packaging | 18:40 |
| kkaaj | How big is a clone of git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git? | 18:40 |
| kkaaj | Or maybe I can directly use https://www.kernel.org/pub/linux/kernel/v3.0/linux-3.3.7.tar.bz2 with the patches from http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.3.7-precise/? | 18:41 |
| rtg | kkaaj, likely | 18:41 |
| kkaaj | rtg, I will try that. Thank you! | 18:41 |
| rtg | rtg@salmon:~/linux$ du -sh linux-stable/ | 18:41 |
| rtg | 682Mlinux-stable/ | 18:41 |
| kkaaj | Will precise get a 3.3 kernel? Or will it stick with 3.2? | 18:43 |
| rtg | kkaaj, there will be an LTS kernel for Precise that is a backported 3.7. You can get linux-lts-quantal from https://launchpad.net/~ubuntu-x-swat/+archive/q-lts-backport | 18:54 |
| === henrix is now known as henrix_ | ||
| rtg | ogasawara, are you happy with Ubuntu-3.5.0-17.28 ? I'm about to fire it up to the LTS PPA. | 19:08 |
| ogasawara | rtg: yep, pushed to master and uploaded. still need to rebase master-next. | 19:08 |
| rtg | ogasawara, ack | 19:08 |
| ogasawara | rtg: we've had a request to get ti-omap4 and armadaxp rebased to it as well. | 19:08 |
| ogasawara | rtg: I figure we can get ti-omap4 rebased, but might need to nudge ike about armadaxp | 19:09 |
| rtg | ogasawara, you wanna do ti-omap4 ? I have a guest showing up soon. | 19:09 |
| ogasawara | rtg: yep, I can do that | 19:09 |
| rtg | ogasawara, thanks | 19:10 |
| rtg | ogasawara, gomeisa arm chroots look like they are back in business. | 19:12 |
| ogasawara | rtg: ack, thanks. | 19:12 |
| === rtg is now known as rtg-afk | ||
| === rtg-afk is now known as rtg | ||
| apw | ogasawara, lowlatency is done and up | 20:34 |
| apw | ogasawara, linux-signed is in | 20:34 |
| ogasawara | apw: ack, thanks | 20:34 |
| rtg | still building ti-omap4. | 20:35 |
| rtg | that is a giant rebase. | 20:35 |
| * rtg -> EOD | 21:06 | |
| bjf | ogasawara, about? | 21:36 |
| ogasawara | bjf: yep, what's up | 21:36 |
| bjf | ogasawara, can you mumble? | 21:36 |
| ogasawara | bjf: just a sec | 21:36 |
| infinity | I can haz linux-meta-ti-omap4 to go with that linux-ti-omap4 in proposed? | 22:46 |
| infinity | ogasawara: ^ | 22:46 |
| doko | ogasawara, rtg, apw: can you commit to start r with 3.6 headers? | 22:49 |
| ogasawara | infinity: yep, prepping it now and will upload | 22:51 |
| infinity | ogasawara: Less than three. | 22:51 |
| ogasawara | doko: our ubuntu-r repo is already based on 3.6, so we should be able to commit to that. I'll discuss with rt and apw tomorrow so we're all on the same page. | 22:52 |
| ogasawara | infinity: uploaded | 22:57 |
| infinity | ogasawara: Danke. | 22:57 |
| ogasawara | doko: oops, I misspoke, our ubuntu-r repo is rebased to v3.7-rc1 already. so you'd prefer if we start with an initial v3.6 upload rather than v3.7 based? | 22:59 |
| doko | ogasawara, no, that fine | 23:09 |
| ogasawara | doko: ok great | 23:10 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!