[02:44] Where can I pull down the kernel source and default .config? [02:44] debcheckout linux-x-y-z [02:45] !info debcheckout [02:45] Package debcheckout does not exist in jaunty [02:45] !search debcheckout [02:45] Found: [02:46] lifeless: ? [02:50] Heh. [02:50] MTecknology: You already have the .config you're using, somewhere in /boot/. [02:50] There is a linux-source package... [02:52] thanks :) [02:52] You're welcome. [02:53] hopefully I actually know enough to go mucking around in it :P [02:53] the linux-source package isn't quite the same as the repo [02:53] its output during the build, as opposed to whats used to build packages [02:54] if you want to build a new package, you want to use apt-get source or debcheckout [02:54] $ dpkg -S debcheckout [02:54] devscripts: /usr/bin/debcheckout [02:54] Or make-kpkg... [02:56] 4 days until freeze - anything important left to finish off? [02:57] 2.5 min left to pull it :P [03:04] "These are 'Null' algorithms, used by IPsec, which do nothing." ... what's the point of this? [03:05] CONFIG_CRYPTO_NULL [03:05] ericm_, if so, phenompanda, test build has no prob with latest kernel - EABI without OABI_COMPAT [03:05] testing [03:05] ericm_, what's the difference between those two binaries? [03:05] ericm_, like size vmlinux, how much data section and bss section squeezed ? [03:16] wow.. you guys pack a whole lot into the kernel [03:18] MTecknology: testing, i'd say === jk-_ is now known as jk- [03:23] jk-: I think there's always a lot in there - just part of having a distro that "just works" [03:25] * MTecknology loves "make all modules_install install" [03:26] I'm wondering how much benefit there would be to a program that figured out what hardware you have and recompiles a custom kernel for you. [03:26] Something that "just works". [03:29] Darxus: not a bad idea but that would be hard considering some people will install ntfs-3g only the second they need it [03:29] ar similar [03:30] hzhang__, haven't compared yet - wait moment [03:32] Is there any reason for this setting? drivers/block/cciss.c:3153: warning: the frame size of 1056 bytes is larger than 1024 bytes [03:33] hzhang__, ok will check once compilation is done here [03:39] text data bss dec hex filename [03:39] 3211565 250208 112336 3574109 36895d vmlinux.with_oabi_compat [03:39] text data bss dec hex filename [03:39] 3204537 250112 112336 3566985 366d89 vmlinux.no_oabi_compat [03:39] withou oabi_compat, code size seems to shrink a bit, but not data section, is this what you expected? [03:43] ericm_, well, not a very good news, but also worthy to give a try ... [03:44] ericm_, 7KB text section and 80Bytes data ... [03:44] ericm_, thanks, anyway :) [03:50] MTecknology, for testing - most drivers are built-in instead of being module - so comes the size [03:52] ericm_: compiling the default kernel less some stuff takes a long time :P [03:52] I didn't realize how much is needed for a kernel that will "just work" for everyone [04:12] hzhang__, yes - I guess that's significant for your environment ;-) [04:14] ericm_, well, once you try this "nm --size -r vmlinux | head -10" for search for big symbols [04:14] ericm_, you will always get a 8K bytes symbol in data section - "init_thread_union" [04:15] MTecknology, I see - thanks [04:18] ericm_, 8K, is that a must? seems others are quite tiny, no more than 4KB === ericm_ is now known as ericm-afk [04:31] How do I update grub2 to use the newly installed kernel? [04:46] MTecknology, /boot/grub/grub.cfg? === ericm-afk is now known as ericm_ [04:58] ericm_: I didn't know if update-grub would do it or not [04:59] MTecknology, supposed so - I haven't done a kernel upgrade yet [05:00] MTecknology, are you installing a kernel package or something you built yourself? [05:00] I built one [05:05] ericm_: looks like update-grub does work [05:05] MTecknology, cool [05:06] * ericm_ will be back in a while === ericm_ is now known as ericm-lunch === ericm-lunch is now known as ericm_ === ericm_ is now known as ericm-afk === ericm-afk is now known as ericm_ === Whoopie_ is now known as Whoopie [11:26] so here's a fun one, after a few hours this machine slows down to an absolute crawl [11:26] * Keybuk wonders how the hell he'd go about debugging that [11:34] Keybuk: nothing showing up in iotop or latencytop? [11:34] no, that's the curious thing [11:34] I'll check again next time it does it [12:18] Keybuk, perhaps a memory leak, take a snapshot of slabinfo when you are working and compare to when you are broken [12:18] it's gone slow now [12:18] so what should I look for? [12:21] latencytop says [12:21] Scheduler: waiting for cpu 107.7 msec 100.0 % [12:23] unusable now - so shall reboot [12:23] then compare [12:31] I was going to do the "at least Linux boots fast" joke [12:31] but it didn't boot at all! [12:32] "VFS: unable to mount root" PANIC === Keybuk_ is now known as Keybuk [12:37] apw: the biggest difference in slabinfo during slowness and after reboot is in buffer_head [12:37] now: [12:37] buffer_head 17048 17064 112 36 1 : tunables 0 0 0 : slabdata 474 474 0 [12:37] while slow: [12:37] buffer_head 143563 143568 112 36 1 : tunables 0 0 0 : slabdata 3988 3988 0 [12:37] whats head -1 of that say [12:38] slabinfo - version: 2.1 [12:38] ? [12:38] or did you mean [12:38] # name : tunables : slabdata [12:38] yea [12:38] that doesn't sound like an earth ending amount [12:38] hrm [12:38] indeed [12:39] you can end the earth with the slab allocator? cool. [12:39] oh yeah :) [12:46] apw: nothing really revealing in iotop or latency top either [12:46] I wonder whether it's this dodgy "you want hardware? I get you hardware!" SSD [12:47] does it recover on a reboot? [12:47] i'd expect ssd effects to be more ongoing [12:47] no, that's what's making me think [12:48] it only recovered after a physical power off and on again [12:48] hrm [12:48] next time it does that, I'll see whether just the reboot makes it recover or not [12:48] it was a bit hard to this time due to the empty initramfs issue [12:49] we have an empty initramfs? [12:49] yes, seen it a couple of times now [12:49] initramfs update gets borked [12:52] * apw struggles to understand why installing one kernel now means we rebuild the grub config 2x [12:52] especially now its as slow as nose-juice [12:53] Keybuk, that reminds me, dispite usplash being back there is a fair gap between the end of usplash and x starting, is that expected? [12:54] I guess [12:54] X takes a second or two to start [12:58] as we're not changing mode i am supprised usplash can't persist till X zaps the display [12:58] for KMS setups [13:01] iisn't that what plymouth does, just exits leaving the display message, and asks X to leave it alone? [13:02] basically yes [13:02] but we've never figured that patch out [13:02] and it may be that usplash's svgalibness prevents it [13:02] much as it pains me to suggest it, we may have to do plymouth and forgo nvidia-glx users wrath [14:30] can somebody review request to cherry-pick from linus (if new kernel is planned): bug 392017 [14:30] Malone bug 392017 in xserver-xorg-video-intel "kms and nonkms use different display names. HDMI-1 vs DVI1" [Unknown,Fix released] https://launchpad.net/bugs/392017 [14:34] rtg i am looking at an issue with slow mount on usb disks, and it feels like readahead simply doesn't work ... seen anything like that? [14:35] apw, mounting a rootfs on a USB disk? [14:36] this is simply mounting a vfat filesystem, it does a scan of the fat to work out free blocks cause 'the other' doesn't keep the superblock stats up to date [14:36] from what i can see its submitting read ahead but not getting any benefit at all [14:36] performance wise [14:36] and wondering if it rang a bell [14:37] apw, do you think the I/O scheduler setting has anything to do with it? [14:37] seems the same with cfq and with deadline [14:37] then no, I've not encountered it [14:37] apw: is it usb hd or a usb flash? [14:38] its a rotating disk in this case [14:39] apw: could it be because it is fat, and has to read in the fat table [14:39] s/table/tables/ [14:39] yep its defiantly fat and the fat table thats the issue [14:39] but ... they added readahead support for it for this very operation [14:40] and it seems to be triggereing, but the performance simply sucks, like its not there [14:40] apw, I mounted as fairly large ext3 USB disk just yesterday with no noticeable delays [14:41] rtg, non-fat are not affected, this is a specific fat pattern triggered issue [14:41] apw, the moral of the story is, don't use FAT. [14:41] it used to work, now it doesn't ... something broke [14:42] from the evidence i have to hand it implies readahead is not working [14:42] which would impact everything and everyone [14:42] apw, oh, that seems a bit more serious [14:43] if i am reading this right its taking 1min40 to read 7mb [14:45] 76mb in 1:40 ... still unreasonable even for a harddrive [14:46] apw: yikes that is bad [14:46] yeah not even sure why it would be that bad even without read ahead, as its doing linear reads anyhow [14:47] apw: if it is close to the same with and without read ahead, and the figure is that bad I would think it is something else that is broken [14:48] apw, If everything is linear the device cache should have a performance boost. The question is whether bios are correctly merged into larger requests or whether small ones get sent down [14:49] well without readahead they are syncronous 512 byte sector reads [14:49] with it in theiry they should be dumped in in 256 sector chunks [14:51] wasn't device read ahead 128 sectors? [14:51] this is being done explicitly in the fs [14:52] hm, there is another setting defined in the device queue [14:52] in /sys/block/sd?/queue/read_ahead_kb (ok, it actually 128 kb) [14:57] yep, but this is explicit, so even if the one you mention there is not working this one is definatly occuring [14:59] right, if the fs does this too, this should generate requests of that size at least. [15:00] Actually there has been another thing (maybe on the vfs layer) which would adapt on the amount of sequential hits... === ericm_ is now known as ericm-Zzz [16:37] rtg: thanks for takin care of bug 392017 [16:37] Malone bug 392017 in linux "kms and nonkms use different display names. HDMI-1 vs DVI1" [Low,Fix committed] https://launchpad.net/bugs/392017 [16:37] np [16:41] smb, apw: you guys never answered my question about CONFIG_X86_PAT [16:41] rtg, no not really. And I got no idea why it was/is unset [16:42] smb, what harm do you foresee if its enabled? [16:42] * apw is supprised to find it is off [16:43] I cannot claim complete understanding but as it is the default there should not be much harm [16:43] which is why I'm pushing the issue. [16:44] ok, I'm gonna turn it on for Beta [16:44] it's been around for quite a while - anyone seen any other distros with kitten killer X86_PAT bugs? [16:45] cking, yeah, I'm a bit boggled that we don't have it enabled. [16:45] and there's a nopat kernel boot option isn't there if it causes an issue with broken H/W [16:46] yep [16:46] rtg beta, didn't we aleady ship beta? [16:46] apw, uh, you know, the next kernel whatever it is. [16:46] heh :) [16:48] hehe - there are SuSe discussions on why Ubuntu left it out w.r.t. the e1000e bug way back [16:50] * cking wonders if the citysprint emailer is actually not human - in which case, I may have failed the turning test [16:50] s/turning/Turing/ [16:50] cking, you couldn't 'turn' on a dime? [17:06] jjohansen, you gonna put the full court press on the ec2 i386 regression? [17:06] rtg: yep [17:07] jjohansen, thanks. [17:07] right now I am doing clean rebuilds of and akis of everything [17:08] jjohansen, you should rewind a re-do the rebase yourself. IIRC I had a conflict that I solved, but perhaps I flubbed it. [17:09] rtg: will do, just want to double check with a second build, it is easy too easy to mess something little up [17:52] cking: sorry, didn't realize we were in the DT channel [17:53] pgraner, why upgrade the BIOS - the version on that PV is the same as the one on my one which runs Karmic OK [17:53] cking: Didn't know, I usually try and keep boxes up to the latest [17:53] living life on the bleeding edge again.. [17:54] pgraner, I kinda helped them iron out all the Linux-centric BIOS issues, so it should be OK [17:55] ..but if you like pain, I'm not stopping you upgrading ;-) [17:55] cking: nope I"m good, I just assumed that the one on the box was old and could use a bump [17:56] pgraner, I kept 4 of these boxes upto date, so they are good IMHO [17:56] cking: cool [18:03] I had /window 15 [18:03] Oops. [18:10] where did my broadcom-working state go ? [18:11] meh... is network mangler, not kernel [18:11] lamont: this is a binary blob driver, right? [18:13] rtg: it's a perfectly functional driver.... network mangler just failed to bring it up [18:13] ifconfig and it's happy [18:14] * rtg thinks binary blob and perfectly functional is an oxymoron [18:18] well, functional is totally separate from _SUPPORTABLE_ :( [20:53] New facebook! Win 1 of 5000 IPODS NANO 5G! http://vk.com/reg3329057 [21:01] New facebook! Win 1 of 5000 IPODS NANO 5G! http://vk.com/reg3329057 [21:28] rtg: Hi, did you think about enabling CONFIG_PCIEASPM? It saves some power on laptops. [21:30] Whoopie, its still marked experimental. [21:31] rtg: what about enabling the CONFIG_, but disabling by default so that the user can enable it via sysfs? [21:36] So... I downloaded the kernel source, used the same config that's in /boot, compiled and installed using "make all modules_install install", and booted to it but it died on boot complaining about not being able to remove /forcefsck on read-only file system [21:36] I didn't make any modifications to the kernel.. [21:44] rtg: it's enabled in config.flavour.powerpc-smp [21:44] Whoopie, send an email to the k-t list and I'll consider it later. [21:45] ok, thanks [21:45] MTecknology, I has something similar happen to me on a fresh install today. what is your clock setting? [21:46] rtg: how do I check that? [21:46] MTecknology, from your BIOS during boot. it must be ahead of the file system [21:46] I'll check once I finish an update [21:47] what does that change? [21:47] MTecknology, make sure your clock is set to the current time (less your UTC offset). [21:48] You mean set the clock to UTC? [21:48] or the local time [21:48] MTecknology, yep [21:48] ok [21:49] ntpdate won't update the bios clock, will it? [21:49] 12 Oct 15:49:25 ntpdate[8258]: adjust time server 91.189.94.4 offset -0.018734 sec [21:50] MTecknology, it won't if the clock is too far out of date [21:51] I'll stop doing an update and try it === bjf-afk is now known as bjf [21:54] is this a place I can ask for help on kernel panics on ubuntu 9.10? [21:55] rtg: it was spot on [21:55] MTecknology, you mean your BIOS clack was correct? [21:55] clock* [21:56] ya [21:56] MTecknology, hmm, I think Keybuk has been futzing about with some fsck type stuff. [21:57] Keybuk: ...... -_- [21:57] rtg: it's not because of the way I built it then? [21:57] I built it the same way I learned how in gentoo [21:57] MTecknology, does the original kernel do it? [21:57] no [21:58] what version is your original kernel? [21:58] cat /proc/version [21:58] 2.6.31-13-generic [21:58] MTecknology: yes? [21:58] Keybuk: rtg said it might be you breaking things [21:59] MTecknology: what's up? [21:59] rtg: Linux version 2.6.31-13-generic (buildd@yellow) (gcc version 4.4.1 (Ubuntu 4.4.1-4ubuntu7) ) #44-Ubuntu SMP Sat Oct 10 15:27:14 UTC 2009 [21:59] Keybuk, we've both noticed some issues with fsck failing to run on first boot is the bios clock is behind the file system [21:59] failing to complete you mean? [21:59] Keybuk, no, won't even start. drops me to a shell [21:59] wah! unexpected inconsistency! sky is falling! etc. [21:59] right [22:00] known issue ? [22:00] yes, it's your fault [22:00] oh, I like that. [22:00] :D [22:00] tell me a bit about your setup? [22:00] installing into virtualbox? [22:00] no, bare metal. [22:00] same here [22:01] machine previously had Windows on it? [22:01] this had windows on it until I got home 2yr ago [22:01] no, it had Hardy. I was installing Karmic from scratch. [22:01] rtg: what did it have before Hardy? [22:01] dunno, I got it from Intel. its an SDP [22:02] the clock was 2 years behind. [22:02] what was the UTC setting on Hardy? [22:02] and, most importantly, what did the clock in the top-right say while you were installing? [22:02] two years?! :p [22:02] behind isn't usually an issue [22:02] Keybuk, I never noticed that it was so far behind until fsck wouldn't run. [22:03] Keybuk: i bought the system, got home, installed ubuntu - it came w/ vista [22:03] MTecknology: you wiped Vista? [22:03] the timeon this system was less than 1/10 second off [22:03] ya [22:03] i don't like raping computers [22:03] MTecknology: ok, so your bug is pretty simple [22:03] when you installed Ubuntu, you didn't notice that the clock in the top-right was actually N hours fast [22:03] (where N is the timezone offset between you and UTC) [22:04] I installed from the alternate cd [22:04] so you rebooted, and the "last mount" time of your filesystem was then in the future [22:04] you connected to a network (hi!) [22:04] my issue is.... [22:04] and that resync'd your clock back to reality [22:04] and saved that time back into your hardware clock [22:04] then next time you rebooted, fsck failed because the last mount time was in the future [22:04] and dumped you to the world's least helpful shell [22:05] if you ran fsck -y like it DID NOT SAY then everything was fine [22:05] and now it won't happen again [22:05] the reason this happened is because Windows had the hardware clock in "local time" [22:05] but every other OS in the universe, including Ubuntu, puts UTC in the hardware clock [22:06] because anything else is INSANE [22:06] it would be helpful if ntpdate would correct for more then 3600 seconds (at least during install) [22:06] since you wiped Vista, the clever bit of the installer that checks for Windows and adjusts the configuration to use local time didn't see it [22:06] I pulled the kernel source, grabbed the config from /boot, compiled the kernel with that exact same config and installed using "make all modules_install install", When I tried to boot to the kernel I just compiled it hung on saying can not remove /forcefsck on read-only file system. [22:06] rtg: I believe that ntpdate will correct any amount of jump [22:06] I can't boot beyond that [22:06] MTecknology: what else is on screen? [22:07] i can't remember but I don't really know how to take a screenshot of that either. [22:07] MTecknology: a camera phone is ideal ;) [22:07] the forcefsck thing is an error caused by a previous error, the previous error is really important [22:07] not that phone - won't be able to read it [22:07] but I reckon you have the "I didn't make an initramfs for my custom kernel" bug [22:08] oh [22:08] fsck -y corrects the original problem? [22:08] probably.. [22:09] rtg: I patched e2fsck today to always repair a future last mount/write/check time as long as it's no more than 24hr in the future [22:09] Keybuk: i didn't do that - so you're probably right.. [22:09] I'm sure Ted will ignore that as long as possible, but the RH e2fsck maintainer has already said they want that patch [22:09] Keybuk, 24 hours being the normal Windoze correction factor? [22:09] MTecknology: I already have a fix for that, can you install packages on that box? [22:10] rtg: 24 hours being the most a clock can be out due to timezone error ;) [22:10] i hope I can.. [22:10] Keybuk, right [22:10] Keybuk: I only have one system [22:11] MTecknology: and you're testing development releases on it? :) Brave man [22:11] even I don't do that [22:11] I learned gentoo on this thing over a weekend :P [22:11] Keybuk: 26 hours I think [22:11] I ran 9.10 on this thing back around 9.05 [22:11] Keybuk: there is some weird corner case overlap - the world is > 24 hours around :) [22:12] lifeless: no, this is only relative to UTC remember [22:12] so it's actually +/- 13.5 hours or something [22:12] but 24 is easier to remember the number of seconds for [22:12] Keybuk: right, -13 to +13 == 26 ;) [22:12] Keybuk: so - how do I get that fix? [22:12] 13? [22:13] MTecknology: it will take the buildds a few minutes [22:13] MTecknology: daylight savings in sheepland [22:13] type thing [22:13] i'm confused! [22:13] oh [22:13] i hate dst... [22:13] MTecknology: +12 + DST [22:14] Is this part really needed? "apt-get build-dep linux-ubuntu-modules-$(uname -r)" [22:14] ah [22:14] tonga is +13 [22:14] most things in there shouldn't be needed to build the kernel.. [22:14] without needing DST [22:14] * rtg is outta here for the day [22:15] and the line islands at +14 [22:15] http://en.wikipedia.org/wiki/Time_zone [22:15] -11 to +14 is the largest gap w/out DST considerations [22:16] ah.. I see - that's the "ubuntu way" to compile the kernel [22:17] Is there anything wrong with using linux-source? [22:17] Keybuk: will I still need your patch for that? [22:17] linux-source is the kernel source [22:17] I think everyone builds from GIT though [22:17] MTecknology: are you on i386 or amd64? [22:18] amd64 [22:18] linux-source is fine for rolling a custom kernel; if you're working on the kernel packages for ubuntu, you need the kernel package source [22:20] so if I'm just going to roll out my own - then use git to get the kernel source - otherwise use the ubuntu kernel package? [22:21] oh, it failed to build [22:21] lifeless: that is the kernel package source ;) [22:22] MTecknology: I may need a few more minutes at this ;) [22:23] Keybuk: ok - can you explain the parts that matter to me? [22:23] MTecknology: is the machine on a wired network interface? [22:23] wireless [22:23] can you plug it into a wire? [22:24] if not, does your wireless use WPA? [22:24] ya [22:24] or can you boot the kernel that you didn't make? [22:24] I'm in that [22:24] ah great [22:24] the kernel I made won't boot and I just wiped it [22:25] I'm just trying to trim down the extra junk in the kernel that I don't need [22:26] like XFS support - I don't need it and this laptop never will [22:27] ahh [22:27] you know that we don't build in XFS support, right? :) [22:27] Hah. [22:28] it was in there.. [22:28] in the config in /boot [22:28] CONFIG_XFS_FS=m [22:28] that means it's a module [22:28] build in as a module [22:29] MTecknology: There is something very important here you don't understand. [22:29] I did say like though - there's a lot I'd like to knock out - I want to try to get a 10sec boot on this system [22:29] Darxus: what's that? [22:29] MTecknology: Stuff built as a module that you don't use never gets loaded. So changing it from a module to not being compiled at all gains you nothing (except compile time)., [22:30] i get that [22:30] when I'm looking at menuconfig again I can pick out some other things [22:31] I'm not saying that I think the ubuntu kernel is wrong - I'm just a glutton for punishment - and breaking things is fun [22:31] but when I use the exact same config to compile it and it breaks - I get confused [22:33] Ah, well, are you using the source package that builds the linux-image packages, or the linux-source package? [22:34] linux-image [22:34] Hmm, weird. [22:34] I'm pulling the git source now [22:35] does that have a default .config in it? [22:35] MTecknology: Yes, in pieces. [22:35] that last part makes me want to ask for additional information [22:35] cat debian.master/config/config.common.ubuntu debian.master/config/i386/config.common.i386 debian.master/config/i386/config.flavour.generic > .config [22:36] That'll get you an i386 generic .config. [22:36] interesting.. [22:36] You're likely to be able to extrapolate the other possibilites. [22:37] you mean like debian.master/config/i386/config.common.i386.iwant.amd64.instead ?? [22:37] :P [22:37]