=== mdz_ is now known as mdz [10:20] does anyone know if Ubuntu 8.10 will have ext4 enabled? [10:38] it does [10:38] not by default though === peterere1 is now known as petererer [10:39] the kernel module is there, and the userland tools are too === asac__ is now known as asac === thunderstruck is now known as gnomefreak === pgraner_ is now known as pgraner [14:53] rtg: so Corey Minyard replied and said Signed-off-by: him ... [14:53] does that mean it's in the upstream kernel now? [14:53] who has to commit it? [14:54] Keybuk: no, I still have to send it to akpm. [14:54] I though they'd have a maintainer's tree, but I guess not. [14:54] how do you do that? [14:55] Keybuk: I'll have to figure it out. Its been awhile since I sent anything to Andrew. [14:56] Keybuk: once I do send it, you'll get 1 or more automated responses that tell you when the patch transitions states [14:59] so, let me work this out [14:59] you first send the patch to the subsystem maintainer [14:59] cc lkml, just for kicks [14:59] and _if_ you think it might be worth a stable update later, add a Cc: stable@kernel.org in the body [14:59] the subsystem maintainer might commit it themselves to their own tree [14:59] or they might just reply with another signed-off-by line? [15:00] at that point, I guess you *reformat* the patch to include that new signed-off-by line, and then send it to someone else? [15:00] Keybuk: the subsystem maintainer _should_ commit to their tree. Seems like Cory is being lazy, or they don't actually have a subsystem tree. After all, isn't that the role of a subsystem maintainer? [15:01] why would you have a subsystem tree for every single driver? [15:01] Anyway, the Cc is to notify stable when the patch is included in Linus tree. [15:02] what if there's no subsystem? [15:02] Reformatting the patch with an extended SOB would normally happen when he maintainer added it to their tree. [15:03] Keybuk: All drivers belong to a subsystem at some layer. [15:03] let's say I get around to writing waitfd() [15:03] who would I send that patch to ? [15:04] Keybuk: you're getting pretty high up the stack. You'll have to get new syscalls ack'd by Linus' first level lieutenants. Guys like Garzik, Miller, Morton, etc. [15:05] suffice it to say, there are no hard and fast rules for patch submission. [15:05] its kind of fluid :) [15:05] generally, someone will tell you what to do if you've transgressed. [15:51] just setup a Netgear WNDR3300 (RangeMax Dual Band Wireless-N Router). Seems like it's working pretty good. [16:22] * apw wonders if you are using it as a router, or have installed intrepid on it :) [16:30] nope, just as a router. I'm using it to test wireless stuff, so I don't want to introduxce any more problems to the mix. [16:36] so is there any chance at all of the freezer control group (just pulled into Linus' tree a few days ago) making it into the intrepid kernel? [16:38] hallyn: this close to release? I don't think so. [16:39] drat [16:41] NCommander: around? [16:41] lamont, yup [16:42] guess i'll see how trivially i can pull those patches into intrepid.git [16:45] lamont, what's up? === thegodfather is now known as fabbione === thunderstruck is now known as gnomefreak === thunderstruck is now known as gnomefreak [18:58] do any of you guys have USB 3g modems? they pretend to be CD drives until you eject them, then they become modems. I just tried two different models on two laptops and they just spew loads of scsi errors [18:59] ./win 245 [18:59] gar [19:19] 100 for /proc/sys/vm/vfs_cache_pressure seems too low at least on laptops [19:19] anyone around for a discussion about it? [19:23] calc: you mean too high? [19:23] jdong: needs to be a larger number it seems or something else is causing problems during high disk io [19:23] jdong: my system is almost completely unresponsive when i do a transmission torrent check [19:23] jdong: can't use evolution or even pidgin at times, not even repainting the screen [19:24] calc: isn't that more of an IO scheduler issue? [19:24] jdong: probably so, someone pointed me at this tuning option but i think its probably a io scheduler issue [19:24] jdong: i don't know enough about how disk io works under linux to know what is the problem [19:25] calc: do you see any evidence of swapping? [19:25] calc: if you're swapping it could possibly be a vfs_cache_pressure or vm.swappiness issue [19:25] 4MB usage in swap [19:25] ~ 2300MB in buffers [19:25] calc: yeah that's not cache pressure then [19:25] er not buffers [19:25] 2300MB in cached, sorry about that [19:25] i'm checking a 28GB torrent [19:25] calc: that's your IO scheduler not prioritizing the smaller requests needed to check your e-mail vs the 5GB of torrent data :) [19:25] err, 28GB :) [19:25] heh [19:26] as this channel is logged i'll leave it at that ;-) [19:26] but it seems in general not to fairly divide up disk io between apps [19:27] to the point that pidgin didn't repaint for what seemed like at least 30s, i finally switched to my terminal window, so i don't know how long it actually took [19:27] calc: well it fairly divides between IO requests [19:27] calc: doesn't help if your torrent client is making a lot of said requests :) [19:27] calc: have you tried to see if the deadline IO scheduler works any better for your workload? [19:28] echo deadline > /sys/block/sda/queue/scheduler [19:28] jdong: how do i switch to it? [19:28] oh ok [19:28] is that safe to do while under heavy disk io? [19:28] yeah [19:28] they're designed to be online-switchable [19:28] ok [19:29] that is amazingly better [19:29] completely responsive [19:30] hmm maybe i should switch back to verify its actually not a placebo thing [19:30] what was the other one called? [19:31] it definitely seems more responsive than it was, but it might be due to other reasons, need to try flipping back and forth [19:32] hrm, it appears deadline is the default scheduler here: [19:32] [denisovich ~]$ cat /sys/block/sda/queue/scheduler [19:32] noop anticipatory [deadline] cfq [19:33] i switched to cfq to see what it does [19:33] calc: cfq is the default for -generic kernel [19:34] ok === steve__ is now known as sbeattie [19:35] ah, right, that machine is running the -kernel server [19:35] calc: in my experience excessive disk-io when using bittorrent is often caused by fragmentation in the (ext3) file system [19:35] well deadline does seem more responsive to a non-scientific click on my mailboxes in evolution test [19:35] paran: well doing a torrent check causes a lot of disk io also [19:36] and apparently helps to expose holes in the scheduler [19:36] hmm when i waited a bit to let cache purge my mailboxes it seems deadline might not help much after all [19:37] so i really can't tell :\ [19:37] calc: of course, but the disk IO will hurt a lot more if it is mostly seeks, cause of fragmentation [19:37] yea [19:37] so is there any way to defrag ext3? :) [19:37] everyone claimed that ext3 never needed to be defraged in the past [19:37] calc: you can use "filefrag " to see how fragmented a file is [19:38] really fraged [19:38] calc: as far as I know you can't. the problem is that ext3 do not like the usage pattern of a typical torrent client [19:38] eg 9915 extents found perfection would be 3 [19:39] calc: you should get a client that pre allocates (writes the whole file full with zeroes) before starting the download [19:39] paran: well the default one in ubuntu that is preinstalled doesn't do that, so maybe i should file a bug :\ [19:40] though i think filing a bug about the lack of a defragger when it is clearly really needed would also be good [19:40] jdong: you made one i see in google ;-) [19:41] calc: haha :) [19:42] calc: something else to consider with CFQ is the use of ionice priorities [19:42] calc: the problem is that most people would then file bugs complaining about their torrents not starting right away [19:42] calc: you can punt your torrent client to best-effort priority 8 [19:42] jdong: ah [19:42] paran: ext4 will support preallocation right? [19:42] i.e. KTorrent with XFS uses the XFS preallocation calls. [19:43] takes a fraction of a second to reserve optimal contiguous space for the entire torrent [19:43] jdong: so is your defrag tool safe? :) [19:43] calc: it is basically a cp a b; mv b a; [19:43] calc: as long as your filesystem treats mv b a atomically it should be safe [19:43] worst case is leaving behind a tempfile [19:43] ok [19:44] where is the official copy of this tool? :) [19:44] jdong: I haven't really followed ext4, but I think so [19:44] calc: you're a pretty experienced hand at UNIX, you probably can just do it by hand for problematic files :) [19:44] calc: it should be in bzr somewhere on launchpad [19:44] (lol) [19:44] lol [19:44] ah i see it [19:44] 3rd hit on google [19:45] it seems you have lots of empty branches on lp that you forgot to delete [19:45] haha that could very well be [19:46] calc: you know, I've always considered writing a hackish script that's the equivalent of Windows's foreground-IO-booster [19:46] the default behavior of workstation builds of Windows is to allocate a higher priority for the foreground app [19:46] that might arguably help interactiveness a bit if you are runing a torrent client and switch to your e-mail client [19:49] yea [19:49] do the defragmenting program you are talking about now simply moves files between file systems? [19:50] jdong: defrag is pretty cool :) [19:50] paran: pretty much it makes a copy of the file and checks if the new copy has fewer fragments according to filefrags [19:50] of course it would be nice if ext3/ext4 or something like that had built in defrag option [19:50] paran: it's the same strategy xfs_fsr uses, except XFS has a efficient way to sort by most fragmented files [19:50] iirc XFS has something like that [19:50] yup [19:51] when i tried it was buggy though and ate my fs ;-) [19:51] calc: well it's as safe as XFS is ;-) [19:51] what it does is basically equivalent of cp and mv operations except it uses an atomic mv [19:51] jdong: heh [19:51] (XFS mv is NOT atomic by default) [19:51] that IMO is insane [19:51] but technically POSIX-compliant :D [19:51] jdong: that is pretty much the only thing you can do from user space. I have defragmented files by moving from ext3 to a tmpfs and back :) [19:52] paran: yeah, I don't know (or WANt to know!) what that does to free space fragmentation [19:52] but empirically it helps a lot with file fragmentation and read performance [19:52] jdong: does defrag have an option to skip files that are mostly defraged already? [19:52] calc: there should be a frags/MB threshold commandline flag [19:52] ok [19:53] calc: by default IIRC it is pretty sane, like around 2frags/MB? [19:53] oh ok [19:53] paran: another neat trick that's worked wonders for me: tar and remove all the files mentioned in readahead/boot, then restore them at once from the tar [19:53] i didn't look to see what it defaulted to [19:53] paran: that's managed to shave 10-15% time off readahead for me [19:54] I'm guessing EXT3 is more likely to allocate files in the same area of the disk if created at the same time? [19:54] what would be neat is a defrag that could work like norton defrag on windows (or how it used to work) where you could even select file order, like what you are basically doing with tar [19:55] jdong: yeah, that should work. [19:56] calc: yeah I wish we actually had a defrag API call! [19:56] (ext4 feature) [19:56] and have option to defrag order most recent to oldest accessed files on the disk, that would help for non ssd's [19:56] jdong: oh its in ext4? [19:56] in the mean time you could always use XFS. [19:56] calc: Windows is nice that it gives an atomic defrag function that basically takes a inode and place to put it [19:57] paran: XFS is too scary for me after it eating my disk multiple times in the past :\ [19:58] calc: oh? we have many TiB on XFS at work, no problems and much better performance than ext3. [19:59] paran: well the last time i used XFS was quite a while back > 4 years ago so it probably has improved [19:59] but it was very very unreliable back then [19:59] calc: XFS used to have a lot of out-of-order write hazard bugs back then [19:59] well s/bugs/'features'/ [19:59] it's better [19:59] not as reliable as ext3 for irresponsible plug-pulling but should be good enough in gneeral [19:59] after seeing its bugs and reiserfs bugs i decided to go with what everyone uses since it tends to be more debugged, eg ext3 for now [19:59] I put all my torrents on XFS for performance [19:59] my data and OS is on ext3 [19:59] XFS's small-file performance still sucks [20:01] the only thing I remember is that XFS on 32 bit x86 used to be problematic, but it was years since we installed anything not x86_64 :) [20:01] jdong: is defrag smart enough not to cross mount points? [20:02] * calc considers running it on / [20:02] paran: i'm still having to use i386 since resume doesn't work right on my laptop on amd64 [20:05] wow transmission manages to fairly consistently frag files ~ 32frag/MB [20:37] WTF/ [20:46] so ext4 became 'stable' as of yesterday? [20:47] apparently fedora 10 is going to use it [20:48] hmm actually that happened on oct 11, wikipedia seems to be conflicting itself [20:57] calc, well, fix the confliction