=== psusi [n=phreak@54.161.205.68.cfl.res.rr.com] has joined #ubuntu-kernel | ||
=== mgalvin [n=mgalvin@ubuntu/member/mgalvin] has joined #ubuntu-kernel | ||
=== mgalvin [n=mgalvin@ubuntu/member/mgalvin] has joined #ubuntu-kernel | ||
bluefoxicy | does anyone think that the Linux kernel mainline should have built-in memory compression for memory stress? | 01:27 |
---|---|---|
bluefoxicy | I did some test with Rodrigo's patch, tweaked it to be able to compress in larger blocks | 01:28 |
bluefoxicy | using the WK4x4 and WKdm compression algorithms (lz0 I didn't like, it corrupted memory on occasion) I got 20% or so at 8k blocks | 01:29 |
bluefoxicy | but with 16k blocks I averaged 35%, and with 32 I got 40%. Beyond there, there wasn't much gain. | 01:29 |
=== mgalvin [n=mgalvin@ubuntu/member/mgalvin] has joined #ubuntu-kernel | ||
airlied | just buy more RAM.. it's cheap... | 01:34 |
airlied | or fix apps to not chew it all up.. | 01:34 |
airlied | I don't think the gain of in-memory compression is that majorly useful for most people.. | 01:34 |
=== airlied remembers win95 tools to do it.. | ||
bluefoxicy | airlied: in-memory compression can get up to 40% memory gains on the compressed area. If you allow typically 25% of memory to be in-memory compressed, this may be significant. | 01:38 |
airlied | but what about the process using that memory? | 01:40 |
bluefoxicy | For example, at 256M this allows 299M memory to be used; at 512M this allows 597M to be used; and at 1 gig this allows 1194M to be used. This is an average of 170M per gigabyte. | 01:40 |
airlied | it doesn't really... | 01:40 |
bluefoxicy | the process using that memory would hit an invalid pagefault trying to access it | 01:40 |
bluefoxicy | which would trigger decompression. | 01:40 |
airlied | fi you are under memory pressure you are going to run out, unless you fix the problem.. | 01:40 |
bluefoxicy | This happens when the linux kernel swaps memory to disk as well; execpt that the the memory is retrieved from disk instead of decompressed. | 01:41 |
bluefoxicy | You won't necessarily run out under memory pressure | 01:41 |
bluefoxicy | Linux is ALWAYS under memory pressure; disk cache typically uses whatever programs don't | 01:41 |
airlied | I'm just not seeing the advantages outweighing the complexity of it.. | 01:41 |
bluefoxicy | If you have ample memory already, then the system just becomes faster, as it can simply compress disk cache until memory pressure increases enough to dump it. | 01:42 |
bluefoxicy | I'm currently thinking of liveCDs, although database applications may benefit largely as well. | 01:42 |
bluefoxicy | LiveCDs may run on systems without swap. While you may say it is "nice" to try to use less than 25% for memory compression, without swap memory pressure suddenly results in one of two things: OOM kills or eating more memory for compressed cache | 01:43 |
airlied | but if I'm running a firefox and an openoffice and I've got no RAM left I don't think I'll be able to do much compression ... | 01:43 |
bluefoxicy | A system with 256M of memory runs somewhat slow on a LiveCD. There is a LOT of reading from the CD, because disk cache is constantly invalidated. | 01:43 |
bluefoxicy | airlied: remember I'm talking about compressing 32K blocks. The WK4x4 algorithm in Rodrigo CAstro's patch used about 128K of memory for this (it errantly over-allocated 300k, which it didn't use) | 01:44 |
bluefoxicy | at any rate, given severe pressure (growing disk cache, growing program memory) and no swap, a livecd system may compensate by compressing memory and using up to 90% of memory for a compressed store. This may slow programs down by perhaps 10% between pagefaults and decompression routines (remember we're spending a fraction of a millisecond decompressing a compressed 32k block or vice versa) | 01:47 |
airlied | well I'll remain sceptical that the good outweights the bad... and I doubt it'll turn up in mainline any time soon.. | 01:47 |
bluefoxicy | however the apparent memory capacity of such a system becomes 410MB | 01:47 |
bluefoxicy | disk access is much, much longer than the compression, decompression, and spaztic pagefaults until you use a lot of memory for a compressed store. I don't like anything over 50%, heh. | 01:48 |
airlied | but if an apps working set is larger than 10% you'll end up compress trashing it.. | 01:48 |
bluefoxicy | yes, you would | 01:48 |
bluefoxicy | but if an app's working set is larger than 10%, you're risking OOM kills. | 01:48 |
bluefoxicy | well, maybe not | 01:49 |
bluefoxicy | that would be 25 megs on a 256 meg system | 01:49 |
airlied | true... I can see it being rejected though as RAM is cheap.. | 01:49 |
bluefoxicy | However, if an application is working with a lot of libraries (openOffice, gnome applets, firefox.. . . these feel this a lot), then every time you do something your CD drive spins | 01:49 |
bluefoxicy | which means you wait about 5-10 seconds, instead of a few miliseconds | 01:50 |
bluefoxicy | on the subject of database systems, many of these would like to mmap() large areas of the database into memory | 01:50 |
bluefoxicy | as your server may have 8GB of memory and a 256GB database (there are multi-TB databases on 64GB of RAM, btw), you could feasibly map in 256GB of the database (64-bit system will do this, you have 256TiB of VM space and the kernel gives access to 87TiB) | 01:51 |
bluefoxicy | but only 8GB could be resident at once, minus other stuff in memory. | 01:51 |
bluefoxicy | database servers such as this could feasibly use half of memory max as a compressed store, since most of that would become disk cache for the database anyway; they'd have about 10GiB (2GiB extra) of effective ram | 01:52 |
bluefoxicy | RAM may be cheap, but CPU clocks are even cheaper | 01:52 |
bluefoxicy | airlied: ram is cheap yeah, it's nice to get something for free though ;) It's been nice talking though, just felt like running through it to see if anyone actually cared. | 01:54 |
airlied | bluefoxicy: yes it's an intersting idea.. just not sure it'll get mainlined :-) | 02:02 |
=== dolson [n=dana@d235-185-252.home1.cgocable.net] has joined #ubuntu-kernel | ||
dolson | if I am compiling a customized dapper kernel that I want to be the default over any of the ubuntu standard ones, what should I put in the EXTRAVERSION field or kpkg parameters to do this? | 02:04 |
cjb | I don't think what you put in EXTRAVERSION will affect where it goes in the grub menu. If you want it to be the grub default, make it that in /boot/grub/menu.lst. | 02:06 |
dolson | well then when a new kernel comes out, it takes over as the default | 02:07 |
=== BenC [n=bcollins@debian/developer/bcollins] has joined #ubuntu-kernel | ||
dolson | BenC: hey man, would you answer a question for me? you said you would in your emails to me :) | 02:22 |
BenC | sure | 02:22 |
dolson | cool. ok, I've got a version of Ingo's patch that applies to Dapper's kernel, and I want to compile my kernel and have it as the default kernel at bootup, even if a new Ubuntu kernel comes out. how do I do this? is it in EXTRAVERSION, make-kpkg, or in menuconfig? | 02:23 |
BenC | well, building requires make-kpkg | 02:24 |
BenC | but whether or not it is the default depends on your boot loader | 02:24 |
BenC | if it's grub, then edit /boot/grub/menu.lst appropriately | 02:25 |
dolson | ok, but then if there's a kernel update, it takes priority and becomes the default | 02:25 |
BenC | not if you setup menu.lst correctly | 02:27 |
BenC | look at the default config option, and read what it says about saveddefault | 02:29 |
dolson | alright. I was hoping to not have to add a section about configuring grub to my wiki, but I guess that's the only way | 02:30 |
infinity | dolson: No one's stopping you from removing the Ubuntu kernels completely once yours is installed. | 02:35 |
dolson | I know, that but I'm not going to advise that in a tutorial | 02:35 |
BenC | the only other way, maybe, is to set the extra version to something like -99-foo | 02:43 |
BenC | --append_version for make-kpkg | 02:44 |
BenC | but I can't gurantee that we wont hit -104-* before dapper is released :) | 02:44 |
dolson | heh. | 02:44 |
dolson | is there a need to change the localversion (in menuconfig) or EXTRAVERSION in the Makefile, or is just --append_version enough | 02:45 |
BenC | --append_version works | 02:52 |
dolson | alright thanks Ben | 02:55 |
=== dolson [n=dana@d235-185-252.home1.cgocable.net] has left #ubuntu-kernel ["leaving..] | ||
mgalvin | BenC: has the fix for #28660 been uploaded yet b/c it does not appear to be fixed on the current (2.6.15-16-686) kernel I have installed? | 04:18 |
=== mxpxpod [n=bryan@unaffiliated/mxpxpod] has joined #ubuntu-kernel | ||
=== karlheg [n=karlheg@131.252.226.60] has joined #ubuntu-kernel | ||
=== karlheg [n=karlheg@131.252.226.60] has left #ubuntu-kernel ["Leaving"] | ||
=== BenC [n=bcollins@debian/developer/bcollins] has joined #ubuntu-kernel | ||
=== HAL [n=HAL@CPE-144-131-214-239.qld.bigpond.net.au] has joined #ubuntu-kernel | ||
HAL | anyone here | 06:13 |
HAL | need a little assistance with kernel stuff | 06:14 |
HAL | Q1: what is the command to report kernel version | 06:14 |
fabbione | uname -a | 06:15 |
HAL | cool | 06:15 |
fabbione | but you should really ask stuff like this in #ubuntu :) | 06:15 |
HAL | tried in there they all seam to be having there own little discussions | 06:16 |
HAL | thanks for that | 06:16 |
fabbione | welcome | 06:17 |
bluefoxicy | holy shit that scared me | 06:23 |
bluefoxicy | for a second i thought I had an e-mail from Richard Stallman and I was like "Oh no not this moron" | 06:23 |
bluefoxicy | sleep | 06:24 |
=== HAL [n=HAL@CPE-144-131-214-239.qld.bigpond.net.au] has left #ubuntu-kernel ["Leaving"] | ||
=== ispiked_ [n=ispiked@Lee-11-133.rh.ncsu.edu] has joined #ubuntu-kernel | ||
=== JaneW [n=JaneW@196.36.161.235] has joined #ubuntu-kernel | ||
=== chmj [n=chmj@196.44.1.98] has joined #ubuntu-kernel | ||
=== Keybuk [n=scott@c51008D91.inet.catch.no] has joined #ubuntu-kernel | ||
=== CataEnry [n=Enrico@host83-39.pool8248.interbusiness.it] has joined #ubuntu-kernel | ||
=== johnm [n=johnm@gentoo/developer/johnm] has joined #ubuntu-kernel | ||
=== raphink [n=raphink@ubuntu/member/raphink] has joined #ubuntu-kernel | ||
raphink | hi guys | 12:02 |
raphink | I'm currently working on a minor bug on update-grub | 12:02 |
raphink | and would like some comments on how to fix it | 12:02 |
fabbione | raphink: i think zul is working on grub, but he is not around right now. he is on .ca time | 12:03 |
raphink | ok | 12:03 |
raphink | fabbione: do you know of update-grub ? | 12:03 |
raphink | I think you might be able to help me the issue is pretty simple | 12:04 |
raphink | update-grub detects existing ^splashimage= lines | 12:04 |
fabbione | raphink: sorry i am busy and i don't have time for it :/ | 12:04 |
fabbione | you will have to wait for zul | 12:04 |
raphink | and add them to the buffer | 12:04 |
raphink | ah ok | 12:04 |
fabbione | otherwise file a bug | 12:04 |
raphink | :( | 12:04 |
raphink | the bug exists fabbione, I'm fixing it | 12:04 |
raphink | ;) | 12:04 |
raphink | but I need some guidelines from the kernel guys | 12:04 |
raphink | cause I don't want to break update-grub | 12:05 |
Keybuk | raphink: fabbione says he's busy, and suggested you speak to zul when he's around | 12:06 |
raphink | Keybuk: yes I can understand what fabbione said ;) | 12:06 |
Keybuk | raphink: ah, then why did you carry on bugging him? | 12:06 |
raphink | I don't need english->english translation :) | 12:06 |
Keybuk | if somebody says they're busy and don't have time to help you, it's generally quite rude to carry on talking to them | 12:07 |
=== chmj [n=chmj@196.44.1.98] has joined #ubuntu-kernel | ||
raphink | I've uploaded a fix for the grub package. Who shall I assign the bug to so the fix gets uploaded ? | 01:01 |
Keybuk | ... if you've already uploaded it ? | 01:02 |
Keybuk | leave it unassigned, it'll get assigned to the appropriate person | 01:02 |
raphink | I've uploaded to malone | 01:02 |
Keybuk | ah, leave it unassigned | 01:03 |
raphink | I doubt it'll get assigned soon, otherwise it would already be | 01:03 |
Keybuk | what makes you think that? | 01:03 |
raphink | cause this bug is not very very new ;) | 01:04 |
Keybuk | there's nobody in particular who knows/cares about grub | 01:04 |
raphink | Keybuk: I mean is there an equivalent to motu-reviewers for main? | 01:04 |
Keybuk | yes, the default assigness | 01:05 |
Keybuk | (ie. "nobody") | 01:05 |
raphink | this is a quite small patch, could you have a look at it ? | 01:05 |
Keybuk | no, sorry | 01:05 |
Keybuk | I don't know grub | 01:05 |
raphink | it's just a piece of bash in update-grub | 01:06 |
raphink | to better detect and replace the splashimage line | 01:06 |
Keybuk | having attached the patch to the bug, it should get noticed on the ubuntu-bugs mailing list | 01:06 |
raphink | it has nothing to do with grub itself | 01:06 |
Keybuk | have you talked to mvo about it? he was dealing with splashimage stuff | 01:06 |
Keybuk | (according to the changelog) | 01:06 |
raphink | really? | 01:06 |
raphink | hmm | 01:07 |
raphink | I'll go talk to him | 01:08 |
=== mjg59 [n=mjg59@cavan.codon.org.uk] has joined #ubuntu-kernel | ||
fabbione | BenC: new OCFS2 rocks :) | 01:50 |
fabbione | it doesn't crash anymore on super load | 01:50 |
BenC | sweet | 01:50 |
fabbione | i am checking if i have something for you | 01:50 |
fabbione | i don't hounestly rememeber if i did some push/pull :) | 01:51 |
fabbione | oh crap | 01:53 |
fabbione | i do have some stuff | 01:53 |
fabbione | i did a pull from david but it did download all the 1500 objects again | 01:54 |
fabbione | and now it's pushing them to roockery | 01:54 |
fabbione | commit 0351e92ae406408854d0a4a6d7251f8e04308a79 | 01:56 |
fabbione | Author: Ben Collins <bcollins@ubuntu.com> | 01:56 |
fabbione | Date: Mon Feb 27 18:39:56 2006 -0500 | 01:56 |
fabbione | Blah | 01:56 |
fabbione | AHHAHA | 01:56 |
fabbione | lovely | 01:56 |
fabbione | BenC: please pull from me | 01:56 |
fabbione | i have a couple of changes/fixed | 01:56 |
fabbione | fixes | 01:56 |
fabbione | more sun4v and one tg3 fix | 01:56 |
BenC | fabbione: that stupid file wouldn't go back to "no change"...I've no idea why it had a delta | 01:56 |
BenC | ok | 01:56 |
fabbione | (for sun blade 2500( | 01:56 |
mjg59 | BenC: Did those diffs I sent look ok? | 01:56 |
BenC | mjg59: yeah | 01:57 |
mjg59 | Rock | 01:57 |
fabbione | receiving file list ... rsync: link_stat "/scm/linux/kernel/git/bcollins/ubuntu-2.6/objects/." (in pub) failed: No such file or directory (2) | 02:14 |
fabbione | something wrong on master.k.o ben? | 02:14 |
fabbione | never mind | 02:16 |
BenC | I'm doing a git-prune | 02:23 |
=== zul [n=chuck@ubuntu/member/zul] has joined #ubuntu-kernel | ||
fabbione | ok | 02:24 |
fabbione | it was only for 30 secs or so | 02:24 |
fabbione | it disappeared immediatly after | 02:24 |
zul | heylo | 02:26 |
fabbione | hi zul | 02:29 |
zul | apparently my ears were burning in the log file | 02:30 |
=== CataEnry [n=cataenry@host83-39.pool8248.interbusiness.it] has joined #ubuntu-kernel | ||
=== doko [n=doko@dslb-084-059-090-097.pools.arcor-ip.net] has joined #ubuntu-kernel | ||
=== mxpxpod [n=BryanFor@unaffiliated/mxpxpod] has joined #ubuntu-kernel | ||
zul | BenC: ping | 03:36 |
BenC | zul: pong | 03:36 |
zul | im getting the following error when im doing a pull | 03:36 |
zul | inux/kernel/git/bcollins/ubuntu-2.6 | 03:36 |
zul | rsync: link_stat "/scm/linux/kernel/git/bcollins/ubuntu-2.6.git/refs/heads/ubuntu-fixes" (in pub) failed: No such file or directory (2) | 03:36 |
zul | rsync error: some files could not be transferred (code 23) at main.c(1173) | 03:36 |
BenC | currently? | 03:37 |
zul | yep...ill try again thought if you want | 03:37 |
BenC | oh, yeah, rm -f .git/refs/heads/ubuntu-fixes | 03:37 |
BenC | actually cd .git/refs/heads/ | 03:38 |
BenC | rm everything except for master and origin | 03:38 |
zul | done | 03:38 |
=== ubuntulog [i=ubuntulo@trider-g7.fabbione.net] has joined #ubuntu-kernel | ||
=== Topic for #ubuntu-kernel: Ubuntu kernel development discussion ONLY | New git tree for dapper: https://wiki.ubuntu.com/KernelGitGuide | 2.6.15-16.23 uploaded (PPC FTBFS fixed) | Daily Diet of Destruction: http://people.ubuntu.com/~bcollins/kernels-daily/ | ||
=== Topic (#ubuntu-kernel): set by BenC at Mon Feb 20 16:39:14 2006 | ||
=== fabbione [i=fabbione@gordian.fabbione.net] has joined #ubuntu-kernel | ||
=== BenC [n=bcollins@debian/developer/bcollins] has joined #ubuntu-kernel | ||
=== BenC [n=bcollins@debian/developer/bcollins] has joined #ubuntu-kernel | ||
=== CataEnry [n=Enrico@host83-39.pool8248.interbusiness.it] has joined #ubuntu-kernel | ||
=== zul [n=chuck@ubuntu/member/zul] has joined #ubuntu-kernel | ||
=== CataEnry [n=cataenry@host83-39.pool8248.interbusiness.it] has joined #ubuntu-kernel | ||
=== mgalvin [n=mgalvin@ubuntu/member/mgalvin] has joined #ubuntu-kernel | ||
fabbione | BenC: | 09:41 |
fabbione | rsync: link_stat "/scm/linux/kernel/git/bcollins/ubuntu-2.6.git/refs/heads/ubuntu-fixes" (in pub) failed: No such file or directory (2) | 09:41 |
fabbione | again? | 09:42 |
=== ispiked_ [n=ispiked@Lee-11-133.rh.ncsu.edu] has joined #ubuntu-kernel | ||
=== netzmeister [n=netzmeis@p549FAC7E.dip0.t-ipconnect.de] has joined #ubuntu-kernel | ||
fabbione | BenC: can you please pull from me again? | 10:14 |
fabbione | i have got the SPARC64 IIIi SMP bug fix | 10:15 |
fabbione | so we will make elmo and the porting machine more happy :) | 10:15 |
lamont | BenC: it would be wonderful if the next upload included the hppa patches - it'll help the DC buildd machines out some, as well as making it possible for me to use my new test machine... | 10:26 |
BenC | fabbione: yeah | 10:59 |
BenC | fabbione: also, cd .git/refs/heads/ and rm everything but master and origin | 10:59 |
BenC | lamont: it will be | 10:59 |
fabbione | BenC: i did try that too... it didn't work.. i had to clone again | 11:15 |
=== netzmeister [n=netzmeis@p549FAC7E.dip0.t-ipconnect.de] has left #ubuntu-kernel ["...] |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!