[00:04] <pwnguin> wtf
[00:07] <CarlFK> can it_rsa hold multiple private keys?
[00:12] <pwnguin> sure
[00:12] <pwnguin> but you probably wanted id_rsa ;)
[00:13] <CarlFK> er, yeah, that.
[00:13] <CarlFK> thaks
[00:14] <pwnguin> actually, i donno about id_dsa
[00:15] <CarlFK> I would think ssh would have an option to specify which key it tried to use
[00:15] <CarlFK> if it supported multiple keys in one file 
[00:16] <pwnguin> or perhaps it tries all of them ;)
[00:16] <pwnguin> but i dont quite understand why -kernel is the place to ask about ssh
[00:19] <CarlFK> it isn't.  I got confused :)
[00:20] <CarlFK> ... for linux?  made me think this was #linuxhelp :)
[00:20] <pwnguin> ok ;)
[08:08] <soren> How do I pass a paramater to the modules that you're now compiling into the kernel?
 on the kernel command line, perhaps?
[08:21] <smb_tp> soren, put them into a file in /etc/modprobe.d. Format "options <module> <name>=<value>"
[08:21] <smb_tp> soren, Oh forget it
[08:21] <smb_tp> Should get my eyes open
[08:21] <smb_tp> Yeah, yours is correct
[08:35] <soren> So there's no way to change these values, since I can't rmmod and then modprobe with new options?
[08:35] <soren> *sob*
[08:37] <smb_tp> soren, If the code has no /proc or sysfs interface I fear not.
[08:37] <soren> Oh, right.
[08:37] <soren> AFAIUI, all module paramaters are exposed in sysfs these days?
[08:37] <soren> Althought, they might be read-only. :(
[08:37] <mjg59> soren: /sys/module/modulename/parameters
[08:38] <mjg59> Should be there even if it's statically linked
[08:38] <soren> mjg59: But that doesn't necessarily mean I can change the value. Some might require a reinitialisation.
[08:40] <mjg59> soren: Right
[08:40] <mjg59> If it's marked read/write then it should work on the fly
[08:41] <soren> Hmm.. I don't even see processor.max_cstate exposed in sysfs anymore.
[08:42] <soren> if the module_param is created with mode 0000, does that mean it doesn't even get exposed in sysfs?
[08:47] <soren> include/linux/moduleparam.h suggests that that is indeed the case.
[08:47] <soren> So I lose. :(
[08:48] <soren> I have a laptop that makes a really annoying noise when on AC in C3 and C4, so I want to set max_cstate=2 when on AC and max_cstate=4 when on battery. I don't see a way to do that now.
[08:50] <mjg59> You can use the pm_qos stuff to mimic that
[08:50] <mjg59> If you set the dma latency to a number smaller than the C3 latency
[08:51] <soren> *chuckle*
[08:51] <soren> Well, yes, that would work.
[08:52] <soren> Ok, so how do I find the relevant numbers?
[08:54] <soren> And once I find it, I just write that as a signed 32 bit integer to /dev/cpu_dma_latency and hold it open until I switch to battery?
[08:57] <soren> mjg59: in /proc/acpi/processor/CPU0/power, the C3 line reads (among other things, of course): "latency[057]", and the same number for C2 is 001... So I just put e.g. 56 in /dev/cpu_dma_latency ?
[09:00] <mjg59> soren: Yeah, though I can't remember if the units are the same
[09:05] <soren> mjg59: I'm not sure how to find out. include/acpi/processor.h just says the latency attribute is a u32. No mention of units.
[09:09] <mjg59> soren: I /think/ it's usec
[09:10] <soren> Lovely. That's what /dev/cpu_dma_latency expects as well, apparantly.
[09:10] <soren> mjg59: Thanks. I'll tyr.
[09:10] <soren> try, even.
[09:46] <soren> mjg59: It seems to work. Thansk!
[09:46] <soren> Thanks, even!
[09:46] <soren> (typing is hard)
[10:16] <mdz> evening
[10:16] <mdz> (wrong window)
[12:27] <marijus> hello, did anyone mage to build 2.6.29-rc1? i tried yesterday but build failed...
[12:27] <marijus> sorry typo... mage=manage
[13:17] <TimStarling> how do you debug the kernel in ubuntu?
[13:17] <TimStarling> just stare at the source until the truth becomes clear?
[13:20] <TimStarling> the buddha method?
[13:47] <apw> TimStarling, some of that yes
[13:48] <cking> TimStarling: a combination of staring at the source and also the use of printk() :-)
[13:48] <apw> some bits have debug you can enable, others you add as you stare
[13:50] <TimStarling> I have a production server with a top that looks like this: http://p.defau.lt/?84j7MTR_v4d7BUgx5E91bw
[13:51] <TimStarling> it works just fine for a few days, then dies
[13:51] <TimStarling> I could add printk() all over the whole kernel (one of us thinks the bug could be pretty much anywhere)
[13:51] <TimStarling> but a few days' worth of output might be a bit ugly
[13:52] <apw> is that mysql really 30GB ?
[13:52] <apw> how much ram do you have?
[13:52] <TimStarling> 32 GB
[13:52] <apw> can you define dies
[13:53] <soren> Erk... Those kswapd processes don't look too good.
[13:53] <TimStarling> connections to mysql time out, attempting to attach with strace hangs forever
[13:54] <TimStarling> soren: no shit
[13:54] <apw> how is mysql aquiring all that memory?  large shm segments?  something else?
[13:54] <soren> You only have 1GBof swap?
[13:54] <TimStarling> 30GB is normal
[13:54] <TimStarling> 31 actually
[13:55] <apw> i would suspect the machine is trying to find something to release cause it has no swap
[13:55] <soren> With 32 GB of RAM and a mysql server that wants to eat 31 GB of memory, 1 GB of swap sounds like a disaster waiting to happen.
[13:55] <apw> and if the memory mysql has is anonymous this can trigger almost infinite list active/inactive list scanning
[13:55] <TimStarling> it may be that the margin is too small, our DBA likes to set it real small
[13:55] <apw> in older kernels, what version is this
[13:56] <TimStarling> Linux db18 2.6.24-22-server #1 SMP Mon Nov 24 20:06:28 UTC 2008 x86_64 GNU/Linux
[13:56] <TimStarling> stock ubuntu kernel
[13:56] <apw> yep ... .24 does not have the fixes for that
[13:57] <soren> What else is running on the machine?
[13:58] <soren> The run queue is ~22..
[13:59] <TimStarling> http://p.defau.lt/?GNY7y_PM7vcQfy4C4bf0xQ
[14:01] <TimStarling> could be FS-related, with the logrotate processes
[14:01] <TimStarling> but I've seen OOM conditions crash ubuntu kernels before so adding a bigger margin is an obvious thing to try
[14:02] <TimStarling> apparently changing the FS was already tried
[14:02] <TimStarling> could be a block driver
[14:02] <apw> i would suspect the unevicatable pages problem at first glance, kswapd's running madly and achieving nothing
[14:02] <apw> note they arn't in D writing stuff, they are running
[14:02] <apw> the changes to fix that are major and so not likely to appear in a hardy kernel
[14:03] <apw> _if_ the memory that mysql is at least in theory swappable adding swap that will never be used may prevent the issue (iirc)
[14:03] <TimStarling> are there fixes in 8.10 that I should know about?
[14:04] <apw> i believe the fixes are in 2.6.28, so that would be jaunty
[14:05] <apw> possibly adding swap would tell you if that is the issue, though getting it back will be hard with only 1gb by default
[14:06] <TimStarling> the easiest thing is to reduce mysql's memory usage back to 28 or so
[14:07] <apw> if it is reliable there yes
[14:08] <apw> if the problem is the spinning lists issue, then that tended to get reported by people who had 32GB of ram or more
[14:08] <apw> and tended to have 0 swap as 'who needs swap with this much ram'
[14:08] <TimStarling> mysql's memory usage is very stable, it just allocates a big chunk at startup and uses it for most things
[14:09] <TimStarling> we know enough to not set swap=0, lots of bugs if you do that
[14:10] <TimStarling> ok, going to reboot it, last chance if you want more data
[14:22] <TimStarling> presumably there's some page table overhead, what's the effective amount of memory available for mmap() if there's 32GB physical?
[14:23] <TimStarling> I'm just wondering if that 30.9GB virtual figure for mysql was hitting it
[14:24] <apw> i tend to work on the rule of thumb that 10% is used by the system but you were clearly 'at the end' as you kswapd was awake
[14:42] <apw> you are only talking about having 1.1GB free, i have some 180MB missing even before the kernel gets going for the kexec kernel, so you are squeezing the entire system (other than mysql) into 900MB
[14:45] <TimStarling> mmm
[14:48] <TimStarling> looks like our (stable) 16GB servers have about 1.7GB free for non-mysql stuff
[14:49] <TimStarling> so I should probably be thinking about keeping the same proportion free, ~3.4
[14:50] <TimStarling> we would like to have graceful OOM behaviour at some point, we've had a lot of trouble with that on our application servers
[14:50] <TimStarling> because they have a load-dependent memory usage
[14:51] <TimStarling> maybe we should package our own kernels
[15:06] <rtg> soren: is there any reason we should enable CONFIG_NFSD_V4 in the hardy -virtual flavour?
[15:06] <rtg> s/should/shouldn't/
[15:09] <soren> rtg: I can't think of any, no.
[15:10] <rtg> soren: thanks. looks like it must have been an oversight.
[16:25] <persia> Good day.  I'm trying to understand the difference between the "lpia" and "lpiacompat" kernel flavours.  A diff -u of the config confuses me, as config.lpia seems to have CONFIG_M586=y and config.lpiacompat seems to have it unset, which is contrary to the information I have previously received.  Is there a set of known devices on which the lpia kernel is not expected to work?
[16:27] <rtg> persia: lpiacompat was built for a standard PC, e.g. a   non-Atom CPU.
[16:27] <rtg> at least, if I recall correctly.
[16:27] <persia> rtg, That's the information I had before, which is why I'm confused at the config values I'm seeing.
[16:28] <persia> Atom doesn't need CONFIG_M586=y and many "standard" PCs (e.g. Via C7M) do.
[16:29] <rtg> where is Amit? He knows all these details.
[16:29] <persia> You ask the hard questions :)
[16:39] <rtg> soren: where does the JeOS kernel image come from in Intrepid? 
[16:39] <soren> rtg: The server image.
[16:39] <soren> It's the same image, with a stack of modules ripped out.
[16:39] <soren> I think BenC calls it subflavours?
[16:39] <rtg> soren: this guy is complaining that NFSv4 doesn't work, but it _is_ enabled.
[16:40] <rtg> ah, perhaps too may debs are ripped out?
[16:45] <apw> sconklin, ok ...  you have filed a bug for the apport changes.  acording to sources (#ubuntu-bugs) it seems the appropriate thing to do is to share that bug for all the changes for this overall feature so that they all have the same number and track together
[16:46] <persia> Well, assuming they are all related to the same issue.
[16:46] <sconklin> apw: that makes a lot of sense
[16:46] <sconklin> I'm glad I gave it a generic title
[16:47] <apw> it can be changed anyhow!
[18:47] <fabbione> hi gusy
[18:47] <fabbione> who is in charge of the hardy-updates tree?
[18:48] <fabbione> rtg: ?
[18:48] <rtg> fabbione: dude
[18:49] <rtg> fabbione: uh, smb_tp is the Hardy guy
[18:49] <fabbione> rtg: hey.. who does pull from linux-2.6.24.x into hary tree?
[18:49] <fabbione> something is hutterly wrong in the kernels you are releasing
[18:49] <fabbione> smb_tp_: you there?
[18:49] <rtg> how so?
[18:49] <fabbione> there are missing commits
[18:49] <fabbione> and changelogs are missing entries
[18:49] <rtg> missing from stable?
[18:49] <smb_tp_> fabbione, I am
[18:49] <fabbione> yes
[18:50] <fabbione> smb_tp_: i am looking at linux (2.6.24-23.46) hardy-proposed; urgency=low
[18:50] <fabbione> according to the changelog:
[18:50] <fabbione> lots of entry for bug: #301608
[18:50] <fabbione> that is merging 2.6.26.4 into hardy
[18:50] <fabbione> but upstream 2.6.24.4 has:
[18:50] <fabbione> commit 68b498d251d97de9adda518fda42cfe1451063b7
[18:50] <fabbione> Author: David S. Miller <davem@davemloft.net>
[18:50] <fabbione> Date:   Thu Mar 6 14:47:20 2008 -0800
[18:50] <fabbione> SPARC64: Loosen checks in exception table handling.
[18:50] <fabbione> [SNIP]
[18:50] <fabbione> that is missing from the hardy tree completely
[18:51] <fabbione> or at least is not in the released tarball
[18:51] <rtg> I think thats because we haven't incorporated all of the stable commits in Hardy as  we have for Intrepid. smb - is that still the case?
[18:51] <fabbione> this is an example of a missing commit
[18:51] <fabbione> broken changelog:
[18:51] <fabbione>   * Linux 2.6.24.6, 2.6.24.7
[18:51] <fabbione>     - LP: #301634
[18:51] <smb_tp_> rtg, No the intention was to gte all.
[18:51] <smb_tp_> Let me see
[18:52] <fabbione> ^^ points to the same bug as 2.6.24.4 and they have no references to the 2 security bugs fixed in those 2 upstream releases
[18:52] <fabbione> smb_tp_: i simply need to know if you are going to pull them all or not..
[18:53] <fabbione> as easy as it, it means for me removing ubuntu from all my machines or keeping it around
[18:53] <smb_tp_> fabbione, They should all get pulled
[18:53] <fabbione> smb_tp_: ok then you should check why that one is not there
[18:53] <fabbione> 68b498d251d97de9adda518fda42cfe1451063b7
[18:53] <fabbione> i know this is missing because without it my hardy sparc firewall doesn't boot
[18:54] <rtg> fabbione: its in the hardy git repo.
[18:54] <fabbione> rtg: it's not in the released tarball
[18:54] <rtg> it may not have been promoted to -updates yet
[18:54] <fabbione> rtg: it is according to the changelog...
[18:55] <fabbione> i don't use proposed.. that kernel has been copied from proposed to updates
[18:55] <rtg> fabbione: gimme a minute....
[18:56] <fabbione> sure even 20
[18:56] <fabbione> i am honestly worried that something in the release process went wrong
[18:56] <fabbione> on the other side i also have the issues to keep my machines running :)
[18:56] <fabbione> i am sure you can understand
[18:56] <fabbione> anyway you'll sort it out :)
[18:57] <rtg> fabbione: 2.6.24-23.46, right?
[18:58] <fabbione> yes
[18:59] <fabbione> do you need any more info?
[18:59] <smb_tp_> fabbione, We well do
[18:59] <fabbione> my wife is about to yell at me if i don't show up
[18:59] <smb_tp_> fabbione, No I don't think so
[18:59] <fabbione> ok
[18:59] <fabbione> i'll pass by in about 2 hours..
[19:00] <fabbione> root@vultus5:~/linux-2.6.24# patch -p1 < ../fix_sparc.diff 
[19:00] <fabbione> patching file arch/sparc64/mm/fault.c
[19:00] <fabbione> ^^^fix_sparc is that commit on top of that kernel..
[19:00] <fabbione> so it is missing somehow :)
[19:00] <fabbione> later!
[19:40] <fabbione> re
[19:41] <fabbione> rtg, smb_tp_: found anything?
[19:42] <smb_tp_> fabbione, Yes, found some patches missing (including yours) for 2.6.24.3->2.6.24-4
[19:42] <fabbione> smb_tp_: ok!
[19:42] <smb_tp_> I did not understand however, the thing about 2.6.24.6/7
[19:42] <fabbione> that's something to do with changelog
[19:42] <fabbione> not sure if it's automatically generated or manually done
[19:42] <fabbione> but:
[19:42] <smb_tp_> automatically
[19:42] <fabbione> then it's wrong :)
[19:43] <fabbione> look at the same kernel changelog
[19:43] <fabbione>   * V4L: Fix VIDIOCGAP corruption in ivtv
[19:43] <fabbione>     - LP: #301634
[19:43] <fabbione>   * Linux 2.6.24.6, 2.6.24.7
[19:43] <fabbione>     - LP: #301634
[19:43] <fabbione> ^^ last two entries for that release
[19:43] <fabbione> .6 and .7 points to the same bug as .4
[19:43] <fabbione> (or .5... doesn't matter)
[19:44] <fabbione> 2.6.24.6 fixes one bug, not reported in the changelog
[19:44] <fabbione> .7 fixes another bug.. not reported in the changelog
[19:44] <fabbione> as long as the fixes are there, this is "minor" but makes the changelog incomplete
[19:44] <smb_tp_> There might be some not in that log because they got fixed before with another name potentially
[19:45] <fabbione> worth checking tho
[19:45] <fabbione> need to reboot my firewall.. brb
[19:45] <smb_tp_> The updates were only to track importing the patches from -stable that not yet went into hardy
[19:49] <fabbione> gotcha
[19:49] <fabbione> re
[19:49] <fabbione> well at least that fix works :)
[19:49] <rtg> smb_tp_: I think I remember pointing out that there was a typo in the changelog for that bug number.
[19:51] <maxb> ooi, what does SAUCE mean in Ubuntu kernel changelogs? Googling was unfruitful
[19:51] <smb_tp_> maxb, That is a patch that is not (yet) upstream
[19:51] <rtg> maxb: we use it to denote patches that are either not upstream, or patches that will never get upstream
[19:54] <smb_tp_> rtg, Hm, somehow I am blind to that. I posted the missing 6 to the list
[19:55] <rtg> smb_tp_: you are blind to what?
[19:55] <fabbione> smb_tp_: did you check that manually or a git pull did show them?
[19:55] <rtg> oh, the typo?
[19:55] <smb_tp_> rtg, yes
[19:55] <smb_tp_> fabbione, I compared the export names and checked all missing from my list
[19:56] <rtg> smb_tp_: maybe it was a different changelog entry.
[19:56] <fabbione> smb_tp_: it might be wise to check the whole tree tho..
[19:56] <fabbione> smb_tp_: just to make sure other bits didn't get lost in other pulls
[19:57] <smb_tp_> fabbione, I will double check that
[19:57] <fabbione> smb_tp_: is linux_2.6.24.orig.tar.gz a pristine copy from upstream or is it generated from your git trees?
[19:57] <fabbione> smb_tp_: if it's a pristine copy from upstream, then it's "simple"...
[19:57] <fabbione> if it's generated from your git tree, you want to check the whole thing from scratch
[19:58] <rtg> fabbione: its pristine from upstream
[19:58] <fabbione> rtg: ok.. then it's simple..
[19:59] <smb_tp_> fabbione, don't think that simple the orig is the tree at point of first release
[20:00] <fabbione> smb_tp_: yes it's simple in git.. you can just branch out v2.6.24 in git and then compare the commits in your tree with whatever is in 2.6.24.x
[20:00] <fabbione> smb_tp_: everything local to ubuntu has proper UBUNTU or SAUCE tags. EVerything else either matching commits (everything in .24.x should be there) or a very detailed commit entry
[20:01] <fabbione> but starting from v2.6.24 tag and going, will reduce the load a lot
[20:01] <fabbione> anyway.. again... you will sort it out :)
[20:02] <smb_tp_> I guess so :)
[20:03] <fabbione> you are all young guys full of energy :)
[20:04] <smb_tp_> who? young? :)
[20:15] <smb_tp_> fabbione, rtg Ok, I went through the remaining commits of 4,5 and 6. Looks like those from 3 were the only missing ones
[20:16] <fabbione> "Remote root kernel exploit..." ...
[20:16] <fabbione> :)
[20:16] <fabbione> ^^doens't exist but can scare the hell out of people ;)
[20:18] <smb_tp_> heh, well you can scare people even by "panic can't find /" as a fortune message. :)
[20:25] <fabbione> speaking of which.. do I get a pony for finding this problem? :)
[20:25]  * fabbione really really wants a pony
[20:25] <rtg> fabbione: I'll buy you a beer the next time we cross paths, how about that? I can't afford a pony.
[20:26] <smb_tp_> Me neither
[20:26] <smb_tp_> But we can make two beers of it
[20:26] <fabbione> rtg: ahha works for me.. 
[20:26] <fabbione> rtg: even if i doubt somebody will ever fly me to a UDS or anything :)
[20:28] <smb_tp_> fabbione, There might be other opportunities ... :)
[20:28] <fabbione> smb_tp_: very unlikely tho.. i am located in DK..
[20:28] <smb_tp_> I'm in DE (just one letter difference) :-P
[20:29] <fabbione> smb_tp_: well you could.. rtg might be problematic :)
[20:29] <rtg> fabbione: Its Barcelona in the spring...
[20:29] <fabbione> rtg: again?
[20:29] <fabbione> we have been close to Barcelona
[20:30] <fabbione> rtg: do you have dates already?
[20:30] <rtg> It was Seville last time in Spain. this one is somewhere in the Catelonia district. May 22 I think.
[20:30] <fabbione> rtg: there was one before Seville .. a long time before Seville.. in a small city close to Barcelona :)
[20:31] <fabbione> rtg: at that time the kernel team was made of me and.. me...
[20:31] <rtg> ah, I wasn't around for that one.
[20:31] <fabbione> nope :)
[20:31] <fabbione> not even Ben was there
[20:31] <rtg> UDS - 25-29 May
[20:31] <fabbione> IIRC it was right after warthy release
[20:31] <fabbione> ok.. i'll keep an eye
[20:31] <fabbione> can't efford to fly on my own.. maybe i can find a sponsor or two
[20:32] <rtg> send an email to Mark :)
[20:32] <fabbione> he is not my only sponsor :)
[20:33] <fabbione> night guys
[20:33] <fabbione> take care
[20:33] <fabbione> and try to get those fixes out :)
[20:33] <fabbione> so i can finally stop building my kernels ;)
[20:34] <rtg> they'll percolate in -proposed for awhile. Should be uploaded in the next day or so.
[20:35] <smb_tp_> Right, I am on it
[20:35] <fabbione> yeah that's fine...
[20:35] <fabbione> day more, day less.... worst case /. will have a nice story of 100's of Ubuntu machines being r00t3d :D
[20:36] <laga> is there a new root exploit?
[22:53] <tjaalton> hum, how come linux-libc-dev is only for i386/amd64/armel? since it contains the drm headers now the xserver build fails on other archs (no drm.h)
[22:54] <tjaalton> although armel fails too..