[04:08] <praveen_> i want to know abt trace flag and i need some pointers\
[11:21] <lesshaste> hi
[11:22] <lesshaste> I am trying to produce some debugging info for kernel panics I get the whole time
[11:22] <lesshaste> is this the right way to run kexec?
[11:22] <lesshaste> sudo kexec -p /boot/vmlinuz-2.6.27-11-generic --initrd /boot/initrd.img-2.6.27-11-generic --append="root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23 ro
[11:32] <lesshaste> no one in?
[11:33] <lesshaste> it seems the root= option is wrong for some reason
[11:33] <lesshaste> I get
[11:33] <lesshaste> Cannot open `--append=root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23': No such file or directory
[11:40] <mnemo> you got an unmatched " in there
[11:41] <lesshaste> mnemo, yes sorry.. sudo kexec -p /boot/vmlinuz-2.6.27-11-generic --initrd /boot/initrd.img-2.6.27-11-generic --append="root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23"
[11:41] <lesshaste> gives
[11:41] <lesshaste> Cannot open `--append=root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23': No such file or directory
[11:42] <lesshaste> mnemo, so I must just have it completely wrong
[11:44] <lesshaste> hi Wellark 
[11:44] <mnemo> so you have one kernel that gives you panic and one that works?
[11:44] <lesshaste> mnemo, no.. I have one that panics every day or so
[11:44] <lesshaste> mnemo, I don't have one that obviously doesn't panic
[11:44] <mnemo> usually you do "kexec -l kern_img"
[11:44] <lesshaste> mnemo, right.. did you see my attempt at the command line above?#
[11:45] <mnemo> yeah thats the thing, you dont have any -l  ??
[11:45] <lesshaste> mnemo, no I am using -p
[11:45] <lesshaste> mnemo, but check out
[11:45] <lesshaste> http://www.mjmwired.net/kernel/Documentation/kdump/kdump.txt
[11:45] <lesshaste> the instructions are to do
[11:46] <lesshaste>   kexec -p <dump-capture-kernel-bzImage> --initrd=<initrd-for-dump-capture-kernel> --append="root=<root-dev> <arch-specific-options>"
[11:46] <lesshaste> so my line is sudo kexec -p /boot/vmlinuz-2.6.27-11-generic --initrd /boot/initrd.img-2.6.27-11-generic --append="root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23"
[11:46] <lesshaste> clearly I have the root= part completely wrong
[11:46] <lesshaste> but I don't understand as that is what I have in menu.lst for grub
[11:47] <lesshaste> and cat /proc/cmdline gives me  cat /proc/cmdline 
[11:47] <lesshaste> root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23 ro quiet splash crashkernel=64M@16M
[11:47] <mnemo> I think the --append part is fine... I think you forgot some other part so that its assuming that "--appendBLAH" is the filename
[11:48] <mnemo> but check out "man kexec"
[11:48] <mnemo> it says "kexec -l img" but for -p it doesnt say "kexec -p img" it just says "kexec -p"
[11:48] <lesshaste> mnemo, sudo kexec -l /boot/vmlinuz-2.6.27-11-generic --initrd /boot/initrd.img-2.6.27-11-generic --append="root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23"
[11:48] <lesshaste> Cannot open `--append=root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23': No such file or directory
[11:48] <mnemo> i dunno, I have never used of kexec before so im really the wrong person to ask :) :)
[11:48] <lesshaste> it doesn't make any differnce
[11:48] <lesshaste> I have the syntax wrong
[11:50] <mnemo> lesshaste: jaunty will have automatic ops crash reporting at least
[11:50] <mnemo> but I guess that's a long time to wait just to report it ;o
[11:50] <lesshaste> mnemo, you see in the man page it says " Passing  the  exact  contents  of
[11:50] <lesshaste>        /proc/cmdline into command-line-options is the  safest  way  to  ensure
[11:50] <lesshaste>        that correct values are passed to the rebooting kernel.
[11:50] <lesshaste> "
[11:51] <lesshaste> so I have
[11:51] <lesshaste>  sudo kexec -l /boot/vmlinuz-2.6.27-11-generic --initrd /boot/initrd.img-2.6.27-11-generic --append="root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23 ro quiet splash crashkernel=64M@16M"
[11:51] <lesshaste> and that is wrong apparently!
[11:51] <lesshaste> grrr!
[11:51] <mnemo> try --command-line=BLAH instead of --append=BLAH then?
[11:54] <IntuitiveNipple> Are you missing "--args-linux" before --append ?
[11:56] <lool> TheMuso: Hey, I checked the build failures of linux-ports on various arches; it seems we want to enable makedumpfile on more arches: kexec-tools is needed first on armel lp #323634, I don't know about hppa, powerpc and amd64 needs the bdep to be updated from makedumpfile [i386] to makedumpfile [!armel]
[11:56] <lool> and I don't know about ia64
[11:56] <lesshaste> IntuitiveNipple, it's not in the compressed kernel version at http://www.mjmwired.net/kernel/Documentation/kdump/kdump.txt
[11:57] <TheMuso> lool: makedumpfile is only available for i386 powerpc ia64 and lpia, ports does not build any armel kernels.
[11:57] <TheMuso> lool: ia64 makedumpfile fails, same with powerpc
[11:57] <TheMuso> lool: haven't investigated why, just want them to build for now.
[11:57] <lesshaste> IntuitiveNipple, ok this is just weird
[11:57] <lesshaste> sudo kexec -l /boot/vmlinuz-2.6.27-11-generic --initrd /boot/initrd.img-2.6.27-11-generic --args-linux --append="root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23 ro quiet splash crashkernel=64M@16M"
[11:57] <lesshaste> Cannot open `--args-linux': No such file or directory
[11:58] <lesshaste> hi Lure 
[11:59] <mnemo> lesshaste: you should use "--initrd=BLAH" instead of "--initrd BLAH"
[11:59] <lesshaste> gaaaaaah!
[11:59] <TheMuso> lool: again we don't build amd64 kernels with ports
[11:59] <lesshaste> thanks :))
[11:59] <mnemo> np :)
[11:59] <lool> TheMuso: Ok
[11:59] <lesshaste> blimey that took a long time
[12:01] <lool> TheMuso: I had in mind that you could probably keep a very similar bdep as the main kernel which should reflect arches where makedumpfile should theoritically be available and only disable it in the rules, but it's fine like that; thanks
[12:04] <lool> TheMuso: Also, I was curious whether you plan to move the tree to ubuntu/ like the other in-archive trees?  or is this reserved to a special team?
[12:05] <TheMuso> lool: ubuntu/ is only for the kernel team afaik, I don't have access to it. In addition, the kernel team don't want very much to do with ports, so I don't know what will be happening about where the official prts tree will end up.
[12:05] <TheMuso> So I am keeping it under my own namespace currently.
[12:06] <lool> TheMuso: Is there a way to share access to this tree to others (I think NCommander also contributed to the git tree?!?)
[12:07] <TheMuso> lool: unless everyone has ubuntu/ access, the easiest way is to create one's own tree, and ask me to merge/pull changes I guess. I know NCommander has done ports work before, but that was for intrepid/early jaunty, and ports is now maintained somewhat differently (how the lpia kernel was maintained before it was included into mainline jaunty).
[12:07] <lool> TheMuso: Understood
[12:09] <TheMuso> lool: If things work out with me maintaining ports in my own time, and if others want to help work on ports, there may be grounds to making a team that has access to something like ubuntu-ports or something similar.
[12:23] <lesshaste> how do ubuntu kernel people normally try to diagnose kernel panics?
[12:44] <lesshaste> hi Lure 
[13:17] <lesshaste> hi abogani 
[13:23] <rtg> amitk: where are you on the config file hierarchy changes and the ARM config file updates?
[13:24] <rtg> I thought I'd take a stab at getting LPIA into the Jaunty archive today.
[13:31] <amitk> rtg: Almost done
[13:31] <amitk> rtg: 10 mins to finish build testing
[13:33] <rtg> amitk: no rush, I've plenty of other minutia to keep me busy
[13:36] <amitk> rtg: it is an abi bumper
[13:36] <amitk> i guess it will be for lpia in any case
[13:38] <rtg> amitk: I had planned on bumping ABI anyway 'cause I want to tuurn on CRDA
[13:38] <IntuitiveNipple> rtg: possible regression in Intrepid (ath5k); you might want to cast an eye over it: bug #327237
[13:38] <ubot3> Malone bug 327237 in linux "Kernel 2.6.27-11 in 8.10 has no WiFi support" [Undecided,New] https://launchpad.net/bugs/327237
[13:39] <amitk> rtg: changes compiled successsfully and pushed out to git
[13:39] <rtg> IntuitiveNipple: contact smb_tp as he is responsible for maintenance and regressions.
[13:39] <rtg> amitk: thanks
[13:39] <IntuitiveNipple> okay
[13:40] <smb_tp> IntuitiveNipple, I look at it
[13:40] <IntuitiveNipple> smb_tp: thanks.
[13:44] <maxb> Hmm... madwifi and ath5k both loaded in that intrepid bug - is that badness?
[13:45] <rtg> yep - madwifi borks the hardware. there is a pending bug fix which includes blacklisting madwifi
[13:46] <smb_tp> rtg, So was madwifi not there before?
[13:46] <rtg> it ought to have been, but its kind of racy.
[13:47] <maxb> The wifi%d line looks a bit odd too - %d not being interpolated?
[13:48] <IntuitiveNipple> maxb: yeah, and that is generated by ath9k :p
[13:48] <IntuitiveNipple> (drivers/net/wireless/ath9k/core.c:1127)
[13:48] <maxb> The pending fix leaves things jockey-able, right? I have some hardware which works with madwifi but not ath5k
[13:58] <amitk> rtg: are you looking at https://bugs.edge.launchpad.net/ubuntu/+source/ixp4xx-microcode/+bug/328188 
[13:58] <ubot3> Malone bug 328188 in ixp4xx-microcode "please include firmware from ixp4xx-microcode in the linux-firmware package for armel" [Undecided,New] 
[13:59] <rtg> amitk: not yet. you wanna do it?
[13:59] <amitk> rtg: not sure what needs doing...
[14:00] <rtg> amitk: ok, I'll get to it as I think I'm assigned the bug
[14:00] <amitk> just add the firmware to the jaunty/firmware directory?
[14:00] <amitk> rtg: ^
[14:00] <rtg> and make sure the license/copyright stuff gets added to the package
[14:01]  * amitk looks at how the firmware udeb is created
[14:04] <smb_tp> IntuitiveNipple, Actually the message comes from lrm:ubuntu-restricted/madwifi/ath/if_ath.c:446
[14:05] <IntuitiveNipple> smb_tp: really? oh, duplicates then! I thought it seemed strange every module trying to grab it :)
[14:05] <smb_tp> They are quite similar. :)
[14:06] <IntuitiveNipple> My brain's mush... didn't go to bed last night so only just functioning right now 
[14:06] <rtg> smb_tp: uh, the bug I was thinking about is https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/208137, but its the same solution for madwifi v.s. ath5k.
[14:06] <ubot3> Malone bug 208137 in module-init-tools "Add vt8623fb to blacklisted framebuffer drivers" [Medium,In progress] 
[14:07] <rtg> In fact, shouldn't jockey handle this?
[14:09] <smb_tp> rtg, So, the solution would be to blacklist madwifi and jockey should handle/load it if required?
[14:10] <rtg> smb_tp: actually, if they just ran jockey it ought to do the right thing.
[14:31] <tjaalton> any idea why the qla2xxx driver takes one minute to initialize?
[14:31] <tjaalton> on jaunty
[14:32] <tjaalton> the default rootdelay is 30 seconds, so unless I change it the boot fails
[14:33] <tjaalton> this is a blade server with a dual-HBA adapter
[14:33] <tjaalton> -adapter
[14:34] <tjaalton> [   62.940070] qla2xxx 0000:13:00.0: Firmware image unavailable.
[14:34] <tjaalton> maybe that has something to do with it?
[14:36] <rtg> tjaalton: but it eventually works?
[14:37] <tjaalton> yes
[14:37] <rtg> tjaalton: well, start a bug report and attach dmesg. maybe we can eyeball it from the driver source
[14:38] <tjaalton> rtg: yeah, started already
[14:39] <rtg> tjaalton: I've got a server that is totally wedging, it looks like igb. It started yesterday after a BIOS update and perhaps an update to the latest Jaunty -7 kernel.
[14:40] <rtg> I should try nomsi
[14:40] <tjaalton> rtg: this has happened since I started testing multipath-booting (late december)
[14:41] <rtg> tjaalton: ah, nothing really new then. have you bugged upstream?
[14:42] <tjaalton> rtg: haven't had time yet, fixed multipath-tools first so it actually boots up without manual intervention
[14:42] <rtg> biab, have to get close to my server in another room
[14:43] <tjaalton> it could be that the slow initialization is normal in this case, since there are so many paths to the device (8)
[14:49] <tjaalton> rtg: filed bug 328550
[14:49] <ubot3> Malone bug 328550 in linux "qla2xxx takes ~one minute to initialize per device" [Undecided,New] https://launchpad.net/bugs/328550
[14:49] <tjaalton> wtf? apt-getting the source for linux-image-2.6.28-7-generic fetches the meta-package
[14:50] <mnemo> i told you yesterday timo :)
[14:50] <tjaalton> mnemo: yes, maybe our mirror was the one that was outdated :)
[14:50] <tjaalton> since I couldn't reproduce it then
[14:51] <mnemo> hehe yea
[14:51] <mnemo> anyway I worked around it with:
[14:51] <mnemo> apt-get source linux=2.6.28-7.20
[14:51] <tjaalton> thanks
[14:53] <tjaalton> rtg: do you know the contact point for qla2xxx upstream?
[14:57] <rtg> tjaalton: doesn't appear to be in the MAINTAINERS file. You might try sending an email to James Bottomley
[14:58] <tjaalton> rtg: actually, Andrew Vasquez looks like a better match :)
[15:20] <tjaalton> rtg: ok, got a reply already. they'd like it not to complain about the firmware :)
[15:20] <tjaalton> but I'm not sure if the initramfs has it or not
[15:24] <tjaalton> update-initramfs complains as well
[15:31] <smb_tp> tjaalton, Actually I would suspect that getting linux-meta as the source for linux-image-2.6.28-7-generic is sort of the right thing
[15:33] <tjaalton> smb_tp: but that only happens as of a few days ago
[15:33] <tjaalton> or yesterday
[15:34] <tjaalton> the docs still suggest to do that when trying out patches
[15:35] <tjaalton> anyway, the qla2xxx still complains even if I copy the firmware in /l/f/2.6.28-7-server
[15:35] <smb_tp> Which is rather strange. That linux-image-2.6.28-7-generic is a meta-package itself and the source of that is linux-meta
[15:36] <tjaalton> no it isn't :)
[15:36] <tjaalton> but maybe it's the new apt that does this
[15:37] <tjaalton> the source of the image is 'linux', and the source for it is 'linux-meta'
[15:37] <tjaalton> so the resolver is different now
[15:38] <smb_tp> tjaalton, Hrm, yeah. You are so right. It just sounded like the linux-restricted-modules trap I fall into so often
[15:38] <tjaalton> hehe
[15:49] <tjaalton> sigh, the qla2xxx driver still complains even if I copy the fw to /l/f
[15:50] <tjaalton> bbl
[16:44] <Kano> rtg: http://kanotix.com/files/kernel/unused-patches/2.6.28-ubuntu-qc-usb-compile-fix.patch
[16:44] <Kano> dont forget this one
[16:46] <Kano> a x-fi driver would be interesting too
[17:42] <Keybuk> ok
[17:43] <Keybuk> something is definitely, absolutely, officially wrong with I/O in 2.6.28
[17:59] <Kano> whats your problem?
[18:06] <Keybuk> Kano: every single bootchart shows almost continuous IO, with one core continually in I/O wait, and no process claiming it
[18:06] <Keybuk> not limited to any particular machine either
[18:06] <Kano> ah
[18:06] <Kano> and 2.6.27 or .29 rc is different?
[18:06] <Keybuk> the intrepid kernel does not show it
[18:57] <rtg> Keybuk: can you try the server kernel? Its got a different I/O sched setting, e.g., deadline 
[18:58] <Keybuk> sure
[19:04] <elmo> or just boot with elevator=deadline
[19:04] <tjaalton> rtg: any idea why the qla2xxx driver still complains about the missing firmware even if I copy it to /lib/firmware (and repack the initramfs)?
[19:05] <rtg> elmo: well, there are some other differences as well, like the time slice quanta
[19:05] <elmo> rtg: ah, ok
[19:05] <rtg> tjaalton: not off the top of my head. are you sure the name is right?
[19:06] <tjaalton> rtg: checking
[19:08] <tjaalton> rtg: yep, correct name
[19:11] <rtg> tjaalton: uh, gimme a bit. I'm in the middle of horking in 2.6.28.5
[19:11] <tjaalton> rtg: sure
[19:14] <Keybuk> rtg: mad I/O still shows up with -server
[19:14] <rtg> hmm, that sucks.
[19:15] <rtg> q:q!
[19:39] <rohan> https://wiki.ubuntu.com/KernelTeam/SuspendResumeTesting --> the script provided here for stress testing suspend/resume is meant to be run on which distribution?
[19:39] <rohan> or is it distro agnostic?
[19:44] <rtg> apw:  ^^
[19:46] <apw> rohan, it mostly relies on pm-suspend existing.  it has one gnome specific test which uses dbus to initiate a suspend for a more complete test
[19:46] <apw> but that test not working is not fatal, and occurs on kde
[19:46] <rohan> apw: ok, so i can use it on kubuntu 8.04?
[19:46] <apw> the failure reporting to launchpad only occurs on the latest jaunty installs and is tied into apport
[19:47] <apw> i have run it on 8.04, it will not report back to launchpad automatically on failure there, but still is expected to work
[19:47] <apw> if it does not contact me and i'll see what i can do to make it so do
[19:47] <rohan> ok, thanks
[19:47] <apw> np
[19:48] <rtg> tjaalton: can you build a kernel? strip the DEBUG2 macro from line 2711 of drivers/scsi/qla2xxx/qla_os.c
[20:05] <tjaalton> rtg: sure, and it shouldn't take long since the beast has eight cores
[20:05] <rtg> tjaalton: it looks like that _has_ to be the failure point.
[20:06] <rtg> tjaalton: you could also modprobe it with ql2xextended_error_logging=1
[20:06] <rtg> use break=top to catch it in initramfs
[20:08] <tjaalton> rtg: if it was set to use the serial port for console, I could. I'll try building the kernel first
[20:08] <rtg> k
[20:09] <tjaalton> hmm, so remove those two lines?
[20:10] <rtg> tjaalton: no, unwrap the DEBUG2 macro so that it prints something
[20:10] <tjaalton> oh, heh
[20:10] <tjaalton> let's see
[20:11] <rtg> this is in the qla2x00_request_firmware() function, just to make sure we're looking at the same sources.
[20:11] <tjaalton> yep
[20:45] <tjaalton> rtg: [   63.010090] scsi(0): Failed to load firmware image (ql2400_fw.bin).
[20:46] <tjaalton> rtg: note that the image is not normally on the initrd at all
[20:46] <tjaalton> ..and actually I didn't remember to copy it manually either
[20:47] <rtg> tjaalton: doh! that might have something to do with it. I wonder if there is an option to build it in. lemme check
[20:47] <tjaalton> update-initramfs complains about the firmwares
[20:49] <tjaalton> W: Possible missing firmware /lib/firmware/2.6.28-7-server/ql2500_fw.bin for module qla2xxx
[20:49] <tjaalton> etc
[20:50] <rtg> tjaalton: thats cause it should be /lib/firmware/ql2500_fw.bin
[20:51] <tjaalton> rtg: that's where they actually are
[20:51] <tjaalton> I'll repack the initrd with the firmware in /l/f
[21:00] <tjaalton> and according to dmesg it really is waiting one minute for the firmware
[21:05] <tjaalton> rtg: same error message with the firmware in /lib/firmware
[21:07] <rtg> tjaalton: ok, I'll dig deeper, but its gonna have to wait until tomorrow.
[21:08] <tjaalton> rtg: take your time :)
[22:21] <tjaalton> rtg: qlogic guy says that redhat has a similar problem related to udev, and he'll get back to me later
[22:47] <tjaalton> rtg: got a reply; "This is an issue with the initrd (initramfs) infrastructure not supporting the request_firmware() interface."
[22:48] <tjaalton> "Basically, the 60 second lag time is the request_firmware() call timing out due to udev being unable to satisfy the call to load firmware"
[23:39] <rtg> tjaalton: seems like we ought to be able to get initramfs to do the right thing.