[04:08] i want to know abt trace flag and i need some pointers\ === praveen_ is now known as shadow_fax [11:21] hi [11:22] I am trying to produce some debugging info for kernel panics I get the whole time [11:22] is this the right way to run kexec? [11:22] sudo kexec -p /boot/vmlinuz-2.6.27-11-generic --initrd /boot/initrd.img-2.6.27-11-generic --append="root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23 ro [11:32] no one in? [11:33] it seems the root= option is wrong for some reason [11:33] I get [11:33] Cannot open `--append=root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23': No such file or directory [11:40] you got an unmatched " in there [11:41] mnemo, yes sorry.. sudo kexec -p /boot/vmlinuz-2.6.27-11-generic --initrd /boot/initrd.img-2.6.27-11-generic --append="root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23" [11:41] gives [11:41] Cannot open `--append=root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23': No such file or directory [11:42] mnemo, so I must just have it completely wrong [11:44] hi Wellark [11:44] so you have one kernel that gives you panic and one that works? [11:44] mnemo, no.. I have one that panics every day or so [11:44] mnemo, I don't have one that obviously doesn't panic [11:44] usually you do "kexec -l kern_img" [11:44] mnemo, right.. did you see my attempt at the command line above?# [11:45] yeah thats the thing, you dont have any -l ?? [11:45] mnemo, no I am using -p [11:45] mnemo, but check out [11:45] http://www.mjmwired.net/kernel/Documentation/kdump/kdump.txt [11:45] the instructions are to do [11:46] kexec -p --initrd= --append="root=

" [11:46] so my line is sudo kexec -p /boot/vmlinuz-2.6.27-11-generic --initrd /boot/initrd.img-2.6.27-11-generic --append="root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23" [11:46] clearly I have the root= part completely wrong [11:46] but I don't understand as that is what I have in menu.lst for grub [11:47] and cat /proc/cmdline gives me cat /proc/cmdline [11:47] root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23 ro quiet splash crashkernel=64M@16M [11:47] I think the --append part is fine... I think you forgot some other part so that its assuming that "--appendBLAH" is the filename [11:48] but check out "man kexec" [11:48] it says "kexec -l img" but for -p it doesnt say "kexec -p img" it just says "kexec -p" [11:48] mnemo, sudo kexec -l /boot/vmlinuz-2.6.27-11-generic --initrd /boot/initrd.img-2.6.27-11-generic --append="root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23" [11:48] Cannot open `--append=root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23': No such file or directory [11:48] i dunno, I have never used of kexec before so im really the wrong person to ask :) :) [11:48] it doesn't make any differnce [11:48] I have the syntax wrong [11:50] lesshaste: jaunty will have automatic ops crash reporting at least [11:50] but I guess that's a long time to wait just to report it ;o [11:50] mnemo, you see in the man page it says " Passing the exact contents of [11:50] /proc/cmdline into command-line-options is the safest way to ensure [11:50] that correct values are passed to the rebooting kernel. [11:50] " [11:51] so I have [11:51] sudo kexec -l /boot/vmlinuz-2.6.27-11-generic --initrd /boot/initrd.img-2.6.27-11-generic --append="root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23 ro quiet splash crashkernel=64M@16M" [11:51] and that is wrong apparently! [11:51] grrr! [11:51] try --command-line=BLAH instead of --append=BLAH then? [11:54] Are you missing "--args-linux" before --append ? [11:56] TheMuso: Hey, I checked the build failures of linux-ports on various arches; it seems we want to enable makedumpfile on more arches: kexec-tools is needed first on armel lp #323634, I don't know about hppa, powerpc and amd64 needs the bdep to be updated from makedumpfile [i386] to makedumpfile [!armel] [11:56] and I don't know about ia64 [11:56] IntuitiveNipple, it's not in the compressed kernel version at http://www.mjmwired.net/kernel/Documentation/kdump/kdump.txt [11:57] lool: makedumpfile is only available for i386 powerpc ia64 and lpia, ports does not build any armel kernels. [11:57] lool: ia64 makedumpfile fails, same with powerpc [11:57] lool: haven't investigated why, just want them to build for now. [11:57] IntuitiveNipple, ok this is just weird [11:57] sudo kexec -l /boot/vmlinuz-2.6.27-11-generic --initrd /boot/initrd.img-2.6.27-11-generic --args-linux --append="root=UUID=7d9f52a6-3632-41cc-aa41-67534f3cdd23 ro quiet splash crashkernel=64M@16M" [11:57] Cannot open `--args-linux': No such file or directory [11:58] hi Lure [11:59] lesshaste: you should use "--initrd=BLAH" instead of "--initrd BLAH" [11:59] gaaaaaah! [11:59] lool: again we don't build amd64 kernels with ports [11:59] thanks :)) [11:59] np :) [11:59] TheMuso: Ok [11:59] blimey that took a long time [12:01] TheMuso: I had in mind that you could probably keep a very similar bdep as the main kernel which should reflect arches where makedumpfile should theoritically be available and only disable it in the rules, but it's fine like that; thanks [12:04] TheMuso: Also, I was curious whether you plan to move the tree to ubuntu/ like the other in-archive trees? or is this reserved to a special team? [12:05] lool: ubuntu/ is only for the kernel team afaik, I don't have access to it. In addition, the kernel team don't want very much to do with ports, so I don't know what will be happening about where the official prts tree will end up. [12:05] So I am keeping it under my own namespace currently. [12:06] TheMuso: Is there a way to share access to this tree to others (I think NCommander also contributed to the git tree?!?) [12:07] lool: unless everyone has ubuntu/ access, the easiest way is to create one's own tree, and ask me to merge/pull changes I guess. I know NCommander has done ports work before, but that was for intrepid/early jaunty, and ports is now maintained somewhat differently (how the lpia kernel was maintained before it was included into mainline jaunty). [12:07] TheMuso: Understood [12:09] lool: If things work out with me maintaining ports in my own time, and if others want to help work on ports, there may be grounds to making a team that has access to something like ubuntu-ports or something similar. [12:23] how do ubuntu kernel people normally try to diagnose kernel panics? [12:44] hi Lure [13:17] hi abogani [13:23] amitk: where are you on the config file hierarchy changes and the ARM config file updates? [13:24] I thought I'd take a stab at getting LPIA into the Jaunty archive today. [13:31] rtg: Almost done [13:31] rtg: 10 mins to finish build testing [13:33] amitk: no rush, I've plenty of other minutia to keep me busy [13:36] rtg: it is an abi bumper [13:36] i guess it will be for lpia in any case [13:38] amitk: I had planned on bumping ABI anyway 'cause I want to tuurn on CRDA [13:38] rtg: possible regression in Intrepid (ath5k); you might want to cast an eye over it: bug #327237 [13:38] Malone bug 327237 in linux "Kernel 2.6.27-11 in 8.10 has no WiFi support" [Undecided,New] https://launchpad.net/bugs/327237 [13:39] rtg: changes compiled successsfully and pushed out to git [13:39] IntuitiveNipple: contact smb_tp as he is responsible for maintenance and regressions. [13:39] amitk: thanks [13:39] okay [13:40] IntuitiveNipple, I look at it [13:40] smb_tp: thanks. [13:44] Hmm... madwifi and ath5k both loaded in that intrepid bug - is that badness? [13:45] yep - madwifi borks the hardware. there is a pending bug fix which includes blacklisting madwifi [13:46] rtg, So was madwifi not there before? [13:46] it ought to have been, but its kind of racy. [13:47] The wifi%d line looks a bit odd too - %d not being interpolated? [13:48] maxb: yeah, and that is generated by ath9k :p [13:48] (drivers/net/wireless/ath9k/core.c:1127) [13:48] The pending fix leaves things jockey-able, right? I have some hardware which works with madwifi but not ath5k [13:58] rtg: are you looking at https://bugs.edge.launchpad.net/ubuntu/+source/ixp4xx-microcode/+bug/328188 [13:58] Malone bug 328188 in ixp4xx-microcode "please include firmware from ixp4xx-microcode in the linux-firmware package for armel" [Undecided,New] [13:59] amitk: not yet. you wanna do it? [13:59] rtg: not sure what needs doing... [14:00] amitk: ok, I'll get to it as I think I'm assigned the bug [14:00] just add the firmware to the jaunty/firmware directory? [14:00] rtg: ^ [14:00] and make sure the license/copyright stuff gets added to the package [14:01] * amitk looks at how the firmware udeb is created [14:04] IntuitiveNipple, Actually the message comes from lrm:ubuntu-restricted/madwifi/ath/if_ath.c:446 [14:05] smb_tp: really? oh, duplicates then! I thought it seemed strange every module trying to grab it :) [14:05] They are quite similar. :) [14:06] My brain's mush... didn't go to bed last night so only just functioning right now [14:06] smb_tp: uh, the bug I was thinking about is https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/208137, but its the same solution for madwifi v.s. ath5k. [14:06] Malone bug 208137 in module-init-tools "Add vt8623fb to blacklisted framebuffer drivers" [Medium,In progress] [14:07] In fact, shouldn't jockey handle this? [14:09] rtg, So, the solution would be to blacklist madwifi and jockey should handle/load it if required? [14:10] smb_tp: actually, if they just ran jockey it ought to do the right thing. [14:31] any idea why the qla2xxx driver takes one minute to initialize? [14:31] on jaunty [14:32] the default rootdelay is 30 seconds, so unless I change it the boot fails [14:33] this is a blade server with a dual-HBA adapter [14:33] -adapter [14:34] [ 62.940070] qla2xxx 0000:13:00.0: Firmware image unavailable. [14:34] maybe that has something to do with it? [14:36] tjaalton: but it eventually works? [14:37] yes [14:37] tjaalton: well, start a bug report and attach dmesg. maybe we can eyeball it from the driver source [14:38] rtg: yeah, started already [14:39] tjaalton: I've got a server that is totally wedging, it looks like igb. It started yesterday after a BIOS update and perhaps an update to the latest Jaunty -7 kernel. [14:40] I should try nomsi [14:40] rtg: this has happened since I started testing multipath-booting (late december) [14:41] tjaalton: ah, nothing really new then. have you bugged upstream? [14:42] rtg: haven't had time yet, fixed multipath-tools first so it actually boots up without manual intervention [14:42] biab, have to get close to my server in another room [14:43] it could be that the slow initialization is normal in this case, since there are so many paths to the device (8) [14:49] rtg: filed bug 328550 [14:49] Malone bug 328550 in linux "qla2xxx takes ~one minute to initialize per device" [Undecided,New] https://launchpad.net/bugs/328550 [14:49] wtf? apt-getting the source for linux-image-2.6.28-7-generic fetches the meta-package [14:50] i told you yesterday timo :) [14:50] mnemo: yes, maybe our mirror was the one that was outdated :) [14:50] since I couldn't reproduce it then [14:51] hehe yea [14:51] anyway I worked around it with: [14:51] apt-get source linux=2.6.28-7.20 [14:51] thanks [14:53] rtg: do you know the contact point for qla2xxx upstream? [14:57] tjaalton: doesn't appear to be in the MAINTAINERS file. You might try sending an email to James Bottomley [14:58] rtg: actually, Andrew Vasquez looks like a better match :) [15:20] rtg: ok, got a reply already. they'd like it not to complain about the firmware :) [15:20] but I'm not sure if the initramfs has it or not [15:24] update-initramfs complains as well [15:31] tjaalton, Actually I would suspect that getting linux-meta as the source for linux-image-2.6.28-7-generic is sort of the right thing [15:33] smb_tp: but that only happens as of a few days ago [15:33] or yesterday [15:34] the docs still suggest to do that when trying out patches [15:35] anyway, the qla2xxx still complains even if I copy the firmware in /l/f/2.6.28-7-server [15:35] Which is rather strange. That linux-image-2.6.28-7-generic is a meta-package itself and the source of that is linux-meta [15:36] no it isn't :) [15:36] but maybe it's the new apt that does this [15:37] the source of the image is 'linux', and the source for it is 'linux-meta' [15:37] so the resolver is different now [15:38] tjaalton, Hrm, yeah. You are so right. It just sounded like the linux-restricted-modules trap I fall into so often [15:38] hehe [15:49] sigh, the qla2xxx driver still complains even if I copy the fw to /l/f [15:50] bbl [16:44] rtg: http://kanotix.com/files/kernel/unused-patches/2.6.28-ubuntu-qc-usb-compile-fix.patch [16:44] dont forget this one [16:46] a x-fi driver would be interesting too [17:42] ok [17:43] something is definitely, absolutely, officially wrong with I/O in 2.6.28 [17:59] whats your problem? [18:06] Kano: every single bootchart shows almost continuous IO, with one core continually in I/O wait, and no process claiming it [18:06] not limited to any particular machine either [18:06] ah [18:06] and 2.6.27 or .29 rc is different? [18:06] the intrepid kernel does not show it [18:57] Keybuk: can you try the server kernel? Its got a different I/O sched setting, e.g., deadline [18:58] sure [19:04] or just boot with elevator=deadline [19:04] rtg: any idea why the qla2xxx driver still complains about the missing firmware even if I copy it to /lib/firmware (and repack the initramfs)? [19:05] elmo: well, there are some other differences as well, like the time slice quanta [19:05] rtg: ah, ok [19:05] tjaalton: not off the top of my head. are you sure the name is right? [19:06] rtg: checking [19:08] rtg: yep, correct name [19:11] tjaalton: uh, gimme a bit. I'm in the middle of horking in 2.6.28.5 [19:11] rtg: sure [19:14] rtg: mad I/O still shows up with -server [19:14] hmm, that sucks. [19:15] q:q! [19:39] https://wiki.ubuntu.com/KernelTeam/SuspendResumeTesting --> the script provided here for stress testing suspend/resume is meant to be run on which distribution? [19:39] or is it distro agnostic? [19:44] apw: ^^ [19:46] rohan, it mostly relies on pm-suspend existing. it has one gnome specific test which uses dbus to initiate a suspend for a more complete test [19:46] but that test not working is not fatal, and occurs on kde [19:46] apw: ok, so i can use it on kubuntu 8.04? [19:46] the failure reporting to launchpad only occurs on the latest jaunty installs and is tied into apport [19:47] i have run it on 8.04, it will not report back to launchpad automatically on failure there, but still is expected to work [19:47] if it does not contact me and i'll see what i can do to make it so do [19:47] ok, thanks [19:47] np [19:48] tjaalton: can you build a kernel? strip the DEBUG2 macro from line 2711 of drivers/scsi/qla2xxx/qla_os.c [20:05] rtg: sure, and it shouldn't take long since the beast has eight cores [20:05] tjaalton: it looks like that _has_ to be the failure point. [20:06] tjaalton: you could also modprobe it with ql2xextended_error_logging=1 [20:06] use break=top to catch it in initramfs [20:08] rtg: if it was set to use the serial port for console, I could. I'll try building the kernel first [20:08] k [20:09] hmm, so remove those two lines? [20:10] tjaalton: no, unwrap the DEBUG2 macro so that it prints something [20:10] oh, heh [20:10] let's see [20:11] this is in the qla2x00_request_firmware() function, just to make sure we're looking at the same sources. [20:11] yep [20:45] rtg: [ 63.010090] scsi(0): Failed to load firmware image (ql2400_fw.bin). [20:46] rtg: note that the image is not normally on the initrd at all [20:46] ..and actually I didn't remember to copy it manually either [20:47] tjaalton: doh! that might have something to do with it. I wonder if there is an option to build it in. lemme check [20:47] update-initramfs complains about the firmwares [20:49] W: Possible missing firmware /lib/firmware/2.6.28-7-server/ql2500_fw.bin for module qla2xxx [20:49] etc [20:50] tjaalton: thats cause it should be /lib/firmware/ql2500_fw.bin [20:51] rtg: that's where they actually are [20:51] I'll repack the initrd with the firmware in /l/f [21:00] and according to dmesg it really is waiting one minute for the firmware [21:05] rtg: same error message with the firmware in /lib/firmware [21:07] tjaalton: ok, I'll dig deeper, but its gonna have to wait until tomorrow. [21:08] rtg: take your time :) [22:21] rtg: qlogic guy says that redhat has a similar problem related to udev, and he'll get back to me later === Guest65438 is now known as mxab === mxab is now known as maxb [22:47] rtg: got a reply; "This is an issue with the initrd (initramfs) infrastructure not supporting the request_firmware() interface." [22:48] "Basically, the 60 second lag time is the request_firmware() call timing out due to udev being unable to satisfy the call to load firmware" [23:39] tjaalton: seems like we ought to be able to get initramfs to do the right thing.