[06:03] <CarlFK1> this is only a little OT: where can I put a script to load netconsole?
[14:25] <CarlFK1> smb_tp: remember my "boot pauses" https://bugs.edge.launchpad.net/linux/+bug/254668 
[14:25] <ubot3> Malone bug 254668 in linux "[2.6.27] pausing during boot (several issues)" [Unknown,Confirmed] 
[14:26] <apw> rtg, did you have a chance to look at the lpia trees i pushed yesterday?
[14:26] <CarlFK1> wondering if it is related to  "Tainted: P   M "   https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/317227
[14:26] <ubot3> Malone bug 317227 in linux "skb_over_panic skbuff.c:128 invalid opcode: 0000 [1] SMP " [Undecided,New] 
[14:27] <CarlFK1> or there is anything related, given you looked at that boxes dmesg more than once 
[14:32] <rtg> apw: its on my todo list for this morning.
[14:32] <apw> no worries
[14:33] <smb_tp> CarlFK1, Hi, yes I do remember, though I have been distracted by other stuff since. From somewhere I heard that people had success with jaunty for that...
[14:33]  * smb_tp looks at the link
[14:42] <CarlFK1> smb_tp: ill try a jaunty kernel and comment both of those bugs 
[14:43] <smb_tp> CarlFK1, Ok, I am not really convinced they are tied together but thats just by quickly looking upon it
[14:45] <rtg> smb_tp: 2.6.27 stable updates pushed
[14:45] <smb_tp> rtg, Ok, thanks
[14:46] <smb_tp> rtg, that was stable update to -11, right?
[14:46] <smb_tp> Doh, 2.6.27.11 I meant to say
[14:46] <rtg> smb_tp: 2.6.27.11
[14:47] <smb_tp> ok
[14:47] <rtg> not to be confused with -11 :)
[14:47] <smb_tp> yeah, thought of it the moment I hit enter
[14:48] <smb_tp> The upload of this will have to wait until we got the current one to -updates
[14:55] <CarlFK1> smb_tp: fixed the pause - yay
[14:55] <CarlFK1> "failed to start X"... not so yay.
[14:55] <CarlFK1> nvidia...
[14:56] <smb_tp> CarlFK1, So rather yayno. First is good, now I only would have to find which of the zillion changes this was.
[20:39] <Keybuk> well
[20:39] <Keybuk> and I stress that this isn't final
[20:39] <Keybuk> with just kernel and udev updates, it looks like jaunty takes 10s less time to boot than intrepid already
[20:39] <Keybuk> (no switching to ext4 or anything like that)
[20:40] <rtg> I've read similar anecdotal evidence. Is that with readahead?
[20:40] <Keybuk> same readahead as intrepid
[20:40] <Keybuk> I did reprofile it to make sure
[20:40] <Keybuk> before both
[20:40] <Keybuk> so that shouldn't effect the numbers
[20:41] <Keybuk> grub to full login is now 61s
[20:41] <rtg> huh, I didn't know we were doing readahead. Is that part of initramfs?
[20:41] <Keybuk> it's part of the normal system booy
[20:41] <Keybuk> /etc/rcS.d/S01readahead
[20:42] <Keybuk> has been since, err, hoary
[20:42] <Keybuk> I'm pretty sure it was Mataro we put that in, the food still haunts my memories
[20:42] <Keybuk> so it wasn't warty
[20:42] <rtg> drat, I was hoping readahead would get us the next 10 second win.
[20:43] <Keybuk> sorting out readahead might buy us more
[20:43] <Keybuk> but this is SSD
[20:44] <Keybuk> readahead counts for less than 1s of the boot to read in the entire profile
[20:44] <rtg> I'm working on LRM boot time link. that ought to buy us 2-4 secs
[20:44] <pwnguin> one thing ive noticed is that boot time degrades over time =/
[20:44] <Keybuk> switching from "read in entire profile then move on" to "read in by access order throughout the boot" may be an improvement of a few seconds, since less ram is required
[20:44] <Keybuk> pwnguin: sure, because your readahead profile gets increasingly out of date
[20:44] <Keybuk> so you need to load more from disk
[20:44] <rtg> Keybuk: have you messed with ext4?
[20:45] <pwnguin> Keybuk: there's also the matter of fragmentation
[20:46] <Keybuk> pwnguin: ext* are largely self-defragmenting
[20:46] <Keybuk> rtg: no, noet yet
[20:46] <Keybuk> my next task is to sort out module-init-tools
[20:46] <Keybuk> I'm hoping for another couple of seconds there
[20:46] <pwnguin> Keybuk: and yet, i have documented proof that super duper hack involving tar retrieves a few seconds
[20:46] <Keybuk> then sort out the hardware clock and console setup (on the sprint agenda), I'm hoping *that's* another couple of seconds
[20:47] <rtg> a few sends here, a few seconds there, ....
[20:47] <Keybuk> pwnguin: sure, but that's for a different reason ;)  it realigns your disk order with your readahead profile (which is in that kind of order)
[20:47] <pwnguin> it could be that the fs groups writes together
[20:47] <Keybuk> there's a sleep in procps I'm going to uncover
[20:47] <pwnguin> so theres fewer seeks
[20:47] <Keybuk> that's almost 2s
[20:47] <rtg> thats an odd place for a sleep
[20:47] <pwnguin> Keybuk: what do you think about removing the png-ification on bootchart, and leaving it as svg?
[20:47] <Keybuk> rtg: I know
[20:47] <Keybuk> pwnguin: SVGs are a pain
[20:48] <Keybuk> name me a program that can view them
[20:48] <pwnguin> inkscape?
[20:48] <Keybuk> inkscape can't view them, it can edit them
[20:48] <pwnguin> firefox
[20:48] <Keybuk> firefox can't view them last time I looked unless they're embeded in html
[20:48]  * Keybuk doesn't understand why people mind the pngification
[20:49] <pwnguin> it basically defeats datamining
[20:49] <Keybuk> rtg: well, with all the bits and pieces everywhere, including ext4 ... I'm hoping for X in 15s for jaunty
[20:49] <Keybuk> pwnguin: I don't follow? bootchart doesn't include itself in its own chart ;)
[20:49] <rtg> now if we could speed up that pig GDM startup
[20:49] <Keybuk> rtg: this is rather less than what I thought we could get - so I'm really happy
[20:50] <Keybuk> you guys have done great work
[20:50] <Keybuk> kernel accounts for roughly half of that improvement from what I can tell
[20:50] <pwnguin> Keybuk: imagine being able to graph the bootchart over time
[20:50] <Keybuk> pwnguin: personally I disable stop-bootchart and run it by hand once I get to the logged in session anyway
[20:51] <Keybuk> pwnguin: again, irrelevant, since bootchart doesn't include itself
[20:51] <Keybuk> you start generating the png at the point you've stopped collecting data
[20:51] <pwnguin> im vastly confused
[20:51] <Keybuk> bootchart generates a chart of your boot
[20:51] <pwnguin> yes
[20:51] <Keybuk> at some point, it decides to stop collecting data
[20:51] <Keybuk> that's when it generates the chart
[20:51] <pwnguin> indeed
[20:51] <Keybuk> so the time to generate the chart is irrelevant to the length of time it takes to boot
[20:51] <pwnguin> and then it writes to $version-$date-$uid
[20:52] <Keybuk> now, some people install bootchart
[20:52] <Keybuk> and leave it installed
[20:52] <pwnguin> i agree, but i never said anything about bootchart itself
[20:52] <Keybuk> and complain that it slows down login or something, because it's generating the png
[20:52] <Keybuk> but well
[20:52] <Keybuk> that's silly
[20:52] <pwnguin> let me re explain
[20:52] <Keybuk> why would you leave it installed if you're not doing work on the boot sequence?
[20:52] <pwnguin> i dont care about bootchart speed
[20:52] <Keybuk> the collector slows down your boot anyway
[20:52] <pwnguin> i care about having access to a vast array of bootcharts
[20:52] <Keybuk> why?
[20:53] <Keybuk> unless you know what you changed between each chart, they're not really helpful
[20:53] <pwnguin> for example, graphing performance over time
[20:53] <pwnguin> maybe to identify common problems outside the defaults tested
[20:53] <Keybuk> but it's not useful
[20:53] <Keybuk> because you don't know what changed to cause any bootchart changes
[20:54] <Keybuk> "my boot is faster/slower" kinda of reports are nice
[20:54] <Keybuk> but not really helpful
[20:54] <Keybuk> "with ext3 my boot takes 25s, with ext4 my boot takes 21s" are useful
[20:54] <Keybuk> you only need two charts for that, one before, make the change, then another one after
[20:56] <pwnguin> well it would at least provide a high level view for people who don't dedicate themselves to the task, and help identify large upward spikes
[20:56] <pwnguin> presumably you arent testing thnigs that would make it worse
[20:57] <Keybuk> but that data isn't helpful
[20:57] <Keybuk> to use a real-world example
[20:57] <Keybuk> it's like taking your weight every morning
[20:57] <Keybuk> and making pretty graphs
[20:57] <Keybuk> and realising you're getting fatter, and identifying spikes
[20:57] <Keybuk> but then not recording what you're eating, or how much exercise you're doing
[20:57] <Keybuk> it's no good knowing that there was a 7-day spike when you weighed 5kg more
[20:57] <Keybuk> if you don't also note down that it's christmas and you stuffed your face
[20:58] <Keybuk> bootchart is only the measurement of time
[20:58] <Keybuk> in order for it to be meaningful, we need to know what changes between successive ones
[20:58] <Keybuk> so they don't identify large upward spikes at all
[20:58] <Keybuk> worse still, they can be self-propelling
[20:58] <pwnguin> they're meaningful
[20:58] <Keybuk> two people can both identify an increase in boot time
[20:58] <pwnguin> they're just not proscriptive
[20:58] <Keybuk> and assume they're for the same reasons
[20:59] <Keybuk> and unfortunately, the chart doesn't tell you they're not
[20:59] <Keybuk> (we've seen this)
[20:59] <pwnguin> it tells you there's a problem, not what the problem is
[20:59] <Keybuk> one person had a legitimate bug
[20:59] <Keybuk> the other's hard drive was dying
[20:59] <Keybuk> exactly, and I don't think that it's a good way to detect the problem
[20:59] <Keybuk> simply logging /proc/uptime at the end of rc.local is as good as anything
[20:59] <Keybuk> you don't need bootchart for that ;)
[20:59] <Keybuk> bootchart is a tool for people actually making changes and wanting to see whether they have an effect
[21:00] <Keybuk> (actually, bootchart is largely a tool for making changes to the readahead profile, but hey)
[21:00] <pwnguin> you could measure the change in readahead time =/
[21:00] <pwnguin> i donno if that would help, readahead is a cache trick with distributed consequences
[21:00] <Keybuk> doesn't quite work
[21:00] <Keybuk> but yes, that kind of thing
[21:05] <pwnguin> Keybuk: so if firefox rendered bootchart svgs correctly in jaunty, is there any reason to continue pngification?
[21:07] <Keybuk> yes
[21:07] <Keybuk> I like pngs
[21:07] <Keybuk> you can view them with zvg
[21:07] <Keybuk> zgv even
[21:08] <pwnguin> well, its trivial to enable / disable it, and you can easily convert later ;)
[21:08] <Keybuk> I'm sorry, but I just don't agree with your use case
[21:08] <Keybuk> you're complaining because it makes your boot slower
[21:08] <Keybuk> well, bootchart in general makes your boot slower
[21:08] <pwnguin> im not
[21:08] <pwnguin> of course its slower
[21:09] <Keybuk> so what's the issue? :-)
[21:09] <pwnguin> it's bigger on disk, throws away potentially valuable information, and in random edge cases, massively delays boot
[21:10] <Keybuk> err, the svg is much bigger on disk actually
[21:10] <pwnguin> svgz
[21:10] <Keybuk> no valuable information is thrown away, the svg is identical
[21:11] <Keybuk> however the png is much more portably viewable
[21:11] <Keybuk> (including can be viewed on the console)
[21:12] <Keybuk> (I'd be interested to see the random edge cases, btw
[21:12] <pwnguin> https://bugs.launchpad.net/ubuntu/+source/bootchart/+bug/218499
[21:12] <Keybuk>  and would wager that the process taking the longer time isn't rsvg but is the java one :p)
[21:12] <ubot3> Malone bug 218499 in bootchart "after fsck, bootchart -> rsvg-convert makes computer slow" [Low,Confirmed] 
[21:12] <pwnguin> you've seen it already
[21:12] <Keybuk> right, and that one, it's mostly the java one that takes longer
[21:13] <Keybuk> rsvg-convert just takes longer too
[21:13] <Keybuk> so by the time people look, it's that one as well
[21:13] <Keybuk> and as I say in the bug
[21:13] <Keybuk> "then uninstall bootchart"
[21:15] <Keybuk> rtg: and I really must stop lsb-release calling apt-cache