[00:00] (zapping the '^' check and copying the entire string one-by-one breaks too) [00:00] that leaves the "^" though, right? [00:01] right [00:01] oh, hmm, I wonder if 'put' needs to be given a pointer type [00:01] I bet it's type-sensitive ... [00:19] cjwatson: it's performing a forall against menuitemmap.text while changing the contents of menuitemmap.text... [00:19] the forall is against the string returned by translate, actually [00:20] er, or not? the forall is the translation -- yeah [00:20] I'm trying to replace it by for though in case forall is the problem [00:20] hmm, that sentence could have used some punctuation [00:20] nope, using a for loop instead doesn't help [00:20] it was easier to parse than gfxboot code ;) [00:21] coffee [00:23] "by for though in case" - who needs punctuation, just declare it to be a new dialect of python [00:43] keescook: as far as I can see, any attempt to write into menuitemmap.text using 'put' breaks it [00:44] jdstrand: re bug 202706 - did you forward the patch to Debian ? [00:44] Launchpad bug 202706 in mysql-dfsg-5.0 "MySQL 5.0.51: ORDER BY not working with GROUP BY" [High,Fix released] https://launchpad.net/bugs/202706 [00:44] mathiaz: not yet, plan to [00:45] cjwatson: odd, very odd [00:47] cjwatson: are there other cases of string-putting working? [00:48] keescook: loads [00:48] it's used all over the place [00:49] cjwatson: what happens if you drop "translate" and leave everything else? [00:49] FilenameType Size Used Priority [00:49] /dev/compcache0 partition 30416 22380 100 [00:49] I don't really see why the source of the put should matter, though [00:49] Uh, sorry. [00:49] keescook: I think that worked, would have to check again [00:54] heya people [00:54] cjwatson: does forall include the terminating null? [00:55] oh, nm, it's nulled at the end. hrmf [00:55] shouldn't [00:55] gfxboot is at 4.0.1 upstream [00:55] yarrr [00:58] though no changes in bincode.asm so I guess that isn't relevant [00:59] Is there a way I can figure out which packages depends on mysql-server (excluding Recommends and Suggests) ? [01:00] mathiaz: apt-cache rdepends? [01:01] (don't know how it deals with Recommends and Suggests off-hand) [01:02] cjwatson: interesting update to /split def [01:03] jdstrand: hum - it includes Recommends and Suggests [01:03] keescook: hmm, where? [01:04] in the upstream system.inc [01:05] + % split does not work if str1 is in a special memory region (where [01:05] + % 'cvp length' does not work). So we dup it first. [01:05] cjwatson: you want the full diff pasted somewhere? [01:06] keescook: no, I have it. I think that might be referring to string literals in the bytecode [01:06] yeah... ow my head [01:07] keescook: stripping out a bunch of keymaps also works around it, as Riddell suggested earlier on #ubuntu-release [01:07] I think we're overflowing gfxboot's available memory, perhaps [01:07] Hi, lool can I ask you a thing about the automount of a crypted partition of a USB HD? [01:08] I might be able to fix it by stripping out some unused strings [01:08] mathiaz: apt-rdepends? [01:10] mathiaz: I think it goes too far though [01:10] * Riddell hugs colin and kees and goes to bed [01:11] I don't understand why reducing the memory footprint would solve it -- both strcpy and put operate on the same already-malloc'd region. hmpf [01:12] yeah, I know [01:12] jdstrand: awesome - that's what I was looking for [01:12] this is all very, very weird [01:13] if I rip out dia_install.inc and everything related to it (since it was unused in Ubuntu anyway), it all works [01:13] cjwatson: the bootloader defines available memory? it's not yet obvious to me where that happens. [01:14] err. somewhere in syslinux and/or gfxboot, I'm not entirely certain where [01:14] looks liek gfxboot reads it from a "sysconfig" area of the boot loader [01:15] I've haven't looked at the memory areas yet, but maybe the stack is crashing into malloc? strcpy is smaller than forall [01:16] I have no idea where code vs stack vs data vs malloc is stored yet :{ [01:16] it could be that, yeah [01:17] 58 files changed, 3 insertions(+), 3042 deletions(-) [01:17] I have to say I'm happy with that kind of change on principle [01:17] hah -- what's that? 4.0 release? [01:17] no, ripping all this stuff out [01:17] oh haha [01:18] including translations [01:18] eek [01:18] cjwatson: so, what produces the syslinux config that launches gfxboot? [01:18] http://people.ubuntu.com/~cjwatson/bzr/debian-cd/ubuntu/ [01:27] cjwatson: my brain is melted enough for one day -- sounds like we have a viable workaround. :P [01:27] mmmm....brains.... [01:27] keescook: me too [01:27] (I don't see any hard-coded memory limits in config files -- perhaps isolinux has one internally, but I'm going to stop looking) [01:27] cjwatson: I feel kind of up to speed on gfxboot now, though. ;) [01:28] yay [01:28] a victim [01:28] :-) [01:28] hehe [01:28] the discussion was useful even if the solution was a lot cruder than we might have hoped [01:28] we might need those associative memories again in the future ... [01:29] yup, I figure I might actually be useful Next Time. :) [01:30] you were :) [01:30] heh [02:21] slangasek: aforementioned fix posted to bugs 197645 and 199960 for validation of the fix before uploading. [02:21] Launchpad bug 197645 in gnome-settings-daemon "gnome-settings-daemon crashed with SIGSEGV in rw_screen_list_outputs()" [Medium,In progress] https://launchpad.net/bugs/197645 [02:21] Launchpad bug 199960 in gnome-settings-daemon "error starting GNOME Settings Daemon" [High,Confirmed] https://launchpad.net/bugs/199960 [02:25] bug #198808, what is up with that? Who though it would be a good idea to pass all the video quirks unless specifically told not to? === lamont` is now known as lamont [03:09] hmm, seems i spend 10 seconds in initramfs on boot [03:09] lol you're really motivated to fix that, huh? :) [03:09] Amaranth: Fujitsu was also seeing something like that (although 30 sec or something) [03:09] jdong: was watching a movie :) [03:09] jdong: and i want my < 20 second boot :D [03:09] Amaranth: lol [03:09] everyone wants my 19s boot :) [03:10] jdong: not fair that you get 19s with 19MB/s and I get 32s with 47MB/s :P [03:11] i can see why you have me start udev on starting readahead though, i guess I have the IO to spare :P [03:12] is this something that shows up before bootchart begins loggin? [03:12] no [03:13] http://www.realistanew.com/random/hardy-20080317-3.png [03:13] jdong: also, you start a lot less stuff [03:13] oh wow [03:13] pwnguin: yeah, it's upstart scripts calling init scripts [03:13] but in parallel and event driven [03:13] jdong did it [03:14] what? [03:14] made it tank? [03:14] Amaranth: nice, your real boot is 22s if initramfs didn't suck. [03:14] oh, the 10 second stall isn't jdong [03:14] i'm pretty sure it's been doing that all along [03:14] pwnguin: the solid blue from 23 to 30 is all me :) [03:14] but look at that green line [03:14] that's all seek =( [03:14] jdong: yeah, i think if i trim down what i start and try that defragment thing i can get 12-15 seconds :) [03:15] Amaranth: try an easy cheap defrag replacement first [03:15] pwnguin: the whole boot is seek except for readahead [03:15] even readahead [03:15] that's why readahead is there :P [03:15] eh? during readahead I peak at 47MB/s [03:15] Amaranth: tar up all the files in /etc/readahead/boot [03:15] Amaranth: then unpack it back into / [03:15] actually cpio probably works better with a list of files ;-) [03:16] that's...sneaky [03:16] heh [03:16] Amaranth: for me that halved readahead time. [03:16] if it works [03:16] Amaranth: cheaper than defragging [03:16] i see why it works [03:16] lol it is really sneaky and cheap :) [03:16] gimme a script or something? :) [03:16] xargs? [03:17] * Amaranth lazy [03:17] Amaranth: lol one sec [03:17] * jdong whips out the command he used [03:17] Amaranth: cat /etc/readahead/boot | xargs sudo tar cvpf /tmp/init.tar [03:18] Wasn't someone working on actually rearranging blocks on the partition based on usage patterns? That would entirely remove the need for readahead, i'd think. [03:18] Amaranth: sudo tar xvpf /tmp/init.tar -C / [03:18] tar cf foo `cat /etc/readahead/boot | xargs` [03:18] alright, i'll use yours [03:18] lol yours works too :) [03:18] I'd use p with tar though [03:18] you never know what permissions some of those files might need :) [03:18] then pipe that back out to tar ;) [03:19] pwnguin: LOL that sounds like... fun :D [03:19] jdong: so how expensive is ext3 defrag? [03:19] im not even aware of an application that does that [03:19] pwnguin: e2defrag [03:19] psusi patched it to work with ext3 [03:19] it works... just is a lot more destructive than this [03:19] this operation is more or less atomic [03:20] define destructive [03:20] wow that went way too fast [03:20] pwnguin: lose power in the middle of the process [03:20] ah [03:20] Amaranth: yeah it's instantaneous and yields a measurable performance for me [03:20] Amaranth: btw I'm really impressed by how deep your bootchart is [03:20] that's amazing [03:21] ok that was scary [03:21] deep? [03:21] must be at least 50 processes in parallel [03:21] ah [03:21] tall [03:21] yeah, my CPUs freak out for a bit :P [03:21] brb, seeing if i destroyed anything [03:21] :) [03:21] now for an after picture [03:22] one hopes that green line shows considerable improvement [03:22] indeed [03:22] my green line is still rpetty abysmal [03:22] pwnguin: http://jdong.mit.edu/~jdong/macbook/bootchart-upstart.png [03:22] actuallly [03:22] http://jdong.mit.edu/~jdong/macbook/hardy-upstart-defragged.png [03:22] that's after the "defrag" [03:23] nice [03:23] pwnguin: yeah, at this point unless readahead or modprobe (udevtrigger) drastically improves, I think I'm at my peak boot speed [03:24] like someone else said, you could potentially group files together based on access patterns [03:24] ie, config files at boot grouped togethre [03:24] pwnguin: I generated a dependency diagram of my upstart boot at http://jdong.mit.edu/~jdong/macbook/depchart/upstart-dep.png [03:24] pwnguin: it's nicely massively parallel [03:24] whee [03:25] jdong: http://www.realistanew.com/random/hardy-20080317-4.png [03:25] peak went down but throughput seems to have gone up overall [03:26] only cut a second off though [03:26] readahead's much shorter that time [03:26] shows some improvement [03:26] theres a wierd exe though [03:26] i think that's upstart [03:26] how the hell is your readhead so short? [03:26] * jdong cries [03:26] because it's faster? [03:26] Amaranth: move up udev [03:26] * lamont wonders why it is that, when I have "shadow: compat db" in nsswitch.conf, the gnome screen lock needs read access to /var/lib/misc/shadow.db before I can unlock the screen. [03:27] jdong: i don't have the sad excuse of a harddrive that apple puts in their laptops [03:27] heh [03:27] 12? ouch [03:27] Amaranth: /etc/event.d/udev, "start on starting readahead" [03:27] jdong: this HD was the fastest you could get for a laptop until a year after i bought it [03:27] Amaranth: try that for size. That should reduce the readahead bottleneck [03:27] (hopefully no races :D) [03:28] i should probably rerun profile [03:28] Amaranth: same for lrm-manager [03:28] yeah but i still can't go fast until i fix initramfs [03:28] * jdong looks at dep chart [03:28] what could make it slow? [03:29] reiserfs? [03:29] /etc/event.d/udev is already "start on starting readahead" [03:29] you told me that one before [03:29] pwnguin: no, ext3 [03:29] Amaranth: oh, do the same for lrm-manager [03:29] alright, done [03:30] i wish this was a pdf so i could search through it [03:30] Amaranth: are you booting verbosely? [03:30] yeah [03:30] don't do that? [03:30] Amaranth: you should be able to see where it hangs up in initramfs then [03:30] right? [03:30] I'm guessing probably the probing of some sort of hardware, either USB or disk [03:31] initramfs runs a udevtrigger IIRC, which could be costly [03:31] it sits at "Starting up" then "Loading", then it seems to start for a little bit at setting up cfq [03:31] you might win by removing all USB stuff from initramfs [03:31] but it does sit for a second or two on "Starting up" [03:31] Amaranth: ok, I *DARE* you to.... [03:32] Amaranth: http://jdong.mit.edu/~jdong/macbook/event.d.new.tar.gz [03:32] my keyboard and touchpad are ps/2 so... :P [03:32] what is that? [03:32] Amaranth: diff that against your event.d [03:32] Amaranth: it's my updated event.d after shaving 2 secs off my boot [03:32] Amaranth: it sets the IO scheduler to AIO and disabled pdflush during boot [03:32] Amaranth: btw, you're responsible in rc.local to revert those two changes :D [03:33] see event.d/pdflush [03:33] Amaranth: that should further reduce seek and write pressure during boot [03:33] and HERE, we cross into the realm of voodoo boot magic [03:33] you changed udevtrigger? [03:33] Amaranth: I did? [03:34] Amaranth: my memory kind of fails [03:34] -start on stopped lrm-manager [03:34] +start on started udev [03:34] Amaranth: oh, I wanted to trigger a bit earler as I didn't have any LRM devices [03:34] ah [03:34] but i do :P [03:34] Amaranth: yeah so you shouldn't do that :D [03:35] gimme your rc.local :P [03:36] Amaranth: my rc.local is slightly scary :) [03:36] is your change in load-modules the same thing? [03:36] +start on stopped udevtrigger [03:36] jdong: As long as you don't say worksforme and then insist everyone should do it this way. [03:37] Amaranth: right, I think load-modules should start on udev instead. [03:38] Amaranth: http://jdong.mit.edu/~jdong/macbook/rc.local [03:38] ScottK2: lol, this is hack central, anyone who dares try this is at minimum insane :D [03:38] Amaranth: useful stuff are the last line and the 2nd line [03:38] Amaranth: the rest of hacks to save about 2W of power on my system [03:39] and note the modprobe usbhid for when I couldn't get udev started correctly :D [03:39] * jdong puts on his hack shame hat [03:39] jdong: should it really be initctl emit shutdown in rc6? [03:39] heya [03:41] Amaranth: well, shutdown and reboot meant the same thing when I just wanted Pulse to save mixer levels. [03:41] hehe [03:41] :) [03:41] i don't ever reboot anyway [03:42] Amaranth: good, I'm not sure rebooting works all that gently [03:42] Amaranth: it's kind of the extreme end of the "faster shutdown" spec from 2 releases ago [03:42] :D [03:42] yeah, i see lots of devmapper errors on shutdown [03:42] because stuff goes away in the wrong order [03:42] but screw it, it's shutdown :P [03:43] :) [03:43] good enough, tear things down, sync, umount, good neough [03:43] Amaranth: since you're conveniently syndicated on planet, you should probably do a quick blogpost once you're done :D [03:44] uh oh [03:44] i knew there was a catch [03:44] :) [03:45] brb, i hope [03:45] heh [03:45] he didn't have it nearly as bad as me [03:45] adventurous to do that one one machine [03:45] let me tell you, screwing up udev mount order on a USB keyboard system is NOT fun [03:46] haha [03:46] you mean apple notebooks dont have a serial debugger port? [03:46] gasp :) [03:46] pwnguin: I'm lucky to have an optical drive and a mouse button. [03:47] but just the one button, no more [03:47] pwnguin: watch multitouch replace that 6 months from now. [03:48] ? [03:48] pwnguin: I doubt trackpads will have buttons at apple's rate [03:48] everyhting's gonna be encapsulated in touch and drag type gestures [03:48] like a multitouch screen, or the trackpad just gets even smarter [03:48] the trackpad getting smarter [03:50] jdong: ok, something went bad [03:50] eep === travis is now known as Amaranth [03:50] I got a desktop but I've got three sleeps running and got no bootchart [03:51] hmm [03:51] Amaranth: is rc.local hanging? [03:51] Amaranth: initctl status, look for anything in starting phase [03:52] jdong: initctl: missing job name [03:52] Amaranth: sorry, list. [03:53] grep says nothing [03:53] oh, wait [03:54] i've got a bunch of stuff in 'waiting' but i think some of it is supposed to be there and some of it isn't [03:54] oh, no, that's all for on stop [03:54] Amaranth: what's bootchart's status? [03:54] man. i have this five second "resume" crap [03:55] jdong: waiting :/ [03:56] Amaranth: follow up the chain: http://jdong.mit.edu/~jdong/macbook/depchart/upstart-dep.png [03:56] Amaranth: usplash->gdm->acpi_support->laptop_mode->rc_local->stop_bootchart [03:56] jdong: http://rafb.net/p/ZelU1G27.html [03:57] it's waiting for gdm? that doesn't make sense [03:57] Amaranth: no stopped is the normal state for that [03:57] i should probably go through here and remove all the things i don't have init scripts for [03:57] Amaranth: those are all "script / end script" relationships, which enter stopped/waiting once they finish [03:57] oh, i did have a typo in my rc.local [03:58] cfg instead of cfq [03:58] :P [03:58] Amaranth: well that'll do it :) [03:58] Amaranth: yay set -e. [03:58] so those jobs should be started to not need ok [03:58] Amaranth: agreed [03:58] err, i hope that made sense :P [03:58] Amaranth: try "start on starting shutdown" for size :) [03:59] stop-bootchart should not depend on rc.local working [03:59] stratus: right, it should just depend on rc.local stopping [03:59] err, wrong person entirely [03:59] wait, waht? [03:59] start on starting shutdown? for what? [03:59] Amaranth: for an oxymoronic statement [03:59] Amaranth: it was quite awkward writing rules that sounded like that :) [03:59] Amaranth: that's where upstart's UI falls [04:00] (and yeah blame it on PEBKAC) [04:00] alright brb again [04:00] good luck [04:02] hmm [04:05] jdong: man you're killing me here: http://www.reaistanew.com/random/hardy-20080317-5.png [04:05] http://people.cis.ksu.edu/~jld5445/after-readahead-tar.png [04:05] Amaranth: ye olde 404 [04:05] http://www.realistanew.com/random/hardy-20080317-5.png [04:05] Amaranth: hmm [04:06] pwnguin: you did the upstart thing too? [04:06] Amaranth: no [04:06] wow [04:06] oh you start almost nothing [04:06] Amaranth: so perhaps we should concentrate all the modprobing together instead, and start readahead after that's done [04:06] i start X and gdm [04:07] Amaranth: it seems like hwenever we do modprobing everyhing else grinds to a halt [04:07] jdong: sounds good to me [04:07] Amaranth: what am i not starting/ [04:07] although i should really take some time to figure out where my initramfs bug is [04:07] Amaranth: so I guess move udev*, lrm load-modules to "start on stopped mount-kernel-filesystems ok" [04:08] pwnguin: i dunno, i seem to start about twice as many things [04:08] Amaranth: and move readahead to the last thing in the module loading chain. [04:08] Amaranth: keep in mind that bootcharts elide short lived processes [04:08] maybe my computer's fast enough that things don't show up in the charts ;) [04:08] pwnguin: Amaranth seems to have a larger bluetooth stack, ntfs mount, and firestarter/iptables [04:08] those are all valid points ;) [04:08] ntfs in particular would be painful [04:09] yeah, firestarter is for connection sharing, ntfs can go away [04:09] and bluetooth? wtf [04:09] pwnguin: your blotchy stuff towards the end of bootup looks like could be optimized. [04:09] jdong: im sure it can be [04:09] pwnguin: it would seem like resources are heavily underutilized the last quarter of your boot [04:10] pwnguin: so.. either join the dark side or creatively use ampersands :) [04:10] jdong: i agree, but im more curious about the first quarter where nothing's going on and a process called resume is sitting there [04:10] pwnguin: yeah that's annoying too [04:12] * pwnguin should drop bluetooth [04:14] don't suppose you guys have any idea where to poke to figure out initramfs troubles [04:14] /usr/share/initramfs-tools [04:14] or -utils [04:14] i forget. ;) [04:14] i recall someone saying there was an interactive boot version of initramfs stuff [04:14] i think thinking more "what would likely cause the stall" :P [04:15] pwnguin: break={init, top, premount, mount, postmount, bottom, magic, other voodoo} [04:16] Ahh. Yes. All that stuff. [04:16] When I'm debugging the initramfs I braek premount usually. [04:16] Then just step through it manually. [04:16] Which is a bit hard with such a crappy shell. [04:24] hmm. well, just touching rc to use CONCURRENCY=shell seems to have broken hal [04:24] which is a shame, since it went about 5 seconds faster [04:24] pwnguin: haha see ya still need those dependency things :) [04:25] it was quick enough to undo :P [04:26] hmm [04:26] "/scripts/local-premount/resume: /scripts/local/premount/resume: 57: log_begin_msg: not found" [04:27] pwnguin: I wonder if we can put some sort of mini-upstart in initramfs :) [04:27] just use make ;) [04:28] im not even sure what that message means, or which file to look for line 57 on =( [04:29] jdong: it's been mentioned a few times that I remember [04:29] I'd like to see upstart used for desktop session stuff. [04:29] pwnguin: /usr/share/initramfs-tools/scripts [04:30] wasabi: well right now I'm wasting over 5s in bootup because udevtrigger is called once in initramfs and once in mount [04:30] wasabi: I'd like to see some of that duplication removed, as initramfs is nearly 1/3 of my bootup right now [04:31] udevtrigger is async though, and does not reload modules... which is the slow part. No? [04:32] wasabi: seems like things that close to touching the kernel aren't async [04:32] wasabi: and right, it doesn't reload modules but it does waste time that could be spent loading the rest of the modules :) [04:32] wasabi: maybe instead I need a really big initramfs with all my modules :) [04:32] Hehe. [04:33] Does it really waste time? I find that odd. [04:33] I'd think it'd be nearly instant to finish it's work after it's already run once. [04:34] jdong: http://www.realistanew.com/random/hardy-20080317-9.png [04:34] we seem to be doing the same amount of work in the same amount of time just in a different order :P [04:34] Amaranth: I guess that's a good sign of hitting a bottleneck :) [04:35] hehe, yeah [04:35] Amaranth: that bootchart would be SO sexy if the first 11s were stripped. [04:35] so to get faster i just need to trim down services and fix initramfs [04:36] i wonder if bootchart is the one breaking my initramfs :P [04:36] Amaranth: lol that would be funny [04:36] Amaranth: have you tried what happens if you zap out readahead altogether? Does your disk seek fast enough ? :) [04:36] because i don't remember whether or not it paused this long before before starting usplash [04:36]