=== Keybuk [n=scott@wing-commander.netsplit.com] has joined #upstart [12:48] Hello. [12:49] I heard that you're doing ubuntu init scripts too. [12:49] <_ion> Welcome back. :-) [12:49] I'm especially interested in the fsck - esp. not parallelising checks on things on the same physical drive. [12:50] I have some ideas I've already implemented, but I'd like to hear yours. [12:52] fsck itself isn't that good at checking multiple fses [12:54] I wonder how does fsck check if the deviced are on the same physical device... [12:59] major number? [01:00] <_ion> That doesn't help much if you're e.g. comparing /dev/sda1 to /dev/volumegroup/foo which happens to be on software RAID-1, which happens to reside on /dev/sda2 and /dev/sdb2. [01:02] I didn't think fsck minded? [01:02] <_ion> I assume fscking /dev/sda1 and /dev/sda2 simultaneously is *very* slow. [01:03] <_ion> I don't see a reason for it not to work, but it's gotta be slow. === AstralSt [n=astralst@host-81-190-179-124.gorzow.mm.pl] has joined #upstart === AstralSt is now known as AStorm [01:09] Ok guys, did I miss something? [01:09] nope [01:09] pfsck uses blkid_get_devname [01:09] Which will fail with LVM or RAID. [01:09] (fail to spot the use of the same device) [01:10] My idea is to do some ls in /sys [01:10] The dependency info is all exported in sysfs [01:12] what's the problem with checking the same device? [01:12] seeking [01:12] Means slowdown. [01:13] <_ion> *Huge* slowdown, most likely. [01:13] Not all that bad, because most fsck checks are small. [01:13] I also wrote a tiny util to detect swap partitions. [01:14] (4K on glibc) [01:14] It could be useful when doing fsck totally automatically. [01:16] (e.g. on HAL event when you hotplug your new shiny USB hard disc) [01:17] Though some fsck options could probably fix it for us... if they are correctly implemented, that is === cortana [n=sam@62-31-146-25.cable.ubr12.azte.blueyonder.co.uk] has joined #upstart [01:17] not just checking the fstab [01:19] Yes, it's broken. Will default to ext2. [01:19] We could really use some FS detector. [01:19] AStorm: vol_id can do that [01:20] How? [01:20] (detect swap partitions) [01:20] Didn't work for me here... [01:21] Ah, now it works. Stupid me. [01:21] even the FS type (if known to the kernel) [01:21] Ok, then it's no problem at all [01:22] the devices will be discerned by sysfs data. [01:23] ( reading /sys/block/*/* and /sys/block/*/*/slaves if present [01:24] Probably there's some tool for that in sysfsutils too. [01:24] <_ion> Has there been a lot of progress with the complex-event-config implementation? [01:26] It works here :-) [01:26] _ion: I've written a parser [01:26] just need to rewrite that into the actual code and make it nice [01:32] is "when x" isomorphic to "start on started x; stop on stopping x"? [01:35] there's no such thing as "when x" ? [01:38] Bah. [01:38] I meant "with x" [01:39] "with x" only exists in the complex-event-config [01:40] it's somewhat analogous to "starting x until stopped x", except that there's no serialisation of the two [01:41] So it's more lax? [01:41] yes [01:41] Nice still. [01:42] I'm looking forward to complex-event-config full implementation. [01:58] me too === _ion [i=johan@kiviniemi.name] has joined #upstart === Amaranth_ [n=travis@ubuntu/member/Amaranth] has joined #upstart === Md [i=md@freenode/staff/md] has joined #upstart === solemnwarning [n=solemnwa@cpc3-fare2-0-0-cust598.cos2.cable.ntl.com] has joined #Upstart [10:37] Is upstart more stable then initng? === pkt [n=knoppel@87.203.145.142] has joined #upstart === Keybuk [n=scott@quest.netsplit.com] has joined #upstart === mbiebl [n=michael@e180122080.adsl.alicedsl.de] has joined #upstart [02:18] http://people.ubuntu.com/~scott/complex-event-config.patch [02:18] \o/ [02:33] hehe, cool [02:34] Keybuk: Do you know, if the old sysv init sets the environment variables PREVLEVEL and RUNLEVEL on startup to N and S? [02:34] I have no idea [02:34] there's an open bug that the example job doesn't do that [02:34] I put the code in, and rc seemed to not work properly [02:34] so I'm not convinced [02:36] It's quite strange, the Debian udev scripts checks for the presence of these variables (and also splashy) [02:36] (this might just be a problem with /var/run/utmp, and /var/run not existing, of course) [02:37] But they are not set by /etc/init.d/rc [02:37] So, imho they must be set by sysv init. [02:38] I also find it strange, that bootmisc resets /var/run/utmp (to :) in S [02:39] yeah [02:39] sysvinit sets those itself [02:39] I think the bug is that we track the runlevel using /var/run/utmp as the authorative reference [02:39] Hard to N and S? [02:40] whereas sysvinit has an integer in memory that tracks it, and just uses utmp as an update [02:40] yeah, this is quite funny [02:41] if /var/run/utmp persists during reboot [02:41] You'd get 2 S from runlevel during S [02:42] pretty much the problem I had [02:42] Do you know, if the behaviour of runlevel and PREVLEVEL/RUNLEVEL is specified somewhere? [02:42] I doubt it [02:43] I just used the sysvinit source as a reference for how they should behave [02:43] 0.3.5 is happily somewhat closer than 0.2.7 was [02:43] since "shutdown" now only generates runlevel change events [02:45] btw, I modified /etc/even.d/rcS for Debian to set PREVLEVEL/RUNLEVEL unconditionally to N/S [02:46] This is inconsistent to the output of runlevel at this stage, but it seems to work fine. [02:48] that's a good idea :) [03:03] Yeah, it's then consistent to runlevel/previous defined in /etc/even.d/rc [03:19] Keybuk, as to that init.. [03:19] Gentoo init depends on the fact that the RUNLEVEL is set to S or 1 on start. [03:20] Had to fix that. [03:23] AStorm: What about PREVLEVEL? [03:24] Does the Gentoo init system make any assumptions about the value of this var? [03:26] Not at all. [03:27] (also, the runlevel names are special, like sysinit, single, nonetwork, default, reboot - not just numbers - but that's Gentoo-specific) [03:27] the special sysinit runlevel is active only in S and there's explicit check for that [03:27] What's the output of runlevel during boot? [03:28] Hmm? [03:28] Or when you have booted your system completely [03:28] Completely. [03:28] I'm running on upstart right now. [03:28] (but totally using Gentoo scripts) [03:28] (except the gettys - that also had to be hacked a bit) [03:34] To be exact, the gettys were running too early [03:35] making the init process look like crap [03:35] Couldn't run them on stopped runlevel ... [03:35] Because it didn't stop [03:35] So I had to hack the last init script to emit that event. [03:36] The always last script is /etc/init.d/local.start [03:36] (due to after * clause in it) [03:36] s/stopped runlevel/started runlevel/ [04:36] runlevel is an event [04:36] not a job [04:36] events don't have a started or stopped :p [04:37] (unless you have an /etc/event.d/runlevel, of course) [04:45] <_ion> Yay for complex-event-config.patch [04:46] it's still not "official" yet [04:46] it's only gdb-tested, and doesn't have test cases [04:46] and the code is messy at best [04:46] the implementation has some interesting side-effects too [04:46] e.g. [04:47] (foo and bar) until baz [04:47] foo happens, nothing [04:47] bar happens, it gets started [04:47] baz happens, it gets stopped [04:47] now, if foo or bar happen, it will be started again [04:47] since "foo and bar have happened" is satisfied [04:47] the until only "forgets" the AND combination [04:48] I actually quite like this behaviour, since the other intent can be expressed as: [04:48] foo until baz and bar until baz [04:48] (which requires foo and bar to both happen after baz, before it will be started again) [04:50] <_ion> Well, that's exactly what i'd have expected to happen on (foo and bar) until baz [04:50] <_ion> :-) [04:51] <_ion> Agh, sorry. I misread your line as "if foo and bar happen, it will be started again" [04:53] <_ion> Perhaps because there's "foo and bar" directly under the "foo or bar" ;-) [04:56] heh [04:56] after baz happens, it doesn't require foo and bar to both happen again [04:56] just one of them [04:57] That's a bug. [04:57] is it though? [04:57] It is. If you want "once" modifier, add it. [04:57] from once foo and bar until baz [04:57] the "and" operator is TRUE if either expression has ever been TRUE [04:58] <_ion> From the computer's point of view, that's the logical behavior. But from a human's point of view, it might be nicer if the code somehow reset the value of the (foo and bar) expression. [04:58] hmm, or even [04:58] from once foo and once bar until baz [04:58] <_ion> ...when baz becomes TRUE [04:58] _ion: right, so the plan was that until would reset the tree on one side to FALSE when the other side went TRUE [04:58] and act as a flip-flop [04:58] So, if both are satisfied _once_, they don't get reset [04:58] but that would have interesting interactions later with "with" [04:58] But then... hmm. [04:58] Weird. [04:59] foo and once bar [04:59] once foo and bar [04:59] But both once degenerates to on. === Seveas [n=seveas@ubuntu/member/seveas] has joined #upstart [05:01] No, it doesn't :-) [05:01] That'd be a nice extension. [05:01] on once abc [05:01] for a single-fire event [05:02] (or even make on optional in this case) [05:04] Like it? [05:05] not sure what the behaviour would be [05:05] With once, or with those ands? [05:05] The and should be like logic and [05:05] And once should set a permanent flag [05:06] It would be transitive too [05:06] once (foo and bar) -> once foo and once bar [05:10] similar with or... [05:10] I still don't get what "once" does? [05:10] thought about adding xor? [05:11] Keybuk, acts when the event is fired, but only once [05:11] so how is "once foo" different from "foo" / [05:11] once foo has been emitted, foo is always TRUE [05:11] foo will fire multiple times [05:11] Hmm. [05:11] sure, it can affect the tree multiple times [05:11] can you give me a use case for "once" ? [05:12] Let me think... [05:12] once startup-finished... mail me some info [05:12] But then, you could create a one-time event [05:12] why not just make it a one-time event in the first place ? :p [05:12] It's much better when you change and [05:13] from bar and once foo until baz [05:13] change and? [05:13] This means that if foo is fired once [05:13] and baz is fired, we get launched [05:14] unlike from bar and foo until baz [05:14] I still don't understand why this is useful, sorry [05:14] which would mean that if both are fired... [05:14] This means "foo got executed and finished, then on baz, do something" [05:15] I think you're confusing events and jobs? [05:15] Probably. [05:15] Due to JobEvents :P [05:16] job events went away [05:16] Good. [05:17] Well, then and works correctly it seems. [05:17] (except that behaviour is not necessarily intuitive and has to be documented) [05:18] documentation is third on my TODO list :p [05:18] :> [05:18] Could lend you a hand there. Which format do you prefer? [05:19] docbook, I guess [05:19] Could be asciidoc generated docbook? [05:19] what's that? [05:19] asciidoc is a plaintext system [05:20] http://www.methods.co.nz/asciidoc/ [05:21] hmm [05:21] I could also help with docs [05:23] <_ion> asciidoc looks quite nice. [05:27] hmm [05:27] the complex-event-config patch looks good [05:28] <_ion> Ooo, nice. http://www.methods.co.nz/asciidoc/music-filter.html [05:28] <_ion> Ooo, nice2. http://www.methods.co.nz/asciidoc/source-highlight-filter.html === j_ack [n=rudi@p508DA3BD.dip0.t-ipconnect.de] has joined #upstart === mbiebl [n=michael@e180122080.adsl.alicedsl.de] has joined #upstart [05:48] Keybuk, how are the new jobs going? or haven't you started work on them? [05:49] slowly, but surely [05:49] cool [05:50] should be done today [05:50] <_ion> Yay [05:51] I have a question: How would one do normal logging to both console and system log? [05:52] Use logger + echo? [05:53] or something like bootlogd or logd, etc. [05:54] Yes, I was thinking about that logd [05:54] How do you use that? [05:55] right now it's disabled [05:55] has a few bugs [06:02] Ok, not a problem. For a while, I'll stick with plain echo. [06:03] The hardest part is writing the network script [06:03] we talked about the possibility of integration with syslog-ng a while back [06:03] the gentoo one is so universal, that it's quite slow :> [06:03] (also, the network devices aren't parallelised) [06:04] <_ion> astorm: It might be wise to start network interfaces based on events emitted by udev, too. [06:04] Yep. [06:04] Gentoo does that, but it has to wait for udev to settle :P [06:04] frugalware's networking script is quite simple, but it's not flexible in terms of bootup. everything is done by an external C program which uses profiles [06:04] We don't have to. [06:04] AlexExtreme, blah [06:05] why blah? :p [06:05] I'd rather use a bunch of shell scripts :> [06:05] and iproute2 [06:05] :) [06:05] and probably wireless-tools [06:05] AStorm: only because currently daemons are started unconditionally when the interface is up. the correct way would be to start daemons when the interface is up, syslogd is running, /var is mounted, etc [06:06] Md, yes, unfortunately. [06:06] Gentoo init can be fixed, yes. [06:06] But is it worth it? It is somewhat cruddy (e.g. /sbin/rc) === _ion still recommends just starting /sbin/rc as the compatibility layer, and rewriting things gradually as upstart jobs. [06:07] _ion, I'm doing it right now, thank you [06:07] well [06:07] :> [06:07] that would be hard [06:07] because [06:07] <_ion> The result is probably a lot better than converting the gentoo script to upstart jobs automatically. [06:08] <_ion> scripts [06:08] hmm wait [06:08] that's harder for Gentoo, because of their intricately interlinked deps [06:08] No, it's the same as running the gentoo init. [06:08] no, forget i said anything [06:08] :) [06:08] they can't just remove one script [06:08] Exactly. That's why they have to be wrapped. [06:08] because then all of its dependers wouldn't work [06:08] where as we can [06:11] I simply wrap them in a wrapper script [06:11] for upstart. [06:11] And add a "runlevel" with a list of the deps. [06:12] Still haven't got to modifying depscan.sh or rc-update to modify the file [06:26] hmm [06:26] that usplash in feisty atm is nice [06:26] Hmm, what about checking a filesystem just before mounting? :> [06:26] (except if a flag is given in the fstab, maybe...) [06:27] upstart could be turned into an automounter too... [06:27] (maybe not for network filesystems, for these there's autofs) [06:29] Except that it wouldn't know who inserted the disc. So here, Unix user rights would have to be used. [06:32] hmm [06:32] does anyone know if kdump is implemented on feisty yet? [07:35] Hmm, does initctl emit block until the event has been totally handled? [07:36] and what about initctl start? [08:06] ping, anyone? === int0x0c [n=ben@128.164.137.210] has joined #upstart [08:20] AStorm, You have upstart running on Gentoo, right? === juergbi [n=juerg@80-219-17-102.dclient.hispeed.ch] has joined #upstart [08:26] int0x0c, in emulation currently [08:26] writing the basic init and a proper init script emulation layer will take time [08:27] I'm doing that currently [08:27] ahhh [08:27] s/currently/right now/ [08:27] heh [08:27] Have you seen Keybuk around recently? [08:27] I'm looking to port the init system fully to upstart [08:27] [19:23:08] * Keybuk has quit ("") [08:28] He's supposed to release a full upstart jobset soon [08:28] I'm doing that too :-) [08:28] bah [08:28] I'd appreciate the init scripts, but then, I'm porting it right now too. [08:28] ahhh, alright [08:29] I'm afraid I don't understand the init system well enough to port the base system scripts without some research [08:30] it would be great to get a full upstart init sequenceon gentoo === int0x0c hates his dying spacebar [08:31] It's actually quite simple. [08:31] mount proc, dev, remount root ro [08:31] start udev [08:31] fsck root [08:31] fsck other fses [08:31] mount them [08:31] that's the end of basic setup. [08:31] The last step is starting gettys [08:32] That's the basic skeleton. You probably want to add there some daemons, network setup, sound setup, such things. [08:33] (forgot about remounting root rw :> ) [08:33] Also mount /dev/pts and /dev/usb optionally [08:33] also, mounting sys [08:35] hmm [08:35] Have you implemented those as pure upstart jobs? [08:35] Doing it right now. [08:36] Actually, mostly done. [08:36] Still working on improving the fsck [08:36] Then, the init level/profile has to be launched. E.g. a script starting everything in some directory [08:37] and stopping [08:37] a runlevel simulation, that is. [08:38] (or eINIT's mode, rather) [08:40] At least until profiles are implemented. === Md [i=md@freenode/staff/md] has joined #upstart [08:58] Well, the actual depscan.sh just launches a bunch of computation using awk [08:58] I'd rewrite it in Python myself. [08:58] (it's on Gentoo anyway) [08:59] The emulation requires complex-event patch [09:00] (otherwise it's impossible to link deps together) [09:01] It's in awk, because the need system is too hard to implement in bash :> [09:33] Do you suppose you could post your base startup job somewhere? [09:51] <_ion> Haha, i just now saw the picture Keybuk has at his launchpad page. :-) [11:34] Hey guys... [11:35] I wonder if this script is correct (with complex-events patch) [11:36] http://wklej.org/id/8aba67fafb [11:36] (check-fs does what you expect :-) ) [11:37] Maybe change the from line to [11:37] start on started udev [11:37] stop on stopped check-fs FS=/ [11:38] Hmm, or do I need something to listen on initctl events? [11:39] Ah, no, I have to add sleep inf probably [11:40] so that it's still running? (even with the fix for jobs w/o execs) [11:49] I wonder... [11:50] my test-event1 doesn't register as a job [11:51] Ah, my bad. Braino [11:53] Ok, got it :-) [11:54] Is there a way to create a one-fire command that will block until the other event happens, or do I have to use that sleep inf hack? [11:58] (which leaves nasty KILL messages around) === mbiebl [n=michael@e180096150.adsl.alicedsl.de] has joined #upstart [11:58] mbiebl, oh, maybe you know. [11:59] AStorm: hi, what's up [11:59] I wish to run a single-fire command, which should block until an event is fired [11:59] sleep inf and using the dependencies leaves KILL messages. [12:00] (looks yucky) [12:01] AStorm: could you explain a bit more detailed what you want to achieve === int0x0c [n=ben@128.164.137.210] has joined #upstart [12:02] I want to launch another event (which is an instance, so can't just wait on something) [12:02] and then, block, until it finished [12:02] "launch an event" ? [12:03] Actually, start a job [12:03] But emitting an event would do the same [12:03] Start command x, which returns [12:03] and still wait until that one event [12:06] AStorm: Sorry, I think you lost me. I'd have to see the bigger picture to understand what you want to do. [12:06] Launch 3 fsck in parallel [12:06] and wait until they complete [12:06] each fsck is a job. [12:06] (because I would like to hook that up to udev events later) [12:08] Why do you have to wait, until all three of them have been completed? [12:08] <_ion> "initctl emit foo" blocks automatically [12:08] _ion, yep, but what will receive it? :> [12:08] mbiebl, because then the mounting may happen [12:08] (automounting that is) [12:08] <_ion> /etc/event.d/test: [12:08] <_ion> start on foo [12:08] <_ion> exec sleep 5 [12:09] And I also want to use it in system startup sequence [12:09] <_ion> % sudo initctl emit foo [12:09] _ion, hmm, I have to pass parameters too [12:09] Well, I'd say, mount each partition separately when fsck has completed. [12:10] <_ion> You probably should make the fsck job emit "filesystem-checked" and the mount job "start on filesystem-checked" or something like that. [12:10] I see no reason to way for / not being mounted because /usr is still checked. [12:10] s/way/wait/ [12:11] mbiebl, that's not the problem [12:11] the problem is /usr/share mounted while /usr isn't [12:11] :-) [12:12] Well, then the /usr/share mount job should "start on fsck-checked /usr" [12:12] yeah right :> [12:12] And I do know that from udev events :P [12:13] <_ion> I'm not sure there should be separate mount jobs for separate partitions. [12:13] _ion, there can, not a problem at all [12:13] If I serialise mounting after the fsck [12:13] _all_ of them [12:14] This is actually quite tricky. [12:14] You can't know in advance the partitioning scheme of a user. [12:14] I can emit some events, right. [12:14] yes, but can make educated guesses :P [12:14] So providing upstart mount jobs for each separate partition won't be possible. [12:15] You could, as _ion suggest, do the mounting in on single job. [12:15] <_ion> Better make the mount intelligent: if MOUNTPOINT is /usr/share and there's a /usr entry in /etc/fstab, block until /usr is mounted. [12:15] <_ion> Or something like that. [12:15] Exactly. [12:15] But not on the fstab check [12:15] on mtab [12:15] or rather, /proc/mounts [12:16] <_ion> No, that doesn't contain /usr until /usr is mounted, right? [12:16] fstab will be checked anyway, the mounts will be serialised separately by an external program [12:16] I've temporarily solved the problem already, thank you. [12:17] (I meant they will wait on each other, the scripts launched with an argument) [12:19] Hmm, I could wait on the pids [12:19] <_ion> % deps() { while read fstab_fs fstab_mountpoint rest; do case "$MOUNTPOINT" in "$fstab_mountpoint"*) echo "$fstab_mountpoint"; esac; done If there's /usr in /etc/fstab, that should print it. [12:20] More or less. [12:20] <_ion> That's not a real implementation, it would print "/usr/loc" for /usr/local, but it demonstrates the idea. [12:20] I know what I have to do... [12:21] but the parsing will be done by an external program anyway. === j_ack [n=rudi@p508DA3BD.dip0.t-ipconnect.de] has joined #upstart [12:21] I just wonder, how to emit x events and wait on them all [12:22] Well, wrong actually [12:22] <_ion> for e in foo bar baz; do initctl emit "$e" & done; wait [12:22] AStorm: The basic question is, when can certain jobs start to run [12:22] I can use complex-events and stop on (fsck / and fsck /usr and fsck /var and ... ) [12:22] But that will leave evil messages [12:23] I will launch them using events, thank you. [12:23] But what will wait w/o args do? [12:23] mbiebl, that will be assured by another program and the deps :-) [12:23] <_ion> wait for all children. To wait for only those initctls: [12:24] Great then. Should've read the manpage better. [12:24] <_ion> pids=; for e in foo bar baz; do initctl emit "$e" & pids="$pids $!"; done; wait $pids [12:24] <_ion> (i didn't test that, though) [12:24] Not a problem with "all children" [12:25] The shell is running only these commands. [12:27] Ok, so now the fsck problem is solved (thanks to emitting events) [12:28] per-disk fsck is done too properly