[02:21] <_ion> Does anyone know which tool Keybuk uses for keeping the ChangeLog file updated in bzr? === theCore [n=alex@ubuntu/member/theCore] has joined #upstart === j_ack [n=rudi@p508D8BB1.dip0.t-ipconnect.de] has joined #upstart [04:26] <_ion> Phew. Today was one of the few days of a year when i actually manage to get some code done despite my health issues. I wrote http://soijabanaani.net/tmp/nih_watch_delayed.diff [04:26] <_ion> Writing that took like ten times the time it would have taken if i could concentrate normally. [04:27] <_ion> Here's also a small parenthesis fix, http://soijabanaani.net/tmp/nih_hash_foreach_parenthesis.diff === Amaranth [n=travis@ubuntu/member/Amaranth] has joined #upstart === peeps [n=peeps@cpe-70-112-25-110.austin.res.rr.com] has joined #upstart === peeps [n=peeps@cpe-70-112-25-110.austin.res.rr.com] has left #upstart ["Leaving"] === Amaranth [n=travis@ubuntu/member/Amaranth] has joined #upstart === j_ack [n=rudi@p508D8BB1.dip0.t-ipconnect.de] has joined #upstart === theCore [n=alex@ubuntu/member/theCore] has joined #upstart === AlexExtreme [n=AlexExtr@frugalware/developer/AlexExtreme] has joined #upstart === int0x0c [n=ben@161.253.46.23] has joined #upstart === wasabi [n=jhaltom@ubuntu/member/wasabi] has joined #upstart === Seveas [n=seveas@ubuntu/member/seveas] has joined #upstart === wasabi_ [n=wasabi@ubuntu/member/wasabi] has joined #upstart === jonibo [n=jonas@213.212.2.215] has joined #upstart === _ion [i=johan@kiviniemi.name] has joined #upstart === popey [n=alan@ubuntu/member/popey] has joined #upstart === j_ack_ [n=rudi@p508DA30C.dip0.t-ipconnect.de] has joined #upstart === jonib1 [n=jonas@ua-83-227-144-18.cust.bredbandsbolaget.se] has joined #upstart === phsdv [n=paul@dyn-83-152-239-92.ppp.tiscali.fr] has joined #upstart [11:30] int0x0c, are you trying to get upstart going on gentoo? === juergbi [n=juerg@80-219-17-102.dclient.hispeed.ch] has joined #upstart [11:45] I just uploaded an ebuild for 0.3.5 into gentoo bugzilla (http://bugs.gentoo.org/show_bug.cgi?id=150190) === Md [i=md@freenode/staff/md] has joined #upstart === phsdv [n=paul@dyn-83-152-239-92.ppp.tiscali.fr] has joined #upstart === Cubano [n=jens@0x55510128.adsl.cybercity.dk] has joined #upstart === Keybuk [n=scott@wing-commander.netsplit.com] has joined #upstart [03:09] so, I came up with a neat idea ... [03:09] you know how we have emits ? [03:09] that'd be a really nice way of defining special events [03:10] job lists the event names with the emits stanza [03:10] and when started, sends a message to upstart saying that it wants active notification of registrations [03:10] upstart sees it's from a pid of a known job, that has listed emits stanzas [03:10] so sends the job back as a reply every use of "on THAT-EVENT" [03:10] and whenever new jobs are registered, or jobs are deleted, keeps that process up to date [03:11] e.g. [03:11] a job called "inetd" [03:11] emits tcp udp [03:11] connects to upstart [03:11] and receives information that there are jobs with [03:11] on tcp 80 [03:11] on udp 53 [03:11] etc. [03:11] so it knows to listen on those ports, and knows to emit events with those arguments [03:12] sounds good [03:14] so, there's most of the work for replacing inetd done? :P [03:16] it means you can take exotic events out of the core [03:16] don't want inetd support? don't run upstart-ientd [03:16] yes [03:17] useful for the dbus proxy too [03:17] on dbus com/redhat/dhcp/eth0 com.redhat.dhcp.down [03:17] so it'd know to listen to that signal, and emit that evnet === j_ack [n=rudi@p508D92A9.dip0.t-ipconnect.de] has joined #upstart [04:06] <_ion> keybuk: Ooo, really nice idea. [04:06] <_ion> keybuk: Any comments about the watch_delayed thing? [04:07] I looked at the code, and it looked pretty good [04:07] haven't had a chance to test it yet; want to get complex-event done today since we want that asap [04:07] <_ion> All right. [04:08] will look at it properly this week : ) [04:09] <_ion> You might want to include nih_hash_foreach_parenthesis.diff immediately though, it fixes a bug. :-) [04:10] yeah, I saw that [04:10] thanks! [04:10] can you paste me a changelog for that one [04:12] <_ion> * nih/hash.c (NIH_HASH_FOREACH, NIH_HASH_FOREACH_SAFE): Added missing parenthesis. [04:12] <_ion> Is that enough? :-) [04:13] <_ion> Duh, nih/hash.h, not .c [04:13] <_ion> What tool do you use to maintain the changelog and have it integrated with bzr commits, btw? [04:13] <_ion> A bzr plugin? [04:14] a shell script [04:15] <_ion> Is it available for downloading? :-) [04:15] http://people.ubuntu.com/~scott/software/bzr-commit [04:15] <_ion> Thanks [04:15] I maintain the ChangeLog first using emacs (C-x 4 a) [04:16] and then just use that to do the commits, with the changelog diff [04:16] actually -- can you commit your watch delayed stuffa [04:16] and push it to a branch of your own? [04:18] something like [04:18] <_ion> Actually i did commit it to a branch of my own in the first place, but decided just to send a diff because i hadn't written changelog entries or commit messages using the format you use in your repository. I'll do that now, using that script. :-) [04:19] bzr push sftp://ion@bazaar.launchpad.net/~ion/libnih/watch-delayed [04:19] I think is the format [04:19] that should automatically register it on https://code.launchpad.net/libnih === mbiebl [n=michael@e180118213.adsl.alicedsl.de] has joined #upstart [04:52] <_ion> keybuk: https://code.launchpad.net/~ion/+branch/libnih/watch-delayed [04:54] excellent [04:54] wing-commander scott% ./parse "foo bar baz until frodo bilbo and eep" [04:54] Operator (AND) [04:54] Event ("eep") [04:54] Operator (UNTIL) [04:54] Event ("frodo", "bilbo") [04:54] Event ("foo", "bar", "baz") [04:54] \o/ [04:54] now to figure out whether I've got the precedence right :p [04:54] <_ion> \o/ [04:55] wing-commander scott% ./parse "foo until bar and frodo until bilbo" [04:55] Operator (AND) [04:55] Operator (UNTIL) [04:55] Event ("bilbo") [04:55] Event ("frodo") [04:55] Operator (UNTIL) [04:55] Event ("bar") [04:55] Event ("foo") [05:10] <_ion> keybuk: You should be able to pull the nih/hash.h fix and nothing else by running bzr merge --pull -r 386 http://bazaar.launchpad.net/~ion/libnih/watch-delayed, i think. [05:11] <_ion> The actual delayed watch code is in revno 387. [05:12] wing-commander scott% ./parse "set timer until (boom or kaboom)" [05:12] Operator (UNTIL) [05:12] Event ("set", "timer") [05:12] Operator (OR) [05:12] Event ("boom") [05:12] Event ("kaboom") [05:12] \o/ [05:13] <_ion> \/ [05:13] :) [05:13] <_ion> keybuk: Have you decided a name for the complex-event-spec stanza yet? :-) [05:14] wing-commander scott% ./parse "fhs-filesystem-mounted until fhs-filesystem-down and network-up until network-down" [05:14] Operator (AND) [05:14] Operator (UNTIL) [05:14] Event ("fhs-filesystem-mounted") [05:14] Event ("fhs-filesystem-down") [05:14] Operator (UNTIL) [05:14] Event ("network-up") [05:14] Event ("network-down") [05:14] _ion: nope [05:22] wing-commander scott% ./parse "with apache or with thttpd" [05:22] Operator (OR) [05:22] Operator (WITH) [05:22] Event ("apache") [05:22] Operator (WITH) [05:22] Event ("thttpd") [05:22] yay [05:22] even unary works [05:22] <_ion> Whee [05:23] <_ion> D'oh, i wrote "patch change events" to the changelog entry. :-) [05:24] oops [05:26] ok, I think the parser works [05:27] got to make config respect parens as newline escapers, then try and recodify my ugly quick C into a stanza parser [05:49] interesting question [05:49] should this: [05:49] FOO blah and (blah until blah # some comment [05:49] or blah until blah) [05:49] be valid [05:49] (ie. comment inside parens [05:50] <_ion> I don't see why not. Someone might want to do something like this, while testing something: [05:50] it's kinda unique in the format, since nothing else uses parens [05:50] <_ion> FOO something something (something # comment out the rest of the line, something something [05:51] <_ion> # comment out also this part, something something) [05:51] <_ion> new stuff) [05:51] and I do think not being required to escape newlines inside parens is useful === AStorm [n=astralst@host-81-190-179-124.gorzow.mm.pl] has joined #upstart [06:06] Hello. === arachnist [i=arachnis@paludis/monkey/arachnist] has joined #upstart [06:06] hey AStorm [06:06] _ion: merged 386 [06:07] I've ran upstart on Gentoo, basic compatibility right now [06:07] (running large /sbin/rc, and requires a small modification of one script) [06:07] sweet, do you have the details of what needs to be done? [06:07] I'm developing a fully event-based init system for it. [06:07] planning to ship some example jobs in the upstart tarball directly with the next version, would be cool to have a gentoo subdir in there [06:07] And I've some questions, esp. regarding mounting [06:07] sure [06:08] Is there something like stop for these? [06:08] Like the default plain exec is start [06:08] then stop would be ran on stopping. [06:08] sorry, not sure I understand [06:08] Or do I have to use post-stop? [06:08] You know, I don't want to split the event in two. [06:09] oh, I see what you mean [06:09] upstart's jobs aren't quite analogous to init scripts [06:09] (could do this with unmount-dev script for "on stopped mount-dev") [06:09] I'd rather have it called directly in one script. [06:09] you can do it one of two ways [06:09] Can do this in post-stop right now... [06:09] have two jobs, mount and unmount [06:09] (or in pre-stop) [06:09] one that mounts things, one that unmounts them [06:09] But it feels unclean. [06:09] and have them otherwise unrelated [06:10] or [06:10] you can have them in one job [06:10] related related [06:10] if you have them in one job, put the mount stuff in pre-start [06:10] and the unmount stuff in post-stop [06:10] By the unmount one would dep "on stopped mount-dev" [06:10] and don't have an exec or a script [06:10] the job would appear to be "running" all the time the disk is mounted [06:10] Exactly. [06:10] (note that there's a sliiight bug there atm) [06:11] Bug? What kind of? [06:11] it won't ever actually stop [06:11] hmm. [06:11] it slipped in at the last minute yesterday before I released 0.3.5 [06:11] Right, because it's no daemon and there's no SIGCHLD, right? [06:11] I didn't have a good enough test case for it [06:11] Funny bug. So, for now, I'll split the script. [06:12] right; because when the stop event comes in, it will only change the goal to stop -- and won't force a state change [06:12] Funny. [06:12] this worked a few days ago [06:12] Not a problem with me, but with the users... [06:13] Now to unmount the device, they'd have to "start" the unmount [06:13] not stop the mount :-) A bit illogical. [06:13] would you have one job per mounted disk, or one job for all of fstab ? [06:13] Rather the first one [06:13] right [06:13] But the job would work using an argument. [06:14] for the first one, you'd do something like: [06:14] instance [06:14] start on block-device-added [06:14] stop on block-device-removed [06:14] or something [06:14] Yes, something like that probably. [06:14] the instance bit is important, it tells upstart that multiple copies can be running [06:14] But right now, it's /dev mounting, which is doubly funny [06:14] (temporary fix [06:14] rather than splitting into two jobs, just have a dummy script [06:14] Aha, nice, undocumented. [06:15] pre-start script [06:15] # mount stuff [06:15] end script [06:15] Fine, very fine. [06:15] post-stop script [06:15] # unmount stuff [06:15] end script [06:15] I know. the bug is just when I exec something, right? [06:15] (in full start) [06:15] script [06:15] while true; do sleep 1000; done [06:15] end script [06:16] <_ion> sleep inf [06:16] the script...end script will mean that there's a process running while the disk is mounted [06:16] _ion: does that work? :p neat [06:16] exec sleep inf [06:16] :p [06:17] <_ion> It works with GNU sleep, but not with e.g. the BSD one. [06:17] Keybuk, uh, a bit evil hack, but temporarily I'll do that. [06:17] then this week, when I fix that bug, you can drop the sleep :p [06:17] _ion, not a problem here, it's Gentoo and it's temporary. [06:18] Good, very good. [06:18] the bug is that it won't force a state change out of running if there's no pid; because the child reaper calls job_change_goal and doesn't want a state change as a side-effect [06:18] The major thing will be forcing udev to generate events [06:18] the default rules don't do that [06:18] this turns out to be wrong, simply because the child reaper actually does want to force a state change as a side-effect :p [06:18] so all I need to is fix the child reaper, and remove the bogus check from job_change_goal [06:18] need to write some test cases for it first though [06:19] Just check the state [06:19] after running some echo in pre-start [06:19] and something else in post-stop [06:19] (e.g. some file write), then check if the file was written [06:19] Without any exec inside. [06:20] I wonder if initctl stop thatservice would hang [06:21] sorry? [06:21] The state would change properly to stopping? [06:21] Is it just that the task doesn't enter "waiting" state? [06:22] or something else totally? [06:23] Or just that it finishes too early? [06:24] post-stop gets run in stopping [06:24] and then it should go to waiting [06:25] initctl hangs is probably the bug [06:27] Hmm, the test would be quite hard. [06:27] Because of the hang, you'd have to use shell job control with some timeout. [06:29] damned halting problem, eh? :P [06:34] AStorm, I have made an ebuild for 0.3.5 (in bugzilla). If you needs some help with your new scripts on gentoo, pls let me know [06:35] Well, the help would be wrt making udev send upstart events [06:36] that's easy [06:36] I've overwritten my init with upstart currently, have the stupid way of compatibility (running full gentoo init) working [06:36] AlexExtreme, so then, describe it to me, I'd be grateful. [06:36] AStorm: RUN+="initctl emit some-event" [06:37] ACTION=="add", SUBSYSTEM=="block", KERNEL=="*[!0-9] ", RUN="/usr/sbin/initctl emit block-device-added %k" [06:37] I'm not a master of init rules... [06:37] damn [06:37] beat me to it :) [06:37] phsdv, ok, so simple :-) [06:37] So I'll have to also do a package of udev rules - expected that. [06:37] I had them ready... wrote them a while ago [06:37] phsdv: add "-eDEVNAME" to that [06:37] -eDEVNAME? [06:37] phsdv, could you mail them to me then? (or put somewhere I can get them) [06:38] The parameter, you know [06:38] or environment variable DEVNAME [06:38] phsdv: put the DEVNAME environment variable in the environment of any job run by the block-device-added event [06:38] <_ion> -eBAR without =quux grabs the value from the current environment? Nice. [06:38] Great. It's so bad the upstart is so undocumented yet :-) [06:38] ACTION=="remove", SUBSYSTEM=="block", KERNEL=="*[!0-9] ", RUN="/usr/sbin/initctl emit block-device-removed %k" [06:39] ^ same for remove [06:39] phsdv, I know, I know. [06:39] AStorm: heh, I do have "document it" high on the TODO list [06:39] though remember, a lot of the features you're using today were only written in the last week === j_ack_ [n=rudi@p508DB9F4.dip0.t-ipconnect.de] has joined #upstart [06:39] Keybuk, hehe, I'll do something like that [06:39] when I finish full gentoo init script compat [06:39] which will consist of modified depscan.sh and runscript.sh [06:40] (to store the dependencies in upstart scripts) [06:40] Maybe also some mapping, like: net="network-device-eth0:network-device-eth1" [06:42] The hardest part will be deps of type "use" [06:42] The script would have to check the list of added services (simple, right :P ) [06:43] Keybuk, is it possible to parametrise the dependencies, so I don't have to generate scripts? [06:43] As in the dependency would be a parameter, some environment value? [06:43] <_ion> Well, you could do as Ubuntu does: keep the old compatibility layer there and gradually rewrite things from the old system to upstart. [06:44] I'm just thinking about that. [06:44] Just moving the most basic layer to upstart now [06:44] and write the compat script [06:45] Keybuk, so, is that possible? To base "dependencies" (events I want get started on) on the environment variable or a parameter? [06:45] AStorm: What about documenting your work on the wiki? [06:45] http://upstart.ubuntu.com/wiki/UpstartOnGentoo [06:45] I'll do that :-) [06:45] When I have something working, I'll do an ebuild and document it. [06:46] Great, seems as many gentooers are already playing with upstart and there is much duplicated effort. [06:46] Not a lot, though. [06:47] Upstart is so nice, the amount of "work" required is low. [06:47] (except that possible conversion script - that requires a tiny bit of bash-fu) [06:48] AStorm: I'm not sure I understand what you mean? [06:48] Keybuk, like that: [06:48] There is an event send, let's call it run-me [06:49] It is emitted with an unknown number of parameters: abc def ghi [06:49] And I want to have them converted to: on abc and on def and on ghi [06:50] Maybe a tiny extension of the syntax - explicit and? [06:50] and support for environment vars in calculation of the events/ [06:51] so you have [06:51] E.g. I could do: initctl run-it DEPS="abc def ghi" [06:51] "run-me abc def ghi" [06:51] and do: on $DEPS [06:51] ? [06:51] I don't have that yet :-) [06:51] I'm still trying to understand what you're trying to do, I'm afraid [06:51] s/initctl/initctl emit/ === Keybuk has never seen the Gentoo init system [06:52] it's *very* complicated :) [06:52] Keybuk, it's a simple "need" based system [06:52] AlexExtreme, not really [06:52] With some "after and before" [06:52] so when a job starts, it causes the jobs it needs to also be started? [06:52] it seemed that way to me :P [06:52] Keybuk, exactly. [06:52] or is it the kind where a job is not started until everything it needs has been started by something else? [06:52] Like that old "depend" trick [06:53] Which was obsoleted some time ago [06:53] Keybuk, the second one rather [06:53] in upstart, if "FOO" needs "BAR" then in FOO's job file, you would write: [06:53] start on started FOO [06:53] stop on stopping FOO [06:53] Exactly. [06:53] err [06:53] s/FOO/BAR/ in that :p [06:53] And I'd add in the main one: start on gentoo [06:54] and stop on gentoo [06:54] that one would emit proper events to run things [06:54] in the main one? [06:55] Something like that, right. [06:55] main what? [06:55] A conversion of Gentoo init to rcS, so it seems. [06:55] It's just that the script "gentoo" would call the wrapper for gentoo-style init multiple times [06:55] maybe I need to rewind here [06:55] are you wanting to know how to rewrite current init scripts into upstart native jobs ... [06:56] and this wrapper would get the deps right (precalculated by script) [06:56] ... or are you trying to find out how to write a generic job that can start any gentoo init script ? [06:56] Keybuk, the last one [06:56] aha! [06:56] right, basis of my confusion here then :p; [06:56] This would simulate gentoo runlevels. [06:57] and the smaller script would just launch things [06:57] well, at the most basic level, you need a job that handles the simple case of starting, running and stopping one job [06:57] Exactly, but with parametrised deps [06:57] ideally, that would need to spawn the job in the pre-start script, and then kill it again in the post-start script [06:57] But then, I'd need a wildcard [06:57] and somehow wedge in the running state while the job is actually running [06:58] Yes. [06:58] first you need to write the job, without any deps [06:58] you could do it all in "script" in fact [06:58] script [06:58] /etc/init.d/$JOB start [06:58] Exactly [06:58] That's what I want... [06:58] while [ -f /var/run/$JOB.pid ] ; do sleep 1; done [06:58] but for deps I'd need to match on the JOB variable [06:58] /etc/init.d/$JOB stop [06:58] end script [06:58] No pidfile support [06:58] though that's not quite right, since stopping the upstart job would not stop the gentoo one [06:59] what I might do there is [06:59] Gentoo is using the pids itself in the scripts [06:59] post-start script [06:59] /etc/init.d/$JOB start [06:59] end script [06:59] script [06:59] Just call /etc/init.d/$JOB stop [06:59] Yes, that's it. [06:59] while /etc/init.d/$JOB is-running; do sleep 1; done [06:59] end script [06:59] pre-stop script [06:59] /etc/init.d/$JOB stop [06:59] end script [06:59] "is-running" is impossible yet. Just sleep instead :P [06:59] right, but how do you know if someone stops the job when you weren't looking? :p [07:00] if you don't care, then omit the "script ... end script" bit [07:00] Well... [07:00] (and replace with exec sleep inf until I fix this bug) [07:00] Exactly, don't care. [07:00] ok [07:00] Stupidity is not rewarded :P [07:00] so this gives you a job that will start and stop any gentoo job when it, itself, is started and stopped [07:00] And init scripts are disallowed to call stop on others explicitly [07:00] (Gentoo policy) [07:00] of course, you need to mark this "instance" because you want multiple copies running [07:00] Yes. [07:00] But I need a match on $JOB [07:01] now, you'll want an event to start up an instance for a given job [07:01] and another event to stop an instance for the same job [07:01] easiest way to define that [07:01] to start its deps earlier [07:01] initctl emit gentoo-start -eGENTOO_JOB=NAMEOFJOB [07:01] initctl emit gentoo-stop -eGENTOO_JOB=NAMEOFJOB [07:01] Yes. [07:01] so stick in the gentoo job file [07:01] start on gentoo-start [07:01] stop on gentoo-stop [07:01] But then, the deps are to be given as the parameters still [07:01] (because of the need system) [07:01] and use $GENTOO_JOB throughout [07:01] or the single script won't work [07:01] (getting to dependencies) [07:02] Can I do a match like: [07:02] on gentoo-start GENTOO_JOB=xyz * [07:02] no [07:02] or would the DEPS variable be ignored? [07:02] (if I don't match on it) [07:02] question [07:02] how would you like deps to be handled? [07:03] if you start FOO, should it cause its deps to be started? [07:03] Yes. [07:03] right [07:03] and then foo itself. [07:03] so now we improve the "gentoo-start" event [07:03] Yep. [07:03] initctl emit gentoo-stop -eGENTOO_JOB=NAMEOFJOB -eGENTOO_DEPS="dep1 dep2 dep3 dep4" [07:03] uh [07:03] s/-stop/-start/ :p [07:03] Not a problem. [07:03] now that instance, when it gets started, has $GENTOO_JOB which is the job we actually want to run [07:04] (used in post-start and pre-stop) [07:04] I'd love to avoid parsing the GENTOO_DEPS though. there's no way yet, right? [07:04] (like just putting on started $GENTOO_DEPS) [07:04] Or rather: [07:04] when $GENTOO_DEPS [07:04] and that instance also has $GENTOO_DEPS [07:04] so the simplest way is to add [07:04] pre-start script [07:04] which would expand to when dep1 and when dep2 and when dep3 [07:04] for DEP in $GENTOO_DEPS; initctl emit gentoo-start $DEP; done [07:04] end script [07:05] that wouldn't do what you want [07:05] that would emulate the second kind of system [07:05] It would do it too :-) [07:05] Ah, right. [07:05] the need would add when [07:05] the use would add on [07:05] :-) [07:05] that's the catch [07:05] Or even better. [07:05] use would still add when [07:05] after would add on [07:05]