=== Amaranth [n=travis@ubuntu/member/Amaranth] has joined #upstart === rgl [n=Rui@84.90.10.107] has joined #upstart === esr [n=chatzill@static-71-162-243-5.phlapa.fios.verizon.net] has joined #upstart === esr [n=chatzill@static-71-162-243-5.phlapa.fios.verizon.net] has joined #upstart === esr [n=chatzill@static-71-162-243-5.phlapa.fios.verizon.net] has joined #upstart === esr [n=chatzill@static-71-162-243-5.phlapa.fios.verizon.net] has joined #upstart === Amaranth [n=travis@ubuntu/member/Amaranth] has joined #upstart === esr [n=chatzill@static-71-162-243-5.phlapa.fios.verizon.net] has joined #upstart === esr [n=chatzill@static-71-162-243-5.phlapa.fios.verizon.net] has joined #upstart === Amaranth [n=travis@ubuntu/member/Amaranth] has joined #upstart === jonibo [n=jonas@ua-83-227-144-18.cust.bredbandsbolaget.se] has joined #upstart === Starting logfile irclogs/upstart.log === ubuntulog [i=ubuntulo@ubuntu/bot/ubuntulog] has joined #upstart === Topic for #upstart: Upstart 0.3.8 | http://upstart.ubuntu.com/ | http://upstart.ubuntu.com/wiki/UpstartOnGentoo === Topic (#upstart): set by Md at Sun May 6 19:59:46 2007 === Md [i=md@freenode/staff/md] has joined #upstart === rgl_ [n=Rui@84.90.10.107] has joined #upstart === AlexExtreme wonders whether it would be better to define profiles in seperate files for each profile rather than a single file [11:24] it would be much easier to work with IMO [11:25] profiles are interesting [11:25] one could almost define them as a state in their own right [11:31] yes [11:31] which means you'd be able to turn them on/off during system runtime [11:31] if I were to use seperate files for each profile, what would be a good directory name for the profiles to live in? i thought /etc/bootprofiles.d, but a) it's a bit long b) profiles wouldn't just be used during boot :) [11:36] hmm, just a random thought: how about a directory structure like this for upstart: [11:36] /etc/upstart [11:36] |-----> profiles.d/ - Profiles defined here [11:36] |-----> jobs.d/ - Jobs defined here [11:36] |-----> states.d/ - States defined here [11:37] So instead, everything would live under /etc/upstart/something rather than scattered in many directories in /etc [11:38] I've been thinking of that, yeah [11:38] and /etc/event.d is incorrectly named anyway, since they aren't events really, they're jobs and states ;) [11:39] yeah [11:39] I'd thought of /etc/init instead of /etc/upstart [11:39] maybe [11:46] shall I write a spec for a new directory structure? ;) [11:46] I don't think it needs one at this point ;) [11:46] it's a one-line code change [11:47] :P === Keybuk debates the ".d" [11:48] the .d is unnecessary, really [11:49] it'd only be necessary really if there was something like: /etc/upstart/jobs - some static file, and /etc/upstart/jobs.d - a directory to do with that static file [11:49] since that static file is not there, and most likely won't ever need to be there, the .d is unnecessary [11:52] yeah that's what I thought [11:52] though it does fit other standard naming [11:52] e.g. /etc/udev/rules.d [11:52] and upstart will actually support single-conf-files [11:53] (I'm gonna push for 0.3.9 at the weekend, while my partner is away [11:53] cool [11:53] so what super-duper-cool features are we gonna have? ;) [11:54] - portability fixes [11:54] - bug fixes [11:54] - works when there's no console support by default (openvz, etc.) [11:55] - internal rework of Event [11:55] - multiple configuration file and directory support [11:55] - "reload" support for configuration [11:55] - new initctl commands to replace use of signals [11:55] - new config layout [11:56] that lot [11:56] - new config layout << what's that one? [12:01] see above ;) [12:01] we just agreed on that, no? :p [12:01] oh [12:02] i thought that was this one: - multiple configuration file and directory support [12:02] anyhow, sounds good :) [12:03] that one is the code support in cfgfile to support tracking what config directories/files have been parsed, and which jobs were found in them [12:03] looks something like [12:03] cfg_source_new (NULL, "/etc/init/jobs.d", NULL); [12:03] cfg_source_new (NULL, "/etc/init/states.d", NULL); [12:03] cfg_reload (); [12:04] ah === Amaranth [n=travis@ubuntu/member/Amaranth] has joined #upstart [12:27] whoo, written my basic profile parser :p === Md [i=md@freenode/staff/md] has joined #upstart [12:54] Keybuk, 0.3.9 will work inside a linux-vserver guest? === rgl_ is now known as rgl [12:59] rgl: in theory [01:00] Keybuk, it will use syslog instead of /dev/console ? [01:03] no [01:03] it reopens /dev/console properly [01:05] Keybuk, humm, but there is no /dev/console [01:07] there's always a /dev/console [01:07] no there isn't ;) [01:08] you have to either link it to /dev/null, or start up syslog-ng to create a pipe at /dev/console [01:08] (linux-vserver does not have a virtualized /dev/console) [01:08] it still exists in the kernel [01:09] sure. but last time I've checked, its not advised to export it to guests. [01:09] syslog won't be running, because this is init :) [01:11] humm, good point *G* [01:11] well, what does sysvinit do? :) === jonibo [n=jonas@ua-83-227-144-18.cust.bredbandsbolaget.se] has left #upstart [] [01:13] by default, a linux-vserver does not start init. it just runs /etc/init.d/rc [01:14] but we can make it run init. but last time I've tried, I failed to make upstart work at all :-( [01:14] AlexExtreme: opens /dev/null [01:14] rgl: then what is the parent of all the processes? surely when rc exits it'll drop dead since process #1 has died === Md [i=md@freenode/staff/md] has joined #upstart [01:15] AlexExtreme, there is a "virtual" #1. the guest will stay awake until all its processes die. (and not when #1 dies) [01:15] ah [01:16] what reaps children? [01:16] I think its that "virtual" #1. [01:17] but I don't known for sure. there is some magic/vudu out there *G* [01:17] anyways, last night, I couldn't make upstart 0.3.8 do what I want :-( [01:18] can you guys look at http://pastie.caboo.se/66469 and tell me if I'm doing something wrong? [01:18] namely, when I "start php-fcgi", the "php-fcgi-1" does not start at all. [01:31] I don't see why that shouldn't work [01:32] run in another window [01:32] "sudo initctl events" [01:32] then do "start php-fcgi" and paste the output of initctl from the other window [01:33] OK. give me a sec to fireup vmware. :D [01:36] Keybuk, odd. it now works! [01:37] Keybuk, do I have to restart upstart after I add a new service file or something like that? [01:40] shouldn't need to [01:40] 0.2.7 had bugs where it didn't notice [01:40] 0.3.8 should work just fine [01:40] if you have experienced bugs, we'd like to get them fixed :p [01:42] I can't reproduce the problem I had yesterday. but sure, if I found some bug I'll let you known. [01:43] I found one quirk that is bugging me, if a process fails to restart for five(?) times in a row, upstart never tries to launch it again, it that normal? can I config it? [01:43] right [01:43] you'll see a "respawning too fast" type warning [01:43] I think it's attempting to respawn 5 times in 10 seconds [01:44] it's configurable [01:44] "respawn limit COUNT PERIOD" [01:44] e.g. [01:44] respawn limit 100 10 [01:44] (stop job if it respawns more than 100 times in any 10s period) [01:44] ah nice :D [01:45] if either of them are 0, then the respawn catching is disabled [01:45] works on 0.3.8? :D [01:45] most things work on 0.3.8 [01:46] cool :D [01:46] what you mean respawn catching is disabled? the service is always restarted? [01:47] yeah [01:47] service is allowed to restart as many times as it likes [01:47] why were you hitting the respawn limit out of interest? [01:47] usually that means that the thing you're exec'ing is dying [01:47] which is bad, no? [01:48] yes it is a bad thing :D [01:48] why was it dying? [01:48] I was trying it :D [01:48] to see how upstart reacted when the service dies [01:49] ah right [01:49] a few notes from scrollback [01:49] "service" [01:49] and "start" doesn't exit [01:49] Upstart has two basic types of jobs [01:49] Tasks and Services [01:49] Tasks are things like "run fsck", or "backup my home directory", etc. [01:49] Services are daemons and stuff [01:50] for tasks, "start" deliberately doesn't exist until it's complete (and events they "start on" aren't deemed finished), since the task is not successful until it exists [01:50] s/exist/exit/ [01:50] for services, you want "start" to exit (and events to finish) when it is running, since that's when it's successful [01:51] (a successful fsck is a finished one, a successful web server is a running one) [01:51] they're otherwise identical in configuration [01:51] Upstart jobs default to being tasks [01:51] if you want them to be services, you need to put either "service" or "respawn" in the config [01:52] woah, thx! that explains things :-) [01:53] there's a third option as well [01:53] but this is not fully implemented yet [01:53] (supervision on processes that fork into the background) [01:53] -- [01:54] why failing services still say "started" [01:54] this is because from Upstart's point of view, it has started -- the script is running [01:54] there's no internal way (yet) for the service to notify Upstart that it is, in fact, really running and really ready [01:54] there's two planned improvements here: [01:55] 1 - better catching of exec() failures. Right now Upstart forks the child and assumes it's running; exec() failures aren't special. This will change so that the exec() has to finish before Upstart moves the job out of the spawned state (and thus can move directly into stopping if the exec() failed for some reason) [01:56] 2 - libupstart readyness notification. services that take a while to be ready will be able to use libupstart to notify upstart that they're accepting connections, etc. if this fails to happen, they can be considered failed, killed, and restarted [02:00] I see. thx for the explanation :-) [02:00] are you also planning on having monit alike-features? :D [02:02] no [02:03] no particular reason to, since monit already has those features [02:03] you can use monit to do the monitoring, and have it tell upstart to restart services, etc. [02:04] ah ok. [02:04] maybe monit could be enhanced to do real time monitoring by listening to upstart events. [02:04] I'm a full believer in the "do only one thing well" [02:04] exactly [02:04] same for heartbeat, et al [02:05] it makes no sense for upstart to grow the ability to communicate across serial ports, etc. [02:05] it makes far more sense for the services that already can do that to link to libupstart and emit events, or even start/stop processes themselves [02:05] if you look at initctl, it's just a wrapper around libupstart [02:05] so any process can do whatever it can do [02:05] (ie. list services, start/stop them, emit events, etc.) [02:06] makes sense :) [02:07] what is libupstart? [02:07] I mean, I see no libupstart package [02:07] the IPC library [02:08] see upstart/message.h in the source code [02:08] it's not separately packaged yet, because the API isn't anywhere near stable [02:08] ah, so its not yet packed as a .so file. [02:08] right, it's an .a file in the source linked to by initctl [02:09] anything initctl does is through libupstart messages [02:09] e.g. "initctl version" [02:10] that's done by something like: [02:10] int sock = upstart_open (); [02:10] NihIoMessage *msg = upstart_message_new (NULL, UPSTART_INIT_DAEMON, UPSTART_VERSION_QUERY); [02:10] nih_io_message_send (msg, sock); [02:11] msg = nih_io_message_recv (NULL, sock, NULL); [02:11] upstart_message_handle (NULL, msg, { UPSTART_VERSION, my_func }, NULL); [02:11] my_func (..., const char *version, ...) { [02:11] printf ("%s\n", version); [02:11] } [02:12] sweet :D [02:12] what is NIH? (not invented here? *G*) [02:12] NihIoMessage *msg = upstart_message_new (NULL, UPSTART_INIT_DAEMON, UPSTART_I_NEED_CAFFEINE_NOW); [02:12] yes [02:12] ion_: UPSTART_EVENT_EMIT, "i-need-caffeine" :p [02:13] :-) [02:13] rgl: libnih is my "Copy & Paste" code library [02:13] that stack of code that most developers have of useful bits and pieces that don't quite exist anywhere else [02:13] it's roughly akin to glib, or the hackerlab lib, etc. [02:13] ah, good old misc. utils lib :D [02:14] I should put that in the FAQ? [02:14] "Q. Why do you use libnih and not glib?" [02:14] "A. typedef int gint" [02:14] Hehe [02:14] whats the pun? :D [02:15] pun? [02:15] I didn't get the answer *G* [02:15] so I suppose its a pun or something like that [02:16] glib provides alternate names for all of the C standard types [02:16] allegedly for portability [02:16] so instead of "int", you have "gint" [02:16] and instead of "char", you have "gchar" [02:17] so what? [02:17] it's silly [02:18] "gpointer is easier to use than void *" [02:18] no it's not [02:18] its like having DWORD, UNIT and the like, no? :D [02:18] it's longer to type, and it's not very C-ish [02:18] DWORD makes sense, since it varies between platforms [02:18] and needs to be defined differently in different circumstances [02:19] the "WORD" is "DWORD" is different for each CPU I guess. so its like silly too. unless WORD is not as in CPU WORD. [02:20] glib tries to hide C from the programmer [02:20] you don't program in C, you program in "glib" [02:20] this strange OO-ish variant of C [02:20] this, to me, seems silly [02:21] C with gobjects :D [02:27] why do jobs default to task instead of service? [02:29] Id expect most of the jobs to be tasks eventually. [02:33] rgl: had to pick one [02:33] why not make it explicit? [02:33] if "service" was the default, you'd need a "task" stanza [02:34] and then you'd have the issue where you'd have to error if both "task" and "respawn", or "task" and "daemon" were specified [02:34] with "task" as the default, there's no need for any conflicts checking - "service", "respawn" and "daemon" are all possible options; and any of them implicitly enable "service" if not already specified [02:35] explicit is the likely end-point, yeah [02:35] but then that's icky, because you'd need "file doesn't state any of 'task', 'service', 'respawn', 'daemon', 'state', 'profile'" etc. [02:35] another option is to configure them in different directories [02:35] /etc/init/tasks.d [02:36] /etc/init/services.d [02:36] or "unknown job type" *G* [02:36] In the long run, multiple directories might be good, as the number of jobs will go up. [02:36] of course, multiple directories has the namespace issue [02:37] True. [02:37] /etc/init.d/task.d/foo [02:37] /etc/init.d/service.d/foo [02:37] which wins? [02:37] humm, havin different directories means you'll have different event namespaces? that is, how you'd wait for a task/service? [02:37] yeah *G* [02:37] Neither. Upstart gives an error and doesnt use either one. [02:38] or foo.task foo.service *G* [02:39] rgl: ick [02:39] :p [02:41] hehe [02:41] one thing I would like is "nested" services, that is, php-fcgi, php-fcgi/1, etc :D === rgl hides === AlexExtreme deserves to be slapped right now. i spent half an hour figuring out why my profile parser was segfaulting, then realised i wasn't putting braces round the else block so only the first line was being run for it, the rest would always be run >< [02:47] rgl: we kinda have that [02:47] your php-fcgi is an excellent use case for something else we've been thinking of [02:47] and let me explain by first talking about that [02:47] right now we have /etc/event.d/tty[1-6] [02:47] 6 files, all of which do the same thing, with different arguments [02:47] and each one would need something like [02:48] while tty-added tty1 until tty-removed $TTY [02:48] exec /sbin/getty 38400 tty1 [02:48] eurgh [02:48] it would be better to have one getty job that runs 6 times, with different arguments [02:48] we have this kind of thing already [02:48] "instance" jobs [02:48] if you put the "instance" keyword in the config file, then every "start" will spawn off a new instance [02:49] so "status php-fcgi" would return as many lines as "start" has been called [02:49] likewise for events [02:49] so, we could have a single getty job: [02:49] instance [02:49] while tty-added tty[1-6] until tty-removed $TTY [02:49] exec /sbin/getty 38400 $TTY [02:50] that'd get run 6 times, since six matching "tty-added" events would occur [02:50] and each one would run until its associated tty was removed [02:50] your example would work very well with file events [02:50] instance [02:51] while file-exists /var/run/php/fcgi.* [02:51] exec /usr/bin/php5-cgi -b $FILE [02:51] you'd get as many events as you had matching files, and thus as many instances of that job [02:51] "status" would show them all [02:51] "start" would have to specify which file it wanted to start with ... [02:52] # initctl start php-fcgi FILE=/var/run/php/fcgi.foo === tale [n=tale@207.235.54.1] has joined #upstart [02:52] "stop" would have to specify which instance to stop; or "stop php-fcgi" would stop them all [02:55] hum, so to start an instance we just need to touch a file? and to stop it we just need to rm it? [02:55] yeah [02:56] all FS have notifications? [02:56] inotify is in the VFS layer, so yeah [02:57] you are targetting linux only? [02:57] absolutely [02:57] Upstart is unashamedly dependant on features of the Linux kernel [02:57] okay :D [02:58] I think the above is a pretty elegant solution, what do you think? [02:59] its pretty neat :D [03:00] at least for this particular use-case :-) [03:00] can I have it now? ;-) [03:01] soon [03:01] the instance bit works now [03:01] the tying instances to events bit depends on complex-event-config [03:01] and you'd want file events as well, which are todo [03:02] cool :D [03:03] bah [03:03] Keybuk, can you help me with NihList quickly? :) [03:03] sure [03:04] hang on [03:06] http://frugalware.org/paste/1453 << ok, that's my code. it's probably really ugly, but anyway :) the loop at the bottom goes through each of the things i've added to the list and prints them out, but it prints out complete rubbish and uses the wrong value for disabled. so obviously i'm adding stuff incorrectly. it's mostly based on what i've seen from notify.c in upstart [03:06] s/disabled/enabled/ [03:09] style: "NihList entry" is nominally the first thing in a struct [03:09] (nothing assumes this, but it's just a style point) [03:09] heh [03:09] well [03:09] that kinda fixed it :p [03:09] you don't need NIH_LIST_FOREACH_SAFE [03:09] just NIH_LIST_FOREACH will do [03:09] aha! [03:10] yes, the iter points you're getting are pointers to the entry member [03:10] so you can't just cast them to upstart_profileentry_t [03:10] (this is the reason I always put the NihList entry first ) [03:10] :) [03:10] well [03:10] now i'm getting this [03:10] [14:09:35] [ alex in ~/Frugalware/work/upstart/profile ] $ ./profile default [03:10] Parsing profile default... [03:10] * Everything will be enabled by default but can be overridden [03:10] * network will be enabled no matter what [03:10] but, it doesn't seem to recognise the fact that there is also an apache entry there somewhere [03:11] [14:10:39] [ alex in ~/Frugalware/work/upstart/profile ] $ cat default [03:11] * = y [03:11] apache = y [03:11] network = n [03:11] you free the job in the loop ;) [03:11] oops [03:11] :p [03:12] [14:11:22] [ alex in ~/Frugalware/work/upstart/profile ] $ ./profile default [03:12] Parsing profile default... [03:12] * Everything will be enabled by default but can be overridden [03:12] * apache will be disabled no matter what [03:12] * network will be enabled no matter what [03:12] yay \o/ [03:12] thanks ;) [03:12] now to make it check against files in /etc/event.d [03:13] "stop on runlevel [!1] " is that a stop on runlevel not 1 ? [03:13] yes [03:13] is so, why the [] ? [03:13] fnmatch() [03:13] standard UNIX glob(7) syntax [03:14] the runlevel arg is a glob? [03:14] yes [03:14] ah alright :D [03:14] arguments to events in "on" are globs [03:14] I can have: stop on runlevel 1 2 3 ? [03:14] so that means stop on any runlevel event with a single-character argument that is not the character "1" [03:14] no [03:14] stop on runlevel [123 [03:14] stop on runlevel [123] [03:14] the runlevel event has a single argument, the runlevel being entered [03:15] ACK :) [03:17] I realised the other day that the runlevel event should have a RUNLEVEL and PREVLEVEL environment variable [03:17] then I could do away with all of the code in the rc jobs [03:18] runlevel would update utmp and emit the event itself [03:19] err, telinit would [03:20] then I could do away with runlevel --set [03:40] how do I make sure the admin does not start an service because another one is not running? [03:40] something like dependencies. [03:42] oh well, scrap that. if it depends on other, and that one is not started, the start of the service will eventually fail because of that. so the admin will notice that somehing is not right. [03:44] how do you mean [03:44] right [03:47] hooray! http://frugalware.org/paste/1454 [04:02] Keybuk, what is the behaviour of jobs that are symlinks? I mean, I have a symlink: php-fcgi-3 -> php-fcgi-1 and when I change php-fcg-1 job file, and do a: stop php-fcgi-3 ; start php-fcgi-3; upstart does not reload the job file. Is that intended? [04:03] err [04:03] yes :-) [04:03] iirc inotify has some bugs with symlinks [04:04] either we need to handle symlinks specially, or inotify needs to handle them specially [04:04] use hardlinks for now :o) [04:04] ok :D [04:04] kernel command line arguments that the kernel doesn't use are passed to init, right? [04:05] AlexExtreme: foo=bar arguments are set in init's environment [04:05] AlexExtreme: anything else is an argument, yes [04:06] ah, cool. so if i were to do what my profiles spec says on the kernel command line, i'd get the requested profile in the environment? [04:08] right [04:08] except upstart doesn't pass those yet (bug) [04:18] Keybuk, ouch, even hardlinks do not work *G* [04:18] heh [04:18] really? [04:18] interesting [04:19] yup, really. I had to touch the file to upstart to reload it. [04:20] that's odd [04:20] since touching the file is the same effect as editing it, no? :p [04:20] maybe it's touch-by-name [04:21] the point is. if I edit the config file (say, on hardlink 1), I have to touch all the other hardlinks too. [04:23] so I really need that "instance" feature you've talked yearlier, or else I might be biten in the arse someday heheh [04:29] i don't need upstart to pass them to the jobs, i need it from upstart itself :) [04:29] (the environment vars) [04:43] hmm [04:45] is the profile override on the kernel command line thing that I put in the profiles spec really necessary? I can't really think of a valid use case for it [04:45] dunno, I can't remember what it was for? [04:46] The profile can be overridden at boot-time by specifying extra kernel command line arguments, like the example below. It tells upstart to use the "noserver" profile but overrides what the profile defines should be done with mysql and postgresql: [04:46] * [04:46] bootprofile=noserver override=mysql:no,postgresql:no [04:46] it's just the override part i'm referring to [04:46] oh right [04:46] you should definitely be able to change the profile at boot :) [04:47] and after booted? :p [04:47] yes, of course :P [04:47] actually === AlexExtreme rewrites the spec ;) [04:47] which begs the question, do the jobs respond to the events, and just not start [04:47] so when you re-enable the profile, they start immediately [04:48] IMO they shouldn't immediately start [04:49] why not? [04:49]