[00:04] well anyways who cares. taking a look around... most of my systems have dozens or even hundreds of days of uptime [00:05] i mean that's great for a laptop but you know what i hardly ever reboot that too. mostly just for kernel updates [00:06] i was thinking the same thing... looking at the lsb-init headers... those contain plenty of dependency information. i never knew init to actually use that data though, found in that header [00:07] and now oracle bought ksplice so maybe they'll mainstream it and i'll just never reboot :) [00:09] i guess over time after lots of updates you accumulate lots of krufty deleted inodes though still in use... but there's kexec anyways [00:13] i actually think upstart is a little overly complicated too but it's quite capable [00:14] still newbie will have really hard time understanding it i think [00:14] unlike sysvinit [00:15] i mean charts and graphs showing state transition diagrams, a cookbook, wikis, huge man pages... [00:16] the gentoo system actually looks pretty simple [00:17] i'll tell you though... it is definitely nice that i can reboot my system in about five seconds after the bios runs... actually less, seems around 3 seconds [00:17] i mean my laptop (which is on ssd) [00:17] (and i don't run any junk) [00:18] i still have problems though with reboot, just hangs when i do shutdown -r now [00:19] doesn't seem to run the rc with 6 as arg [00:26] i have to go in another terminal and run /etc/rc.d/rc 6 manually [01:42] er, that was /etc/init.conf not /etc/init/init.conf [05:10] hi [05:11] i have a job that has multiple instances, and another job that starts those instances [05:11] i'd like to be able to emit an event that causes all the instance jobs to restart [05:11] http://dpaste.com/636318/ is my instance job conf [05:11] the two important lines are 12 and 13 [05:12] i was under the impression that if you have a "stop on" and "start on" stanza that has the same requirement, the "stop on" will be performed first, followed by the [05:12] "start on" [05:13] however when i do "initctl emit genocide DOMAIN=foo", it job instance stops, but it does not start [05:13] i presume this is something to with it not having the ${COMMAND} parameter? [05:15] i'm just looking for some help :) [10:45] hi! is there a best practice how to handle processes that are not able to fork themselves into background? [10:47] when doing "service xyz start", upstart always tells me "xyz start/running, process 12345"... even when the process has an exit status != 0 [10:54] so, the Upstart job you're trying to start via the SysV interface starts then exits with status != 0? What is "xyz" and what's the .conf file look like? [11:15] jhunt: like this: http://pastebin.com/USY8B5nb [11:15] jhunt: so exit status 1 and 2 are examples [11:21] jhunt: oh and xyz is a Go program, Go unfortunately doesn't support programs that can daemonize themselves :S [11:21] (yet) [11:23] I can reproduce the same effect with this program: http://pastebin.com/Qfm7Ps9a [11:26] If these programs are not long-running, you need to use the "task" stanza. [11:27] Yes, I've seen that. I mean under normal circumstances the program will run for weeks. [11:29] But this scenario might happen: I stop the daemon, a small feature is added to the software or its configuration, and I start the daemon, thinking everything is ok... but it failed right from the start. Or another scenario: the server is rebooted and the server admin forget to check whether all services are running perfectly. [11:30] well, it sounds like the go program is forking atleast once, but you haven't told upstart how many times ("fork" for 1, or "daemon" for 2). You might want to strace xyz on the command-line and see how many forks it's doing and then update the .conf accordingly. [11:32] No it's not... actually I can reproduce the problem with this C-program: int main() { return 1; } [11:33] (but yes, upstart manages do daemonize the program nicely, in case everything is ok) [12:35] I am rather confused by what you've said so far. To be clear - you are starting daemon xyz, Upstart says it is running, but you are saying it has actually exited (even though you say it should be long-running)? [12:35] your main example clearly isn't a daemon so can't be compared with the daemon scenario [13:03] jhunt: precisely [13:04] jhunt: why not? [13:04] jhunt: when you have a daemon that does no forks and that exits right after starting because of an error, this is the same [13:05] jhunt: just to be clear: any daemon, even those that fork one or two times, might have a fatal error *before* doing any forking [13:07] the point is, my "daemon" is not to throw away the "main()". so there is no forking involved in my scenario [13:07] s/is not to/is not able to [13:08] though obviously upstart does forking [13:15] maybe this mininmal version of my "daemon" (I prefer to call it server, anyway) is less confusing: http://pastebin.com/BtzM92LP [13:20] hi, is it possible to set dependencies for runlevel 6? ie. i want to execute a script on shutdown/reboot but ensure that rsyslog isn't stopped until the script finishes [13:22] I think your confusion comes from the fact that Upstart *does* successfully start your daemon. At some later date, your daemon decides something went bad and exits. However, you haven't told Upstart to auto-restart it ("respawn"). If admin intervention is required to say fix the daemons broken config, respawn won't help :) [13:22] philip__: maybe you could use a pre-start to perform some sort of sanity check on the config, print a message and fail. [13:23] * jhunt makes mental note to self to update the cookbook on some of these issues. [13:24] jhunt: ;) [13:24] negev: if you're using Ubuntu, you could make your script "start on stopping rsyslog". Since rsyslog stops on shutdown/reboot, your script will run before rsyslog stops (and before the system shuts down/reboots). [13:27] philip__: I can only think that the machine you're running that http://pastebin.com/Qfm7Ps9a example on is incredibly slow since you wouldn't normally even see a pid if the job immediately exits. [13:33] jhunt: yeah, though I wonder whether it's the 3.3Ghz cpu or the 16 Gigs ram... sorry just joking :D [13:33] jhunt: but this is interesting, so you say the timing is the problem... [13:34] cool thanks jhunt [13:50] jhunt: that doesn't seem to work; i put this http://pastebin.com/zW2s1F4z in /etc/init/myshutdown.conf [13:50] its executing the script but the syslog isn't happening, rsyslog must have already stopped when it runs [14:00] negev: are you sure your script is running successfully to completion? Try running "sh -n /etc/init.d/my-shutdown.sh" and checking that it is really doing what you think. [14:00] negev: See http://upstart.ubuntu.com/cookbook/#develop-scripts-using-bin-sh [14:00] negev: and http://upstart.ubuntu.com/cookbook/#debugging-a-script-which-appears-to-be-behaving-oddly [14:03] well i put "touch /home/negev/test" at the top of the script and the file is being created so it must be running [14:04] negev: no - *that* line is running, but subsequent lines might be failing due to Upstart running your script using "sh -e". [14:05] -e or -n? -n says it doesn't execute the commands, it just reads them [14:06] it works if i execute it with -e, fails with -n [14:06] I've suggested -n to ensure your script is syntactically correct (as far as the shell can check without running it). [14:06] the script is just one line that executes a ruby script [14:06] the bang line is correct and the ruby script is mode 755, ive even tried prefixing it with /usr/bin/ruby [14:07] should i maybe just put the ruby script and its argument in the script section of the init config file? [14:08] negev: any particular reason you're sourcing the script? [14:08] negev: I was wondering why you weren't :) [14:08] ok well i just did that, still didn't work though :( [14:08] oh do i need that . before it ? [14:09] If you're asking, the answer is "no" :) [14:10] hrmm then why is it still failing :( [14:10] kind of annoying that this server takes forever to boot [14:11] have you traced it with -x as I suggested above? [14:11] i can't see -x anywhere in the history :/ [14:11] also, can you not just try "start " (force it to run without waiting for rsyslog to be about to stop). [14:12] yeah doing that works fine [14:14] could it be that the "stopping rsyslog" event never happens and rsyslog is just killed without being gracefully stopped? [14:14] negev: oops! here it is - http://upstart.ubuntu.com/cookbook/#obtaining-a-log-of-a-script-section [14:27] all i got in that log was a + before the line that executes the ruby script [14:27] nothing else at all [14:29] Have you tried "start "? [14:29] yeah i said, that works fine [14:30] and its definitely executing when i reboot or shutdown because if i put "touch " at the top it creates the file [14:30] it just doesn't log anything [14:31] It's possible /etc/init.d/sendsigs might be killing your ruby process/rsyslog if they are taking "too long" to finish. You could put a temporary sleep in sendsigs to establish that fact. [14:31] that seems unlikely given that it runs and finishes instantly if i run it manually [14:31] all it does is write an entry to the syslog and then exits [14:32] negev: um, any reason you're using ruby to do that? [14:32] we have one script that does loads of things, it seemed sensible to add this to it as we already use it for other stuff which also logs to syslog [14:33] rather than having multiple scripts [14:40] negev: what does "status rsyslog" show? [14:40] rsyslog start/running, process 1071 [14:44] not sure what else to suggest without seeing what you're actually running. How about changing shutdown.conf to call 'exec logger -i -t negev "hello"' just to see if syslog is usable at that point. [14:44] negev: you could also have shutdown.conf run "ps -ef" or something to convince yourself the logger is still running. [14:45] negev: is it possible your ruby script is making use of some other service which has already stopped when your shutdown job is started? [14:45] possible i guess, although i don't think it is [14:45] i was just looking at simplified syslogging from the cmdline, worth a try [14:46] negev: also, see if the behaviour you are seeing is consistent across systems. [14:49] well these three servers were installed and built identically so probably not much point testing the other two lol [14:49] i tried putting: logger -i -t oe "system shutdown" [14:49] sorry, exec logger -i -t oe "system shutdown" [14:49] in the script section [14:49] didn't work ;9 [14:49] :( [14:50] well at least didn't make it to the remote syslogs, need to wait for it to come back up before i can see if it got it locally but i doubt it [14:50] im convinced rsyslog is already gone by the time this execugtes [14:51] nope, nothing in the local syslog either [14:52] interesting, if i stop rsyslog manually it doesn't execute either [14:53] sh -n exec logger -i -t oe "system shutdown" gives: sh: Can't open exec [14:55] negev: negev put the commands in a file and run "sh -n file" [14:56] including exec? [14:57] exec is recognized by the shell and by upstart. [15:07] sh -n