[09:53] <bottiger> I've tried to manage my upstart services with both "rcconf" and "bum" to make some services start (or not start) automatically. However, it simply does not work. If I disable "mysql" it shows up as disables in both rcconf and bum, but it still starts up when I boot
[09:54] <bottiger> any idea what the problem could be
[13:12] <sadmac2> bottiger: that's more of a debian/ubuntu question
[15:41] <donEduardo> hi there.
[15:41] <donEduardo> is this the correct group for upstart in ubuntu?
[15:41] <donEduardo> or is this group rather general?
[15:42] <Keybuk> this is the general group for the different distribution maintainers developing upstart
[15:42] <Keybuk> if you're having ubuntu problems, you should ask in #ubuntu
[15:47] <donEduardo> if got an upstart problem in ubuntu 10.04 alpha... ok, i'll try at #ubuntu
[15:47] <donEduardo> s/if/i've
[20:03] <Ng> so process supervision :)
[20:03] <Ng> is there any way for upstart to indicate that something has been respawned other than its syslog entries?
[20:04] <Ng> (I'm thinking that as a sysadmin I'd want realtime alerts that $IMPORTANTDAEMON has died and been restarted, ideally not by having to do realtime syslog analysis)
[20:09] <sadmac> Ng: you could detect the starting event from another job and have that fire off whatever
[20:10] <Ng> oh interesting
[20:17] <Ng> oh the starting event, not a specific respawning event?
[20:18] <Ng> actually what I'd care about is the *dying* of a process, but that's implied by it having been restarted ;)
[20:19] <sadmac> Ng: I think you get a stopping event too
[20:19] <sadmac> there's no respawning event though
[20:19] <sadmac> you definitely don't get a stopped event when respawning. it doesn't make it all the way down.
[20:20] <Ng> it might be a little annoying if booting a machine triggered a bunch of alerts ;)
[21:45] <wasabi> `script` stanza could use an extension:
[21:46] <wasabi> script /bin/bash
[21:46] <wasabi> \tfoo
[21:46] <wasabi> end script
[21:46] <Keybuk> but then we don't know whether what you give supports /proc/self/fd
[21:46] <Keybuk> and we don't know whether it supports "-e"
[21:46] <wasabi> True.
[21:47] <Keybuk> we don't know whether it supports being able to give single commands with "-c"
[21:47] <wasabi> I'd probably just say: yeah. if it support -e, you have to add it
[21:47] <Keybuk> I like the fact that script -es by default
[21:47] <Keybuk> script sh
[21:47] <Keybuk>  ...
[21:47] <Keybuk> end script
[21:47] <Keybuk> would bypass that
[21:47] <Keybuk> and be remarkably inconsistent
[21:47] <wasabi> that is true. 
[21:48] <wasabi> well, you see where I'm going.
[21:48] <wasabi> script python
[21:48] <wasabi> blah
[21:48] <wasabi> I'd probably just say adopt the same semantics any +x file has
[21:54] <sadmac> Keybuk: what if we allowed you to pipe part of the job definition to the exec stanza's argument?
[21:54] <sadmac> exec python <<<END
[21:54] <sadmac> import foo
[21:54] <sadmac> main()
[21:54] <sadmac> END
[21:54] <wasabi> That looks weird.
[21:55] <wasabi> It might be an argument for not using 'script', but instead extending 'exec' into a block stanza./
[21:55] <wasabi> with script being locked to sh -e
[21:55] <wasabi> s
[21:55] <sadmac> wasabi: it does, but it skirts most of Keybuk's arguments semantically. I'm in the habit of doing this. Eventually it results in Really Cool Things.
[21:55] <wasabi> exec python
[21:55] <wasabi>   import foo
[21:55] <wasabi>   main()
[21:55] <wasabi> end exec
[21:56] <wasabi> Since exec is already defined as not passing any arguments.
[21:56] <wasabi> I think it might be more appropiate too
[21:56] <sadmac> that makes the grammar ambiguous if you keep the one-liner exec
[21:56] <wasabi> as 'script' might not always even be the right name for it.
[21:56] <Keybuk> sadmac: I'm just wondering to myself what that exec line would do now ;-)
[21:56] <Keybuk> since technically exec is just a shortcut for
[21:56] <Keybuk> script
[21:56] <Keybuk>   exec ...
[21:56] <Keybuk> end script
[21:56] <Keybuk> ;-)
[21:56] <wasabi> Heh
[21:57] <wasabi> what about a utility that reads from stdin to process commands... that you want executed every now and then
[21:57] <wasabi> but it's clearly not a scripting language
[21:57] <sadmac> Keybuk: it would do the same as always, but make stdin a pipe first, then it'd write the python script into it
[21:57] <wasabi> shit, nc
[21:57] <wasabi> exec nc host
[21:57] <wasabi> \tstuff
[21:57] <wasabi> end exec
[21:57] <Keybuk> right
[21:57] <Keybuk> there's three different types of program
[21:58] <Keybuk> 1) those that take commands on command-line, e.g. sh -c "foo;bar;baz"
[21:58] <sadmac> wasabi: I spent a week in my security class trying to find ways to trick users into doing that. Making it easier to do voluntarily seems wrong
[21:58] <Keybuk> 2) those that take a script filename on command-line, e.g. sh /foo/bar.sh
[21:58] <wasabi> heh
[21:58] <wasabi> sendmail then. :)
[21:58] <Keybuk> 3) those that take commands on standard input
[21:58] <wasabi> exec sendmail admin@isillc.com
[21:58] <wasabi>   shit's busted
[21:58] <wasabi> end exec
[21:58] <sadmac> Keybuk: shock moment, what if we just supported shebang lines
[21:58] <sadmac> script
[21:58] <sadmac> #! /bin/python
[21:58] <sadmac> etc
[21:58] <sadmac> end script
[21:58] <wasabi> heh.
[21:59] <Keybuk> sadmac: no way to execute that ;-)
[21:59] <Keybuk> chmod +x /proc/self/fd/4
[21:59] <Keybuk> doesn't quite work
[21:59] <wasabi> What about symbols in front of the cmd name to specify how the body is passed to it?
[21:59] <Keybuk> symbols would be ick
[21:59] <sadmac> what does the command have to do to supprt /proc/self/fd?
[21:59] <Keybuk> but I like the idea that you should be able to run things with stdin
[21:59] <wasabi> maybe. but there's um
[21:59] <wasabi> what's the word for it
[21:59] <Keybuk> I already want to be able to run things and capture stdout after all
[21:59] <wasabi> crap. brain dead.
[22:00] <Keybuk> sadmac: to do #! you need a filename
[22:00] <wasabi> exec - sendmail jhaltom@isillc.com
[22:00] <wasabi>  foo
[22:00] <wasabi> end exec
[22:00] <wasabi> that gets passed to stdin
[22:00] <wasabi> (making stuff up)
[22:00] <Keybuk> why not perlish?
[22:00] <wasabi> because i don't know perl.
[22:00] <Keybuk> exec |sendmail jhaltom@isillc.com
[22:00] <wasabi> heh
[22:00] <wasabi> That's sort ofe stablished elsewhere too
[22:00] <wasabi> /etc/aliases
[22:00] <sadmac> Keybuk: same reason we couldn't do
[22:01] <sadmac> script /bin/python
[22:01] <sadmac> end script
[22:01] <sadmac> then
[22:01] <Keybuk> one sec, changing machines
[22:02] <sadmac> ah, wing-commander. I missed that box
[22:02] <wasabi> exec >sendmail jhaltom@isillc.com
[22:02] <wasabi> or | I guess
[22:02] <wasabi> Got me
[22:02] <Keybuk> sadmac: lol, "missed it" ?
[22:02] <wasabi> I don't like symbols like that.
[22:02] <sadmac> Keybuk: there's a lot you don't know that happened between me and your computer.
[22:03] <sadmac> Keybuk: don't judge us. we were young. we did what felt right.
[22:03] <Keybuk> should I dunk my laptop in bleach?
[22:04] <sadmac> Keybuk: yes. and you might want to get a new refridgerator.
[22:04] <Keybuk> syntax aside for a moment
[22:04] <Keybuk> what magic input/output/commandline things should we support here?
[22:05] <sadmac> stdin makes sense
[22:05] <sadmac> whatever the most standard syntax is, it should work by the same mechanism as #!, that is if an interpreter works in a #! line it should work in a job definition with minimal effort
[22:06] <Keybuk> right #! is "append filename to the #! line"
[22:06] <Keybuk> which should mean we can append /proc/self/fd/N
[22:06] <sadmac> really. thought it was pipe-through
[22:06] <Keybuk> no, cause you can do
[22:06] <Keybuk> echo "#!/usr/bin/sendmail" > casey.dahlin@redhat.com
[22:06] <Keybuk> chmod +x !$
[22:06] <sadmac> yeah, you're right
[22:06] <Keybuk> ./casey.dahlin@redhat.com
[22:06] <Keybuk> Subject: foo
[22:06] <Keybuk> etc.
[22:06] <wasabi> stdin and last argument make sense to me.
[22:07] <wasabi> anything else can be done by hand
[22:07] <wasabi> exec /proc/fd/0
[22:07] <wasabi> exec foo --name=/proc/fd/0 anyways
[22:07] <Keybuk> I also want to support capture-standard-output
[22:07] <wasabi> same.
[22:07] <Keybuk> for programs that write FOO=bar as standard output
[22:07] <wasabi> Where would you send it, though? :)
[22:07] <Keybuk> it'd be part of the job's environment
[22:07] <sadmac> Keybuk: that's always felt a bit special to me
[22:07] <wasabi> Hmm.
[22:08] <Keybuk> sadmac: it's used by just about everything "agent-esque"
[22:08] <Keybuk> dbus downards
[22:08] <wasabi> Yeah. That would be much nicer.
[22:08] <wasabi> Simply scrape the env exported by dbus when launching a dbus service
[22:08] <wasabi> no work done.
[22:08] <wasabi> except to export it in dbus
[22:08] <wasabi> and make sure your job somehow fires with that env.
[22:08] <Keybuk> right, dbus-daemon writes its session address and stuff to stdout
[22:08] <sadmac> Keybuk: I like a more general solution
[22:09] <Keybuk> so any job with "while dbus-daemon" would have that in its environment
[22:09] <wasabi> Yeah. That's rad. 
[22:09] <sadmac> exec dbus-session-bus | xargs upstart-set
[22:09] <Keybuk> xargs? :)
[22:09] <Keybuk> I guess
[22:10] <Keybuk> then I have to care about process groups
[22:10] <sadmac> yeah, the forking is a bit funny around that
[22:12] <Keybuk> we do need some kind of initctl to set environment though
[22:12] <wasabi> I'm fine with parsing name=value pairs.
[22:12] <wasabi> that's super standard.
[22:13] <wasabi> It would be better in some way if you could have dbus communicate in another way that didn't involve any parsing
[22:13] <Keybuk> lots of people seem to try and parse things in pre-start then punt to exec
[22:13] <Keybuk> but I think that's partly because exec sucks and you can't use cat if you're using fork-following
[22:14] <sadmac> Keybuk: look at augeas any? speaking of parsing.
[22:14] <Keybuk> sadmac: no, not yet
[22:14] <Keybuk> but I have now sent all my patches to halfline, so I suck less, so can look at that in a bit
[22:14] <sadmac> cool
[22:15] <Keybuk> the good news is we found a major bug with pre-stop in upstart ;)
[22:15] <Keybuk> (as a result of plymouth issues)
[22:15] <sadmac> I'm figuring this JSON stuff we've been discussing will need a parser anyway, so I'm going to complete nih-parse either way
[22:15] <Keybuk> not crasher - as in "clearly this approach is insane"
[22:15] <sadmac> ah, cool
[22:15] <Keybuk> and pre-stop was the only thing stopping me from getting rid of all four scripts
[22:15] <Keybuk> so now I can get rid of all four scripts and be happy
[22:16] <sadmac> Keybuk: so we have just one job now?
[22:16] <wasabi> JSON stuff
[22:16] <wasabi> So, JSON parsing will be in upstart.
[22:16] <wasabi> So it's hardly surprising that name=value parsing is
[22:16] <sadmac> wasabi: yeah, talking about using it for cross-upgrade stuff
[22:16] <Keybuk> just one process per job
[22:16] <Keybuk> so job and process become the same thing
[22:16] <sadmac> name=value is hardly even parsing. Its regular for chrissake.
[22:17] <wasabi> Yeah
[22:17] <Keybuk> but multiple jobs/processes defined in one conf file
[22:17] <Keybuk> given what we talked about the other day:
[22:17] <Keybuk>   on foo
[22:17] <Keybuk>   exec ...
[22:17] <Keybuk> it doesn't seem unreasonable to have a kinda "on self"
[22:17] <Keybuk>   on self starting
[22:17] <Keybuk>   exec ...
[22:17] <Keybuk> that's a task as we chatted about
[22:17] <Keybuk> so that's basically identical to the current pre-start script
[22:17] <sadmac> Keybuk: where do names come from then?
[22:17] <Keybuk> sadmac: names?
[22:18] <sadmac> Keybuk: yes, names for jobs
[22:18] <Keybuk> right, so then i got thinking well
[22:18] <sadmac> Keybuk: they come from file names now
[22:18] <Keybuk>   pre-start exec ...
[22:18] <Keybuk> that's a named process called "pre-start"
[22:18] <Keybuk>   reload exec ...
[22:18] <wasabi> Oh. Odd.
[22:18] <Keybuk> those would fire automatically
[22:18] <Keybuk> it's not "there" yet
[22:18] <Keybuk> (syntaxwise)
[22:18] <Keybuk> but I think it works process-wise
[22:18] <sadmac> so with the hierarchy thing we get all of it
[22:19] <sadmac> foo exec whatever in a file called stuff.conf creates a stuff foo job
[22:19] <sadmac> which is a silly name, but real world examples would make sense
[22:19] <Keybuk> right exactly
[22:20] <sadmac> we can parse it, to be sure.
[22:21]  * sadmac needs to finish up the parse tool
[22:22] <sadmac> my code quality has been wildly inconsistent. I'm going to throw away what I'm writing, but then maybe I'll copy paste some of it. I can't decide if I give a shit whether it leaks memory or not.
[22:22] <sadmac> the joys of writing a program who's first and only task is to rewrite itself :)
[22:22] <Keybuk> ll
[22:22] <Keybuk> lol
[22:23] <Keybuk> memory's cheap these days
[22:23] <Keybuk> they can buy more
[22:23] <sadmac> the good news is it won't need many tests. It tests itself just by being compiled :)
[22:23] <sadmac> Keybuk: that's not funny. Java people still say that with a straight face
[22:24] <sadmac> no wonder 2/3 the core dumps I get in during $dayjob report more memory than my hard disk.
[22:27] <Keybuk> heh
[22:27] <wasabi> trying to do this: which is failing. not sure why.
[22:27] <wasabi> script
[22:27] <wasabi> \t /bin/bash << EOF
[22:28] <wasabi> bunch of crap. blah blah.
[22:28] <wasabi> EOF
[22:28] <wasabi> end script
[22:28] <wasabi> Seems to break when I add my first complicated line of script: l=( backup users profiles )
[22:28] <wasabi> Can't imagine why any of it would need to be escaped for any reason
[22:29] <Keybuk> two shells?
[22:29] <Keybuk> you need to escape it from the first shell for the sceond to see it
[22:29] <Keybuk> script
[22:29] <wasabi> sh is reading from a file in proc, right?
[22:29] <Keybuk>   FOO=bar
[22:29] <Keybuk>   /bin/bash << EOF
[22:29] <Keybuk>   FOO=baz
[22:29] <Keybuk>   echo $FOO
[22:29] <Keybuk>   EOF
[22:29] <Keybuk> end script
[22:29] <Keybuk> will echo bar not baz
[22:29] <Keybuk> wasabi: yes
[22:30] <wasabi> I see.
[22:30] <Keybuk> the outer shell applies full expansion to everything
[22:30] <Keybuk> even the bits between << EOF and EOF
[22:30] <wasabi> still no worky though
[22:30] <wasabi> /bin/bash << EOF
[22:30] <wasabi> FOO=bar
[22:30] <wasabi> EOF
[22:30] <wasabi> fails. returns 2
[22:31] <wasabi> << isn't like, not sh compatible, is it?
[22:31] <Keybuk> it is afaik
[22:31] <wasabi> Ahh.
[22:31] <wasabi> Comments in the script body
[22:31] <wasabi> even unindented ones
[22:31] <wasabi> not sure why they're breaking it, but they are.
[22:32] <wasabi> oh.
[22:32] <Keybuk> can't have comments in strings maybe?
[22:32] <wasabi> they're probably being expanded when they shouldn't?
[22:32] <Keybuk> right
[22:32] <wasabi> would have thought upstart would have removed them
[22:32] <wasabi> guess not
[22:32] <Keybuk> why?
[22:32] <Keybuk> upstart passes everything between script and end script
[22:32] <wasabi> including leading indents? k
[22:32] <Keybuk> actually
[22:32] <Keybuk> no ;)
[22:32] <wasabi> heh.
[22:32] <wasabi> see.
[22:32] <Keybuk> it kinda de-dents everything
[22:32] <wasabi> i knew it didn't do that
[22:33] <wasabi> so I assumed it realized that a non-indented line was an upstart command
[22:33] <Keybuk> but that's actually good for you
[22:33] <Keybuk> cause
[22:33] <Keybuk> exec sendmail
[22:33] <Keybuk>   blah
[22:33] <Keybuk>   blah
[22:33] <wasabi> and upstart would fail if it wasn't # or end script
[22:33] <sadmac> Keybuk: could be a slight problem if we allow different interpreters, including python
[22:33] <Keybuk> end exec
[22:33] <Keybuk> gets de-dented
[22:33] <Keybuk> and
[22:33] <Keybuk> exec python
[22:33] <Keybuk>   blah
[22:33] <Keybuk>     blah
[22:33] <Keybuk>   blah
[22:33] <Keybuk> end exec
[22:33] <Keybuk> gets de-dented too
[22:33] <Keybuk> sadmac: it only removes the initial whitespace common to *all* lines
[22:33] <Keybuk> so that comes out as
[22:33] <wasabi> I'd adopt python rules. Each line must be indeneted with the same amount and type of whitespace. :)
[22:33] <Keybuk> blah
[22:33] <Keybuk>   blah
[22:33] <Keybuk> blah
[22:33] <wasabi> And that is auto removed by one level.
[22:34] <Keybuk> wasabi: that's what it does
[22:34] <wasabi> So why is # being passed to the script
[22:34] <sadmac> context sensitive lexical languages make me cry
[22:34] <Keybuk> wasabi: because it doesn't remove comments
[22:34] <wasabi> But the comment is not indented.
[22:34] <wasabi> So it should be invalid.
[22:34] <wasabi> And refuse to run the task
[22:35] <Keybuk> why?
[22:35] <wasabi> OR it should remove it. :)
[22:35] <Keybuk> neither
 I'd adopt python rules. Each line must be indeneted with the same amount and type of whitespace. :)
 wasabi: that's what it does
[22:35] <Keybuk> the comment means your block is simply not de-dented
[22:35] <wasabi> oh.
[22:35] <wasabi> hah.
[22:35] <Keybuk> it doesn't throw out your entire script because you used a space instead of a tab
[22:35] <Keybuk> that's just un-necessarily pedantic
[22:35] <Keybuk> it just doesn't de-dent
[22:36] <Keybuk> it finds the common prefix from each line that consists entirely of whitespace characters
[22:36] <Keybuk> and removes that
[22:36] <Keybuk> if all lines begin \t\t then that is removed
[22:36] <Keybuk> if all lines begin " \t    " then that is removed
[22:36] <Keybuk> if, as you've done there, one line isn't indented - then nothing is removed
[22:36] <Keybuk> it probably doesn't need to screw with it at all ;-)
[22:36] <wasabi> gotcha
[22:36] <Keybuk> but it does
[22:36] <Keybuk> it makes debugging perttier
[22:37] <wasabi> Trying to remember what i have to escape in this script to script then
[22:37] <sadmac> is that french?
[22:37] <wasabi> $ becomes $$...
[22:37] <Keybuk> no that's something else
[22:37] <Keybuk> $ becomes \$ in shell ;-)
[22:37] <Keybuk> $ -> $$ is make
[22:37] <wasabi> ahh
[22:37] <wasabi> duh
[22:38]  * sadmac remembers windows programming
[22:38] <sadmac> C:\\Windows\\System....
[22:38] <sadmac> or \\\\Shared\\Folder
[22:38] <wasabi> windows programming is all I do
[22:38] <wasabi> @"C:\Windows\System"
[22:39] <sadmac> wasabi: I'm sorry :)
[22:39] <wasabi> Fine with me.
[22:39] <sadmac> anyway, time to go eat.\
[22:39] <wasabi> I enjoy getting things done. :0
[22:39] <Keybuk> wasabi: didn't you end up at Google?
[22:39] <sadmac> wasabi: I don't have time to flame you. please pick this fight again later that we might finish it :)
[22:40] <wasabi> No. Didn't take it.
[22:40] <Keybuk> ahh
[22:40] <wasabi> They keep asking though.
[22:40] <Keybuk> they do that
[22:40] <Keybuk> Mountain View seems like such a long time ago!
[22:40] <wasabi> yeah, it does.
[22:40] <wasabi> I think about how much fun I had talking with people who knew what the hell I was talking about
[22:41] <wasabi> often
[22:42] <Keybuk> heh
[22:42] <Keybuk> UDS again in not-so-long
[22:42] <wasabi> I'm not remotely involved enough to get travel paid for, work is super busy (and has been for 2 years).
[22:42] <wasabi> I'd love to go though.
[22:43] <wasabi> Can't afford the out of the states trips though.
[22:44] <Keybuk> yeah
[22:44] <Keybuk> this one's in the middle of NOWHERE
[22:44] <wasabi> I didn't make it to the Dallas one.
[22:44] <wasabi> Because of work.
[22:44] <wasabi> I LIVE IN DALLAS
[22:45] <Keybuk> heh, fail
[22:45] <wasabi> Total.
[22:45] <wasabi> I saw desrt though. Made him come out and drink.
[22:45] <wasabi> Which he did little of.
[22:45] <wasabi> And then left early.
[22:45] <wasabi> Oh well.
[22:46] <wasabi> double escaping this is becoming hard to think about
[22:46] <Keybuk> was he at Dallas?
[22:46] <wasabi> Yeah
[22:46] <Keybuk> UDS has got so big now
[22:46] <wasabi> Dallas can be a great city too. I wanted to come and be the local guy showing everybody where to party.
[22:46] <Keybuk> I don't think I went into the desktop room once
[22:46] <wasabi> Because it's completely non-obvious where to go.
[22:47] <wasabi> \${n//-/--}
[22:47] <wasabi> Hmmm
[22:47] <wasabi> Hah. Got it.
[22:47] <wasabi> \${n//\\-/\\-\\-}