#upstart 2006-10-16
!alindeman:*! Regional server split.  We're looking at it
#upstart 2006-10-17
<Keybuk> mbiebl: did your upstart packages never make it through Debian NEW?
<mbiebl> Yes, they were rejected because the Conflicted with sysvinit.
<Keybuk> interesting...
<Keybuk> who rejected them?
<Keybuk> Random-J?
<mbiebl> jvw
<Keybuk> *nods*
<mbiebl> It's because sysvinit is essential.
<Md> mbiebl: will you use diversions then?
<Keybuk> arguably, sysvinit should not be Essential
<mbiebl> Md, yes I'll use that for the meantime and target experimental first.
<Md> be prepared for a world of pain. m-i-t was not fun...
<mbiebl> Keybuk: Yes, but this won't change for etch.
<mbiebl> Md: I don't like this idea either.
<mbiebl> On the other hand that seems to be the only way to get upstart into the archive now.
<Keybuk> mbiebl: you targetted it to experimental, yes?
<mbiebl> The one that got rejected? No, it was for unstable.
<Keybuk> ahh
<Keybuk> if it were experimental, it may have got through
<Keybuk> though one of jvw's comments amuses me
<mbiebl> Propably.
<mbiebl> which one?
<Keybuk> - Being not yet an init replacement it's also of limited use yet,
<mbiebl> This was a misunderstanding of jvw. He though upstart installed the init binary as /sbin/init and so was not a "drop-in" replacement, yet.
<mbiebl> s#/sbin/init#/sbin/upstart#
<Keybuk> so he rejected it coming AND going?
<Keybuk> he rejected it for not replacing /sbin/init and for replacing it? :p
<mbiebl> The main point was, that upstart conflicted with sysvinit, which is essential and that's not allowed by policy.
<Keybuk> heh
<Keybuk> so is it Debian policy that they will never allow sysvinit to be replaced?
<Md> not in etch, and in etch+1 it may still be hard
<mbiebl> As long as the essential tag is not removed from sysvinit, yes.
<mbiebl> I made a proposal, how we could deal with this by introducing a new base package called "init"
<mbiebl> which is essential and depends on "sysvinit | init-system".
<mbiebl> This package would be essential and the essential flag from sysvinit would be removed.
* ..[topic/#upstart:Keybuk] : Upstart 0.3.0 | http://upstart.ubuntu.com/ | http://upstart.ubuntu.com/doc/getting-started.html | https://wiki.ubuntu.com/UpstartDesignChanges | irc logs: http://people.ubuntu.com/~fabbione/irclogs | http://www.lugradio.org/episodes/61 | http://enterprise.linux.com/enterprise/06/09/18/1623244.shtml
<mbiebl> Other sysvinit replacements (like upstart, runit etc.) could then add a Conflicts/Replaces/Provides: init-system.
<mbiebl> I hope to get that done for etch+1.
<Keybuk> the interesting trick will be complying with the invoke-rc.d/update-rc.d policy :)
<mbiebl> Yes, the "init" package would provide invoke-rc.d/update-rc.d as mere wrappers.
<Keybuk> but how do they work in upstart?
<mbiebl> The init systems would install the actual implementation.
<Keybuk> "stop 0 23" doesn't mean much
<Keybuk> anyway, dinner, bbl
<mbiebl> Md, I have asked jvw if he'd accept an upstart package in experimental which does not use diversions but Conflicts/Replaces: sysvinit.
<mbiebl> He hasn't answered yet.
<mbiebl> Depending on his answer I'll decide to use diversions or not.
<Md> cool
<Md> aw
<Md> oops
<maro_> is my setup screwed or is initctl list supposed to look like this? ;) http://borkware.net/~mark/initctl_list.txt
<maro_> (0.2.7 looks fine)
<mbiebl> maro_: I noticed the same.
<mbiebl> Maybe you just file a bug.
#upstart 2006-10-19
!LoRez:*! Apologies for the huge netsplits.  Our .eu hub has decided to drop off the face of the earth.  I've routed around it for now.  Thanks for using freenode!
* johnnybuoy is off
* johnnybuoy|away is on
* johnnybuoy|away is off
* johnnybuoy is off
* johnnybuoy is on
* johnnybuoy is off
* johnnybuoy is on
* johnnybuoy is off
* johnnybuoy|awa- is off
* Keybuk wondered what that smell was
* johnnybuoy is on
<_ion> Sigh, away nicks.
<_ion> And flooding the channel with "is on, is off" messages.
<_ion> johnnybuoy: I would really appreciate you turning off the "is on, iss off" spam and away nicks.
<johnnybuoy> I have, now
<johnnybuoy> sorry for that
<_ion> Thanks a lot.
<johnnybuoy> I know every channel I was logged onto wants my skin...
<johnnybuoy> and I can understande that...
<_ion> IRC has an "away" feature. Whoever is interested of your away status can /whois you, or add you to her notify list.
<johnnybuoy_> okay, I think I have disabled that sux thing now
<johnnybuoy_> _ion| as I have to everyone, I say sorry!
<johnnybuoy_> I have enerved many many ppl...
<johnnybuoy_> :(
<_ion> The actual away feature never sends anything to any channel. I recommend using that.
<johnnybuoy_> yes
<johnnybuoy_> _ion| I don't really ned to use it, right
<johnnybuoy_> is this the idle thing?
<Keybuk> *sigh* people still use dircproxy
<Keybuk> heh
<Keybuk> trivia for amusement value ...
<Keybuk> upstart uses the dircproxy networking layer to communicate with initctl
<johnnybuoy_> heh
<_ion> johnnybuoy: Let's not continue this discussion on this channel, it's offtopic. The command is 'away': /away message when you leave and /away without parameters when you come back. dircproxy, which you seem to be using, probably supports setting that automatically based on whether you're connected.
<johnnybuoy_> wtf?
<johnnybuoy_> :D
<johnnybuoy_> okay
<johnnybuoy_> thx
<johnnybuoy_> upstart uses the dircproxy networking layer to communicate with initctl  <-- wtf? :D
<Keybuk> johnnybuoy: it's true
<Keybuk> modified after five years of experience, of course
<Keybuk> but I see no point throwing away something that worked
<Keybuk> I got it reasonably right back then
<Keybuk> and it got genericified into libnih
<johnnybuoy> Keybuk| wow, that's amazing.
<johnnybuoy> is dircproxy in any way special?
<Keybuk> "special" ?
<johnnybuoy> is it still doing it?
<Keybuk> not sure I understand what you mean?
<johnnybuoy> well, I guess if you included some messaging code from it to upstart it must have something "specual" ..
<Keybuk> not particularly
<Keybuk> I just wrote it, that's all
<Keybuk> so when I needed async networking code for upstart, there was no point writing it *again*
<Keybuk> I already had the code
<Artanicus> theres only one cure for away nicks, ignore d:
#upstart 2006-10-20
<zenlinuxNH> The example jobs tarball at http://upstart.ubuntu.com/download/example-jobs.tar.gz is missing from the web site.
<zenlinuxNH> On this page: http://upstart.ubuntu.com/doc/getting-started.html
<maro_> I'm wondering how "one job at a time" will work in scenarios where a second event happen before the first one has been processed (say, an usb hub with a couple of disks is plugged in)
<maro_> bah, missed the "Events will no longer share a namespace with jobs" part og DesignChanges
#upstart 2006-10-21
<rmcc316> i literally just heard out about upstart a few minutes ago.  is there any work w/ the redhat/suse side of the linux world?
-ChanServ(ChanServ@services.)- [#ubuntu-server]  Ubuntu Server Discussions (development and support)
-ChanServ(ChanServ@services.)- [#ubuntu]  Welcome to #ubuntu! Please read the channel topic and consider spending some time on the FAQ mentioned there
-ChanServ(ChanServ@services.)- [#ubuntu-server]  Ubuntu Server Discussions (development and support)
-ChanServ(ChanServ@services.)- [#ubuntu]  Welcome to #ubuntu! Please read the channel topic and consider spending some time on the FAQ mentioned there
<maro_> is it just my setup or is initctl really broken in 0.3.0?
<maro_> I have a respawning job, and initctl stop doesn't seem to have any effect
<maro_> initctl status doesn't show anything
<maro_> finally got it stopped by changing the config to "stop on foobar" and emitting that event
#upstart 2006-10-22
* netjoined: irc.freenode.net -> brown.freenode.net
!LoRez:*! Hi fellow freenoders!  I'm about to do some _minor_ rehubbing of a couple of smaller servers.  Affected users is < 400.
!RichiH:*! If you are interested in history, come and visit ##history, the history and humanities channel for the Freenode network. Features daily 'This Day in History' factoids (in the /topic) and more great things coming... quiz, wikipedia lookup etc.!
#upstart 2007-10-15
<JohnFlux> Hey all
<JohnFlux> to any developers..  could you include a package with the header files etc?
<JohnFlux> it's currently impossible to write a program that queries upstart etc
* Starting logfile irclogs/upstart.log
<soren> JohnFlux: Huh? How do you figure that?
<JohnFlux> soren: ah hey!
<JohnFlux> soren: I have ubuntu
<JohnFlux> soren: If I look at what packages there are, there's no libupstart of libupstart-dev etc
<soren> JohnFlux: Why do you need them?
<JohnFlux> soren: if I look at the files that come with it just "upstart" there's no libraries nor header files
<JohnFlux> soren: so that I can make a gui for upstart
<soren> JohnFlux: Er..
<soren> JohnFlux: You don't need header files or libraries to do anything like that.
<JohnFlux> soren: in particular, I want to be able to get a list of running services, for example
<soren> status
<JohnFlux> I'd much prefer a library to trying to parse the output of programs :/
<soren> Sorry, I meant "initctl list".
* JohnFlux nods
<JohnFlux> soren: so how about?  a libupstart ? :)
<soren> There's no such thing. You can try convincing Keybuk that it'd be useful and you might get lucky :)
<JohnFlux> he's comes in here right?
<JohnFlux> I'll give that a shot thanks
<soren> Yeah, he'll be around in an hour or so, I'd guess.
<JohnFlux> btw I'm the maintainer of the kde "task manager" type thinig
<JohnFlux> thing
<JohnFlux> just looking to use upstart info to improve the info of what's shown
<Jc2k> JohnFlux: i think libupstart is about to be killed off
<Jc2k> see the mailing list post that the topic refers to
<Jc2k> "libupstart goes away, which nobody else is using anyway."
<Jc2k> instead there will be dbus aplenty
* Keybuk amuses at the upcoming feature list for Bazaar
<Keybuk> "Pack-based repositories"
<Keybuk> THE RAPTORS ARE COMING!
* soren wonders if Ubuntu 13.04 will be the rowdy raptor release.
<Keybuk> ravenous raptor
<soren> Good one.
<Keybuk> I need a brain-to-tomboy sync plugin
<Keybuk> so when I think of neat code changes in bed, I don't need to have to try and remember them in the morning
<soren> Any reasonable person would suggest some sort of intermediate storage like a piece of paper, but I haven't managed to implement that either.
<soren> Althought it is a bit alarming to imagine how many good ideas have just vanished because I haven't bothered to write them down before falling asleep.
<Keybuk> aye
<JohnFlux> Jc2k: nobody is using it because it's impossible to use
<JohnFlux> Jc2k: it's not actually shipped
<JohnFlux> Jc2k: I've been wanting to use it for ages
<JohnFlux> Jc2k: I asked on the mailing list about it like 6 months ago
<JohnFlux> Jc2k: it will be a shame if it's just killed off 
<Jc2k> why? isnt libupstart mostly about ipc? which will be handled via dbus, and thus *easily* accessible?
<Keybuk> it's entirely about IPC
<Jc2k> so therefore JohnFlux has nothing to fear by libupstart death, in fact has much to gain from it.
<Keybuk> he won't have to rewrite his front-end with every minor release, for a start ;)
<JohnFlux> actually that does sound good
<Keybuk> JohnFlux: libupstart has never been a stable API
<Keybuk> initctl uses it, but that's it
<JohnFlux> how far along is it btw?
<JohnFlux> the dcop thing
<Keybuk> ?
<Keybuk> how far along is what?
<Jc2k> s/dcop/dbus/
<JohnFlux> right hehe
<JohnFlux> Does it offer a dbus service at the moment?
<Jc2k> Keybuk: how long till 0.5 basically
<Keybuk> no
<Keybuk> far along = not even started
<JohnFlux> heh
<JohnFlux> fair enough
<JohnFlux> do any other distros use upstart btw?
<JohnFlux> or just ubuntu?
<Keybuk> just ubuntu at the moment
<Keybuk> I've made a few commits to Upstart in the last few days
<Keybuk> and lots to libnih :p
<Keybuk> so I'm at least starting on the roadmap to 0.5
<Jc2k> \o/
<Jc2k> oh
<Jc2k> questionage
<Jc2k> can upstart be forced to run as a slave of "old" init?
* Jc2k is thinking of a less 'controversial' migration path and can't spell
<soren> Having two init implementations running side by side is hardly less controversial than what Ubuntu currently does?
<Jc2k> if something depended on upstart and upstart could be installed without ripping the heart out of your distrothen it would be a lot more viable
<Jc2k> *shrugs*
<Keybuk> the problem with being a slave is that it isn't init
<Keybuk> so doesn't get afforded init's special privileges
<Jc2k> bleh
* Jc2k kicks unix
<soren> Jc2k: You should know that UNIX kicks harder. :)
<Jc2k> :)
<Keybuk> you could write an upstart parser for inittab :)
<Keybuk> then it could just be a drop-in replacement for sysvinit
<Keybuk> on trunk, that should be quite easy
<Jc2k> interesting
<Keybuk> inittab would be a ConfSource, register jobs with useful names like "inittab/$id", etc.
<Keybuk> certainly upstart supports a superset of inittab's features
* Starting logfile irclogs/upstart.log
<ion_> :-)
<Keybuk> now, of course, I have to work around the ways I was attempting to work around the lack of something I've now implemented
<Keybuk> yay, ok init builds again
<Keybuk> now for the tests
<Keybuk> ==13312==  Address 0x1C is not stack'd, malloc'd or (recently) free'd
<Keybuk> that would be bad, then
* Keybuk is having strange valgrind bugs
<Keybuk> ==32728== Invalid write of size 4
<Keybuk> ==32728==    at 0x8093F89: nih_list_cut (list.c:202)
<Keybuk> ==32728==    by 0x809405D: nih_list_destroy (list.c:246)
<Keybuk> ==32728==    by 0x8092C09: nih_free (alloc.c:330)
<Keybuk> ==32728==    by 0x8092C34: nih_free (alloc.c:336)
<Keybuk> ==32728==    by 0x808D243: parse_job (parse_job.c:278)
<Keybuk> ==32728==    by 0x805DF5B: test_stanza_start (test_parse_job.c:2185)
<Keybuk> ==32728==    by 0x8087030: main (test_parse_job.c:7303)
<Keybuk> ==32728==  Address 0xBEEFA174 is just below the stack ptr.  To suppress, use: --workaround-gcc296-bugs=yes
<Keybuk> oh I see
#upstart 2007-10-16
<Keybuk> hurrah
<Keybuk>  36 files changed, 667 insertions(+), 13930 deletions(-)
<ion_> :-)
* Starting logfile irclogs/upstart.log
* Starting logfile irclogs/upstart.log
<eehouse> are there any linux distros using upstart as intended, rather than in sysvinit-compatibility mode?
<Keybuk> rah
<ion_> har
<ion_> eehouse: Probably not until 0.5.
<Keybuk> what was the question?
<Keybuk> oh, I see
<ion_> ke002103 < eehouse> are there any linux distros using upstart as intended, rather than in sysvinit-compatibility mode?
#upstart 2007-10-17
<Keybuk> tests/test_conf.c:600: error: âJobConfigâ has no member named âgoalâ
<Keybuk> tests/test_conf.c:601: error: âJobConfigâ has no member named âstateâ
<Keybuk> *sigh*
<Keybuk> repeat x 10,000
#upstart 2007-10-18
<ion_> Pretty. And it only takes a kilobyte. http://pouet.net/prod.php?which=32194
<Keybuk> what is it?
<ion_> An intro. Thereâs a link to a video capture on the page.
<ion_> http://heh.fi/heroes/heroes-scrape/pipeline.png
<Keybuk> heh
 * Keybuk finally discovers the fuck-up
<Keybuk> parse_job () will free the old job with the same name
<Keybuk> yet so will conf_item_destroy ()
<Keybuk> the difference being that parse_job () frees the job without freeing the ConfItem for it
<Keybuk> which means that job_change_state () must do the same!
 * Jc2k is somewhat baffled but nods and smiles
<Keybuk> this bug must have always been present, the newer code happens to tickle it more
 * Keybuk tries to think how you'd replicate it
<Keybuk> job of the same name in two different directories
<Keybuk> then reload
<Keybuk> that'd cause older code to core dump
 * Keybuk frowns
 * Jc2k bounces
 * Keybuk almost hacks it
<Keybuk> hurrah
<Jc2k> :O
<Jc2k> victoreh?
<Keybuk> not quiet
<Jc2k> :(
<Keybuk> I'm tempted to ignore the problem and move on
<Keybuk> since it's in code I intend to completely rewrite
<Jc2k> sounds like a plan :O
<Keybuk> odd, I made the bug go away
<Keybuk> la la la
<Keybuk>  woo, only test_job left to fix
<Keybuk> all 5,300 lines of it :-/
<Jc2k> woah
<Jc2k> large test then?
<Keybuk> yeah
<kelsin> Hey, is there a way to specify a start script/ stop script and then set the PID directly for jobs? Basically have a service that comes with a normal "script start/stop/status" as the way to start it, didn't want to have to work a lot with it if possible
<kelsin> actually might have answered my own question by finding the wait command :) I'll ask again if this doesn't work
#upstart 2007-10-19
<Keybuk> right, so that's all the tests able to be built again
<Keybuk> just got to fix a few more test failures
<Keybuk> and then I can push the commit of doom!
<Keybuk>  18 files changed, 2271 insertions(+), 3012 deletions(-)
<ion_> :-)
<Keybuk> being the commit that splits Job into Job and JobConfig
<Keybuk> though I might try and turn it into lots of bi-sectable commits
<Keybuk> not sure how likely that is
<Keybuk> WOO!
<Keybuk> ALL TESTS PASS!
<Keybuk> I cannot tell you how happy this makes me! :D
<Keybuk>  18 files changed, 2549 insertions(+), 3105 deletions(-)
<ion_> Whee
 * Keybuk checks the build with optimisations
<Keybuk> (and fixes the usual gcc "I'm stupid" issues)
<Keybuk> ok, now the coverage check
<phsdv> it seems that the link on http://upstart.ubuntu.com/download.html to the readme file for 0.3.9 is broken...
<Keybuk> phsdv: thanks, fixed
<phsdv> thanks, starting to read it
<CNW8835> hello
<Keybuk> hello
<CNW8835> do you  happen to know how to make upstart listen to kernel boot options (like "3" to make init/upstart boot into runlevel 3)?
<CNW8835> It's my understanding that you just have to put a "3" as a kernel boot parameter, but this doesn't seem to work with upstart
<Keybuk> right, it doesn't work
<Keybuk> it's a bug
<CNW8835> oh, ok
<CNW8835> thanks
<Keybuk> https://bugs.edge.launchpad.net/upstart/+bug/74664
<Keybuk> (and also)
<Keybuk> https://bugs.edge.launchpad.net/upstart/+bug/85014
<Keybuk> depending which side of the bug you care about ;)
<CNW8835> thanks for the links
<Keybuk> http://codebrowse.launchpad.net/~keybuk/upstart/main/revision/scott%40netsplit.com-20071019214909-iyq8ua9epn5wsmfw?start_revid=scott%40netsplit.com-20071019214909-iyq8ua9epn5wsmfw
<mbiebl> wow
<ion_> Hehe
<Keybuk> ba
<Keybuk> I hate it when I find bugs just by looking
<Keybuk> wing-commander init% ./test
<Keybuk> test (#0) goal changed from stop to start
<Keybuk> test (#0) state changed from waiting to starting
<Keybuk> event_new: Pending starting event
<Keybuk> test (#0) goal changed from start to stop
<Keybuk> Handling starting event
<Keybuk> event_finished: Finished starting event
<Keybuk> test (#0) state changed from starting to waiting
<Keybuk> event_new: Pending stopped event
<Keybuk> Handling stopped event
<Keybuk> event_finished: Finished stopped event
<Keybuk> ...
<Keybuk> if the goal changes to stop while the starting event is blocked, you never get a stopping event
<Keybuk> just a stopped one
<Keybuk> bugger
<Keybuk> need two different next states from waiting
<Keybuk> from starting, I mean
#upstart 2007-10-20
<joshk> Keybuk: can upstart parse quoted strings correctly on the kernel command line?
<joshk> or even if it can't do it correctly, is there any way to make it happen?
<Keybuk> popular question today
<Keybuk> why the sudden interest in Upstart's ability to parse the kernel command-line?
<joshk> never mind - it's apparently busybox init i'm worried about
<joshk> whichever init is used to bootstrap the desktop CD
<joshk> i did strings /sbin/init in the live cd environment and found a bunch of upstart, so I imagined that was it
<joshk> but cjwatson proved me wrong
<Keybuk> :-)
<Keybuk> *yawn*
<Keybuk> and damn, I'm almost the entire way through this notebook already
<Keybuk> che: hi
<che> Keybuk, heyyas
<che> Keybuk, actually i see an rpath problem with 0.3.9: http://rafb.net/p/zvDiEk46.html
<Keybuk> did you make install?
<che> Keybuk, 0.3.9 example jobs link on the homepage is broken (yet i guess)
<Keybuk> or did you copy files out of the build directories?
<che> Keybuk, i used make install with DESTDIR set
<Keybuk> example-job: fixed
<che> Keybuk, nice.
<Keybuk> can you do "objdump -p" on one of those files?
<che> Keybuk, just give me 3 mins please. phone.
<che> Keybuk, thanks btw
<Keybuk> also some information about how you built them would be useful
<Keybuk> I just checked the 0.3.9 packages here and they don't have those rpaths that I can tell
<che> Keybuk, i will dump the info into pm
<Keybuk> rafb urls are fine too :)
<Keybuk> che: so the option that turns on those additional warnings was -Wp,-D_FORTIFY_SOURCE=2
<Keybuk> meh, I hate header loops
<Keybuk> bah
<Keybuk> I'm going to have to make a header file for just one struct
<Keybuk> I can tell
<Keybuk> :'(
 * Jc2k offers Keybuk a shoulder to cry on
<AlexExtreme> mmm, interesting
<AlexExtreme> tesco are selling ubuntu PC's
<AlexExtreme> *PCs
<AlexExtreme> http://daviey.mooo.com/uncategorized/tesco-every-little-helps.html
<Keybuk> yeah I saw that
<Keybuk> hmm, back to the code then
<Keybuk> the config stuff feels like it's getting unnecessarily complicated
<Keybuk> trying to decide
<Keybuk> is it really necessary that Upstart handles a job of the same name being defined in two places?
<ion_> Probably not.
<ion_> Unless there are some use cases, which i canât think of. :-)
<ion_> s/canât think of/donât notice immediately/
<Keybuk> so what should it do?
<Keybuk> error when you try and load the second one?
<Keybuk> the trouble then is if you HUP, you might get the other one surprisingly
<ion_> True...
<Keybuk> which would be odd
<Keybuk> of course, I could take away the ability to define multiple configuration directories entirely
<Keybuk> that would solve the conflict problem, since the filesystem namespace would apply
<ion_> Yeah
<Keybuk> hmm
<ion_> http://arcanux.org/lambdacats/tail-recursion.jpg
<AlexExtreme> hehe
<Keybuk> :)
#upstart 2007-10-21
<Keybuk> ?
<ion_> Â¿
<Keybuk> heh
<Keybuk> random key press to get out of the screensaver
<Keybuk> went straight into IRC
#upstart 2008-10-13
<sadmac2> Keybuk: why did you choose Linked List for NihList?
<ion_> Why not?
<sadmac2> ion_: I don't have a reason why not, but if there's no reason why there may be some options that don't have the problems we're running into
<sadmac2> vectors for example
#upstart 2008-10-14
<sadmac2> how can a netsplit between the same 2 servers cause 2 wholly different sets of people to drop?
<suihkulokki> freenode is problably masking what servers really did split
<sadmac2> ah
<sadmac2> well now we're onto them
<suihkulokki> yeah, whenever there is a netsplit it just says split between myserver.freenode.net <-> irc.freenode.net
<sadmac2> ah
<sadmac2> kornbluth sounds like it'd be in the UK
<sadmac2> if it were a town
<sadmac> notting_: just sent Scott a patch for our crasher bug
#upstart 2008-10-15
<Keybuk> sadmac: I'll digest it over the next day or so, but that looks ... inspired
<Keybuk> sadmac: around?
<sadmac> Keybuk: ya?
<Keybuk> sadmac: so what I think you're doing here, after a quick glance, is placing a second "iterator head" in the linked list immediately after the node being iterated
<sadmac> immediately before
<Keybuk> before?
<sadmac> yes
<Keybuk> ah, that explains why it's more complicated than I expected
<sadmac> we're always operating on the node after the "cursor" element
<sadmac> hmm, after would make a few things easier....
<sadmac> I need to get on campus, but I should be online in a couple minutes.
<sadmac> brb.
<Keybuk> yeah, just thinking about the difference
<Keybuk> I'd thought about putting the cursor after, since then the node to return is cursor->prev and the next node to visit is cursor->next
<Keybuk> and the worst they can do is remove all of the nodes
<Keybuk> in which case, cursor->next is the list head, which will end the loop
<Keybuk> if they remove the next node, that just means cursor->next is different
<Keybuk> if they remove the visiting node, that just means cursor->prev is different - but we don't even look at that on the loop iteration
<Keybuk> the bug is caused by "caching" the next node, so the visiting node can be removed
<Keybuk> which means if you remove the next node as well, the cache dereferences free'd memory
<Keybuk> by using a cursor element, we only ever cache the cursor, which in theory the code in the loop knows nothing about
<Keybuk> -- 
<Keybuk> cursor after does mean that if new nodes are inserted after the visiting node, they will not be visited
<Keybuk> not sure if that's a feature or not
<Keybuk> cursor before means that new nodes inserted after *are* visited
<Keybuk> but also, strangely, new nodes inserted before might be visited too
<Keybuk> except you have a more complicated "step the cursor" case
<Keybuk> and I'm not sure that "step the cursor" is robust if nodes are inserted before the visited node
<Keybuk> in fact, I have a slight feeling that you'll visit the node twice
<Keybuk>   C-V-X on beginning of loop, becomes
<Keybuk>   C-A-B-V-X on end of loop
<Keybuk> you'll see that C-next is not V, so won't step the cursor
<Keybuk> so will visit A, B then V again
<Keybuk> cursor after is safe from that
<Keybuk>   V-C-X becomes V-A-B-C-X or A-B-V-C-X
<Keybuk> in neither case will A or B be visited in that iteration
<sadmac> hm
<sadmac> we could mark the cursor as special, either with a flag or by checking the address, and then modify the insert functions to behave as we'd expect.
<sadmac> so V-C-X becomes A-B-V-C-X or V-C-A-B-X
<Keybuk> I can't decide whether it's a good thing or not ;P
<Keybuk> if it works with insert() you'd kinda expect it to work with insert_before() too
<sadmac> the assumption from the "user" is that the list is being iterated linearly.
<sadmac> before the visited node is not in the damage path
<sadmac> after is
<Keybuk> there's an even more fun way of doing it
<Keybuk> actually, no, that's the same way you're trying to do it, sorry
<sadmac> heh
<Keybuk> I was thinking of actually removing the visited node, and use the cursor as a list head only
<Keybuk> but that has exactly the same semantics anyway
<sadmac> having nodes inserted after the visited node get iterated and before not is how it would work in the "oracle" case, where we just could divine where we were in the list by magic.
<Keybuk> err, that wasn't english?
<sadmac> putting it another way:
<sadmac> X-A-X-X-X-V-X-X-B
<sadmac> ^if we're at V, and we just inserted A and B, we'd expect A not to get visited and B to get visited
<sadmac> that's what regular old NIH_FOREACH would do
<Keybuk> right
<sadmac> so we should emulate that behavior
<sadmac> we want to visit things inserted after, but not things inserted before
<Keybuk> agree
<sadmac> so the way to do that is if a node is before a cursor and we try to insert_after, to insert after the cursor instead of right after the node
<suihkulokki> is there some plan to add dh_installevent style helper 
<sadmac> suihkulokki: I have no idea what that is
<suihkulokki> that's a debianism
<sadmac> that'd do it
<Keybuk> suihkulokki: no, none
<suihkulokki> Keybuk: do you think it would be a good idea?
<suihkulokki> this is the third package i'm adding a native upstart event file and I'm noticing a pattern ;)
<sadmac> suihkulokki: who knows what the pattern will look like next release though ;)
<Keybuk> suihkulokki: I do not think dh_ anything is a good idea
<Keybuk> what's wrong with just shipping the file as a conffile?
<suihkulokki> the point is just to cut down the lines written to each debian/rules file
<Keybuk> sadmac: of course, I forgot that nih_list is kinda backwards
<Keybuk> nih_list_add actually adds before :p
<Keybuk> it's only nih_list_add_after that's special
<sadmac> Keybuk: right
<Keybuk> suihkulokki: why do you need any lines in debian/rules?
<Keybuk> sadmac: so you have to try very hard to get the node *after* the one you're looking at
<Keybuk> and because you tried hard, you're likely to expect that it gets iterated
<sadmac> Keybuk: so in order to dtrt we need to be able to tell cursors from normal elements
<sadmac> Keybuk: and we can do that one of 2 ways
<sadmac> Keybuk: 1) the obvious way: add a field to the element with a flag
<Keybuk> expensive
<Keybuk> 64 bits! for a 1 bit flag
<sadmac> Keybuk: 2) The evil way: check the address. If its allocated on the stack, its a cursor. if its in heap, its not
<Keybuk> list elements can be alocated on the stack
<sadmac> you're only technically right, I don't think they ever /are/
<Keybuk> yeah they are
<sadmac> but, hence the evil
<sadmac> hm
<sadmac> we need darker magic
<Keybuk> parse_job.c has one
<Keybuk> parse_job.c:    NihList        stack;
<sadmac> that'll do it
<sadmac> we could add the flag field, and just come up with lots of ultra cool features so we need more flags :)
<sadmac> nih_list_set_flag(list, NIH_EAT_BABIES_AND_KITTENS);
<Keybuk> mmm, kittehs
<Keybuk> your dual for loop really confuses me :p
<Keybuk> this is you trying to do "before and after" code, right? :)
<sadmac> yeah
<Keybuk> couldn't the "test" part of the outer for loop just be 0 ?
<Keybuk> for (NihList cursor = { &cursor, &cursor }, *_iter = nih_list_add_after ((list), &cursor); FALSE; nih_list_remove (&cursor))
<Keybuk> something like that
<Keybuk> ie. don't set the pointer to NULL, then you don't need to test it
<Keybuk> I suspect the compiler will optimise that properly anyway, but it makes it a bit more clear
<Keybuk> I don't think you even need _iter in there
<Keybuk> except as something to allow you call nih_list_add_after
<sadmac> no that won't work
<sadmac> then the loop will never run
<sadmac> setting to null makes it run exactly once
<sadmac> because its not null for the first test
<Keybuk> ah, you're quite right
<sadmac> Keybuk: I'm going to leave it up to you to figure out how to remove the handler function patch from bzr. Its tangled up in quite a few things.
<Keybuk> handler function patch?
<sadmac> Keybuk: our previous patchwork fix for a single instance of this error
<sadmac> with the marking and sweeping and such
<Keybuk> that's easy
<Keybuk> bzr diff -r -2..-1
<Keybuk> or just bzr diff -r -1..-2 then you don't need -R to patch ;)
<sadmac> Keybuk: its tangled with some other changes
<Keybuk> ah, so it is
<sadmac> Keybuk: its got the change of the dbus timeout to INT_MAX in there, and it also seems you moved some files
<Keybuk> something like bzr diff -r before:594.1.2 I think
<sadmac> tried that but the patch looked weird
<sadmac> I think you renamed nih_dbus_tool.py to nih_dbus_tool in the same timeframe
<Keybuk> no, the latter is just made from the former
<sadmac> the diff I got back was re-adding the INT_MAX change
<sadmac> like it wasn't there before
<Keybuk> that merged from you at the same time I think
<sadmac> hm
<Keybuk> +                       if (_##iter->next == iter) { \
<Keybuk> +                       } \
<Keybuk> +                       iter = _##iter->next; \
<Keybuk> *cough*
<Keybuk> shouldn't that last "=" be inside the if? :p
<Keybuk> no
<Keybuk> sorry
<Keybuk> confusing iter and _##iter
<Keybuk> you don't need the "tmp" inside though ;)
<Keybuk> you know that _##iter->next and iter are the same
<Keybuk> +                       if (_##iter->next == iter) { \
<Keybuk> +                               nih_list_add_after (iter, \
<Keybuk> +                                       nih_list_remove (_##iter)); \
<Keybuk> +                       } \
<Keybuk> +                       iter = _##iter->next; 
<Keybuk> in fact
<Keybuk> you don't even need it that complicatedf
<Keybuk> if (_##iter->next == iter)
<Keybuk>         nih_list_add_after (iter, _##iter);
<Keybuk> iter = _##iter->next;
<Keybuk> yeah
<Keybuk> confirmed
<Keybuk> your method is unsafe when using nih_list_add ()
<Keybuk> if the caller does nih_list_add () on the entry itself, the cursor will be pushed back in the list
<Keybuk> it correctly won't step
<Keybuk> but will visit the entry twice
<Keybuk> not that I can think of any time off-hand where we do that (I tend to call it on the list head), but it's valid - as it's a way of doing sorting and stuff
<sadmac> ok
<sadmac> so we need to have the cursor after
<sadmac> and then add_after needs to have something done to it
<Keybuk> cursor afterwards just has opposite problems
<sadmac> how so?
<Keybuk> it doesn't notice the adds at all :)
<Keybuk> I'm actually wondering if there's an evil way of breaking the pointers :p
<sadmac> which is half-right
<sadmac> not noticing the adds is right for add(before) but not for add_after
<sadmac> the latter is fixable
<sadmac> hmm..actually you're right. we just need to fix one of the add methods to account for cursors
<sadmac> the issue is the cursor getting separated from the iterated node in cases where the iterated node is still in the list
<Keybuk> right, we really want to add them after the cursor
<sadmac> assuming we switch to cursor after
<sadmac> we need to add before the cursor if we keep it as is
<Keybuk> right, which means you need to know where they started adding
<Keybuk> am thinking
<Keybuk> what if iter->next is cursor->next
<Keybuk> so the list is slightly broken
<Keybuk> (iter->next would normally be cursor)
<sadmac> I'm afraid
<Keybuk> no, there's no safe way to do that
<Keybuk> the only way is to hide the cursor entirely
<Keybuk> but then the cursor isn't actually updated
<sadmac2> there's one 8-byte solution
<sadmac2> we just need to make it smaller
<sadmac2> only thing I have so far is put a flag in the MSB of one of the poiters
<sadmac2> *pointers
<Keybuk> pointers don't have MSBs
<sadmac2> they can be made to look as though they do.
<Keybuk> no, they can't
<Keybuk> you can't fiddle with a pointer, it's in scary undefined territory according to the spec
<sadmac2> yeah yeah
<sadmac2> this is what I get for reading kernel code
<sadmac2> now to class
<Keybuk> #define NIH_LIST_FOREACH_SAFE(list, iter)                               \
<Keybuk>         for (NihList _##iter = { &_##iter, &_##iter },                  \
<Keybuk>                  *iter = nih_list_add_after ((list)->next, &_##iter)->prev; \
<Keybuk>              iter != (list) && iter != &_##iter;                        \
<Keybuk>              iter = nih_list_add_after (_##iter.next, &_##iter)->prev)
<Keybuk> no, that doesn't quite work, since it leaves the cursor in the list
<sadmac2> Keybuk: that's what the outer for was for
<Keybuk> #define NIH_LIST_FOREACH_SAFE(list, iter)                               \
<Keybuk>         for (NihList _##iter##_cursor = { &_##iter, &_##iter },         \
<Keybuk>                      _##iter = &_##iter##_cursor;                       \
<Keybuk>              _##iter;                                                   \
<Keybuk>              nih_list_destroy (_##iter), _##iter = NULL)                \
<Keybuk>                 for (NihList *iter = nih_list_add_after ((list)->next, _##iter)->prev; \
<Keybuk>                      iter != (list) && iter != _##iter;         \
<Keybuk>                      iter = nih_list_add_after (_##iter->next, _##iter)->prev)
<Keybuk> #
<sadmac2> Keybuk: is it ok to call nih_list_destroy on something stack-allocated?
<sadmac2> Keybuk: and either way, why isn't it nih_list_remove?
<Keybuk> yeah, it just cuts it out of he list
<Keybuk> destroy is cheaper than remove :p
<sadmac2> ah
<Keybuk> destroy => nih_list_cut ()
<Keybuk> remove => nih_list_cut (), nih_list_init ()
<sadmac2> yeah
<sadmac2> Keybuk: don't you need a * on line 2 before _##iter ?
<Keybuk> yes
<sadmac2> Keybuk: it looks good. It has the issue of missing insertions that we expected
<sadmac2> Keybuk: so its just down to that
<sadmac2> adding the flag costs us 64 bits (128 if you allocated it with a free-list-based malloc)
<sadmac2> Keybuk: could we use a char for the flag, and then add the gcc flag for struct packing to get rid of the overallocation?
<sadmac2> damn, but putting it in an unpacked struct would still round it up again
<Keybuk> indeed, very expensive for a single bit flag
<sadmac2> Keybuk: we could keep a global list of cursors
<Keybuk> that's 64-128 bits each
<sadmac2> Keybuk: but only for the cursors
<sadmac2> Keybuk: not for /every/ list node
<Keybuk> true
<Keybuk> you need to check for cursorness on every insertion though
<sadmac2> Keybuk: only on insert_after
<sadmac2> Keybuk: which as you said is the far rarer case
<Keybuk> no, both
<Keybuk> nih_list_add (iter->next, entry)
<Keybuk> you need to check whether iter->next is a cursor
<sadmac2> yeah
<sadmac2> the check is one comparison
<sadmac2> not a terrible expense
<sadmac2> we get a jump when it /is/ a cursor
<sadmac2> but other than that its 2 instructions
<sadmac2> oh, wait
<sadmac2> we're still on the list of cursors idea
 * sadmac2 is on to his next, more evil thought
<sadmac2> what if we added the flag field, but only when it was a cursor
<sadmac2> ?
<sadmac2> so when we call nih_list_new, which means we are getting a struct which is not attached to an object, we over-allocate by 64 bits
<sadmac2> and write 0xdeadbeafcafebabe /after/ the struct
<Keybuk> unreliable
<sadmac2> where's the hole?
<Keybuk> you don't know that all structs in the list are that long
<Keybuk> and you don't know the user isn't using that exact string :p
<Keybuk> e.g. the list head is smaller, you'd read unallocated memory
<sadmac2> user using the exact string is one in several billion billion
<sadmac2> struct not that long isn't possible due to struct member allignment
<sadmac2> (well, 32-bit folk get jacked a bit)
<sadmac2> we have a different value for cursors, and regular stack-allocated lists have stack space in front of them supplying the bullshit
<Keybuk> no, it's entirely possible
<Keybuk> I don't like even remote chances
<Keybuk> it must only succeed _if_ it's a cursor
<Keybuk> not just something that looks like one
<sadmac2> better odds? make the value a hash of the prev and next pointers
<Keybuk> no
<sadmac2> you wouldn't do well in cryptography :)
<Keybuk> it must only succeed if it is a cursor, not if it looks like one
<Keybuk> and, pointedly, it mustn't crash on the list head <g>
<sadmac2> we could set aside some memory at startup, and allocate all cursors in it
<sadmac2> then we could check address range
<Keybuk> what if you run out of that memory?
<sadmac2> get a bigger chunk and rename
<sadmac2> Keybuk: the check if on stack method works if we just enforce via documentation
<sadmac2> Keybuk: are you sure the example you found was on the stack, not a global?
<Keybuk> definitely on the stack
#upstart 2008-10-16
<aent> hey... I'm not too familiar with upstart currently, but I'm currently working on a project where I need something to manage running processes from the project, start them, do dependency management, basically what upstart does, but on a higher level
<aent> is upstart's code suitable for that? or is it tightly integrated into being specifically for init? is there something better for what I'm trying to do?
<aent> its basically needs to monitor processes to make sure they don't die, if they do, bringing down the dependencies and bring them back up on the proper order after the other processes have started, allow different processes to be started and stopped, etc
<sadmac> aent: upstart doesn't do dependencies in the traditional sense. monit might be more what you need
<aent> hmm ok
<sadmac2> Keybuk: what is the purpose of the allocater being pluggable in nih_alloc?
<Keybuk> future stuff
<sadmac2> Keybuk: I was toying with rewriting nih_alloc, and that would have to go
<Keybuk> rewriting it how?
<sadmac2> Keybuk: replace malloc + header with mmap + free list + trailer
<sadmac2> basically re-implementing malloc inside libnih (in such a way that we get information out of it that lets us cull most of the header overhead)
<notting> is there a reason to not use the system malloc? desperately needed in signal handlers?
<Keybuk> is the header overhead really that worrysome?
<sadmac2> Keybuk: for smaller strings it can be
<sadmac2> Keybuk: I think I can get it down to 1 or 2 bytes for most blocks
<sadmac2> and I can do the multi-parent deal
<Keybuk> this doesn't seem worth optimising at this point? :p
<sadmac2> meh. prolly not
<sadmac2> Keybuk: I think in libnih head you somehow reverted the dbus timeout change by mistake
<sadmac2> that'd explain why reverting looked so weird
<Keybuk> it's entirely possible
<Keybuk> in fact, I think you're right
<sadmac2> it'd also explain why the timeout issues seemed to be resurfacing
<sadmac2> also build RPM packages.
<sadmac2> wrong window/button/appendage
<sadmac2> general desk disaster
<sadmac2> Keybuk: are you thinking about rolling another libnih? I think what we have now in terms of foreach is far better than what's there
<sadmac2> it does something a little unexpected, but nothing idiotic.
<sadmac2> also I think a 0.5.1 upstart might be welcome, if only to update the manpages
#upstart 2008-10-17
<sadmac2> Keybuk: I think I have an idea for the linked list issue
<Keybuk> oh?
<sadmac2> Keybuk: for cursor objects, why not set the prev pointer to something weird?
<sadmac2> Keybuk: and then have them keep the real prev value in a 3rd element
<sadmac2> so in the insert/delete/etc methods if (prev == (void *)0x2) //the real prev is off the end of the structure, and this is a cursor
<sadmac2> we could even set it NULL, since I think no valid list structure should ever have null pointers
<Keybuk> and have add_after/add know to check for it?
<sadmac2> those, and possibly a few others
<sadmac2> it could touch quite a few functions given how drastic a change it is
<sadmac2> or rather how drastic a departure from the list structure it is
<sadmac2> but all the information you need to operate on the list is always available
<sadmac2> the other thing we need (and we need this for any cursor-based solution) is an NIH_LIST_PREV and NIH_LIST_NEXT set of macros, since just grabbing the pointers is unreliable from within the linking application
#upstart 2008-10-18
<ST47> OK, I give up.
<ST47> I'm trying to use upstart to run and maintain a perl script - restart it if it dies, etfc
<ST47> I created a file in /etc/event.d
<ST47> start on botstart
<ST47> and then the exec /home/st47/Perl/test.pl
<ST47> However that script never executes
<ST47> main process terminated with status 126
<ST47> never mind, I suck, that's not really the error. It returns status 127. 
<ST47> That's supposed to be a $path error, however there shouldn't be any missing dependencies...
<ST47> when I run the command from bash as root, even from the /etc/event.d working directory, it executes properly.
<ST47> When I change it from a straight /home/st47/Perl/test.pl to perl /home.... it returns error code 2
<laga> hey guys.
<laga> where can i find documentation for the 0.3.9 event syntax? i'm told upstart.ubuntu.com only has documentation for 0.5.0
#upstart 2008-10-19
<Knoobuntu> Hi.  I was snooping around the Upstart web site looking for bug/status of logd (bootlog.)  Anyone know where I should look or have info on when this will be addressed?  Found nothing on Launchpad for search results: logd or bootlog.
#upstart 2009-10-12
<seanius> hi guys, is there an authoritative guide for job file syntax somewhere?  most of the stuuff on the upstart wiki seems more "proposal-ish"
<seanius> specifically wrt the version in hardy
<Keybuk> man 5 init
<Keybuk> oh, in hardy, no
<seanius> yeah i've already heard through the grapevine that it might have changed since then
<Keybuk> yes, and it will change again
<seanius> should i just forget about trying then, and use something like rc.local?
 * seanius would prefer inittab, but that's gone now it seems
<seanius> or, is inittab supported in the -compat-sysv package but just not shipped by default?
<seanius> ...it seems not (beyond default runlevel at least)
<Keybuk> it's not supported, no
<seanius> sorry for flooding teh channel btw :)
<seanius> i have a service i'd like to run with supervision (i.e. "respawn") which should start after ssh/networking/postgres is brought up
<seanius> is this possible to do in hardy's 0.3.9 upstart?
<ion> Probably best to start it when rc2 has finished. But i donât remember the precise syntax for 0.3.
<ion> Something like
<ion> start on EVENTSPEC
<ion> service
<ion> respawn
<ion> exec yourservice
<seanius> okay, i htink i can take that and what's sitting in event.d to try something, thanks!
<ion> where EVENTSPEC is the correct string to match whatever happens when rc2 has finished.
<seanius> ion: okay so that would be something like "start on started rc2"?
<seanius> ...which is output from initctl events when i run "telinit 2"
<ion> I take it thereâs a /etc/event.d/rc2?
<seanius> yeah
<ion> Do you get a âstopped rc2â when rc2 finishes?
<seanius> i think i get that if i switch to runlevel 3
<ion> Youâll probably want âstart on stopped rc2â or something like that.
<ion> Hmh, not that, then...
<seanius> yeah this is what i get http://paste.debian.net/48855/
<seanius> this is switching from 2->3 and then back from 3->2
<seanius> oh, it looks like it is saying "stopped rc2" last then isn't it...
<ion> Did stopped rc2 happen before or just after switching to a new runlevel?
<seanius> it looks like it's the last thing that happens when switching *to* that runlevel
<seanius> so then i'd want "start on stopped rc2"
<ion> Alright, that should work.
<seanius> and if i want to stop it on reboot/shutdown would that be "stop on runlevel [06]" ?
<ion> Look at what the tty* jobs do, that should work.
<seanius> ah, just multiple stop stanzas
<ion> The [016]-style pattern might only be supported in a newer version of Upstart.
<seanius> there was a runlevel [!2] in the rc2 file at least
<ion> Ok, it probably would work then.
<seanius> allright, looks like we're up and running
<seanius> thanks for the help
<Keybuk> ion: I think I always used fnmatch() for that
<akk> Hi! I updated karmic yesterday and now I can't boot with my self-compiled kernel -- mountall complains about /proc/filesystems, I think because I'm not using an initrd.
<akk> I'm wondering if this is a permanent change, and if there's any workaround to boot without initrd?
<sadmac> Keybuk: what are operations that one does on a device node that don't start with opening it and acting on the resulting descriptor?
<JanC> akk: I've been looking at it a bit & I think a solution/workaround would be to edit the existing "mountall.conf" upstart script to add a 'pre-start' script that makes sure /proc (and maybe other requirements?) exist before mountall runs
<JanC> but Keybuk or somebody else probably knows more about what the "right" solution would be  ;)
<akk> JanC: Thanks for looking! Wouldn't I have to delay running mountall at all until it had mounted root readonly?
<akk> Though d found that mountall 0.1.8 worked (two revs back).
<JanC> akk: whatever requirements mountall has should go in the pre-start script (or in a separate upstart job on which mountall depends?)
<chergert> can i have multiple "start on " lines in an upstart config? or do they need to be on one line?
<mugwump> Anyone have any tricks for debugging early boot process with upstart?
<Keybuk> sadmac: none that I can think of
<sadmac_> Keybuk: well the mount command takes the path, but that can be gotten around...
<sadmac_> Keybuk: a shocking proposal: axe device nodes. replace with connect(AF_DEVICE...)
<Keybuk> sadmac_: then you can't alias them ;)
<Keybuk> the network people (who have this) are trying to go the other way
<Keybuk> akk: this is generally a channel for Upstream issues, try #ubuntu
<Keybuk> akk: though because I'm nice, see bug 447947
<sadmac_> Keybuk: sure you can. DNS can alias IP addresses, why can't some similar service alias major/minor pairs?
<Keybuk> sadmac_: how so?
<Keybuk> chergert: no.
<Keybuk> mugwump: --debug on the kernel command-line is always fun
<Keybuk> but you need quick reactions on the old scroll lock key ;)
<Keybuk> (and a scroll lock key, bloody netbooks!)
<mugwump> heh
<mugwump> ctrl+s does the same thing on the console, no?
<Keybuk> no...
<Keybuk> :(
<akk> Keybuk: Thanks!
<sadmac_> Keybuk: basically you have a gethostbyname equivalent like getdevicebyname. you pass it a name, and it communicates to some service (probably a very re-invented udev) and returns a major/minor in a struct socaddr_device
<mugwump> well, that's useful, thanks.  I actually ended up reinstalling because I couldn't figure out why the system wouldn't boot...
<Keybuk> sadmac_: device filesystems are a more popular way of doing that <g>
<sadmac_> Keybuk: but they suck.
<mugwump> it would be nice to have details like that on the upstart man page
<Keybuk> sadmac_: the only API I can think of that sucks more is, err, DNS
<Keybuk> mugwump: it's a good point that init(8) doesn't document the options, could you file a bug about that?
<sadmac_> Keybuk: if you tried out a web browser, and when you launched it instead of a website it gave you a blank window and the glade interface designer and said 'draw yourself a UI', and it did this every time you started it would you keep it?
<mugwump> to which bugtracker?
<mugwump> launchpad?
<sadmac_> Keybuk: by that logic, where the fuck does the kernel get off giving userspace a some-assembly-required interface to hardware?
<Keybuk> mugwump: http://launchpad.net/upstart/+reportbuf
<Keybuk> mugwump: http://launchpad.net/upstart/+reportbug
 * mugwump reports a buf
<Keybuk> it might be +filebug ;)
<mugwump> I'll find it :)
<mugwump> Keybuk: https://bugs.launchpad.net/upstart/+bug/449883
<mugwump> though it would be nice to have an option which starts a shell or something and requires you to prod it to make the next event fire
<mugwump> quite often in that sort of situation I'll be doing diag checks between init scripts
<Keybuk> yes, we should have that
<Keybuk> in fact, my thought is that init will look at the kernel command-line options
<Keybuk> and if any job exists with the names given, it will start that
<mugwump> sure, why not
<Keybuk> so you could have /etc/init/emergency.conf containing the code for a console root shell
<mugwump> the decision to run as real init, it makes that based on the pid, right?
<Keybuk> then if you stuck emergency on the kernel command-line then it would give you that
<mugwump> that sounds pretty clean
<Keybuk> that'd same code would make implementing things like "single" really easy too
<Keybuk> (and most importantly let the mobile phone developers *not* have that <g>)
<mugwump> do you use lxc for testing already?
<mugwump> It should let you start a process that thinks it's pid 1
<mugwump> and can't see any other processs
<Keybuk> no, the test suite is pretty comprehensive though
<mugwump> ah, good to know
<mugwump> presumably you mock that somehow in the test suite
<Keybuk> actually, very little requires that init be pid 1
<Keybuk> the only reason, in fact, is because sysv let people call init to change the runlevel ;)
<Keybuk> if you build upstart a certain way, you can cheerfully run it as any user
<Keybuk> and use that one for testing
 * Keybuk does that a lot
<mugwump> reparenting I would have thought was pretty important
<mugwump> or I guess not, maybe you only care about immediate children
<Keybuk> not really, we don't get told about it
<mugwump> do you get SIGCHLDs from them?
<Keybuk> eventually yes
<Keybuk> but there's no use for that, because we don't know what process it was ;)
<Keybuk> pid 4131 died ... that's nice
<mugwump> heh
<Keybuk> kernel helpfully reaps info from /proc first
<Keybuk> not that it would be reliable anyway
<mugwump> or portable... any bsd users?  :)
<Keybuk> I'm utterly uninterested in portability
<Keybuk> Upstart is a Linux program
<mugwump> fair enough, you won't find me complaining or submitting portability patches
<Keybuk> I've suggested to the Debian/kFreeBSD nutters that they branch the code and rewrite it to be a BSD program ;)
<sadmac_> ugh. I keep getting to places where I want to commit, but I don't want to because it would mean committing or digging out the piles of printfs that I want to still be in place for the next phase of implementation.
<ion> keybuk: Btw, thereâs the mnt->device "none" change sitting in my branch. Not very important, but anyway. :-)
<ion> keybuk: The latest commit in the mountall repo breaks the code. Iâm not quite sure what itâs attempting to do.
<ion> Ah, now i get it. Yeah, just a missing ||
<ion> keybuk: Fix in my branch.
<Keybuk> oh, the fix is already in mine
<ion> Alright
<ion> uncommited and pushed; my branch still contains the (not very important) mnt->device "none" change.
<Keybuk> the whuh?
<Keybuk> oh that, I'll merge that in
#upstart 2009-10-13
<mugwump> sadmac2: re: committing printf's, doesn't bzr have an equivalent of 'darcs record' / 'git add -p'
<mugwump> ?
<mugwump> assuming upstart is in bzr..
<sadmac2> mugwump: its a bit too interleaved for that to be easy
<mugwump> I tend to rebase a lot with git to preen my changes down
<mugwump> it's a longer path, but hugely satisfying once you've travelled a while down it...
<mugwump> eg, knowing that no change breaks the test suite makes finding regressions pretty easy
<ion> Yeah, i constantly use rebase -i for unpushed changes.
<sadmac2> ion: any idea why the macruby interpreter rakefile would run dtrace
<sadmac2> ?
<ion> sadmac: I have no idea.
<ion> Something to do with unit tests maybe?
<sadmac2> ion: something to do with probe points I think
<Daviey> What is the best way of having an upstart job depends/requires on another process, that is still sysv?
<Daviey> Am i correct in saying that upstart checks "start on starting xyz" sagainst upstart jobs only?
<Daviey> Additionally, does anyone have an example of a job that sources variables from a conf file, for the exec line?
<Daviey> is it as trival as you would expect?
<sadmac> Daviey: you can add "initctl emit" lines to your sysv scripts to make them emit events, which offers upstart a way to interact with them
<ion> keybuk: A random thought about prioritizing fscks and mounts: with the current system, if weâre already running fsck for /notimportant and then a partition for /important on the same device appears, itâs not possible to get /important queued any sooner than after the /notimportant check. How about this instead: donât lock anything and start everything in parallel, but for each device, have only one fsck (the one deemed most important at any moment) running ...
<ion> ... with normal ionice prioritiy, and ionice all other instances âidleâ. If weâre running a fsck for /notimportant and /important appears, the fsck for /notimportant would be changed to idle priority until the check for /important finishes.
<ion> keybuk: I take it that change would be too big for karmic at this point?
<ion> if i came up with a patch, say, today :-P
<Keybuk> can you change the io priority of other processes?
<Keybuk> if you can make the patch work, I can merge it
<Keybuk> still have the fsck progress patch to merge
<ion> /usr/bin/ionice has a -p PID parameter, so it should be possible.
<Keybuk> neat
<sadmac> either something's wrong with my git tree, or somethings wrong with cscope, or sys_ioprio_set is weird
<sadmac> Keybuk, ion: sys_ioprio_set(int which, int who, int ioprio)
<sadmac> I could disambiguate that a bit if I could find the actual definition
<sadmac> ... is it written in ASM?!
<sadmac> no, its just missing...weird...
<Keybuk> it's probably under arch/
<sadmac> Keybuk: I've grepped. its nowhere
<sadmac> Keybuk: every entry.S mentions it. syscalls.h mentions it. No source file mentions it.
<ion> The ioprio_set manpage is clear enough. :-)
<Keybuk> sadmac: fs/ioprio.c:75 no?
<sadmac> Keybuk: ah. figured it was something like that.
<sadmac> worthless macros making the world a better place.
<ion> keybuk: Is a simple check for (mnt->tag > TAG_OTHER) a good rule for prioritizing running fsck instances?
<Keybuk> ion: yes
<TomJaeger> Bug #439138 is getting to be real frustrating.  Could someone explain what the deal with console owner is?
<Keybuk> ok
<Keybuk> it's about time I wrote this down
<Keybuk> and please do feel free to tidy up what I write and paste it liberally ;)
<Keybuk> in Linux, we have consoles
<Keybuk> but really we mean Virtual Terminals (VTs)
<Keybuk> and we have TTYs too
<Keybuk> not to mention Pseudo-Terminals
<Keybuk> (PTYs)
<Keybuk> it's all a bit of the kind of jumble sale you get after 40 years of different solutions to different problems
<Keybuk> we can lump them together under the description "terminals" for the next bit
<Keybuk> we also have processes
<TomJaeger> okay, so far so good
<Keybuk> now, processes have a lot of odd little details
<Keybuk> they have a parent
<Keybuk> they have a session
<Keybuk> and a session has a foreground process group
<Keybuk> and a background process group
<Keybuk> now Stevens devotes an entire chapter for this
<Keybuk> and nobody but me apparently understands it ;)
<Keybuk> (and hopefully the guy who maintains the kernel side)
<Keybuk> but it works something like that
<Keybuk> each process is part of a session
<Keybuk> a process may begin a new session by calling setsid()
<Keybuk> every process that init creates is in its own session
<Keybuk> the process is also then the leader of the foreground process group of that session
<Keybuk> new processes are also in that session
<Keybuk> and in that process group
<Keybuk> unless otherwise placed into a new process group
<Keybuk> (setpgrp)
<Keybuk> any new process group is a background process group
<Keybuk> so now, you have a bunch of sessions
<Keybuk> each session has a bunch of process groups, one of which is the foreground process group
<Keybuk> each process group has a bunch of processes, one of which is the leader
<Keybuk> with me so far?
<TomJaeger> okay
<Keybuk> right
<Keybuk> so this all has to do, fundamentally, with terminals
<Keybuk> and who gets the signals
<Keybuk> when the leader of the foreground process group of a session opens a terminal device (without O_NOCTTY) that becomes the CONTROLLING TERMINAL of that process group
<Keybuk> (in fact of that session)
<Keybuk> the terminal and the session become bound to each other
<Keybuk> you can fake this another way by opening a terminal device without O_NOCTTY (or having one passed to you) and then calling the TIOCSCTTY ioctl
<Keybuk> ok
<Keybuk> so
<Keybuk> terminals, controlling terminals and processes
<Keybuk> here's where this gets fun
<Keybuk> if the controlling terminal is hung up, SIGHUP is sent to the foreground process group
<Keybuk> if ^C is pressed on the controlling terminal, SIGINT is sent to the foreground process group
<Keybuk> if ^Z is pressed on the controlling terminal, SIGTSTP is sent to the foreground process group
<Keybuk> (you're getting the idea here I guess)
<TomJaeger> yup
<Keybuk> so this is how the relationship between magic key presses and signals gets established
<Keybuk> shells care about this a lot
<Keybuk> (and yes, when you use & that becomes a background process group :p)
<Keybuk> (and when you use | they are all in the same process group)
<Keybuk> anyhoo
<Keybuk> onwards
<Keybuk> now
<Keybuk> this controlling terminal business applies to all terminals
<Keybuk> whether they be true terminals (which linux doesn't have)
<Keybuk> or virtual terminals
<Keybuk> or pseudo terminals
<Keybuk> so this is as true for your ssh login as VT1
<TomJaeger> okay
<Keybuk> you can always access your *controlling terminal* using /dev/tty
<Keybuk> it's a badly named device node
<Keybuk> it may also be called /dev/ttyS0 or /dev/pts/4 etc.
<Keybuk> so right
<Keybuk> Linux has a bunch of virtual terminals
<Keybuk> these are the things we think of when we say "console" but we're using that wrong
<Keybuk> virtual terminals behave just like ordinary terminals
<Keybuk> they can be the controlling terminal for a process group
<Keybuk> but, unfortunately, stacked on top is the linux vt API
<Keybuk> (they didn't think to make it separate)
<Keybuk> so the stuff to set fonts
<Keybuk> to place it in raw or graphics mode
<Keybuk> to create new vts
<Keybuk> to switch vts
<Keybuk> etc.
<Keybuk> is all loaded into the tty api
<Keybuk> so, in order for X to function, it needs a VT
<TomJaeger> right
<Keybuk> and it needs access to that VT in interesting and familiar ways to place it into raw and graphics mode
<Keybuk> and so on
<Keybuk> it also needs to know if the *current VT* is switched
<Keybuk> so it opens the VT device it wants (/dev/tty7)
<Keybuk> and that becomes its controlling terminal
<Keybuk> so if you were to delete VT7, X would get SIGHUP :)
<Keybuk> now, on VT1-6 you have getty
<Keybuk> on VT8 you have usplash
<Keybuk> and so on
<Keybuk> this is all fine and dandy
<Keybuk> except there's this last mystical piece
<Keybuk> /dev/console
<Keybuk> /dev/console is, like /dev/tty, a fake device
<Keybuk> it points at the currently active VT
<Keybuk> whatever that is
<Keybuk> but it *behaves* like a terminal in its own right
<Keybuk> (whereas /dev/tty just behaves as a proxy for the underlying terminal)
<Keybuk> now, Upstart has a few knobs to customise the standard input, output & error file descriptors
<Keybuk> normally it just starts all jobs with them as /dev/null
<Keybuk> but for *emulation of sysvinit* it has two other options
<Keybuk>  1. set them to /dev/console
<Keybuk>  2. set them to /dev/console and issue the TIOCSCTTY ioctl()
<Keybuk> (console output, console owner)
<Keybuk> now, if your current VT is 7 (X)
<Keybuk> and you start a job that has console owner in it
<Keybuk> the new process will *take the terminal away from X*
<Keybuk> X gets SIGHUP
<Keybuk> and either hits 100% CPU or crashes
<Keybuk> so this is clearly BAD
<Keybuk> the problem really is that things need a "console" at all
<Keybuk> jobs that require interaction should DO IT THEMSELVES
<Keybuk> they should open a VT, switch to it, and ask there
<Keybuk> or they should use usplash to do it
<Keybuk> or they should use X
<TomJaeger> hmm
<TomJaeger> this (switching to an unused VT, asking for a pass, and then switching back) doesn't sound too hard
<Keybuk> "start usplash"
<Keybuk> you can do that under X
<Keybuk> (please don't try it unless you have an Intel card right now <g>)
<TomJaeger> "start usplash" doesn't do anything here (i965)
<Keybuk> really?
<Keybuk> that's odd :)
<TomJaeger> I get this: usplash stop/pre-start, process 2357
<TomJaeger> but either way, that doesn't seem to be an option right now if it breaks on non-intel hardware.
<Keybuk> that's a bug we have to fix for release anyway
#upstart 2009-10-14
<TomJaeger> is there a bug #?
<Keybuk> not sure :p
<Keybuk> I don'
<Keybuk> I don't track things using LP
<TomJaeger> but usplash is the plan for fixing the issue at hand?
<Keybuk> no idea
<Keybuk> don't have a plan yet
<TomJaeger> there's also the issue that if X is started while cryptsetup is asking for a password, it'll activate its own virtual terminal.
<Keybuk> yes, that's fine
<Keybuk> cryptsetup is asking with usplash on vt8
<Keybuk> X starts on vt7
<TomJaeger> won't X switch to vt7?
<Keybuk> yes
<TomJaeger> how is this not a problem?
<Keybuk> it is
<Keybuk> do you have a solution?
<TomJaeger> no
<Keybuk> me neither ;)
<TomJaeger> okay then, thanks for your input, I'll go ahead and post the relevant parts of the conversation on the bug tracker, so that everybody is on the same page.
<ion> keybuk: http://bazaar.launchpad.net/~ion/ubuntu/karmic/mountall/fsck-locking/revision/109
<Keybuk> ion: that looks a bit hairy :-/
<ion> keybuk: Please elaborate.
<Keybuk> as in that's O M G that's a large patch
<Keybuk> I'm scared to apply that without a massive amount of testing
<ion> The deletions basically rip out the queuing, so that checks are started in parallel again, and the additions basically keep a list of active checks sorted by priority and call setpriority and ioprio_set, which are allowed to fail.
<Keybuk> right, that's what I'm worried about ;)
<sadmac2> ion: did you have an ioprio_set from glibc? last time I used it it was a manual syscall(__NR_ioprio_set)
<ion> I used syscall()
<sadmac2> ion: yeah. someone should send patches for that.
<ion> keybuk: How about putting the patch to ~ubuntu-boot PPA and trying to get a lot of people to test it? :-P
<katre> I'm having some trouble trying to grab the upstart source (to track down a bug in upstart on sparc)
<katre> Anyone mind helping me figure what's up?
<Keybuk> how are you trying to grab it?
<katre> so I have a machine (also sparc) running Jaunty
<katre> I installed bzr, which grabbed version "Bazaar (bzr) 1.13.1"
<katre> I then ran "bzr get https://code.launchpad.net/upstart/trunk"
<katre> and got an error: bzr: ERROR: Unknown repository format: 'Bazaar repository format 2a (needs bzr 1.16 or later)\n'
<Keybuk> you almost certainly need bzr 2.0
<Keybuk> there you go then
<katre> okay, is that installable on jaunty?
<Keybuk> yes
<Keybuk> http://bazaar-vcs.org/
<katre> how would I do that?
<katre> is that installable on jaunty from a package?
<Keybuk> the documentation is on the bazaar website
<Keybuk> it is, yes
<katre> because, let me be frank, if I cannot use the current release of ubuntu to do ubuntu development, that is amazingly limited
<Keybuk> for Upstart, my rule tends to be that it should compile and build on the current stable release
<Keybuk> but you might need the current *development* release to run it
<Keybuk> since the next stable release actually uses upstart much more fully than before, that's going to be increasingly true
<katre> yeah, I can't install the development release, because of this sparc bug I'm looking into
<Keybuk> *nods*
<Keybuk> bzr is a bit of a special case sadly
<Keybuk> the bzr devs have pushed 2.0 quite strongly
<Keybuk> and there are lots of strange issues when interoperating
<katre> so, anyway, I don't see a package for bzr 2.0. where do I grab it?
<Keybuk> not to mention LP has already been upgrade to 2.0
<katre> yeah
<Keybuk> katre: if you're on sparc, you may have to grab the source from the PPA and build it youself
<Keybuk> https://edge.launchpad.net/~bzr/+archive/ppa certainly has the 2.0 source package
<ion> keybuk: So... What to do with the fsck priority patch? Do you see a way to make it considerably smaller or to get enough testing, or should i just forget about it? :-P How about putting a it to the ubuntu-boot PPA for testing?
<katre> thanks
<Keybuk> ion: I just don't see myself being able to convince the release manager to let me ship that the day before we freeze
<Keybuk> especially since, right now, it looks like we've fixed all the bugs
<Keybuk> we literally freeze tomorrow, from then on it's all about getting a set of cd images that work
<Keybuk> introducing changes like that ... difficult to sell
<ion> Alright
<Keybuk> but I'll certainly merge it straight into lynx ;)
<ion> Yeah
<katre> Geez, bzr wants to install 61 packages as build dependencies?
<katre> including x windows fonts, I see
<katre> probably due to wanting graphviz, I guess
<katre> Okay, so now I have a working bzr, I can grab the upstart trunk and the libnih trunk.
<katre> How do I build those? Neither the README nor the HACKING files actually answers that
<katre> the README refers to an INSTALL file that does not exist
<katre> I tried running 'autoconf' in each, and it rana  bit and then errored out
<katre> (I can provide the error message if that helps)
<katre> actually, I misspoke. autoconf runs fine, then ./configure runs a bit and finally gives "configure: error: cannot find install-sh or install.sh in "." "./.." "./../..""
<sadmac> katre: you configure libnih first. Then you go into the upstart folder and do ../relative/path/to/libnih/nihify and then you configure upstart.
<sadmac> see if that helps
<katre> sadmac: Thanks. I get the same error for libnih
<sadmac> katre: then its a system thing. what platform is this that you had to compile your own bzr?
<katre> Looks like my problems are firstly that libtool isn't installed and didn't get pulled in as part of build-dep
<sadmac> oh. that'll do ya
<katre> and secondly that I've never used autoconf before and there's no documentation of the exact commands to run to get from a fresh source tree to something compilable
<katre> sadmac: I'm on sparc
<sadmac> katre: what distro?
<katre> ubuntu server, jaunty
<sadmac> katre: autoreconf -i
<katre> thanks
<katre> autoreconf wants cvs? okay, let's install that
<sadmac> wat
<katre> it gave an error that cvs wasn't found. I installed it and now it's running
<katre> between that and bzr pulling in all of X I've installed a heck of a lot of stuff just to build upstart
<katre> I feel like I'm getting closer
<katre> autoreconf looks like it worked fine, but when I run configure it ends with this error
<katre> | /configure: line 20354: PKG_PROG_PKG_CONFIG: command not found
<katre> | ./configure: line 20367: syntax error near unexpected token `0.22'
<katre> | ./configure: line 20367: `PKG_PROG_PKG_CONFIG(0.22)'
<katre> I did run aclocal before I ran autoreconf
<katre> Ah, google suggests an unresolved dependency on pkg-config
<katre> let's try it...
<katre> nope
<sadmac> katre: sounds like a missing macro... Scott's really better with this stuff...
<katre> yeah, I had to re-run aclocal and autoreconf
<katre> and now it dies complaining it needs a newer dbus
<katre> I give up
 * sadmac thought autoreconf -i did the aclocal thing
<katre> if I can't actually build upstart in the currently released ubuntu distro, I can't do it
<sadmac> ubuntu doesn't ship the newer dbus either?
<sadmac> hmm
<katre> maybe it does, I'm deep into cargo cult compiling right now
<sadmac> Fedora didn't, but I'dve thought ubuntu did
<katre> | checking for DBUS... configure: error: Package requirements (dbus-1 >= 1.2.16) were not met
<katre> | apt-cache policy dbus: Candidate: 1.2.12-0ubuntu2.1
<sadmac> katre: upstart uses a lot of dbus that other applications never have. A result has been that we found problems with dbus that nothing else ever stumbled on. That dbus release has the resulting pile of fixes
<katre> Sure
<katre> I don't doubt there's a good reason
<katre> I just know that I've been banging at this for a bit and determined that there's no easy way to get into mupstart development
<sadmac> katre: what version of upstart do you have on the system now?
<katre> and since no one else seems particularly worried that the next version of upstart crashes on sparc, it'll stay until someone else tries to fix it
<katre> apt-cache policy says 0.3.9-8
<katre> this all started when I tried upgrading my sparc system to karmic, and the new upstart crashed in the install
<katre> so I will either keep these servers at jaunty or find a new distro that actually supports sparc (I am aware that sparc is not officially supported by ubuntu but it's been fine til now)
<sadmac> katre: how did you end up building upstart because karmic crashed?
<sadmac> katre: fedora's sparc support is semi-official. that's a bit of an improvement...
<katre> I grabbed the upstart source from bazaar, thinking if I can at least build it I could try and run down the problem
<katre> but apparently I can't build upstart on jaunty without installing, so far, about 70 packages and building bazaar from source
<sadmac> katre: wait, how did the new upstart crash the install?
<katre> I could try grabbing dbus-1 and building that too, but frankly I give up, this is too frustrating
<katre> https://bugs.launchpad.net/ubuntu/+source/upstart/+bug/436758
<katre> I was trying to get to the point where I can run 'make check' and see if I can duplicate the failure mentioned here
<katre> (I did grab the source for the 0.3.9 version in jaunty and it passes all checks, as expected)
<sadmac> katre: you could disable the dbus check in the makefile. you'll get piles of errors in make check though (since it'll trip over all those bugs that were fixed)
<katre> yeah
<sadmac> katre: you could also try a scratch-install of karmic on a vm.
<sadmac> katre: or you could mount a drive from a karmic live cd (if they make such things for sparc)
<katre> they don't
<sadmac> katre: I'd find whoever told Scott that make check failed and have him reproduce it wherever he reproduced it
<katre> yeah
<katre> Thanks very much for the help
<sadmac> np
<ion> Misc. rambling: instead of a âstartupâ event, 0.10 should have a âsystemâ state. Instead of âstart on startupâ, jobs would have âwhile systemâ. Shutdown/reboot would happen by running âstop system ACTION=rebootâ; all âwhile systemâ jobs would stop and there would be a halt.conf job that looks something like âaround systemâ, âpost-stop exec if [ "$SHUTDOWN_ACTION" = reboot ]; then reboot -f; else halt -f; fiâ.
<sadmac> ion: there's neither. to start on startup you just don't put any while or on stanzas
<sadmac> ion: a job with no on stanzas starts a soon as its conditions are met.
<ion> How about if you only want to start a job manually?
<sadmac> ion: you set the job in manual mode via a separate configuration file.
<ion> A âsystemâ state would be an elegant solution for halting IMO. :-P
<sadmac> ion: maybe
<sadmac> ion: its actually a nice addition to the current mechanism. All your crash-only jobs that don't need to be killed before cutting the power can just omit the while system stanza
<sadmac> ion: so "while system" is how a job indicates its sensitive to sudden power evaporation
<ion> True, true
<ion> How to tell the halt job whether to halt or reboot? Iâm not sure whether environment from the âstop systemâ command should be available to halt.confâs post-stop script.
<sadmac> ion: it would be as of now
<ion> Oh? Alright :-)
<sadmac> ion: it would /not/ be in 0.3. We fixed this.
<ion> In fact, Upstart wouldnât really need to treat the system state in a special way, there could simply be an empty /etc/init/system.conf that comes with the package. People would just be told to point to that for anything that requires some action before halting the system.
<sadmac> ion: maybe. There could be advantages to internalizing it too, since it starts so friggin early
<sadmac> ion: an idea: suppose upstart ran from initrd
<sadmac> ion: system pre-start becomes the core script of dracut as it is now. when system itself starts, upstart re-execs on the new root.
<ion> Yeah
<ion> keybuk: A summary of what we discussed above: a âsystemâ state in 0.10 could provide an elegant way to handle shutdown. Any job that requires some action just before halt/reboot would do âwhile systemâ and possibly have a pre/post-stop script, whereas on-startup jobs that donât mind the computer suddenly disappearing would not have any while/on stanzas. To reboot, one could run âstop system SHUTDOWN_ACTION=rebootâ; halt.conf could contain something ...
<ion> ... like âaround systemâ, âpost-stop exec if [ "$SHUTDOWN_ACTION" = reboot ]; then reboot -f; else halt -f; fiâ. In fact, Upstart wouldnât necessarily need to handle the âsystemâ state in any special way, a standard /etc/init/system.conf with no contents would suffice, but an implicit job would be nicer IMHO. sadmac pointed out that Dracutâs core script could be a pre-start script for the system job if Upstart is started from initramfs.
<Keybuk> how would you deal with unmounting the root filesystem?
<Keybuk> which needs all processes killed?
<Keybuk> can you mail the log to the ML so I can digest? :p
<ion> Iâm probably missing something, but couldnât one do a lazy unmount?
<Keybuk> if events had before/after periods and you had priorities that could really work
<Keybuk> ion: no, you can't lazy remount <g>
<sadmac> Keybuk: this is just before shutdown?
<Keybuk> yes
<ion> The filesystem state could have âwhile system and stuffâ and anything that needs to close files would have âwhile filesystemâ. stop system â anything depending on filesystem would stop, the filesystem job would stop (handle unmounts etc. here), the system job would stop, the halt job would stop (and shutdown the system).
<ion> keybuk: https://lists.ubuntu.com/archives/upstart-devel/2009-October/001097.html
<Keybuk> after system
<Keybuk> mount -o ro,remount /
<Keybuk> reboot -f
<Keybuk> type thing?
<ion> That could actually be just in system.confâs post-stop script, no real need to have another job just for that.
<Keybuk> good point
<ion> Iâd probably like Upstart to internally initialize a âsystemâ job with an empty job with no while/on conditions (so that it simply starts immediately), but anything in system.conf would extend that (e.g. the post-stop script to shutdown the box).
<ion> If system.conf is missing for any reason, any jobs with âwhile systemâ would still start.
<Keybuk> not a bad idea
<Keybuk> ion: that fits a pattern I've been thinking about
<Keybuk> have init parse its arguments, and each "word" is the name of a job that will be started
<Keybuk> if a config exists, it's started with the config
<Keybuk> otherwise it's just started with no config
<Keybuk> ie. you'd end up with system states for "quiet", "splash", "emergency", "single", etc.
<ion> Good idea
<Keybuk> we could just build in system like that
#upstart 2009-10-15
<stevenshiau> Hi
<stevenshiau> I am new to upstart 0.6.3
<stevenshiau> I'd like to know if any "service" in /etc/init/ will mount /dev/pts, /dev/shm... (in Ubuntu 9.10)
<stevenshiau> I can not find it
<stevenshiau> If anybody knows that, please tell me
<stevenshiau> thanks
<mbiebl> stevenshiau: the mountall binary is responsible for that â /etc/init/mountall.conf
<stevenshiau> mbiebl: Thanks
<stevenshiau> But I did not see any devpts mode or something similar in /etc/init/mountall.conf
<stevenshiau> description     "Mount filesystems on boot"
<stevenshiau> start on startup
<stevenshiau> expect daemon
<stevenshiau> task
<stevenshiau> emits virtual-filesystems
<stevenshiau> emits local-filesystems
<stevenshiau> emits remote-filesystems
<stevenshiau> emits all-swaps
<stevenshiau> emits all-filesystems
<stevenshiau> emits filesystem
<stevenshiau> Any one of them is responsible for mounting /dev/pts?
<mbiebl> It's in the source code of mountall
<mbiebl> apt-get source mountall
<stevenshiau> ok
<stevenshiau> tkx
<stevenshiau> got it
<stevenshiau> Actually I am implementing a diskless system based on this
<stevenshiau> Trying to figure out how to start the correct "services"
<stevenshiau> Haven't figure it out
<stevenshiau> Any hint will be appreciated
<stevenshiau> The problem now I have is, looks like mountall.conf is run, but there is no /dev/pts, /dev/shm...
<stevenshiau> how can I force mountall to mount them automatically?
<mbiebl> stevenshiau: you can run the mountall binary by hand
<mbiebl> it should mount those file systems automatically
<stevenshiau> yes?
<stevenshiau> Could you please show me how? Sorry for asking such a question
<mbiebl> maybe you get an error message, why it fails for you
<mbiebl> Make that "mountall --verbose"
<mbiebl> for even more debug output use "mountall --debug"
<mbiebl> you can change the mountall.conf job file and add that parameter to the mountall call
<stevenshiau> got it
<stevenshiau> Let me try here
<stevenshiau> mbiebl: looks like mountall will umount the filesystem first, then mount them again?
<stevenshiau> Is that possible I can keep the existing ones?
<ion> http://lwn.net/Articles/356764/
<Keybuk> lol
<Keybuk> saw it
<ion> Looking at http://lkml.org/lkml/2009/1/17/37 and the others like it (http://stanse.fi.muni.cz/bugs.html), perhaps they should use the GCC __attribute__ cleanup extension to handle the mutex unlocking. :-P
<sadmac> "Even if I don't unpull, I don't _ever_ want to see a driver like this 
<sadmac> outside the merge window. And dammit, James, you should have realized 
<sadmac> that.
<sadmac> And dammit, James, I'm a release engineer, not a doctor.
<ion> sadmac: Out of curiosity: is RH using or transitioning to Upstart?
<ion> RHEL
<notting> all existing releases of RHEL use sysvinit; that's obviously not going to change in an update
<sadmac> ion: ^^let notting tell you
<notting> rhel does get derived from fedora, though.
<ion> Nice, http://www.gnu.org/software/gdb/news/reversible.html
<sadmac> ion: kind of scary
#upstart 2009-10-16
<Keybuk> *cough*
<Keybuk> on job pre-start 
<Keybuk> respawn
<Keybuk> script
<Keybuk>  ...
<Keybuk> end script
<Keybuk> RESPAWNING pre-start! :p
<Keybuk> the more I think about single-process-per-job, the more I like it
<ion> :-P
<Keybuk> it just makes sense
<Keybuk> if we could get the sub-job syntax right, it might even be easy
<Keybuk> I feel sorry for the eucalyptus guys
<Keybuk> they're working so hard on their upstart jobs
<Keybuk> and I'm going to break them again <g>
<ion> :-)
<Keybuk> ooh, this new Garam Masala is good
#upstart 2010-10-18
<Benkinooby> hi, i don't have a /etc/init.conf file (the upstart man page (== init man page) lists it under FILES). what is it good for? i have all /etc/init/*.conf files which seem to save the start/stop instructions for each service.
<Benkinooby> hello?
<phretor> hi, this script http://paste.pocoo.org/show/276963/ fires an error due to exec: "Sorry, root is not allowed to run "/usr/bin/Xvfb :99 as foo:bar on host". Any help?
<phretor> anybody?
<Keybuk> yo
<phretor> hey Keybuk
<Keybuk> how goes it?
<phretor> Keybuk: ah, not bad. Could be better, though
<phretor> Keybuk: for instance, I could have my Upstart script working properly :) *that** would be "going good"
<Keybuk> :-)
<phretor> Keybuk: http://paste.pocoo.org/show/276963/ in case you can spare a few minutes
<Keybuk> there are many many problems with that script
<phretor> Keybuk: I bet, it's my first one.
<Keybuk> "start on startup" is before anything like filesystems are checked or mounted, you better not need to use the disk
<Keybuk> you don't specify an Upstart version, in the current the second start on will replace the first
<Keybuk> stop on "[016]" doesn't mean anything
<Keybuk> there is no "pid file" stanza
<Keybuk> everything between script...end script is a shell script
<Keybuk> calling exec on the first line means the second won't ever run
<Keybuk> there is no "emit" command - did you mean "initctl emit" ?
<Keybuk> there's no need to emit an event when a job is running - Upstart takes care of that for you
<Keybuk> (on started YOUR_JOB_NAME)
<Keybuk> oh, and does Xvfb _really_ fork?!
<phretor> Keybuk: it doesn't fork, but from a tutorial I've understood that expect fork has to do with creating a daemon. I felt it wasn't a good idea, but I decided to give it a try.
<phretor> Keybuk: (actually, there is a pid file stanza)
<Keybuk> there isn't :p
<phretor> mh, line 10?
<Keybuk> Upstart doesn't recognise it
<Keybuk> it was an early idea for doing fork following
<Keybuk> but never implemented
<phretor> Keybuk: ah, you mean that I cannot use it.
<Keybuk> right
<Keybuk> at best it'll ignore it
<Keybuk> at worst it'll error
<Keybuk> I'm pretty sure it'll just error
<phretor> Keybuk: this is slightly better, not final, though. http://paste.pocoo.org/show/277005/
<phretor> (it doesn't complain on 10.04 server)
<Keybuk> right, that's better
<Keybuk> stop on [016] should probably be stop on runlevel [016] I'm guessing
<phretor> still I'm not sure about sudo - I just need to run Xvfb as worker:worker and keep it running in case it fails (respawn)
<Keybuk> since it's clearly a user thing, maybe you mean:
<Keybuk>   start on runlevel [2345]
<Keybuk>   stop on runlevel [!2345]
<Keybuk> ?
<Keybuk> yeah, you need sudo or su there
<Keybuk> or start-stop-daemon which has insane syntax
<phretor> Keybuk: w/o exec?
<Keybuk> exec doesn't make much difference there, it'll save a pid though
<phretor> I was told not to mix Upstart with start-stop-daemon
<phretor> and damn it "Sorry, user worker is not allowed to execute '/usr/bin/Xvfb :99' as worker:worker on hostname"
<Keybuk> that's a sudoers issue
<phretor> and it clearly won't start
<phretor> well, I have "root    ALL=(ALL) ALL"
<Keybuk> might be an X issue of course
<Keybuk> but that's not an upstart problem :)
<phretor> Keybuk: let me try with another command
<phretor> Keybuk: "Sorry, user worker is not allowed to execute '/usr/bin/lsof' as worker:worker on hostname"
<Keybuk> then it's sudoers
<phretor> Keybuk: dear lord :)
<Keybuk> maybe it's deciding based on the arguments ?
<Keybuk> ie "as worker execute foo" so worker needs permission to execute foo?
<phretor> -rwxr-xr-x 1 root root 1688032 2010-07-21 08:53 /usr/bin/Xvfb
<phretor> and worker has ALL=(ALL) ALL
<Keybuk> no idea
<Keybuk> I'm not a sudo expert
<phretor> Keybuk: ok, I will look at that. So, Upstart doesn't expose any feature for lowering the privileges, right?
<Keybuk> right
<phretor> Keybuk: I need to spawn the same server (i.e., same executable) on different ports and have Upstart to respawn each instance if it dies. Do I have to create multiple scripts (one for each port) or what?
<Keybuk> no, you can use INSTANCE
<Keybuk> though you would obviously need a master script to start each one
<Keybuk> e.g. you could add
<Keybuk>   instance $DISPLAY
<Keybuk> and then
<Keybuk> start YOUR_JOB DISPLAY=:100.0
<Keybuk> start YOUR_JOB DISPLAY=:101.0
<Keybuk> etc.
<Keybuk> and you'll get two copies with different $DISPLAY variables
<Keybuk> they'll show up in lists as YOUR_JOB (:100.0) ...
<phretor> Keybuk: and Upstart will take care of each of them individually?
<Keybuk> yes
<phretor> so sweet
<phretor> Keybuk: and with "start on ls_screen" I will make it starting when my other script ls_screen has started. Right?
<Keybuk> err, no, because that will only start one instance
<Keybuk> you'd have to have an /etc/init/somejob.conf that had:
<Keybuk> start on filesystem
<Keybuk> script
<Keybuk>   start otherjob DISPLAY=:100.0
<Keybuk>   start otherjob DISPLAY=:100.1
<Keybuk> end script
<Keybuk> etc.
<Keybuk> in it
<phretor> mh, but I want each otherjob to start as soon as ls_screen is running
<phretor> ah ok, I got that
<phretor> so I will have to make allotherjobs.conf
<Keybuk> right you have one start the other
<Keybuk> if you need it to wait, use an event, e.g.
<Keybuk>   initctl emit --no-wait start_other_job DISPLAY=:100.0
<Keybuk> and then
<Keybuk> start on start_other_job
<Keybuk>   \ and some_other_event
<Keybuk> (you'll get $DISPLAY from the event for free)
<phretor> the pool of otherjob is actually a set of firefox instances, and the first job is Xvfb
<phretor> so, yes, I need Xvfb to be available before starting the pool of browsers.
<Keybuk> right, so you could have the job that starts the others started on xvfb
<phretor> so, start browser PORT=1234, start browser PORT=2345, ...,  within script/end script or outside script/end script?
<Keybuk> right
<Keybuk> within
<phretor> Keybuk: and, how do I prevent it to run "spontaneously"? I just omit the start on [...] right?
<Keybuk> right
<phretor> Keybuk: in this case I am lucky, because if Xvfb dies also firefox dies. But, what if I wanted to make process P2 stop when P1 stops? Is "stop on stopped P1" implemented already?
<Keybuk> yes
<Keybuk> or on stopping
<Keybuk> if you want "stop Xvfb" (explicit) to explicitly wait for the firefoxes to die
<phretor> Keybuk: and I can leave the stop on runlevel [!2345] on firefox right?
<Keybuk> you can
<phretor> I know, it won't ever happen
<phretor> because Xvfb stops when !2345 (i.e., before firefox)
<Keybuk> same time as ?
<phretor> yep
<phretor> Keybuk: http://paste.pocoo.org/show/277025/
<Keybuk> ok?
<phretor> ls_browser PORT=4242 works fine, but for P in $PORTS; do start ls_browser PORT=${P}; done in ls_screen's script/end script won't start themm
<Keybuk> err I'm not sure what ls_screen is?
<Keybuk> isn't that ls_screen above>
#upstart 2010-10-19
<phretor> good day
<phretor> anyone on this issue? http://serverfault.com/questions/192416/whats-the-correct-way-to-start-a-ubuntu-upstart-script-from-another-upstart-scri
#upstart 2010-10-20
<Wil_Syd> How do I find what events are available for Upstart?
<gansbrest_> hi. Is there a way I can debug why my app fails to start after reboot?
<gansbrest_> it is defined in /etc/init.d
<phretor> Keybuk: hey!
<Keybuk> hi
<phretor> Keybuk: I had it all worked out, except for two tiny things. (1) I _really_ need the pid stanza :), (2) http://serverfault.com/questions/192416/whats-the-correct-way-to-start-a-ubuntu-upstart-script-from-another-upstart-scri
<Keybuk> it looks correct to me, should work
<phretor> it really doesn't. It's like start browser PORT=4242 is never invoked when I launch start browsers
<Wil_Syd> can you start a job if one of the start criteria is not met?
<phretor> Wil_Syd: what do you mean?
<Wil_Syd> When you test them individually are your variables set?
<Wil_Syd> Disregard >> can you start a job if one of the start criteria is not met?
#upstart 2010-10-21
<twb> Silly question: is there anything like a 1:1 correspondence between LSB dependency headers (e.g. Depends: $network) and upstart start/stop conditions?
<twb> Er, "Required-Start: $network", I mean.
<ion> Some future release of Upstart will parse those files and generate jobs with equivalent semantics internally. At the moment, no.
<twb> Hum, OK.
<twb> I was basically noticing that 1) Debian's LSB headers are correct enough that startpar is on by default now; 2) a lot of those scripts aren't upstartized in Ubuntu yet; so 3) can Ubuntu leverage pere's work for those not-yet-upstartized init scripts?
<ion> As i said, thatâs the plan, but itâs not implemented yet.
<twb> Okey dokey.
#upstart 2010-10-22
<leibs> I have an upstart task which specifies: "start on starting mountall", but mountall seems to run before it finishes.  Any thoughts on possible reasons?
<phretor> hellow
<mbiebl> leibs: maybe you want start on started
<mbiebl> on the other hand, mountall is probably not what you want
<mbiebl> start on filesystem is probably better
#upstart 2010-10-24
<ectospasm> I'm having real trouble getting upstart to work properly.  I thought I had my upstart command working OK, but it didn't survice a reboot.  Here's my config:  http://paste.ubuntu.com/518989/
<ectospasm> s/survice/survive/
<ectospasm> I'm also trying to get this upstart script to respawn the service when it dies, but it ends up starting twelve instances if igmailAgent.php:  http://paste.ubuntu.com/518992/
<ectospasm> I see what was wrong with my first one, upstart tries to parse a quoted string erroneously, so "Soma (Ubuntu PBX)" fails.  Changing the VM name to "soma" fixes it, but initctl can't find soma.conf until I comment out "console logged".  Seems like a bug to me...
#upstart 2011-10-17
<pmjdebruijn> hi again
<pmjdebruijn> does start on stopping networking
<pmjdebruijn> block networking until the job where it is defined finishes
<pmjdebruijn> because it seems it's not
<jhunt> pmjdebruijn: yes - as shown in upstart-events(8), "stopping" is a hook type event, which blocks.
<pmjdebruijn> hmmm very odd
<pmjdebruijn> start on (stopping networking) should execute this script before bringing down the networking
<pmjdebruijn> oh wait
<pmjdebruijn> networking is a boot only job
<pmjdebruijn> which explains me issue
<pmjdebruijn> networking down is an rc job
<pmjdebruijn> hmmm
<pmjdebruijn> jhunt: thanks
<vaxerdec> hello, i am looking for a way to modify environment of upstart itself, so that all children will get certain environment variables.  i.e. global export
<vaxerdec> either that, or some way to run a job that exports stuff in the environment prior to execution of any job
<vaxerdec> i read section on environment in cookbook, and in manual, and doesn't seem to be a way.  i'd have to manually change the environment in all job scripts
<vaxerdec> which is enormous duplication of course...
<vaxerdec> any way to do this?
<SpamapS> vaxerdec: interesting idea... while I look at the source code to see whats possible.. what environment variable do you want to override?
<SpamapS> vaxerdec: far as I see its not really doable in an obvious way
<vaxerdec> it's perfect candidate for use of /etc/init/init.conf
<vaxerdec> unless there are some other plans for that
<vaxerdec> well in particular i want SYSTEMCTL_SKIP_REDIRECT=1 and DBUS_SESSION_BUS_ADDRESS=
<vaxerdec> because (1) i have to contend with some systemd infection on this (fedora15) system and (2) i do not use dbus at all for any purpose and don't want it starting up as it seems to want to do, annoyingly
<vaxerdec> probably i'd use it for pulseaudio too, another irritation 
<vaxerdec> although /etc/pulse/client.conf using autospawn=0 might be all i need for the latter case
<SpamapS> vaxerdec: yeah I agree, /etc/init/init.conf would be a good place for global environment variables
<vaxerdec> everyone seems to forget that not everyone is using a huge bloated gnome operating system on their machine
<vaxerdec> i mean what about people that just want to run twm or something
<vaxerdec> at least you can do it with upstart.  systemd just puts its hands in everything.
<vaxerdec> pulls in all kinds of requirements
<vaxerdec> i mean look at this statement in shutdown(8) on a systemd-infected  system:
<vaxerdec> NOTES This is a legacy command available for compatibility only.
<vaxerdec> i mean "shutdown" is a legacy command?
<vaxerdec> shutdown?
<vaxerdec> next /bin/sh will be "legacy"
<vaxerdec> it's going to be unrecognizable... not unix anymore
<vaxerdec> it's nice mr pottering wants to make his own operating system but why does everyone else have to be sucked into whatever willy nilly new paradigm and program requirements that he feels like 
<SpamapS> vaxerdec: thats why we like upstart .. it does one thing well.. glad you see that. :)
<vaxerdec> yes, i'm trying to run my fedora system with it and it's working but i don't know if i'll be successful with fedora16
<vaxerdec> i may have to jump ship to ubuntu, i dare say
<vaxerdec> or gentoo or something
<SpamapS> slangasek has been working on making upstart an alternative in debian
<vaxerdec> that's great, i hope they don't go the systemd way
<vaxerdec> upstart versus systemd is a big split...
<vaxerdec> big pain for upstreams too
<vaxerdec> well, upstart does have their own too
<vaxerdec> native init scripts that is
<vaxerdec> i guess upstreams will be forced to make a sysv, an upstart, and a systemd
<SpamapS> Yeah, it would be good if the two shared a common "start things this way" format. :p
<SpamapS> LSB-init wasn't so bad.. this idea that the shell slows things down too much is kind of crazy IMO.. dash is lightning fast.
#upstart 2011-10-18
<vaxerdec> well anyways who cares.  taking a look around... most of my systems have dozens or even hundreds of days of uptime
<vaxerdec> i mean that's great for a laptop but you know what i hardly ever reboot that too.  mostly just for kernel updates
<vaxerdec> i was thinking the same thing... looking at the lsb-init headers... those contain plenty of dependency information.  i never knew init to actually use that data though, found in that header
<vaxerdec> and now oracle bought ksplice so maybe they'll mainstream it and i'll just never reboot :)
<vaxerdec> i guess over time after lots of updates you accumulate lots of krufty deleted inodes though still in use... but there's kexec anyways
<vaxerdec> i actually think upstart is a little overly complicated too but it's quite capable
<vaxerdec> still newbie will have really hard time understanding it i think
<vaxerdec> unlike sysvinit
<vaxerdec> i mean charts and graphs showing state transition diagrams, a cookbook, wikis, huge man pages...
<vaxerdec> the gentoo system actually looks pretty simple
<vaxerdec> i'll tell you though... it is definitely nice that i can reboot my system in about five seconds after the bios runs... actually less, seems around 3 seconds
<vaxerdec> i mean my laptop (which is on ssd)
<vaxerdec> (and i don't run any junk)
<vaxerdec> i still have problems though with reboot, just hangs when i do shutdown -r now
<vaxerdec> doesn't seem to run the rc with 6 as arg
<vaxerdec> i have to go in another terminal and run /etc/rc.d/rc 6 manually
<vaxerdec> er, that was /etc/init.conf not /etc/init/init.conf
<bradleyayerswork> hi
<bradleyayerswork> i have a job that has multiple instances, and another job that starts those instances
<bradleyayerswork> i'd like to be able to emit an event that causes all the instance jobs to restart
<bradleyayerswork> http://dpaste.com/636318/ is my instance job conf
<bradleyayerswork> the two important lines are 12 and 13
<bradleyayerswork> i was under the impression that if you have a "stop on" and "start on" stanza that has the same requirement, the "stop on" will be performed first, followed by the 
<bradleyayerswork> "start on"
<bradleyayerswork> however when i do "initctl emit genocide DOMAIN=foo", it job instance stops, but it does not start
<bradleyayerswork> i presume this is something to with it not having the ${COMMAND} parameter?
<bradleyayerswork> i'm just looking for some help :)
<philip__> hi! is there a best practice how to handle processes that are not able to fork themselves into background?
<philip__> when doing "service xyz start", upstart always tells me "xyz start/running, process 12345"... even when the process has an exit status != 0
<jhunt> so, the Upstart job you're trying to start via the SysV interface starts then exits with status != 0? What is "xyz" and what's the .conf file look like?
<philip__> jhunt: like this: http://pastebin.com/USY8B5nb
<philip__> jhunt: so exit status 1 and 2 are examples
<philip__> jhunt: oh and xyz is a Go program, Go unfortunately doesn't support programs that can daemonize themselves :S
<philip__> (yet)
<philip__> I can reproduce the same effect with this program: http://pastebin.com/Qfm7Ps9a
<jhunt> If these programs are not long-running, you need to use the "task" stanza.
<philip__> Yes, I've seen that. I mean under normal circumstances the program will run for weeks.
<philip__> But this scenario might happen: I stop the daemon, a small feature is added to the software or its configuration, and I start the daemon, thinking everything is ok... but it failed right from the start. Or another scenario: the server is rebooted and the server admin forget to check whether all services are running perfectly.
<jhunt> well, it sounds like the go program is forking atleast once, but you haven't told upstart how many times ("fork" for 1, or "daemon" for 2). You might want to strace xyz on the command-line and see how many forks it's doing and then update the .conf accordingly.
<philip__> No it's not... actually I can reproduce the problem with this C-program: int main() { return 1; }
<philip__> (but yes, upstart manages do daemonize the program nicely, in case everything is ok)
<jhunt> I am rather confused by what you've said so far. To be clear - you are starting daemon xyz, Upstart says it is running, but you are saying it has actually exited (even though you say it should be long-running)?
<jhunt> your main example clearly isn't a daemon so can't be compared with the daemon scenario
<philip__> jhunt: precisely
<philip__> jhunt: why not?
<philip__> jhunt: when you have a daemon that does no forks and that exits right after starting because of an error, this is the same
<philip__> jhunt: just to be clear: any daemon, even those that fork one or two times, might have a fatal error *before* doing any forking
<philip__> the point is, my "daemon" is not to throw away the "main()". so there is no forking involved in my scenario
<philip__> s/is not to/is not able to
<philip__> though obviously upstart does forking
<philip__> maybe this mininmal version of my "daemon" (I prefer to call it server, anyway) is less confusing: http://pastebin.com/BtzM92LP
<negev> hi, is it possible to set dependencies for runlevel 6? ie. i want to execute a script on shutdown/reboot but ensure that rsyslog isn't stopped until the script finishes
<jhunt> I think your confusion comes from the fact that Upstart *does* successfully start your daemon. At some later date, your daemon decides something went bad and exits. However, you haven't told Upstart to auto-restart it ("respawn"). If admin intervention is required to say fix the daemons broken config, respawn won't help :)
<jhunt> philip__: maybe you could use a pre-start to perform some sort of sanity check on the config, print a message and fail.
 * jhunt makes mental note to self to update the cookbook on some of these issues.
<philip__> jhunt: ;)
<jhunt> negev: if you're using Ubuntu, you could make your script "start on stopping rsyslog". Since rsyslog stops on shutdown/reboot, your script will run before rsyslog stops (and before the system shuts down/reboots).
<jhunt> philip__: I can only think that the machine you're running that http://pastebin.com/Qfm7Ps9a example on is incredibly slow since you wouldn't normally even see a pid if the job immediately exits.
<philip__> jhunt: yeah, though I wonder whether it's the 3.3Ghz cpu or the 16 Gigs ram... sorry just joking :D
<philip__> jhunt: but this is interesting, so you say the timing is the problem...
<negev> cool thanks jhunt 
<negev> jhunt: that doesn't seem to work; i put this http://pastebin.com/zW2s1F4z in /etc/init/myshutdown.conf
<negev> its executing the script but the syslog isn't happening, rsyslog must have already stopped when it runs
<jhunt> negev: are you sure your script is running successfully to completion? Try running "sh -n /etc/init.d/my-shutdown.sh" and checking that it is really doing what you think.
<jhunt> negev: See http://upstart.ubuntu.com/cookbook/#develop-scripts-using-bin-sh
<jhunt> negev: and http://upstart.ubuntu.com/cookbook/#debugging-a-script-which-appears-to-be-behaving-oddly
<negev> well i put "touch /home/negev/test" at the top of the script and the file is being created so it must be running
<jhunt> negev: no - *that* line is running, but subsequent lines might be failing due to Upstart running your script using "sh -e".
<negev> -e or -n?  -n says it doesn't execute the commands, it just reads them
<negev> it works if i execute it with -e, fails with -n
<jhunt> I've suggested -n to ensure your script is syntactically correct (as far as the shell can check without running it).
<negev> the script is just one line that executes a ruby script
<negev> the bang line is correct and the ruby script is mode 755, ive even tried prefixing it with /usr/bin/ruby
<negev> should i maybe just put the ruby script and its argument in the script section of the init config file?
<jhunt> negev: any particular reason you're sourcing the script?
<jhunt> negev: I was wondering why you weren't :)
<negev> ok well i just did that, still didn't work though :(
<negev> oh do i need that . before it ?
<jhunt> If you're asking, the answer is "no" :)
<negev> hrmm then why is it still failing :(
<negev> kind of annoying that this server takes forever to boot
<jhunt> have you traced it with -x as I suggested above?
<negev> i can't see -x anywhere in the history :/
<jhunt> also, can you not just try "start <yourjob>" (force it to run without waiting for rsyslog to be about to stop).
<negev> yeah doing that works fine
<negev> could it be that the "stopping rsyslog" event never happens and rsyslog is just killed without being gracefully stopped?
<jhunt> negev: oops! here it is - http://upstart.ubuntu.com/cookbook/#obtaining-a-log-of-a-script-section
<negev> all i got in that log was a + before the line that executes the ruby script
<negev> nothing else at all
<jhunt> Have you tried "start <yourjob>"?
<negev> yeah i said, that works fine
<negev> and its definitely executing when i reboot or shutdown because if i put "touch <file>" at the top it creates the file
<negev> it just doesn't log anything
<jhunt> It's possible /etc/init.d/sendsigs might be killing your ruby process/rsyslog if they are taking "too long" to finish. You could put a temporary sleep in sendsigs to establish that fact.
<negev> that seems unlikely given that it runs and finishes instantly if i run it manually
<negev> all it does is write an entry to the syslog and then exits
<jhunt> negev: um, any reason you're using ruby to do that?
<negev> we have one script that does loads of things, it seemed sensible to add this to it as we already use it for other stuff which also logs to syslog
<negev> rather than having multiple scripts
<jhunt> negev: what does "status rsyslog" show?
<negev> rsyslog start/running, process 1071
<jhunt> not sure what else to suggest without seeing what you're actually running. How about changing shutdown.conf to call 'exec logger -i -t negev "hello"' just to see if syslog is usable at that point.
<jhunt> negev: you could also have shutdown.conf run "ps -ef" or something to convince yourself the logger is still running.
<jhunt> negev: is it possible your ruby script is making use of some other service which has already stopped when your shutdown job is started?
<negev> possible i guess, although i don't think it is
<negev> i was just looking at simplified syslogging from the cmdline, worth a try
<jhunt> negev: also, see if the behaviour you are seeing is consistent across systems.
<negev> well these three servers were installed and built identically so probably not much point testing the other two lol
<negev> i tried putting:   logger -i -t oe "system shutdown"   
<negev> sorry, exec logger -i -t oe "system shutdown"
<negev> in the script section
<negev> didn't work ;9
<negev> :(
<negev> well at least didn't make it to the remote syslogs, need to wait for it to come back up before i can see if it got it locally but i doubt it
<negev> im convinced rsyslog is already gone by the time this execugtes
<negev> nope, nothing in the local syslog either
<negev> interesting, if i stop rsyslog manually it doesn't execute either
<negev> sh -n exec logger -i -t oe "system shutdown"    gives:  sh: Can't open exec
<jhunt> negev: negev put the commands in a file and run "sh -n file"
<negev> including exec?
<jhunt> exec is recognized by the shell and by upstart.
<negev> sh -n <script> returns nothing
<jhunt> negev: good news then.
<negev> well not really, still no clues as to why this isn't working
<SpamapS> jhunt: hey, long time no chat. :)
<jhunt> SpamapS: howdi!
<SpamapS> jhunt: did we ever sort out the user sessions in 11.10?
<SpamapS> I know they were somewhat suspect in 11.04
<jhunt> SpamapS: they are in 11.10 but not enabled by default.
<SpamapS> still unstable?
<jhunt> SpamapS: they are problematic to test but I've now got a test script that's in upstream and Ubuntu, but not packaged for ubuntu as yet.
<jhunt> SpamapS: well, we could do with more testing but afaik it's pretty solid now.
<SpamapS> jhunt: ok, was just trying to think up ways they might be made more useful.
<jhunt> SpamapS: it would be gr8 for user sessions if NetworkManager emitted Upstart events. You could do cool things like start firefox only when you're connected to wifi say.
<SpamapS> jhunt: net-device-up *is* emitted in that case.
<jhunt> Sure - I was thinking more of being able to filter on ESSID, encryption type, etc. And we should already be able to do things like fire up a user app when say a user plugs in their usb headset using the udev bridge so there are a few use cases already.
<rrva> hi
<rrva> I need to su to another user, start a shell script which sets some env variables and then start a java app
<rrva> upstart fails hard in getting the correct PID
<rrva> i use expect daemon
<SpamapS> rrva: can you tell the java app to not daemonize?
<SpamapS> rrva: its far simpler to just run things in the foreground
#upstart 2011-10-19
<jY> i have to start a ruby script via rvm.. and it uses chdir to get into the correct directory.. but it runs as root
<jY> how can i run it as another user.. but still honor the chdir?
<Ush4O> hi, can anyone give me an example how to bring up an wlan interface (ifconfig, iwconfig, wpa_supplicant) using upstart?
<stsmith3> i have a service that calling reload on does the wrong thing. how can i go about customizing the reload action?
<stsmith3> been trying to google but can't figure out how to ask the question
<stsmith3> found this: https://bugs.launchpad.net/upstart/+bug/94873
<stsmith3> but cant find any docs on custom actions
<wraiden> "initctl reload service" as far a i know is currently only using pid of a running service and sends a SIGHUP to it...
<wraiden> there are plans to allow a reload customization...
<jhunt> wraiden, stsmith3: yes, we're considering that feature.
<stsmith3> ok. unfortunately the service im working with does the wrong thing for SIGHUP. the master exits and leaves a bunch of children around.
<stsmith3> so a way to customize would really help in this case
#upstart 2011-10-20
<twb> As at lucid, what is the best practice way to start a job when a particular USB device (ffff:0000) comes into existence?
<twb> This is for a server to launch a nut driver
<Ush40> hi, can anyone give me an example how to bring up an wlan interface (ifconfig, iwconfig, wpa_supplicant) using upstart?
#upstart 2011-10-21
<tylerflint> anyone awake?
<tylerflint> I have an executable that doesn't daemonize itself
<tylerflint> so I figured I could use start-stop-daemon
<tylerflint> however, upstart thinks the daemon is dead when it is quite alive
<tylerflint> I'm assuming it's because upstart doesn't know about the pidfile
<ion> Why do you want it to daemonize itself?
<ion> Pid files are a *very* unreliable way to keep track of processes, Upstart ignores such things.
<tylerflint> I see
<tylerflint> I only want it to daemonize so that it won't block the upstart process
<tylerflint> unless I am misunderstanding how upstart works
<ion> It definitely wonât block Upstart.
<tylerflint> interesting
<ion> âexec program-that-doesnt-daemonizeâ is the most trivial case of running a service.
<tylerflint> wow
<tylerflint> I guess I was making this way too complicated
<tylerflint> if I use env FOO="bar"
<tylerflint> will FOO be accessible by my program?
<tylerflint> just as export FOO="bar" in a shell?
<ion> yeah
<ion> The cookbook has some examples.
<tylerflint> thanks
<tylerflint> I think I
<tylerflint> 've got something that will work
<tylerflint> seems really clean
#upstart 2012-10-17
<geek65535> Hope someone can help: I'm having trouble with gluster where the daemon 'glusterd' is starting before the network is properly up, despite using 'start on static-network-up'. My network contains a bridge (for KVM use), and it appears to appear up to upstart (ouch) before it's actually up and passing packets. Is there any way I can make glusterd wait on actual network connectivity through the bridge?
<SpamapS> geek65535: how are you configuring the bridge?
<geek65535> SpamapS: so sorry! Got distracted by the real world. (I hate when that happens). This is my current config:
<geek65535> iface bridge0 inet static
<geek65535>         address 10.8.1.21
<geek65535>         netmask 255.255.255.128
<geek65535>         network 10.8.1.0
<geek65535>         broadcast 10.8.1.127
<geek65535>         gateway 10.8.1.1
<geek65535>         # dns-* options are implemented by the resolvconf package, if installed
<geek65535>         dns-nameservers 10.8.1.33 10.8.1.44
<geek65535>         dns-search houston8.topgolf.com
<geek65535>         bridge_ports eth0
<geek65535>         bridge_stp off
<geek65535>         bridge_maxwait 6
<TimothyA> I seem to have a slight problem with upstart when trying to run a python script as a service; when I try to manually invoke the process, it will just lock up the screen until I press ctrl-c. any ideas how to fix this?
<TimothyA> what is the recommended approach for starting python scripts with upstart?
<TimothyA> and now the server has become unusable...
<SpamapS> TimothyA: python scripts aren't special over any other thing
<SpamapS> TimothyA: can you pastebin your upstart script?
<SpamapS> geek65535: do you have 'auto bridge0' in /etc/network/interfaces ?
<TimothyA> SpamapS: I can't. the server has become unreachable.
<geek65535> SpamapS: yes. I just missed it on the copy-and-paste
<SpamapS> geek65535: ok, static-network-up *should* wait for that.. you can see the exact time static-network-up was emitted by looking at the mtime of the dir /run/network/static-network-up-emitted ...
<SpamapS> geek65535: perhaps ifup returns before the bridge is actually working?
<TimothyA> SpamapS: removed the file and rebooted server
<TimothyA> but it's nothing more than a basic respawn and exec python main.py lines
<geek65535> SpamapS: let me take a look...
<SpamapS> TimothyA: yeah, that should work. If you don't have any post-start or expect lines, it should return immediately on 'start your-job-name'
<geek65535> run/network/static-network-up-emitted is empty...
<geek65535> er, sorry, missed the 'mtime' part...
<SpamapS> geek65535: right, its the dir itself that is used as a flag
<SpamapS> geek65535: you should also see the ifup.bridge0 there, with an earlier mtime/ctime
<geek65535> root@vmhost1:/run/network# ls -l --full-time
<geek65535> total 4
<geek65535> -rw-r--r-- 1 root root 22 2012-10-17 17:26:27.217981181 -0500 ifstate
<geek65535> -rw-r--r-- 1 root root  0 2012-10-17 17:26:27.209981181 -0500 ifup.bridge0
<geek65535> -rw-r--r-- 1 root root  0 2012-10-17 17:26:24.089981092 -0500 ifup.lo
<geek65535> drwxr-xr-x 2 root root 40 2012-10-17 17:26:27.209981181 -0500 static-network-up-emitted
<SpamapS> geek65535: right, so ifup is the problem then
<SpamapS> geek65535: it must return before the bridge is actually working
<geek65535> That's what I had been thinking. I just need to find something that emits a signal after the bridge is active. (Or live with the 'sleep 10' stuck in my /etc/init/glusterd.conf)
<SpamapS> geek65535: thats a bug.. perhaps we should find it and fix it :)
<geek65535> What can I do to help? (I'm an experienced admin, but I don't code outside of scripting languages.)
<SpamapS> geek65535: I'm looking now.. it may be an issue with the way ifupdown delegates this duty to the bridge plugin
<TimothyA> SpamapS: it did not
<TimothyA> and it locks up my screen when I try to do it manually
<SpamapS> TimothyA: "locks up my screen" meaning the terminal does not return?
<TimothyA> yep
<SpamapS> geek65535: ok, this is handled by /etc/network/if-pre-up.d/bridge ...
<SpamapS> geek65535: of particular interest to you may be the MAXWAIT= area
<SpamapS> geek65535: bridge_maxwait 10 would solve your problem ;)
<SpamapS> geek65535: (from man bridge-utils-interfaces)
<SpamapS> geek65535: actualy, bridge_fd seems more appropriate
<SpamapS> geek65535: I suspect yours is failing because brctl showstp bridge0 is not reporting forward delay / bridge forward delay properly
<SpamapS> geek65535: its usually 15.00 which should be 15 seconds.. so maybe you need more?
<geek65535>  SpamapS: I have maxwait at 6  and bridge_fd at 0, allong with bridge_hello at 0 (although that shouldn't matter with stp off).
<SpamapS> TimothyA: ok, well that still should work fine. When you can, pastebin the upstart job file and we can try to help
<TimothyA> SpamapS: the python script already implements a signal handler.
<SpamapS> TimothyA: thats not realy important in 'start'
<SpamapS> TimothyA: signals are only sent on 'reload' and 'stop' (HUP, and TERM, then KILL)
<TimothyA> exec python main.py
<TimothyA> that's about all
<geek65535>  SpamapS: upped maxwait to 20, rebooting to see if it make a difference.
<geek65535>  SpamapS: no joy.
<SpamapS> geek65535: how strange
<SpamapS> geek65535: another possibility is that brctl is showing it as ready, when its not
<geek65535>  SpamapS: I'd buy that.
<SpamapS> geek65535: because maxwait is just how long it waits for brctl showstp to show 'listening, learning, forwarding, blocking'
<SpamapS> oh oh oh .. I think that maxwait is 10th's of seconds
<geek65535>  SpamapS: 10ths? That would make a difference!
<SpamapS> yeah
<SpamapS> well it sleeps 0.1, then inc's count +1
<SpamapS> so yeah, 10ths
<SpamapS> ok, I have to run
<geek65535>  SpamapS: me, too, actually. dinnertime...
<SpamapS> geek65535: I think you have enough to take a look. If you find that the script is doing something wrong, or brctl is, I urge you to do an 'ubuntu-bug bridge-utils' or 'ubuntu-bug linux' ;) good luck
<geek65535>  SpamapS: thanks!
#upstart 2012-10-18
<lurch_> hi, having an issue with stopping a service through upstart on ubuntu. The app is a very basic sinatra server that listens on some port. Starting goes fine, i can access the service, but there's a problem with restarting/stopping the service. One process stays running, which keeps the  port in use. Here's the output i get for the commands: http://pastebin.com/k9hdNJLE
<jodh> lurch_: you've probably mis-specified (or not specified) the 'expect' stanza: http://upstart.ubuntu.com/cookbook/#expect
<lurch_> jodh: thx. let me look at that
<jodh> lurch_: after starting your job, try running 'status app01' and checking that the pid shown is correct (confirm it using 'ps'). If not, it's an expect issue.
<lurch_> jodh: the status command isn't showing any pids
<lurch_> # status app01
<lurch_> app01 start/running
<jodh> lurch_: right, so I suspect your daemon forks either once or twice, but you haven't added an 'expect' to allow Upstart to track the forks.
<lurch_> thx. added an 'expect fork' now, testing
<lurch_> added the 'expect fork' (and also tried 'expect daemon'), but then the start / stop just hangs. Must be doing something stupid, but not that familiar with upstart. Here's the config: http://pastebin.com/RuUwyn8A
<lurch_> the upstart scripts are generated by 'foreman'
<jodh> lurch_: don't add respawn until you are 100% convinced that you've got the rest of the config correct as it will just cause confusion.
<lurch_> ok
<jodh> lurch_: also, you can simplify that exec line by doing 'setuid app' and 'chdir /opt/apps/app01'. Note too that you may be able to get rid of all that redirection since Upstart will automatically log all output to stdout/stderr to /var/log/upstart/app01.log.
<jodh> lurch_: Um. where is $PORT being defined? I suspect that might be the cause of your problem.
<jodh> lurch_: either set it via 'env PORT=1234' or pass in via command line 'start app01 PORT=1234' or source your apps config file as shown here: http://upstart.ubuntu.com/cookbook/#sourcing-files
<lurch_> jodh: ok, i'll try that. thx
<aaronlevy> I'm having trouble with restarts of a python process that spawns other worker processes.
<aaronlevy> The process cant have 2 instances of itself running because a socket will already be in use by the other process. What seems to be happening is that upstart is re-starting the process before it has actually stopped the original (the parent will not exit until all children have completed their work). 
<aaronlevy> Using start/stop works fine. Restart, however, will periodically cause the above to happen and then the process never exits, but upstart reports that is is not running
<ion> Add something like kill timeout 300
<ion> Or whatever number makes sense.
<aaronlevy> Hmm. I mean, shouldn't it track the PID of the parent process?
<ion> yes
<ion> But it sends SIGKILL if it doesnât terminate in a certain number of seconds after SIGTERM.
<aaronlevy> Then just assumes it is closed?
<ion> SIGKILL kills it for good, so the process wonât exist anymore.
<aaronlevy> Hmm.. but the process is still there (as are the children). 
<ion> You can also add a pre-stop exec/script if the server provides a command that tells it to stop and waits until it does.
<ion> huh
<aaronlevy> no re respawns either. Let me double check that it is the same as originating pid
<aaronlevy> So the zombie process is not the same PID as what restart reports
<aaronlevy> I made a stub of my actual program that re-creates the issue. https://gist.github.com/7cf1c9ef87a50ef08c9c (requires pyzmq)
<aaronlevy> and upstart conf: https://gist.github.com/ecc65af894d72c91cecb
<aaronlevy> repeatedly restarting the service quickly will cause it to get an error because the socket is still in use. Upstart will report the process as stopped, but there will be a zombie process (doing nothing)
<aaronlevy> This is probably a mistake on my part, but I'm not sure where :/
#upstart 2012-10-19
<bizhanMona> !help
<jodh> bizhanMona: ?
<bizhanMona> jodh: I am trying to get some info on upstart and how to write services
<jodh> bizhanMona: have you tried the cookbook yet? http://upstart.ubuntu.com/cookbook/
<bizhanMona> jodh: thx that is all I needed
<bizhanMona> Hi would ubuntu will eventually migrate to systemd? could upstart and systemd co-exist? 
<SpamapS> bizhanMona: "maybe" and "maybe" are the answers to both questions :)
<SpamapS> bizhanMona: There was some discussion and a reasoning behind sticking with upstart posted to ubuntu-devel early in the 12.10 dev cycle
<bizhanMona> SpamapS: thx for response.  are there any blogs or pointers on how this has been done (upstart and systemd coexist)? thx
<SpamapS> bizhanMona: AFAIK, it has not been done
<SpamapS> bizhanMona: the only reason to do it would be if systemd continues to usurp all other daemons into it, and it becomes worthwhile to have a copy of systemd running rather than dbus/udev/etc.
<bizhanMona> SpamapS: thx
#upstart 2012-10-21
<[twisti]> hello, im working on my own upstart job because theres nothing decent for mysql out there, and i got it working with start and stop, but when i do restart job, it just stops it, and doesnt start it, what could cause that ? how could i debug that ?
<SpamapS> [twisti]: you don't like the official ubuntu mysql upstart job?
<[twisti]> no, of course not, it starts a raw mysqld instead of the proper mysqld_safe, so a bunch of options are outright ignored
<SpamapS> [twisti]: such as?
<SpamapS> [twisti]: I'm in charge of maintenance of that package, so I'm definitely interested to hear your thoughts
<SpamapS> [twisti]: tho I also triage the bugs and I haven't seen any complaints of that nature come by
<[twisti]> all of those
<[twisti]> http://dev.mysql.com/doc/refman/5.0/en/mysqld-safe.html
<[twisti]> though of course some settings are duplicates
 * SpamapS skips forward to 5.5 .. 5.0 is so.. 2006
<[twisti]> for me the biggest issue was nice level
<[twisti]> oh yeah, sorry, wrong link
<SpamapS> nice level is built into upstart
<SpamapS> [twisti]: as in, add a line 'nice X'
<SpamapS> [twisti]: you can even do it in a non-intrusive way by putting that in /etc/init/mysql.override instead of mysql.conf
<[twisti]> ill keep that in mind for next time
<[twisti]> in any case, ignoring the actual mysql setting and doing it a different, proprietary way does not seem like a good way to me
<SpamapS> [twisti]: I suspect the reason you might be seeing problems with restart is that after stopping you need to wait for mysqld_safe to actually exit, or it will complain about conflicting pidfiles or something like that.
<[twisti]> ill look into that
<[twisti]> thanks for the tip
<SpamapS> [twisti]: upstart's job file syntax is meant to replace all of the complexity of shell scripts like mysqld_safe
<SpamapS> [twisti]: I wouldn't call something using published unix API's like "nice" as "proprietary"
<[twisti]> i understand that, but considering that when you google for mysql nice info, you will get the official "niceness = 0" or whatever the setting is, which then wont work, i dont think this is an appropriate case
<[twisti]> but im not using published unix APIs, im using a flag in a config file for upstart
<SpamapS> [twisti]: I do see your point. Perhaps we should issue log warnings if we find mysqld_safe only configs in my.cnf
<[twisti]> it would be one thing if you were transitioning away from ubuntu specific things, but mysqld_safe is part of mysql
<[twisti]> that would be a good compromise i suppose
<SpamapS> [twisti]: its part of mysql that we're not using because it is redundant with upstart.
<SpamapS> Such is the burden we bear to have a more tightly integrated system. :-/
#upstart 2013-10-15
<jamescarr> is there a way to ensure 100% that an upstart script when run when shutdown begins?
<xnox> jamescarr: man 7 upstart-events
<xnox> jamescarr: i don't quite understand what you mean. Can you explain it in more than one sentence?
<xnox> jamescarr: start on RUNLEVEL=[06]
<xnox> task
<xnox> script
<xnox>    ....
<xnox> end script
<xnox> should work for you.
<jamescarr> xnox: sorry, I figured it out through the cookbook earlier ;)
<xnox> jamescarr: =) ok cool! glad that cookbook solved it all ;-)
#upstart 2013-10-16
<jamescarr> hey guys, another upstart question
<jamescarr> is there a way to ensure something executes on startup when EVERYTHING ELSE has started?
<jamescarr> I have a service that needs to start when at least two other services have started
<jodh> jamescarr: "start on (started service-a and started service-b)". There is no "everything else" - jobs start and stop for the lifetime of the system boot.
<jamescarr> jodh: yeah I think what you said is what I need
<jamescarr> basically, I need this script to run once docker is running and networking has been configured ;)
<jamescarr> the problem was it was running before networking was configured or docker was running
<jamescarr> so sometimes it worked sometimes it didnt
<jodh> jamescarr: take a look at upstart-events(7) or http://upstart.ubuntu.com/cookbook/#ubuntu-well-known-events-ubuntu-specific as there are generally events for most stages of the boot.
<SpamapS> jamescarr: I'd do 'start on runlevel [2345]' and then in pre-start do 'exec start wait-for-state WAITER=myjob WAIT_FOR=docker'
<SpamapS> jodh: btw any movement on state awareness for upstart?
<jamescarr> SpamapS: didn't know about that pre-start thing
<jamescarr> thanks!
<SpamapS> jamescarr: it is less than obvious, but it solves the problem in a more robust way (it will force docker to start if it is not yet starting)
<jamescarr> yeah I like that
<jodh> SpamapS: not recently I'm afraid. Still something we're considering as mentioned in the LinuxCon talk.
#upstart 2013-10-17
<kiorky> hi; im trying to debug upstart inside a docker, and i would like to attach a gdb to it, but as it crashes directly, i cant
<kiorky> (idea is to launch one docker with the compiled init and attach a gdb from another shell)
<kiorky> if someone has an idea on how i can debug ...
<jodh> Kiorky: I've never used docker. Do you get any crash output? You could boot your docker environment specifying bash as your init, then start upstart in a separate pid namespace running with --no-startup-event, then attach gdb to that pid.
<kiorky> jodh: well the difficulty is the docker magic between
<kiorky> jodh: and that upstart wants pid - 1
<kiorky> jodh: well, i want to debug the start process, that's the real problem amongst all
<jodh> kiorky: in all likelihood, the crash is probably due to some environment issue: try installing procenv into the docker environment (its in the debian and ubuntu archives, or https://launchpad.net/procenv/), and boot the docker container so that procenv runs first, logs the environment output, then execs upstart. I've blogged about how to do this for lxc here:
<jodh> http://ifdeflinux.blogspot.co.uk/2013/04/observing-initial-lxc-environment-using.html
<kiorky> jodh: i dont know why but init dies, without logs, so my idea was to gdb it
<kiorky> jodh: lookin' thx
<jodh> kiorky: yes, you can run upstart in the docker environment as *pid 1* if you put it into a new pid namespace within your docker environment.
<kiorky> jodh: ha i see!
<jodh> kiorky: are you sure there is no stdout/stderr output being logged by docker somewhere? no docker option to record the console?
<kiorky> jodh: normally, all goes to docker logs, but anything is captured
<kiorky> jodh: i have some logs, but nothing fatal
<kiorky> just that the docker boot and after a little while, dies
<kiorky> without explanation
<kiorky> but i will look your blog post and go forward into trying to exec init inside a new pid namespace
<jodh> kiorky: I've covered the technique in the cookbook too: http://upstart.ubuntu.com/cookbook/#debugging-another-instance-of-upstart-running-as-root-with-pid-1
<kiorky> jodh: in your setup, init starts and then you attach with gdb isnt it ?
<kiorky> jodh: problem there is that init dies before i can even attach to it
<kiorky> jodh: i mean on the cookbook
<jodh> kiorky: Well, in the lxc example yes, but not in the first example using my clone utility.
<jodh> kiorky: I still think it might be quicker to use the procenv trick to get a dump of the docker environment, then run procenv in a "normal" environment and diff the files. 
<kiorky> jodh: the clone is same, you one already running, isnt it ?
<kiorky> :)
<modafinil> I'm really at my wits-end here -- how can I, in an unprivileged upstart job, emit an event to start another job?
<modafinil> I have a job to make logs for a service, so I want the service to start this job 'make sure my logs dir is owned by serviceuser' and then run itself unprivileged (using setgid/setuid)
<modafinil> Really annoying that pre-start runs as the setuid user :/
<modafinil> obviusly sudo doesn't work
<jodh> modafinil: easy - turn it round: have the job that handles your logs run as root, then make the job that actually runs the service (with setuid/setgid), specify 'start on stopped <log_job>'.
<modafinil> jodh: the problem is that i have a bunch of jobs, and i want them all to use this one job (and set the variable of where the logs dir is)
<modafinil> i think i figured it out though
<modafinil> i can 'start on starting acmeco-*' to wildcard match in the log creation job
<jodh> modafinil: have the root job that starts first 'export logdir' so that "$logdir" is available to all the non-priv jobs.
<modafinil> jodh: i agree that would work, but its not really workable for this specific situation -- theres hundreds of different services and i don't want to have one thing that needs to know about them all
<modafinil> this wildcard match definitely works though
<modafinil> i appreciate your help! thanks!
<kiorky> jodh: simpler solution; add a sleep(3) and recompile upstart ;) so i have the time to attach gdb with lxc-execute
#upstart 2013-10-18
<ansu> hey all :) I'm trying to run this command in an upstart script. Does anyone of you see whats wrong with it? "exec start-stop-daemon --start -c youtrack --exec /usr/bin/java -jar /usr/local/youtrack/youtrack.jar 10002"
<jodh> xnox, stgraber: could you take a look at the o/s MPs when you get a chance?
<elmo> jodh: ' If a job is running, the changes to the definition will not occur until the job has been stopped; for instance jobs, all instances must be stopped. '
<elmo> jodh: what does 'instance job' mean there?
<elmo> jodh: and will 'initctl reload' override the above?
<elmo> jodh: never mind, the job I care about is an instance job
<elmo> jodh: or rather, ignore the first question; I'd love to know if 'initctl reload' enough to overide
<jodh> elmo: 'initctl reload' simply sends the reload signal (SIGHUP by default) to the job. If you change a .conf file, Upstart gets notified immediately, but if a job is actually running, *it* won't get a new copy of the configuration unless you stop and then restart it.
<elmo> jodh: so, the issue is I have a ceph osd instance job
<elmo> jodh: and I *really* don't want to restart all 11 of them
<elmo> jodh: is there no way to get upstart to use the new ceph-osd.conf for any future restarts?
<elmo> (short of stopping all 11)
<elmo> jodh: e.g could I use 'initctl reload ceph-osd' without risk of it stop/starting all the instances?
<Spads> jodh: I suspect there's some reload vs reload-configuration confusion propagated by http://upstart.ubuntu.com/faq.html#reload
<jodh> elmo: yes, are you talking about reloading the upstart (.conf) config, or the ceph config (assuming there is one?)
<Spads> just the upstart config
<Spads> basically right now it's starting up the OSDs with incorrect command-line arguments calculated in the upstart conf itself
<Spads> we fixed that
<Spads> but if someone bounces one of these, it'll come back with incorrect settings (to potentially disastrous results)
<Spads> and I'd expect that diagnosing that problem would be nigh impossible without the context we just got about how upstart caches the configuration just now
<jodh> Spads: confused - I thought you said you fixed the issue and that the running instance(s) have the wrong arguments?
<jodh> Spads: the whole point of upstart instances is that they are "templates" - every instance is "identical", atleast to upstart, apart from the value of the 'instance' variable(s).
<Spads> jodh: yes, that's what we're saying.  We fixed ceph-osd.conf, but when we restart an individual instance job it uses the bad information from the old broken conf
<Spads> sure
<Spads> how do we make them all use the new version of the config without taking them all down together?
<jodh> Spads: you need to stop, tehn start not "restart".
<Spads> yes, we stopped, slept for ten seconds, then started
<Spads> for one instance job
<Spads> stop ceph-osd id=${id}; sleep 10; start ceph-osd id=${id}
<jodh> right, but since there were presumably other instances still running, upstart will not give the newly started instance the *new* config since that would mean the newly started instance would/could be completely different from the running instances (bad).
<Spads> and that start caused it to start with the same broken arguments (which are not the ones in the conf on disk any more)
<Spads> sure
<Spads> how do we tell upstart to switch to the new one while we do rolling restarts of the instance jobs?
<jodh> stop all instances (which allows upstart to propagate the new config), then start the new instances.
<Spads> â¹
<Spads> is there not even a hacky way to force the issue here?
<jodh> we don't like hacks :) seriously, not that I can think of, sorry.
<Spads> Hm, no way to (say) take an instance job down and start it up in a new instance group somehow?
<Spads> some kind of staging upstart job for these to live in for a time?
 * Spads is clutching at straws
<xnox> Spads: you can cheat!
<Spads> xnox: Tell me how.
<xnox> Spads: cp /etc/init/ceph-osd.conf /etc/init/seph-osd-transition.conf
 * Spads nods
<xnox> Spads: stop 5 instances of ceph-osd, and start 5 instances of ceph-osd-transition
<Spads> that's what I was hoping would work.  have you done this?
<xnox> Spads: stop remaining 5 instances of ceph-osd, start 5 instances of ceph-osd (new config)
<xnox> Spads: stop & delete all ceph-osd-transition
<xnox> Spads: start ramaining 5 of ceph-osd.
<xnox> Spads: but you just need to make sure that the first 5 instance of ceph-osd-transition are operational and do cephy things that ceph-osd is suppose to do.
 * Spads nods
<jodh> actually, that technique is in the cookbook (http://upstart.ubuntu.com/cookbook/#using-expect-with-script-sections), but maybe we should elevate it to its own section :)
<Spads> interesting.
#upstart 2014-10-13
<fly-away> hi guys
<fly-away> how I supposed to use expect with su ?
<fly-away> its just hanging now
<fly-away> Im trying to start my non-forking script under unprivileged user
<fly-away> its a upstart 0.6.5
#upstart 2014-10-14
<gena2x_> Hi all. May be little bit unrelated question.
<gena2x_> We have a perl script which runs as a daemon like "projectalarms.pl".
<gena2x_> What would be good name for the service in this case from point of view of upstart developers?
#upstart 2014-10-15
<kylinlzy> hello every body
<kylinlzy> would someone tell me the difference betwen 14.04 and 14.10 about upstart
#upstart 2014-10-17
<staykov> hey i guess im not understanding how session jobs work
<staykov> should service "sessionjob" start work?
<staykov> ive tried placing it it in $HOME/.init and $HOME/.config/upstart
#upstart 2015-10-13
<skmar> hi, is it possible to propagate custom environment variables from the init process (set during container startup) to jobs?
#upstart 2015-10-14
<gerow> Hi folks. I'm trying to figure out how to write an upstart scipt that is started after another service is started but *only* if that service is installed. The cookbook makes mention of this but doesn't really go into any detail about it.
<gerow> From the cookbook: "Does the application need to start before or after a service which might not be installed?"
#upstart 2015-10-15
<geomyidae_> http://upstart.ubuntu.com/cookbook/#job-lifecycle
<geomyidae_> Shouldn't step 17 say "started on" and "stopped on" ?
<geomyidae_> looks lke a copy/paste mistake
