#upstart 2006-09-15
* Starting logfile irclogs/upstart.log
-ChanServ(ChanServ@services.)- [#ubuntu-server]  Ubuntu Server Discussions (development and support)
-ChanServ(ChanServ@services.)- [#ubuntu]  Welcome to #ubuntu! Please read the channel topic and consider spending some time on the FAQ mentioned there
* ..[topic/#upstart:fabbione] : Upstart 0.2.6 | http://upstart.ubuntu.com/ | http://upstart.ubuntu.com/doc/getting-started.html | https://wiki.ubuntu.com/UpstartDesignChanges | irc logs: http://people.ubuntu.com/~fabbione/irclogs
<fabbione> logs will be available from the next hours or so
<DMTM> hello
<DMTM> i would like to register my nickname
<thom> why do people think upstart is a support channel?
<DMTM> hmm
<Md> because it's the first or last channel I joined, so stupid people use /who to look for staff members and then think it's a good idea to ask support questions in a random channel
<DMTM> ok
<LarstiQ> Md: smooth :)
<Coldbyte> Md are you there
<Coldbyte> ?
<NthDegree> i'm using ubuntu edgy eft, and i'm having trouble sorting out a script to mount my dm-crypt loopback drive at boot
<NthDegree> can someone help me with getting it added to the new upstart runlevels?
#upstart 2006-09-16
<Coldbyte> Md are you there?
<Coldbyte> staff members here?
<neuralis> Coldbyte: this is not a support channel.
<Coldbyte> i know that but Md is listed as staff member
<Coldbyte> and i can't pm him about my registration issues as im not registered
<zorglub> you should use the dedicated channels
<zorglub> I suppose #help exists
<Coldbyte> doesnt
<Coldbyte> anyway type /who freenode/staff/*
<Coldbyte> and you will see
<neuralis> Coldbyte: you can always private message staff members.
<Coldbyte> the message i get says smth else
<Md> Coldbyte: everybody can send messages to staff, I have no clue why you believe this is not possible. anyway, if somebody is not answering usually insisting will not help much
<dem> is there documentation on the job format; i'm trying to write a job to start the mythtv backend server
<LarstiQ> dem: http://upstart.ubuntu.com/doc/getting-started.html ?
<dem> yeah, but it dosen't really list (all) the events that I can use
<LarstiQ> from what I gather, there is no limit
<dem> i'm trying to figure out exactly what respawn does
<dem> since it's in some jobs but not really documented anywhere
<dem> for example can i respawn something as a certain user
<dem> basicaly i'm trying to figure out, how to start the mythbackend deamon, and have it respawn in the case it goes down, as the mythtv user
* LarstiQ has near to zero upstart knowledge
<LarstiQ> dem: sorry not to be able of help :/
<dem> and a second script to start an Xserver with the mythtv-frontend, also repsawn that in the case it goes down, but only if the mythtv-database up
<dem> that's okay, maybe some one knows
<dem> i figured, hell there's this new stuff edgy, might as well use/get on the bandwagon
<_ion> "respawn" respawns a daemon in case it dies.
<dem> _ion: how do you variables in the job?
<_ion> Do you mean environment variables?
<dem> yeah
<dem> well that too, but how would i do a deamon that respawns when it dies and also runs as a particular user?
<alp> from a blog entry i saw, "The IPC mechanism used by upstart is not currently D-BUS because of various problems" -- i was wondering if there's a summary of what these problems are?
<_ion> dem: env foo=bar
<_ion> dem: Seems like there's no support for setting uid in jobs. It wouldn't be difficult to implement, though.
<_ion> alp: There is lengthy discussion about that in the mailing list.
<alp> _ion: any idea which list/timeframe?
<alp> ah, there is an upstart-devel
<alp> _ion: i don't see any pertinent threads
<alp> i'm working on a dbus-compatible ipc implementation and want to make sure any limitations of libdbus and its daemons aren't carried on to mine
<alp> the only discussion i could find (on the dbus list) relates to process activation
#upstart 2006-09-17
<pepsiman> Is it intended that single user mode in edgy runs dash as root's shell?  I'd much prefer bash.  /etc/passwd says root's shell is bash
#upstart 2007-09-11
<sadleder> hi all, what happened to upstart lately
<Keybuk> oddly enough, I'm writing a mail to the list right now about that ;)
<ion_> :-)
<Keybuk> though, since it has a lot to say, it may be a few hours off yet
#upstart 2007-09-13
<Keybuk> silence on the ML :-/
<AlexExtreme> Keybuk: I'll try to look through your mail properly, I only skimmed it as I've been quite busy
<Jc2k> hi all
<Jc2k> i'm working on some things for Conduit and one of the Ubuntu sync specs. right now i'm trying to make plug and play more "magic". for a lot of target devices, they are only useful once a ppp session has been established over their USB serial connections. its similar for windows mobile syncing too, various user space daemons need starting.
<Jc2k> does this sound like a good match to upstart?
<Keybuk> to some degree, yes
<Jc2k> \o/
<Jc2k> do i sense a but :)
<Jc2k> or a "but watch out for"
* Jc2k puts on his debian hat and tries to setup upstart
<Keybuk> determining direction of Upstart
<Jc2k> googling that leads to your blog :P
<Keybuk> https://lists.ubuntu.com/archives/upstart-devel/2007-September/000463.html
<Jc2k> interesting read
<Jc2k> what do you suggest?
<Keybuk> as to which regard?
<ion_> I have tried to read that, but not succeeded so far because of problems with concentration. :-(
<Jc2k> Keybuk: sorry, in my case for Conduit. am i ok starting with upstart now?
<Jc2k> if you are planning to rework it.
<Keybuk> Upstart probably doesn't support the features you would need to use it now
<Jc2k> ahh
<Jc2k> big shame :(
* Keybuk honestly doesn't know what to do
<Jc2k> :(
<Jc2k> i'm probably going to have to resort to py-upstart because i need something to manage my pppd and other daemons...
<Keybuk> py-upstart?
<Jc2k> :) some home grown hack that would suck on HAL and manage services for me..
* Jc2k wishes he had an idea stick to poke Keybuk with
<Keybuk> don't you just want Network Manager?
<Jc2k> i dont think so
<Jc2k> certainly not the current network manager
<Jc2k> for example, if i plug in my phone it registers as a network connection and my wifi cuts out
<Jc2k> yay \o/
<Keybuk> yes
<Keybuk> that's what NM does
<Jc2k> and i dont think the current NM can auto start ppp for the older phones
<Jc2k> also in the windows mobile 5 case, i'd like to start sync-engine.. so its about more than bringing up network interfaces
<Keybuk> perhaps you have ideas about which direction to take Upstart?
<Jc2k> to be honest im just a hapless noob who use to excitedly read your blog :)
<Jc2k> ok, i'll describe how i'd imagined this all working.. see if it inspires?
<Jc2k> i plugin my windows mobile, usb driver kicks in, hal rules detect WM5, magic, upstart starts the two user space daemons needed, new hal entry is created "Windows Mobile Sync Endpoint" or something. this is picked up by Conduit/OpenSync which can just sync.. instead of the user having about 10 commands to execute. then i disconnect the phone and upstart kills off the daemons
<Jc2k> same process for pppd stuff really
<Jc2k> both types of device are an absolute horror for a new user to get to grips with
<Jc2k> this could magicify it i think, and create a lot nicer user experience..
<Jc2k> implementation wise this is a horror of course
* Jc2k ponders
<Jc2k> what about using XML in messages/events?
* ion_ thinks he just vomited a bit.
<Jc2k> yah
<Jc2k> me too
<Jc2k> Keybuk: can i make an fdi file that triggers an event in upstart?
<Jc2k> and when a device is removed too..
<Keybuk> dunno
<Keybuk> the two ways of thinking about this are
<Keybuk> 1) the upstart job definition specifies hal objects in some magic way, which listens to d-bus messages
<Keybuk> 2) the hal fdi files specify upstart events in some way
<Keybuk>   and the job definitions specify the upstart event
<Keybuk> neither seems quite right
#upstart 2008-09-08
<sadmac2> notting: don't suppose you've heard anything about LPC?\
<notting> nope
<sadmac2> wouldn't have thought
<sadmac2> don't suppose you've looked at the bundle either?
<notting> i did. upstart crashed going into single-user mode
<notting> is cvs up-to-date with the dbus-related fixes?
<sadmac2> ?
<sadmac2> cvs dbus should have what it needs...
<sadmac2> upstart cvs, let me check...
<sadmac2> notting: looks like its right
#upstart 2008-09-09
<pingveno> I'm trying to get an Upstart script working. I used a soft link to put the script I made into the /etc/event.d directory but I'm getting an "Unknown job" error from the start command
<pingveno> Is there something else I need to do?
<pingveno> I'm trying to get an Upstart script working. Do I just need to drop it into /etc/event.d for it to be registered?
<sadmac2> notting: what does your crash look like?
<notting> init: Unable to set root directory: Resource temporarily unavailable
<notting> init: Caught segmentation fault, cure dumped
<notting> Kernel panic - not syncing: Attempted to kill init!
<sadmac2> hm
<sadmac2> notting: tried with -v ?
<notting> ... where?
<sadmac2> notting: argument to init
<sadmac2> notting: in kernel args init=/sbin/init -v
<Keybuk> try --debug instead ;)
<sadmac2> oic
<sadmac2> Keybuk: theres some guys on fedora devel list threatening to boot F9 in 5 seconds at LPC
<Keybuk> though those messages are all part of the core dump routine
 * sadmac2 really wants to go now
<sadmac2> hm
<notting> sadmac2: yup, there's a lot of debugging noise. whatcha want? 
<notting> rcS-sulogin changed from post-start to running
<notting> event_new: Pending started event
<notting> Handling started event
<notting> event_finished: Finished started event
<notting> Unable to set root dir... (etc)
<sadmac2> hmm
<sadmac2> strange place for that issue
<sadmac2> notting: did you install over rawhide or F9?
<notting> rawhide
<sadmac2> ok
<sadmac2> notting: is this box at work? I can't reproduce it in my VM
<notting> yes
<sadmac2> notting: are /you/ at work?
<notting> yes
<sadmac2> cool
<sadmac2> Keybuk: we're hitting that list issue elsewhere :(
<sadmac2> Keybuk: I'm going to see if I can just stuff semaphores into NihList
<sadmac2> treat it like a real concurrency problem
#upstart 2008-09-10
<asdaf> hi all, anyone ever tried to compile upstart with uclibc?
<asdaf> I get some errors, such as WEXITED ans WNOWAIT undefined
<keesj> asdaf: what version? it somewhere rings a bel
<asdaf> uclibc 0.9.28, upstart 0.3.9
<asdaf> keesj, I have a clue, it seems uclibc doesn't define them in bits/waitflags.h
<asdaf> and glibc does
<keesj> not a simlink issue?
<keesj> it's defined in the include/linux/wait.h  in het kernel 
<asdaf> i don't think so, I checked the differences between glibc and ucilbc and it simply seems not to be defined in uclibc
<ion_> http://heh.fi/lhc-status (offtopic)
<sadmac2> ion_: lol
#upstart 2008-09-11
<sadmac2> where's Keybuk?
#upstart 2008-09-12
<ack> Hi, I have a strange error from upstart, being killed and respaned
<ack> this is the error: error.c:177: Assertion failed in nih_error_raise_system: errno > 0
<ack> on upstart 0.3.9
<ack> then it prints out init: Caught abort, core dumped
<ack> mhm, that's wierd, if i put some debug print (using nih_warn) around nih_error_raise_sistem calls, in some cases init works fine
<ack> is there any known stack problem with 0.3.9 ?
#upstart 2009-09-07
<Keybuk> sadmac: am around now
<Keybuk> mbiebl_: if the SELinux loonies get too much, feel free to direct them upstream
<Keybuk> (ie. to the ML)
<Keybuk> or to LP
#upstart 2009-09-08
<ion> #6  0xb7ff3eee in nih_file_read (parent=0x0, 
<ion>     path=0xb800b094 "/sys/dev/block/8:1/../dev", length=0xbffff53c) at file.c:134
<ion> mountall:file.c:134: Unhandled error from nih_file_read: Invalid or incomplete multibyte or wide character
<ion> Wtf? :-P
<ion> keybuk: A bug in nih_file_read: various errors cause a âgoto errorâ, some of which set errno and some donât. The error handler does nih_error_raise_system, potentially leading to nonsensical error messages.
<ion> keybuk: I was trying to nih_file_read a sysfs file. Its stat says its size is 4096, whereas read actually returns 4 bytes. That caused the âInvalid or incomplete multibyte or wide characterâ message.
<ion> keybuk: Ah, you do set errno manually. I missed that.
<ion> keybuk: I still think the error message for this case isnât very helpful. :-)
<ion> To ensure Keybuk notices this: hey, look at what i said on the other channel. :-P And now for some sleep.
<ion> Morning
<ion> keybuk: Online? :-)
<Keybuk> yup
<ion> Any comments about my patch?
<Keybuk> patch?
<ion> ti052526 < ion> keybuk: Hereâs an untested implementation. It probably breaks your system and leaks memory, but itâs a start. Hey, at least it compiles without warnings. :-P  http://heh.fi/patches/mountall/
<ion> ti052620 < ion> keybuk: Iâll have a better capability to test mountall patches as soon as i can get my system to run *some* version of mountall. ;-)
<ion> ti052638 < ion> keybuk: As in, boot using it
<Keybuk> oh cool
<Keybuk> at a quick glance, isn't there a libudev way of doing this rather than dealing with sysfs by hand?
<Keybuk> you have mnt->udev_device ;)
<ion> Good point. Iâll take a look at that struct.
<Keybuk> http://www.kernel.org/pub/linux/utils/kernel/hotplug/libudev/
<Keybuk> e.g. while (dev = udev_device_get_parent ())
<Keybuk>         if (dev is what you're looking for)
<Keybuk>             break;
<Keybuk> etc.
<ion> How about slaves?
<ion> udevadm info --export-db doesnât seem to contain slave information.
<Keybuk> you can still get things like that as properties, attributes, etc.
<ion> udevadm info --query=all --name=/dev/alku-main/root: http://pastebin.com/f49b5a4b
<ion> udevadm info --attribute-walk --name=/dev/alku-main/root: http://pastebin.com/f5c3b19a
<ion> I fail to notice any mention of the md slave.
<ion> I made made the physical_dev_ids function work on Mount * directly, patch updated. I didnât implement the udev thing yet. Iâll have to research more.
<ion> keybuk: Whoops, forgot to add the entry to the list in queue_fsck. Patch updated.
<ion> It would be nice to see whether the actually code works in the current form.
<Keybuk> am figuring out why it doesn't now ;)
<ion> Hehe
<ion> Any debug output?
<Keybuk> OMG
<Keybuk> IT BUILT
<Keybuk> ion: --debug ;)
<ion> I mean, when you tested it
<Keybuk> ?
<ion> You said youâre figuring out why it doesnât work. Does it print debug output i could take a look at?
<Keybuk> oh, I mean mountall in general
<ion> Ah
<[psy]> so whats is the RightWay(tm) to turn off a service on boot?
<[psy]> and: how do i make my ldap-service start automaticly when the user wants to start apache, for example?
<[psy]> we wanna rewrite our daemontools based distro to upstart
<[psy]> oh and: how do i make everything that 'depends' on ldap, stop, and start again automaticly, when the user wants to restart ldap?
<[psy]> (just like gentoo's init system does)
<[psy]> i've read the wiki, but i'm not sure how to do it with events.
<Keybuk> if you're thinking in terms of "depends", you need to take a few steps backwards
<[psy]> right
 * [psy] steps back
<[psy]> back to daemontools you mean? :)
<Keybuk> no, just your way of thinking
<Keybuk> Upstart isn't a dependency-based system
<[psy]> its event based, right?
<Keybuk> if you're interested in Upstart, you need to learn not to think in terms of dependencies
<Keybuk> execatly
<[psy]> my question was how to do it with events..is it possible?
<Keybuk> yes
<[psy]> e.g.: user wants to start apache, so ldap is started first
<Keybuk> but you're still thinking in terms of dependencies
<Keybuk> you need to unlearn that
<Keybuk> that's not how Upstart works
<[psy]> ok
<Keybuk> in Upstart, we would say that apache is started when ldap is started
<Keybuk> *not* the other way around
<[psy]> so i need to but a start on ldap in the apache event-config
<[psy]> however: i dont want apache to be started as soon as ldap is started all the time
<[psy]> sometimes i just wanna start ldap, without apache
<Keybuk> why?
<[psy]> dont think in terms of why
<[psy]> ldap+apache is just an example
<Keybuk> what are the conditions in which apache would not be started at boot?
<Keybuk> we don't do hypothetical examples here, only real world ones
<[psy]> ok
<[psy]> so i cant have ldap start automaticly as soon as i try to start apache? e.g. in the pre-start script of apache or something?
<[psy]> dont tell me i need to unlearn dependencys
<[psy]> maybe its time for upstart to implement dependencys 
<sadmac2> [psy]: why?
<[psy]> why?
<sadmac2> yeah, why do you need them?
<[psy]> for example: winbindd cant start if smbd isnt running yet
<[psy]> currently with daemontools it keeps restarting until smbd is started
<[psy]> what i want is a way to start smbd automaticly if winbindd is started
<[psy]> and stop winbindd automaticly when smbd is stopped
<[psy]> by the sysadmin
<sadmac2> ok. upstart does that...
<[psy]> how?
<sadmac2> have smbd start winbind
<[psy]> thats the wrong way around
<[psy]> package-maintainer-wise
<sadmac2> ok
<[psy]> now this will happen: for some reason ldap is started. ldap starts smbd, and smbd starts winbindd
<sadmac2> have winbind be started by smbd
<sadmac2> yes. that will happen. so?
<[psy]> but the sysad doesnt want smbd and winbindd running!
<sadmac2> why not?
<[psy]> he only wants ldap
<[psy]> you wont get peace with that attidue
<[psy]> attitude 
<sadmac2> its not a rhetorical question
<[psy]> i just told you: samba IS installed, but the sysad doesnt need it or use it. however, ldap IS needed for something
<sadmac2> so he should uninstall samba
<ion> keybuk: Patch updated. Using libudev now. It still needs to be tested, i only know it compiles. :-P
<[psy]> furthermore: the package maintainer of openldap doesnt want all kinds of references to smbd in his ldap-event script
<sadmac2> [psy]: the references can go in the other scripts
<sadmac2> start on starting ldap
<[psy]> ah ok
<[psy]> well let me ask another question: how do i keep a service down without removeing or moving the event definition?
<Keybuk> you'd place the service in manual mode (0.10 era)
<sadmac2> [psy]: at the moment you don't. solution coming.
<[psy]> oki 
<[psy]> i'll wait a while then
<[psy]> i dont see any advantages to daemontools at the moment
<sadmac2> Keybuk: one more 3-day weekend and event_operator will be gone from my tree :)
<[psy]> some new features i dont need, some missing features i do need :)
<Keybuk> sadmac2: I know I still have your mail to review :-/
 * Keybuk is behind with lots of things atm
<[psy]> and you really should implement dependencies
<sadmac2> Keybuk: considering that changing event_operator was a precursor to that proposal, the fact that I figured out how to get rid of it entirely is kind of surprising.
<[psy]> yeah..you could life without them...but then we you could say: hey..who needs packages, just ./confiugre;make install everything by hand because 'thats how we see it'
<sadmac2> [psy]: there are some more powerful relationships coming. I won't say "dependencies" though.
<[psy]> oki
<[psy]> well the most important thing is the ability for the sysad to shutdown something permanently
<[psy]> but we could work around that by using symlinks like /etc/event.d/servicename -> /etc/events/realservicescript
<[psy]> sysv init style syslink ;)
<[psy]> m
<sadmac2> [psy]: that's a good for-now solution. Nicer things are coming.
<[psy]> yup
<ion> keybuk: Another small update, better debug logging in add_device.
<ion> keybuk: The dev_ids code seems to work on a box that has lvm over md over two HDDs. I havenât managed to test the fsck queue code yet.
<ion> keybuk: I still had to call udev_device_get_syspath, add "/slaves" to its result and call udev_device_new_from_syspath for each entry under that path. I didnât figure out a way to traverse the slaves using plain libudev API.
<sadmac2> ion: what is this you're working on?
<ion> sadmac: Keybuk implemented mountall, a program that handles the fscking and mounting of fstab entries in the proper order, parallelizing fsck as much as possible. Well, too much, since all fsck instances run in parallel, with a /* FIXME */ for having each fsck instance use a lock for each underlying physical device to avoid thrashing. My patch fixes the FIXME.
<sadmac2> ion: oic
<ion> keybuk: How does the patch look now?
<Keybuk> ion: haven't looked yet ;)
<ion> keybuk: I created the file /looptest, made an ext2 filesystem on it and added â/looptest /mnt ext2 loop 0 2â to fstab. Mountall seems to end up waiting forever in the main loop select().
<sadmac2> ion: missing an nih_main_loop_interrupt ?
<Keybuk> ion: isn't it just waiting for the root filesystem to be ready?
<Keybuk> udevadm trigger --subsystem-match=block
<ion> http://pastebin.com/f2483aa5a
<ion> It says mountpoint_ready: /
<ion> Oh, now would be a good idea to test without my patch. :-P
<ion> Nope, it still does the same thing.
<ion> The same issue with a swapfile: i created the file /swaptest, did mkswap and added âswaptest none swap sw 0 0â to fstab. Mountall says mountpoint_ready: / and then blocks forever in select().
<ion> â/swaptest none swap sw 0 0â
<ion> (having commented out the /looptest entry)
<sadmac2> Keybuk: strange thought. What if instead of start on started <name> <env> it was start on started <env>, where the first item in <env> was always UPSTART_EVENT_NAME= ?
<Keybuk> sadmac2: ugh
<Keybuk> besides, 0.10 that's start on <job> started again ;)
<sadmac2> Keybuk: yeah, I was wondering about the semantics for that. Is event.name == job.name and started is an argument?
<Keybuk> jobs, states and events are the same fundamental thing
<Keybuk> event is just an edge of a state
<Keybuk> it's not so much an argument, as a sub-state if that makes sense
<sadmac2> the edge of a state is not the same fundamental thing as a state.
<Keybuk> I think it is
<sadmac2> Keybuk: if you buy an apple and I give you the core of an apple are you happy>
<Keybuk> *baffled*
<sadmac2> Keybuk: better: if you buy a table and I give you the edge of a table, are you happy?
<Keybuk> you're being silly
<Keybuk> stop it
<sadmac2> Keybuk: what does muddling states and events gain us?
<Keybuk> symmetry
<sadmac2> Keybuk: why not make states and malloc'd regions the same too then?
<Keybuk> you're still being silly
<sadmac2> If I'm being silly by following your logic to its conclusion...
<Keybuk> for example:
<Keybuk> one job
<Keybuk>   while network-device eth0 up
<Keybuk> another job
<Keybuk>   while apache running
<Keybuk>   on network-device eth0 up
<Keybuk> that's why it makes sense for events and states to be symmetric
<Keybuk> it's simple, obvious and elegant
<sadmac2> does network-device eth0 up correspond to starting, started, stopping, stopped, or something in the middle?
<Keybuk> to whatever you want
<sadmac2> and how do you specify which?
<sadmac2> Keybuk: I see what you're doing, but I think you describe it poorly. I think this model has no events at all (at least not ones visible to the user)
<Keybuk> on is an event
<sadmac2> on itself?
<Keybuk> no
<Keybuk> on matches events
<Keybuk> while matches states
<Keybuk> states are either true or false
<sadmac2> but events are states
<sadmac2> or so you say
<Keybuk> events do not have any value
<Keybuk> events are when states change
<sadmac2> so are events the same as states or different?
<Keybuk> events occur when states change
<sadmac2> ok, so events aren't states
<sadmac2> that kind of ruins your explanation of on's arguments
<Keybuk> why?
<sadmac2> Keybuk: because that explanation began "events are states"
<sadmac2> Keybuk: and we've now arrived at "events aren't states"
<Keybuk> only the way you describe them
<Keybuk> to me they're one and the same thing
<sadmac2> Keybuk: then why do we have two words for it?
<sadmac2> not a rhetorical question.
<Keybuk> because I haven't thought of a better word yet
<sadmac2> Keybuk: I could follow you if there were no events at all, just states
<Keybuk> I don't like the word "states"
<sadmac2> Keybuk: on apache started => start at the earliest point in time at which apache started is true
<Keybuk> right
<sadmac2> Keybuk: if you're into the true/false thing, "conditions" would be the next point.
<Keybuk> I'm not as much of a fan of PhD level logic problems are you are ;)
<sadmac2> Keybuk: all of this makes 0.6 compatibility reeeely grody. Are you still interested in 0.6 compatibility?
<Keybuk> I don't see it as a problem
<sadmac2> Keybuk: I think "conditions" is a fairly sane word
<sadmac2> Keybuk: information coded as conditions/states for 0.10 as you describe is coded as environment variables in 0.6. That gets scary fast.
<sadmac2> unless these two things don't interact seamlessly
<ion> keybuk: Whatâs missing from the ubuntu-boot PPA so that one could upgrade to it and have a booting system? Perhaps i could do something.
<sadmac2> Keybuk: I actually prefer conditions precisely because its /less/ prominent in PhD-land. States is a reeeeely loaded word from the formalist side of things. Especially when we talk about "state machines"
<sadmac2> which are rarely what "state machines" really are when we're talking about them.
<sadmac2> Keybuk: also I wonder about the usefulness of your example. What wants to start "on network-device"? If it needs the network device to start why doesn't it care if it goes away halfway through initialization?
<sadmac2> so if on x means "at the earliest point when x is true," then I'd assume we'd have an after x for "the earliest point when x is false"
<sadmac2> from...until would want a from after...until, and a from...through counterpart as well
<Keybuk> ion: figuring out why it's not working ;)
<Keybuk> it works on one mini
<Keybuk> but packaged up it doesn't
<Keybuk> trying to figure out what step I've missed
<Keybuk> sadmac2: ntpdate is started when a network device comes up
<Keybuk> I don't think your "after" case is invalid, I've been thinking the same thing
<sadmac2> Keybuk: beyond that, I'd keep the term states. It has heritage in the project and its not really "wrong" just yet. (Events I'd stop saying for awhile. They're still present here, but they're hidden."
<sadmac2> honestly I'd be ok with calling them jobs.
<sadmac2> that's what the code calls them now (or what will become them)
<Keybuk> right, I suspect that's what the code will always call them ;)
<sadmac2> Keybuk: so in the phrase "on foo" foo is a state, but on infers an event from it. "after" would take the same state and infer a different event.
<Keybuk> don't know
<Keybuk> I've used after for temporary events
<Keybuk> <time period> after <event>
<sadmac2> Keybuk: either way. Point is events don't really have names and arguments anymore. We don't talk about them directly in the config. We mention states, and an event is implied by context.
<sadmac2> Keybuk: I'd also imagine initctl emit doesn't make much sense anymore. Things like udev can just start states for their devices.
<Keybuk> no it still makes sense I think
<Keybuk> there are still events that don't have attached states
<Keybuk> events used as commands
<sadmac2> Keybuk: how do we talk about them in job definitions. Do we just put them in the same namespace as states so "on fooevent" works too?
<Keybuk> that's a pretty good question
<Keybuk> I can't find my notes on this
<Keybuk> but basically you have events that are really states
 * sadmac2 gets nervous
<sadmac2> how long do they last?
<sadmac2> also you should publish these "notes" :P
<Keybuk> I mean in the current scheme
<sadmac2> were we all perfect :)
<Keybuk> to publish involves typing
 * Keybuk writes a lot with pen, paper and whiteboards
<Keybuk> so
<Keybuk> events that are states
<Keybuk> "job started" type things
<Keybuk> there are events that are real events that would never have states
<Keybuk> control-alt-delete being a good example
<Keybuk> and there are events that we use a bit like commands
<Keybuk> where the *completion* is important to the emitter
<Keybuk> maybe hooks is more appropriate than commands as a description
<Keybuk> "I'm going to reboot, does anyone need to do anything first?"
<sadmac2> Keybuk: so theres still the question of this time-interval (state) => point-in-time (event) mapping.
<sadmac2> For the first case, job started clearly maps to an interval. if we need an event we say "The earliest point at which 'job started'"
<sadmac2> for case two... that's hard.
<Keybuk> I don't think it's earliest point at which
<sadmac2> for case three, that looks a lot like injecting case 1.
<Keybuk> it's "after 'job started' has happened"
<Keybuk> the key point is that 'job started' may no longer be true anymore
<sadmac2> Keybuk: so when the job stops
<Keybuk> becoming false doesn't cancel the fact that the event is still queued
<sadmac2> Keybuk: I would say job started is a convenience alias for job running
<Keybuk> right, so that's the interesting bit
<Keybuk> isn't "job running" just "job" ? :)
<Keybuk> or is "job" true when the job *should* be running, but "job running" true when the job *is* running
<Keybuk> etc.
<sadmac2> Keybuk: assuming "job foo" can't be true unless "job" is true, then just job would have to be job starting U job running U job stopping
<sadmac2> Keybuk: which means job is true when it should be running
<sadmac2> Keybuk: which I like
<Keybuk> right
<Keybuk> that's pretty much the direction I was thinking
<sadmac2> Keybuk: the idea that on tests for presence also gets around the "what if you start on started but its already running when you insert your job" thing.
<Keybuk> huh?
<sadmac2> Keybuk: suppose I have a running system, where apache is running. If I add a new job and have "on apache started" in it, if on is looking for an event, my job won't be running, because the "apache started" event has already passed. If on checks for earliest point at which a state is true, though, it can figure out to start my job right away.
<sadmac2> when I say new job I mean new job and class. Completely new .conf
<Keybuk> err
<Keybuk> but you don't *want* that to be started
<Keybuk> that's the whole point
<Keybuk> if you're using "on" it should NOT look-behind like that
<Keybuk> your "on apache started" job should only be run *when* apache is started
<Keybuk> not if apache happens to be running
<sadmac2> yeah. out-moded problem.
<Keybuk> this is exactly how it differs from "while" :)
<Keybuk> (and that while also implies a stop condition)
#upstart 2009-09-09
<mbiebl> Keybuk: hi
<Keybuk> hey
<mbiebl> tested your ubuntu-boot ppa
<mbiebl> noticed two issues:
<mbiebl> With FSCKFIX=no, it fails to boot
<mbiebl> and I get a warning during boot from mountall, that it couldn't write the pid file
<mbiebl> as the fs was ro
<Keybuk> err, as in you *set* FSCKFIX=no?
<mbiebl> mountall also mounts stuff which is later attempted again by the apparmor sysv init script
<mbiebl> Yes, I had set FSCKFIX=no
<mbiebl> With FSCKFIX=yes it boots fine
<mbiebl> I'm getting a warning though: init: mountall main process (927) terminated with status 1
<Keybuk> oh, that's not good ;)
<ion> mbiebl: Do you have multiple partitions in fstab, btw? Are they on the same disk?
<ion> mbiebl: Oh, actually, do you have multiple pass=2 partitions in fstab?
<mbiebl> interestingly I can't always reproduce this warning message
<mbiebl> The: "mountall: Unable to write pid file: Read-only file system" is reproducible
<mbiebl> The other one not
<ion> keybuk: Does mountall handle the pass=1 / partition first and pass=2 ones later in parallel? I havenât really looked at that part of the code.
<mbiebl> ion: I tested inside virtualbox installation with a / and swap partition
<Keybuk> no
<ion> Ok, no need to test my patches then. I probably should install virtualbox myself and test different partition configurations.
<Keybuk> pass= is fairly nonsensical for mountall
<ion> Ok
<mbiebl> Keybuk: another issue I observed is, that the upgrade of at hangs
<mbiebl> The "start at" command in postinst does not complete
<mbiebl> so the upgrade process is stuck at this stage
<mbiebl> I manually had to kill the start command so the upgrade process could finish
<mbiebl> I also remember problems with services that use restart in postinst instead of start/stop (like rsyslog), when the initial upgrade to the upstart job is made
<Keybuk> I thought I'd fixed all those issues
<Keybuk> it'll complain but not fail the postinst
<mbiebl> lemme check again. I set a snapshot in virtualbox, so I can easily test an upgrade again.
<mbiebl> Keybuk: looking at the bootchart of the default alpha5 install now
<mbiebl> What is this weird couchdb doing. It spawns xulrunner and takes over 5 secs
<mbiebl> Keybuk: I think FSCKFIX=no is the default
<Keybuk> it is
<sadmac2> mbiebl: couchdb wat?
<mbiebl> sadmac2: no idea what this is doing
<mbiebl> and why this is in a default install
<sadmac2> mbiebl: I'd love to know
<Md> couchdb is a key-value data storage
<Md> I fear that somebody tought it would be a great backend for some desktop application...
<mbiebl> Md: why does it need to run xulrunner for that?
<Keybuk> I know
<Keybuk> I saw that today
<Keybuk> BIG SIGH
<sadmac2> Md: is it just key-value? last I read up on it I thought it was a bit shinier.
<mbiebl> Keybuk: damn, I'm hitting the superblock mount time in the future issue in my virtualbox install
<mbiebl> I guess this is also where the new mountall fails
<sadmac2> mbiebl: your drive has negative write latency?
<Keybuk> hahaha
<mbiebl> could this be a ext4 or util-linux bug?
<Keybuk> did the new mountall fail properly to a shell
<Keybuk> ?
<Keybuk> it's a bug in the ext3 and ext4 filesystem code in the kernel
<Md> no clue about both issues, since it is written in erlang I stand clear from it
<mbiebl> Keybuk: when is it triggered?
<mbiebl> I don't seem to encounter it on every boot
<sadmac2> Md: something particular against erlang, or just not a Î» guy?
 * sadmac2 still has haskell on his list of languages to find something to do with.
<Keybuk> mbiebl: east of UTC, hardware clock in local time, system powered off without clean unmount
<mbiebl> It's the clean unmount that bothers me, because I did a regular shutdown/reboot
<Md> sadmac2: I had ejabberd explode puke erlang core dumps on me too many times...
<sadmac2> Md: shame. That's supposed to be one of the better jabber servers.
<Md> and indeed it is. you figure :-)
<Md> (I jave not tried the java ones, but I hate java with a passion)
<sadmac2> Md: you're a good man
<ion> CouchDB seems awesome and Erlang is awesome. :-P
<sadmac2> I'm very drawn to Î» programming. I need something to do in ocaml/haskell/erlang
<sadmac2> something that isn't such a wacky idea that I'm afraid of dooming it further by writing it in a language people are hostile to by policy
<Keybuk> mbiebl: hah, that FSCKFIX is never going to work, is it?
<mbiebl> Keybuk: what do you mean, setting it to yes?
<Keybuk> no, I mean that if you do FSCKFIX=no it'll still act as if it's yes
<mbiebl> dunno if this is the case with the new mountall command
<mbiebl> With the old initscripts, it made a difference
<Keybuk> huh? :)
<Keybuk> I mean the code in /etc/init/mountall.conf
<Keybuk> [ -n "$MOUNTALL" ] && fsck_fix="--fsck-fix"
<Keybuk> err
<Keybuk> [ -n "$FSCKFIX" ] && fsck_fix="--fsck-fix"
<Keybuk> is wrong ;)
<Keybuk> I just changed it
<Keybuk> [ "$FSCKFIX" = "yes" ] && fsck_fix="--fsck-fix"
<Keybuk> ;)
<mbiebl> ok
<sadmac2> Keybuk: I tried to test an upstart build the other day. It didn't grab the dbus name (because I didn't run it as root and left out the upstart.xml file), but moreover I couldn't look at it at all with d-feet. System logs said something about not having permissions to call the introspection methods. Will putting the permissions file in place and running as root miraculously fix this?
<mbiebl> Keybuk: found a few other issues: at vs atd
<mbiebl> after the upgrade, there is /etc/init.d/atd and /etc/rcS.d/S89atd because the upstart job is called "at"
<mbiebl> Directly after the upgrade, this is the list of initctl list: http://paste.debian.net/46172/
<mbiebl> As you can see, a few services are not listed as running, like rsyslog, udev or avahi-daemon
<mbiebl> interestingly dbus is marked as start/running without a pid
<mbiebl> There is also still a symlink /etc/rcS.d/S37udev-finish after the upgrade
<mbiebl> on shutdown, I also get the following messages
<mbiebl> init: rc main process (2383) stopped by STOP signal
<mbiebl> init: rc main process (2383) continued by CONT signal
<mbiebl> init: rc main process (2383) stopped by STOP signal
<mbiebl> init: rc main process (2383) continued by CONT signal
<mbiebl> ...
#upstart 2009-09-10
<Keybuk> mbiebl: the shutdown messages are normal
<Keybuk> that's unrelated
<Keybuk> at -> atd - good catch
<Keybuk> udev-finish - good catch
<Keybuk> the directly after upgrade stuff isn't ideal
<Keybuk> not sure why you have a running dbus though
<Keybuk> but it's because those are services are can't shutdown, so we can't "inherit" them
<Keybuk> they're still running under sysv conditions until you reboot
<mbiebl> Keybuk: I'd also suggest to remove dbus-reconnect.conf and run telinit q in dbus.conf's post-start
<Keybuk> mbiebl: doesn't work ;)
<ion> Why is that?
<Keybuk> because telinit q reloads all the configs
<Keybuk> and throws away the accumulated "start on" events ;)
<Keybuk> it was a bad choice for dbus reload, in hindsight
<ion> So, there should be a way to tell Upstart just to reconnect to dbus?
<Keybuk> yes
<ion> keybuk: rc-sysinit starts again if one runs mountall from a system that has already booted up. Any way to fix that?
<ion> The job starts running rcS scripts
<ion> keybuk: Why is anacron a task?
<PuffTheMagic> can someone point me to some docs on what events i can emit
<PuffTheMagic> like how do i start stop services
<PuffTheMagic> using initctl emit
<PuffTheMagic> or like... cant i get a list of services this way?
<mbiebl> ion: anacron is not a daemon
<mbiebl> It's triggered by cron (and other event sources)
<mbiebl> <Keybuk> and throws away the accumulated "start on" events ;)
<mbiebl> Do you intend to approach it differently then or keep the current solution?
<ion> mbiebl: Ah. In that case, âtaskâ is right, but something else must be done to prevent âstart anacronâ from blocking for ages, especially during the installation of the package.
<Keybuk> oh, is that what it's doing?
<Keybuk> hah
<ion> to053953 < ion> keybuk: rc-sysinit starts again if one runs mountall from a system that has already booted up. Any way to fix that?
<ion> to054029 < ion> The job starts running rcS scripts
<Keybuk> no, I mean anacron
<ion> I mean, thatâs another issue i talked to you about.
<Keybuk> oh
<Keybuk> Don't Do That Then
<ion> Hehe
<Keybuk> mountall will emit events like filesystem, which will start other bits
<ion> Indeed
<Keybuk> I guess rc-sysinit should have "start on startup and filesystem"
<Keybuk> because it should be only run-once
<ion> I think thereâs a slight design problem with my fsck-queue patch. Iâm just installing a virtual system to test. If thereâs a LVM VG that resides on /sda1 *and /sda2, the code will probably output {"8:0", "8:0", NULL} which then will get both locked (i.e. added to the lock hash).
<ion> That is, iâll have to make the get-physical-dev-ids code not output duplicates.
<Keybuk> *nods*
<Keybuk> mbiebl: you were saying something about /forcefsck not working ?
<keesj> so anybody has a good dbus-deamon starting service?(that waits for the service to be started?)
<Keybuk> keesj: ?
<Keybuk> not sure what you mean
<keesj> something that checks if the daemon is ready 
<Keybuk> you don't need to
<keesj> I need to sue fork for that?
<Keybuk> right, "expect fork" and "exec dbus-daemon --system --fork" are sufficient
<keesj> thanks
<Md> Keybuk: I have noticed myself as well that /forcefsck does not work since I switched to upstart
<ion> Ah, Mount *mnt isnât nih_allocâed. No wonder it segfaulted. :-P
<sadmac2> Keybuk: Yesterday you said that in the expression on foo started that started was a sub-state of foo (or sub-job or whatever)?
<sadmac2> Keybuk: I was wondering if you'd thought of a way to create more of these sub-states
<sadmac2> in terms of syntax etc.
<Keybuk> syntax is the wishy bit
<Keybuk> but yes, you can create them
<Keybuk> you can also create them deliberately inside logs
<Keybuk> err inside jobs
<Keybuk> e.g. apache logrotate
<keesj> I have an other meta physical question, suppose I have a dbus server that I want to launch using dbus activation, would'nt it be nice if is wat a upstart server?
<keesj> service
<Keybuk> yes
<keesj> (so I have respawn and such)
<Keybuk> it's relatively trivial to patch d-bus to use Upstart to start services
<sadmac2> Keybuk: heres a few thoughts then: Suppose the existence of a stanza called "exclude" which takes as its argument a few sub-states. It prevents those sub-states from being true at the same time
<Keybuk> why would you want that?
<Keybuk> and how would such a thing be resolved?
<sadmac2> Keybuk: if you tried to start one state it would stop any of the others in the list
<sadmac2> Keybuk: the "why" I'm getting to :)
<Keybuk> isn't that the same as "while not <state>" ?
<Keybuk> starting that implies that <state> has to be stopped
<Keybuk> (for the same reason that starting "while <state>" implies that <state> has to be started)
<sadmac2> Keybuk: for two states yes. for N or more its a bit more difficult. Plus this is a push maneuver. It implicitly /makes/ it true
<Keybuk> assuming you're talking about the physical "start" command
<Keybuk> which is the only case where we want that kind of behaviour
<Keybuk> I don't think we ever want auto behaviour like that
<sadmac2> Keybuk: ah, but we do, let me continue
<Keybuk> could you explain on the ML?
<Keybuk> I'm really busy at the moment
<sadmac2> Keybuk: sure. It'll be out of context there since we started the discussion here, but that's our fault to begin with anyway :)
<Keybuk> you can provide context ;)
<sadmac2> Keybuk: not to mention I'll see you in two weeks anyway :)
<Keybuk> indeed
<ion> keybuk: Okay, now everything seems to work â i have tested the fsck queue as well (in a virtual machine). Patches updated.
<Keybuk> awesome
<ion> Actually, i just noticed a missing nih_assert that should be there. Patches updated again. :-P
<Keybuk> ion: can you get rid of the LIKELY/UNLIKELY bits
<ion> Okay
<Keybuk> also no need to nih_warn() on repeating update_mount_dev_ids
<Keybuk> if it's not supposed to happen, assert
<Keybuk> if it's safe, just return
<Keybuk> actually, it's quite easy for that to be called multiple times
<Keybuk> e.g. a disk change for an md device
<Keybuk> (would you need to recalculate then, I wonder?)
<ion> update_mount_dev_ids is called just before fsck is spawned, and the dev_ids are only used for fsck locking.
<Keybuk> it looks like you call it in the udev device watcher
<Keybuk> in which case
<Keybuk> say the fsck is queued
<Keybuk> and a new update for the udev device comes in
<Keybuk> you'd need to dequeue the existing fsck (I don't think you even check for it right now)
<Keybuk> rescan the ids
<Keybuk> then queue again
<ion> Do md or lvm partitions ever become visible if they are not ready/fully initialized?
<Keybuk> yes
<Keybuk> though the watcher thing checks for that
<Keybuk> but they can *change* remember
<Keybuk> imagine:
<Keybuk> boot, mdadm initialised, in degraded
<Keybuk> user notices while fsck of / is happening
<Keybuk> (md is /var maybe)
<Keybuk> and pulls the disk out and replaces it
<Keybuk> you'll get a change for the queued md
<Keybuk> forcefsck - not sure this hook is right
<Keybuk> mountall gets passed this as --force-fsck on the command-line, it doesn't check for the file
<Keybuk> removing that should go in post-stop of the conf I think
<ion> Ok
<PuffTheMagic> so is there anyway to start a service using 'initctl emit'?
<sadmac2> PuffTheMagic: yes. initctl emit <something> where the service is defined with start on <something>
<PuffTheMagic> oh, but no equiv of 'initctl start <serfvicename>'
<sadmac2> PuffTheMagic: no
<PuffTheMagic> :(
<sadmac2> PuffTheMagic: why?
<PuffTheMagic> cause stupid palm dev had to use upstart 0.3.x on the Pre....
<PuffTheMagic> so there is no dbus interface
<PuffTheMagic> and i wanted to make a gui
<PuffTheMagic> for starting stopping services
<PuffTheMagic> so i have to make a service that wrapps around upstart/initctl
<PuffTheMagic> and relays stuff over dbus
<PuffTheMagic> and i wanted to just use events
<PuffTheMagic> but i guess i could just call initctl start/stop
<PuffTheMagic> just seems ugly
<PuffTheMagic> is there a libupstart or anything in the older upstarts that will give me more functionality?
<sadmac2> PuffTheMagic: there was a time when such a thing as libupstart existed. Not sure if its still rattling around in that code.
<PuffTheMagic> well im using 0.3.8
<sadmac2> PuffTheMagic: still.
<Keybuk> yes, there's a libupstart in 0.3.8
<Keybuk> (it's how initctl talks to upstart)
<PuffTheMagic> good
<PuffTheMagic> so i can use that to do start/stop :D
<Keybuk> sadmac2: *cough* isn't "that code" what you're going to be supporting for the next seven years? :p
<PuffTheMagic> haha
<sadmac2> Keybuk: 7? You're some crazy kinda optimist. I'm betting on it being around in 2040.
<sadmac2> I'll retire before 0.3 does.
<Keybuk> you could always ship 0.6 instead :p
<Keybuk> then, when you have some problem with it, I probably won't just point and laugh <g>
<PuffTheMagic> so the palm devs must have static build initctl then cause i see no libupstart on this userland
<PuffTheMagic> how lame
<Keybuk> PuffTheMagic: upstart always static-builds initctl and init
<sadmac2> Keybuk: you overestimate my authority on these issues. notting is the one to bug. maybe plautrba.
<PuffTheMagic> oh
<sadmac2> Keybuk: 8ball says don't count on it though :(
<sadmac2> Keybuk: I'm support engineering now though. I don't build the product. I just get stuck with the consequences.
<notting> Keybuk: didn't land in time (heck, i still haven't seen any patches for integrating it yet, and i do not have time to look at it myself right now)
<Keybuk> I guess you're not using it any differently than in F10?  No plans to convert more jobs over?
<notting> nope. but i'd assume it's still not a no-op to land it
<sadmac2> notting: its missing the state transfer thing, which means the ttys-don't-respawn thing will come back. other than that its not hard.
<sadmac2> not a no-op, but not hard.
<sadmac2> Keybuk: I wasn't planning on moving anything until 1.0
<sadmac2> just gonna have to fiddle with them again then anyway.
<sadmac2> 1.0/0.10 depending on how stable we feel.
 * Keybuk is moving things now, it's fun
<ion> keybuk: Patches updated.
<ion> Shutdown seems to be delayed by perhaps 10 seconds by this: http://heh.fi/tmp/upstart-rc-stop-cont-stop-cont
<ion> LVM seems to be nice enough to compute which physical partitions each logical volume spans, and only report those partitions in /sys/.../slaves instead of e.g. every single partition the volume group uses.
<ion> keybuk: Did you receive the above messages?
<Keybuk> ion: I can't answer that question ;)
<Keybuk> shutdown: the "10 seconds" is killall5 running
<Keybuk> the upstart output is just a side-effect of what killall 5 is doing
<Keybuk> some other process is the one not stopping
<ion> Ok
<ion> Why SIGSTOP?
<ion> Can you review the new patches, btw?
<mbiebl> ion, Keybuk: who is responsible for mountall?
<ion> Dunno about responsible, but Keybuk is the author. :-P
<mbiebl> ok, there is a bug wrt to ntfs mounts
<mbiebl> I have a ntfs-3g auto mount in my fstab and subsequently mountall fails
<mbiebl> which stops the boot process
<mbiebl> I think there are two issues here: 1.) failing to mount ntfs(-3g) mount points
<mbiebl> 2.) the mount point is non-critical (mounted as a shared partition under /mnt/data)
<mbiebl> the boot should not fail because mount failed for a non-critical mount point
<mbiebl> Keybuk: and a more cosmetic issue: the output of mount differs between the old-style mounting using the sysv initscripts and mountall
<mbiebl> http://pastebin.ca/1561541
<mbiebl> Keybuk: btw, is there a bug tracker for this sort of issues or should I send it to the m-l or via pm?
#upstart 2009-09-11
<Keybuk> ion: I believe that killall5 sends SIGSTOP to all processes so that the process tree is unlikely to change while it iterates it
<Keybuk> mbiebl: haven't seen the ntfs-3g problem before, any idea why it fails?
<Keybuk> mbiebl: mountall should just call mount the same way mount -a does
<Keybuk> does the boot "fail" or does the boot hang?
<Keybuk> /mnt/data won't stop the "filesystem" event being emitted, but it would cause a shell
<Keybuk> and as to the output difference, it's simply cosmetic
<Keybuk> mountall leaves the device field as "none" for nodev filesystems deliberately
<ion> Btw, using the vbox guest filesystem that mounts host directories in fstab caused mountall-shell to activate because the module wasnât loaded yet. The system booted fine otherwise, and after the init script loaded the vboxvfs module, just exiting the shell finished the job. I wonder what could be done?
<ion> Hardcoding special cases for everything like that doesnât sound very appealing.
<Keybuk> the shell is probably over-sensitive
<Keybuk> really you only want to know whether mountall failed to mount an FHS filesystem
<Keybuk> sorry, want to *fail* if
<ion> Yeah
<Keybuk> if some other filesystem marked "auto" fails, it's not realllly critical
<Keybuk> but I'm not quite sure yet how to strike the balance
<Keybuk> esp. since debugging, it's sane to leave it slightly over-keen for now
<ion> How about /etc/mountall/fs/networking: âuntil network: nfs nfs4 smbfs...â and /etc/mountall/fs/vbox (installed by vbox-guest-utils): âuntil vbox: vboxsfâ and have the if-up.d and vbox-guest-utils scripts send a message to mountall about ânetworkâ or âvboxâ being available?
<ion> And have mountall delay the mounting of such filesystem types until the message
<ion> Perhaps instead of sending a message, launch a parallel mountall with a ânetworkâ/âvboxâ parameter.
<ion> Or something :-P
<Keybuk> well, that's basically how network works now
<Keybuk> you send it SIGUSR1 and it retries them
<Keybuk> we could just add vboxsf to that set
<Keybuk> the bit that needs fixing is that failure to mount them can be temporary ;)
<Keybuk> (ie. it's ok to try again later)
<ion> True
<Keybuk> might be worth applying the same to outside-of-FHS filesystems?
<Keybuk> it's ok for them to fail to mount, just retry?
<ion> Yeah
<ion> keybuk: http://pastebin.com/m17081a2f
<mbiebl> Regarding the ntfs issue: The boot simply hangs at this point
<mbiebl> LABEL=Data      /mnt/data       ntfs-3g user,auto     0       0
<mbiebl> is the fstab entry
<ion> (An irrelevant comment: /mnt is for temporary mounts. If itâs in fstab, itâs not temporary. :-P)
<soren> I think, actually, /mnt is for temporary mount (singular).
<ion> True
<soren> (i.e. not mounting stuff on subdirs of /mnt, but on /mnt itself)
<ion> Unless you need to have two temporary mounts at the same time. :-P But yeah, if you only need one, mount foo /mnt is quick and nice.
<ion> keybuk: Regarding the pseudocode above, the try-again-if-mount-failed thing would become a simple âwhileâ loop around some code in a single function instead of additional logic buried in the a callback function somewhere.
<mbiebl> ion: fwiw, it doesn't really matter if I mount it at /mnt/data or /data ;-)
<ion> You lose the ability to mount a temporary partition without having to do a whopping amount of work to make a new directory first. ;-)
<mbiebl> ion: What I meant was, that it doesn't matter wrt my failing boot
<ion> As i said, my nitpicking was an irrelevant comment. :-)
<mbiebl> It seems, robbiew just found the cause for the failing ntfs mount
<ion> What was it?
<robbiew> ;)
<mbiebl> mountall tries to run fsck.ntfs
<mbiebl> which is not available
<mbiebl> it also seems to pass a -n option to ntfs-3g, which it doesn't understand
<ion> Huh. If iâm reading the code right, when passno = 0, it shouldnât try to fsck it.
<ion> keybuk: The equivalent could be nice for libnih as well. Too bad nih_log_message doesnât know the callerâs going to abort. http://launchpadlibrarian.net/31650305/glib2.0_2.21.6-0ubuntu1_2.21.6-0ubuntu2.diff.gz
<ion> http://developers.slashdot.org/story/09/09/11/1139212/Apple-Open-Sources-Grand-Central-Dispatch
<sadmac2> ion: just so long as their blocks implementation doesn't require trampolines
<ion> Dunno
<ion> Iâve only read http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/12 and the next page so far.
<sadmac2> ion: it'd probably have to get as far as gcc before we knew
<Keybuk> mbiebl: I don't see what mountall is doing that mount -a doesn't here
<Keybuk> remember, mountall isn't passing -n to ntfs-3g, _mount_ is
<Keybuk> likewise mountall isn't calling fsck.ntfs, _fsck_ is
<robbiew> Keybuk: in any case...*somethings* broken that we would need to fix before release, right?
<Keybuk> oh, definitely
<Keybuk> I just won't yet understand why mountall is failing when the old script didn't
<Keybuk> but that's ok
<Keybuk> this is just a bug
<Keybuk> in some ways I'm glad we're finding some now :p
<robbiew> yep
<Keybuk> robbiew: can you mail me your /etc/fstab as well?
<robbiew> sure
<ion> keybuk: I take it you noticed my messages above? :-)
<Keybuk> fsck /windows [1581] exited normally
<Keybuk> ah
<Keybuk> so it complains about no fsck.ntfs but doesn't actually consider it an error
<Keybuk> ion: I did, but now is not really the time to suddenly think about rewriting something with some neat library that would need an MIR, etc. etc.
<Keybuk> mountall: mount /windows [1590] terminated with status 11
<Keybuk> mountall: Filesystem could not be mounted: /windows]
<Keybuk> hmm
<sadmac2> ion: I'm betting the gcc implementation of blocks would end up using trampolines
<Keybuk> even the exit status makes no sense
<sadmac2> ion: and consequently be a force of overwhelming evil
<Keybuk> mount status 11 means
<Keybuk> "user interrupted a system error whilst permission denied"
<Keybuk> !!
 * Keybuk thinks ntfs-3g is just broken wrt compatibility with mount
<robbiew> I didn't touch anythin 
<robbiew> I can run 'mount /windows' after login...no problems
<robbiew> fwiw
<Keybuk> yeah that's what doesn't make sense
<Keybuk> because that's basically what mountall does ;)
<Keybuk> it seems it doesn't like the "-n"
<robbiew> /dev/sda1 on /windows type fuseblk (rw,nosuid,nodev,allow_other,default_permissions,blksize=4096)
<ion> I didnât mean that would have to be done for karmic, but it would be... interesting. :-) In fact, i might be interested in refactoring (wouldnât really need a full rewrite) the code within the next n years, if you ever bless that idea. :-P
 * Keybuk may have to fix ntfs-3g to support -n
<Keybuk> ion: I'm hoping to understand the corner cases enough to recode most of mountall into Upstart itself - it should be largely flexible enough eventually
<Keybuk> depends really
<ion> Alright
<Keybuk> right, according to mount ntfs-3g should support the -s, -f, -n, -v & -o options
<Keybuk> since those are the mounts that mount passes to mount.*
<Keybuk> and looking at the ntfs-3g code, it's probably ok to workaround in mountall either
<Keybuk> so this should be simple enough
<ion> keybuk: About my patches, i ended up creating a hash of dev_ids and then converting it to a string array in update_mount_dev_ids, and then iterating over the string array when locking. Thatâs only because i implemented it as a string array first and then started using the hash to dismiss duplicate dev_ids. I might as well put the hash thatâs generated anyway to the Mount struct and iterate over that when locking. Should be a quick enough change. Do you agree?
<Keybuk> yup
<ion> Testing in a vm...
<ion> Ah, no wonder i got no debug output. console output missing.
<ion> Seems to work. http://heh.fi/patches/mountall/ updated.
<ion> Oh, i think i noticed a bug with the fsck_queue list. Iâll fix.
<sadmac2> Keybuk: please see new upstart-devel email (and old one for that matter :)
<ion> keybuk: Updated once again.
#upstart 2009-09-12
<mbiebl> Keybuk: regarding the sigstop/sigcont issue
<mbiebl> it seems that sendsigs tries to kill its parent process and upstart respawns it
<mbiebl>  so I naively tried to add -o $PPID to the killall5 call
<mbiebl> but that didn't help
<mbiebl> have you investigated this further already?
<ion> Curious. If i use mountall --debug ... >/dev/mountall.log 2>&1 in mountall.conf, itâs as if two separate instances are writing to the log file at the same time.
<Keybuk> mbiebl: it doesn't try and kill it
<Keybuk> the bug is simply that sendsigs is doing something ridiculous
<Keybuk> but it isn't even a bug, nothing bad is happening
<Keybuk> what happens is that killall5 (called by sendsigs) sends SIGSTOP to all processes
<Keybuk> then it reads /proc
<Keybuk> then it sends those processes SIGTERM
<Keybuk> then it sends SIGCONT
<Keybuk> the idea of the STOP/CONT being that it prevents new processes from being spawned while it's reading /proc
<Keybuk> it has to stop all processes
<Keybuk> and then it does this over and over again in a loop, until no processes are left
<Keybuk> while it ignores some pids (including its parent) for the purposes of sending SIGTERM and may be left afterwards, it still sends them SIGSTOP
<Keybuk> it's silly
<Keybuk> because it's trying to win a race
<Keybuk> but it's just doping it
<Keybuk> most particularly, because init itself ignores SIGSTOP ;)
<Keybuk> so init can always spawn new processes if it likes
<Keybuk> so rcS isn't dying or being respawned
<Keybuk> the shell associated with it is just getting hammered by SIGSTOP/SIGCONT signal pairs until all *other* processes on the system have died
<Keybuk> (sendsigs "ignores" rcS itself, since it's its parent)
<Keybuk> if you see it going on for a long time, it means that _some_other_ process is not terminating
<Keybuk> even with SIGKILL
<Keybuk> that implies one in uninterruptible sleep in the kernel
<Keybuk> sendsigs/killall5 don't handle this
<mbiebl> Keybuk: thanks for the explanation
#upstart 2010-09-13
<TheBiggerGuycom> Hi, is there anyone who can help me with https://bugs.launchpad.net/upstart/+bug/637172
<bvleur> Hi. I'm wondering why a script with "start on started ssh" can launch processes with a lower pid than "sshd"
<ashb> bvleur: because pids are fairly random
<ashb> and the sshd might well fork
<bvleur> okay. fair enough. I've created a script that runs autossh with remote forward to localhost:22. It seems that running it before sshd is up makes the forward fail, so I want to make sure they start in order. PIDs don't tell me the boot order then right?
<yaaang> is it possible to use upstart (init, start, etc) as non-root?
<yaaang> would like to daemonize & manage processes without requiring root; supervisord seems to fit the bill but i was wondering if i should also take a deeper look at upstart 
<JanC> yaaang: why not use sudo to manage who can start/stop/etc. system daemons without needing to give them general root access?
<yaaang> JanC: that can also restrict to starting/stopping specific services?
<JanC> yaaang: yes, see the sudoers manpage for examples
<JanC> you can restrict people to specific commands including parameters etc.
<JanC> plus, with sudo every time they use this, it will be logged
<ion> A future release of Upstart will handle usersâ jobs as well.
<yaaang> JanC: how does a non-root user add/remove jobs?
<yaaang> ion: ah, i see
<JanC> you could create a script to add a new job and allow users to use that script with sudo, but of course that has serious security implications
<yaaang> JanC: right
<JanC> I suppose it wouldn't too difficult to create a system-wide upstart job that looks for user-specific job files though
<yaaang> maybe even easier than that is the supervisord route - just let users run their own process management daemons
<JanC> I don't know supervisord, but probably it could be that upstart-job  ;)
<JanC> at least until upstart grows these features itself
#upstart 2010-09-17
<warly> I am stuc/win 8
<warly> oops, sorry
<warly> What could make a initscript block when doing a 'start XX'
<warly> I just take the sshd initscript, replace the exec with another binary, and the 'start XX' or 'stop XX' block indefinitively
<ashb> warly: not forking properly?
<warly> ashb, hum, maybe, still I am using libdaemon and everything seems fine, but maybe it takes too long to fork?
<ashb> no idea
<ashb> just a guess
<warly> where are stored the status of running or stopped script
#upstart 2011-09-12
<jhunt> PuffTheMagic: http://upstart.ubuntu.com/cookbook/#running-upstart-as-a-non-privileged-user
#upstart 2011-09-13
<c9er> hey... i am trying to make an upstart script for rabbitmq but it blocks it starts the service and when i stop it the service is stopped but the stop process hangs and if killed it and try to start it again the start hangs
<jhunt_> c9er: sounds like Upstart is unable to find the pid of the rabbitmq process. Have you specified "expect fork" or "expect daemon"? http://upstart.ubuntu.com/cookbook/#expect
<c9er> yes i did
<jhunt_> which one? maybe you specified the wrong option (they are different)
<c9er> expect fork
<jhunt_> I don't know rabbitmq, but if it forks twice, that won't work.
<c9er> what if it forks more than twice?
<jhunt_> it wouldn't need to. 2 forks is the maximum number of forks any daemon needs to perform to initialize.
<c9er> will then i guess i will try the expect daemon
<jhunt_> it might subsequently fork off process of course to handle incoming requests, but we're talking solely about daemon initialisation this point.
<jhunt_> That *should* work, but if it doesn't, you can cheat: http://upstart.ubuntu.com/cookbook/#alternative-method
<c9er> another question is there anyway to make the upstart reread the files without rebooting 
<c9er> because you know i define an instance then i remove it then i stop ther service but the upstart insists that i have to define that instance 
<jhunt_> there is a caveat though - although that will ensure the process starts, upstart still won't know the pid, so won't be able to stop it gracefully.
<jhunt_> you want upstart to re-read its .conf files?
<jhunt_> removing a .conf file when an instance is running, is going to cause you problems :)
<jhunt_> upstart re-reads its .conf files automatically whenever any of them changes.
<c9er> no they don't 
<jhunt_> ?
<c9er> and i mean by instance the instance variable inside the file
<c9er> it keeps telling me that variable need a value
<c9er> while i have already removed it's line from he file
<jhunt_> If you have a running job that needs an instance variable, yes, you'll need to specify the value of that instance variable to stop that running instance.
<jhunt_> See http://upstart.ubuntu.com/cookbook/#instance
<jhunt_> Also see: http://upstart.ubuntu.com/cookbook/#attempting-to-stop-a-job-that-requires-an-instance-variable
<c9er> i don't mean so
<c9er> i have already removed the line of the variable after stopping the service
<c9er> the next time i need to start it it asked for the value
<jhunt_> c9er: sounds like you might have another instance of that job still running?
<c9er> nope i check the ps output each time
<c9er> i think i may report it as a bug
<jhunt_> please do, ensuring you specify exactly how to reproduce the problem.
<jhunt_> Going back to what you say above: "i define an instance then i remove it then i stop ther service"
<jhunt_> this is wrong - you have to start the instance, stop then instance *then* remove the variable from the file.
<c9er> that's exactly what i have done
<c9er> :)
<c9er> i have wrote upstart script on ubuntu 10.10 version 6.6 it'w working perfectly but when i use the same script on upstart 9.7.3 on ubuntu 11.04 it freezes
<c9er> any help?!
<jhunt_> c9er: please raise a bug with the details: https://bugs.launchpad.net/ubuntu/+source/upstart/+filebug
<c9er> any idea why would the upstart script hang when i use stop?
<c9er> kay does upstart have any kind of a data base locking file or caching thing?
<JanC> c9er: AFAIK upstart keeps all its "state" data in RAM, and the only thing resembling a database is the collection of *.conf files?
<c9er> JanC, is there anyway to access these data in RAM?
<JanC> not really
<JanC> you can get some of it using some 'initctl' commands like "list"
<c9er> well do you know why would stop hangs?
<JanC> and of course you can turn up log verbosity & check the log messages
<c9er> JanC, how is that?
<JanC> c9er: not without context
<JanC> c9er: read "man initctl" for the command to change log verbosity
<c9er> k
<JanC> it's named 'log-priority'
<JanC> you can also set it on the kernel commandline in Ubuntu (see the cookbook)
<c9er> JanC, so any idea why would the script hang on stop?
<JanC> c9er: script?
<JanC> c9er: did you read http://upstart.ubuntu.com/cookbook/#debugging ?
<c9er> yea i did
<[dlp]> Enhancement request:
<[dlp]> At boot time, with "noisy" output (and no splash screen).
<[dlp]> ..
<[dlp]> Information about what process is being started left justified...
<[dlp]> Indication of success: right justified.
<[dlp]> Most monitors are wide enough to make it impossible to correlate one to the other.
<[dlp]> Either print them together and be jaggedy or pad it with dots.
<[dlp]> If anyone agrees and is prepared to point me inthe direction of exactly the right source file I'll do it myself.
#upstart 2011-09-14
<jMCg> I'm thinking out loud here, but would it make sense to support something like taskset(1) or sched_setaffinity(2) for that matter, in upstart?
<phirox_> looks like upstart follows fork or spawned children of a process it's monitoring, any way to turn that behaviour of? it leads to a lot of overhead coming from the init process
<jhunt> out of interest, have you measured that overhead and found it to be too high for your environment?
<phirox_> it is the #1 process measured in cputime :)
<jhunt> Upstart only uses ptrace for "expect fork" and "expect daemon". The overhead of using ptrace is negligible for starting such jobs.
<phirox_> ah ok, well I made a nagios init. what happens is that everytime nagios spawns a process, somehow init gets triggered
<phirox_> this is what init does in a strace: http://pastebin.com/JSkKFE7p
<jhunt> upstart reaps zombie processes (as do all init systems). That is correct behaviour and not something you can (or would want to) turn off.
<phirox_> ok thanks, I'll look for another way then
<jhunt> I'm confused. Is there a problem with upstart on your system?
<phirox_> actually no, I think I just understand what you mean. doesn't matter how I start these 1000's of processes. init still has to check them
<phirox_> so this isn't really an issue, probally more of a misunderstanding on my part :)
<jhunt> yes. If init didn't do this, your process table would eventually fill up with zombies, which might eventually kill the system.
<phirox_> yeah, makes sense. thanks for explaining
<jhunt> no problem
#upstart 2011-09-15
<codebeaker> hi, I'm searching (fruitlessly) in the docs for some info about any provision upstart makes for restarting processes without sending them a TERM and then booting them again, many bits of software (I'm talking Unicorn and Resque) here can be gracefully restarted using signalsâ¦
<jhunt> codebeaker: "man 5 init" has lots of information on what I think you're asking for.
<codebeaker> ah, cool - thanks
<jhunt> codebeaker: if you're referring to maybe sending SIGHUP to a process, "man initctl" and search for SIGHUP.
<codebeaker> thanks, it's not solely about SIGHUP, example "Resque" (background workers for web apps) expects USR1 for graceful restarts
<codebeaker> but thank you in advance for the guidance
<jhunt> np. in init(5), also see the section on "kill signal".
 * codebeaker offers a bow
<codebeaker> cheers
<jhunt> any time
<JanC> codebeaker: the codebook also has lots of good info
<codebeaker> thanks JanC 
<JanC> teh cookbook I mean  ;)
<JanC> http://upstart.ubuntu.com/cookbook/
<codebeaker> cookbook > wiki
<codebeaker> I hadn't realized the wiki was so badly outdated until I happened to click the wiki homepage
<codebeaker> also, I recognized there's no obvious stanza for running a job as a specific user, is it the script author's responsibility to `su` and run? (Writing a wrapper for Unicorn, which is a forking server supporting specifying which user to run as, but opens it's log files as root before forking, and that's making a problem)
<jhunt> yes. we're planning to add setuid and setgid stanzas as the code is already in upstart for user jobs.
<jhunt> sounds like Unicorn needs to be fixed.
<codebeaker> ok, jhunt yeah, looks like, after the first request it hits (failed, or otherwise) then it sets the log permissions correctly, (and owner)
<codebeaker> but it's kinda lame
<codebeaker> thanks again jhunt - I really appreciate your time
<SpamapS> jhunt: maybe its time we took the wiki down, or at least added a massive disclaimer that its there for posterity only?
#upstart 2011-09-16
<fudoreaper> running Ubuntu 11.04, i want to delay the startup of DHCPd, because it starts before all my interfaces are up
<fudoreaper> as a result, it doesn't answer DHCP requests for 2 of the 3 interfaces it should
<fudoreaper> the network comes up just a couple of seconds after dhcpd does
<fudoreaper> hrm.. dhcpd doesn't seem to be an upstart job, actually
<fudoreaper> this may be the problem, essentially
<fudoreaper> i added a dhcp restart to rc.local
<fudoreaper> if anyone is watching my conversation
<fudoreaper> i'm out
#upstart 2011-09-17
<savid> Hi, I'm looking into possibly using upstart to manage my website processes (using nginx + gunicorn), but I don't want to have a separate script for each website.  Is it possible to have a single script manage multiple processes based on a given name?
<savid> For example, if my upstart script is called "website",  I would want:  "service website www.example.com start",   and I would want upstart to be able to monitor and respawn that individual process for that website separately.
#upstart 2011-09-18
<simenbrekken> Anyone know why this init script might fail? "exec sudo -u ec2-user /usr/local/bin/node /home/ec2-user/myapp/application/api/app.js 2>&1 >> /home/ec2-user/myapp/log/api.log", I don't get any output (even with start -v) and the logfile stays empty.
<jMCg> si.. meh. You foolks these days, ay too impatient.
#upstart 2012-09-10
<delinquentme> alllriight
<delinquentme> soo how do I launch mysql w upstart?
<delinquentme> how can I check to see that its installed
<delinquentme> a google on " check upstart ubuntu 10.04 version " gave me nada
<SpamapS> delinquentme: sudo apt-get install mysql-server should be sufficient
<SpamapS> delinquentme: or did you mean something else?
<delinquentme> SpamapS, seems that doesn't configure things correctly =/   I'm trying to figure out if mysql is running ( sudo netstat -tap | grep mysql )
<delinquentme> ... this is what the manpage says to do ...and the command simply hangs
<SpamapS> delinquentme: service mysql status should show something like 'start/running, pid 9999
<delinquentme> SpamapS, it gives me " mysql start/running "
<delinquentme> I mean thats not behaving correctly is it?
<SpamapS> delinquentme: no, if there is no pid listed, then something has gone wrong
<delinquentme> So how do I check that im actually running on upstart?
<SpamapS> delinquentme: what OS are you running?
<delinquentme> ubuntu 10.04
<SpamapS> delinquentme: ok, you are definitely "on upstart".. 10.04 will not work any other way.
<SpamapS> delinquentme: as far as mysql, have you modified /etc/init/mysql.conf in any way?
<delinquentme> I have, i have backups... but the issue predates the changes made
<SpamapS> delinquentme: and the issue is... ?
<SpamapS> delinquentme: has it always said "start/running" but not actually made mysqld run?
<delinquentme> mysql.sock is not being created
<joppla> hi, I need a little help figuring out some upstart quirks
<joppla> I have an upstart script that I start on networking and this works fine so far, but for some reason as soon as script is finished, it initiates stop, which should only be called on runlevel [!2345]
<joppla> do i need to have a service running for upstart to not call stop on a script? (the job is not defined as a task)
<SpamapS> delinquentme: well that is likely because mysqld is not running
<joppla> what the script does, basically, is this: on (networking and filesystems) mount container from network share into loopback, stop on runlevel [!2345] unmount loopback device
<SpamapS> delinquentme: but its also possible that you've changed the path in /etc/mysql/my.cnf or even /etc/my.cnf or (the very seldom considered ) /var/lib/mysql/my.cnf
<SpamapS> joppla: "start on networking" ?
<delinquentme> SpamapS, what is mysqld?
<SpamapS> delinquentme: *mysql*
<delinquentme> Ok check
<SpamapS> delinquentme: what did you modify in /etc/init/mysql.conf ?
<SpamapS> joppla: can you pastebin your upstart job?
<delinquentme> exec /usr/sbin/mysqld >> exec sudo -u mysql /usr/sbin/mysqld
<SpamapS> delinquentme: *that* is why you are failing
<SpamapS> delinquentme: change it back. You must start as root. mysqld will become mysql on its own
<SpamapS> delinquentme: user        = mysql
<SpamapS> delinquentme: that should be in /etc/mysql/my.cnf , and causes the switch to that user
<delinquentme> SpamapS, changes outlined here: http://www.bbhart.com/848/ubuntu-and-upstart-and-mysql/
<delinquentme> I mean he says to modify something else in my.cnf
<SpamapS> delinquentme: you're just blindly trusting that, without any evidence as to why?
<delinquentme> also its been reverted
<delinquentme> and still the same issue
<delinquentme> no mysql
<SpamapS> delinquentme: once reverted, you have to do 'stop mysql' then 'start mysql'
<SpamapS> delinquentme: 'restart mysql' will not re-load the upstart job
<delinquentme> $ stop mysql >> stop: Rejected send message, 1 matched rules; type="method_call", sender=":1.90" (uid=1000 pid=18202 comm="stop) interface="com.ubuntu.Upstart0_6.Job" member="Stop" error name="(unset)" requested_reply=0 destination="com.ubuntu.Upstart" (uid=0 pid=1 comm="/sbin/init"))
<SpamapS> delinquentme: sudo stop mysql
<delinquentme> damnit lol thats what that error meant?
<SpamapS> delinquentme: also I just checked, and the default my.cnf for 10.04 does not have skip-bdb in it
<delinquentme> yeah mine didn't so.. well at least thats good
<delinquentme> now sudo start mysql ... seems to be hanging
<joppla> SpamapS: http://pastebin.ubuntu.com/1197044/
<SpamapS> joppla: this is actually not correct, even though it sounds good.. "start on filesystem and started networking"
<SpamapS> joppla: runlevel [2345] is actually what you want (unless you're on pre-11.10)
<SpamapS> delinquentme: its likely failing because mysqld is printing errors of some kind
<SpamapS> delinquentme: do 'stop mysql', and then try 'sudo /usr/sbin/mysqld' and see if it prints any errors
<delinquentme> nada from $ sudo /usr/sbin/mysqld
<SpamapS> delinquentme: oh, check /var/log/mysql/error.log
<joppla> SpamapS: the problem is not get it started, it starts fine. The problem is that it mount the container and then directly unmounts it again
<delinquentme> theeere we go
<delinquentme> looks like i need write access for mysql
<delinquentme> that would do it.
<delinquentme> chmod 777?
<joppla> SpamapS: made the changes as you suggested, problem still exists though
<SpamapS> joppla: ok hm
<SpamapS> joppla: oh, yes, of course it does
<SpamapS> joppla: the main process (script or exec) is what controls whether it is running or stopped
<SpamapS> joppla: so the mount command exits, and upstart sees that as your job exitting normally
<SpamapS> joppla: you want that all to be in pre-start
<joppla> SpamapS: eureka :)
<joppla> thanks a lot, I would probably spent hours on this
<SpamapS> joppla: glad you could get it working
<delinquentme> http://pastebin.com/  so heres the error output  ... however now I'm unable to run " sudo stop mysql "
<SpamapS> delinquentme: /???
<delinquentme> SpamapS, it runs .. just only when mysql is in its semi-running state
<SpamapS> delinquentme: you pasted a pastebin.com link w/o output
<SpamapS> like, just pastebin.com/
<delinquentme> ROFL
<delinquentme> ohhh man
 * delinquentme deep breath
<delinquentme> 1 cat
<delinquentme> 2 cat 3cat .. OK http://pastebin.com/Cjarts72
<SpamapS> delinquentme: do you have any data in the mysql database?
<SpamapS> delinquentme: 120910  6:08:10  InnoDB: Operating system error number 13 in a file operation.
<SpamapS> InnoDB: The error means mysqld does not have the access rights to
<SpamapS> delinquentme: Most likely because of your sudo changes and such
<delinquentme> SpamapS, ok so I gave my user " thrive " permissions to the DB
<SpamapS> delinquentme: like, you chowned the files to thrive?
<SpamapS> delinquentme: this is a study in "all the ways to break your mysql server"
<delinquentme> yeah
<delinquentme> i should have a separate user huh?
<SpamapS> delinquentme: no, you should leave it alone and let it run as mysql
<SpamapS> delinquentme: if you need access to the data files, you're doing it wrong
<delinquentme> I mean the only thing that I've done which is out of the ordinary is drop in my own database files
<delinquentme> sudo apt-get install mysql-server
<delinquentme> according to the documentation should effectively " do what its supposed to do "
<delinquentme> however everything after that command ... on the official documentation (https://help.ubuntu.com/10.04/serverguide/mysql.html) is busted
<delinquentme> like official docs are saying to use init.d
<delinquentme> sudo /etc/init.d/mysql restart
<delinquentme> O_o;
<SpamapS> delinquentme: thats just a backward compatibility thing
<SpamapS> delinquentme: it works fine actually
<delinquentme> how?
<SpamapS> delinquentme: the init.d stuff works fine I mean.
<SpamapS> delinquentme: it calls a compatibility layer which in turn calls the appropriate upstart actions
<delinquentme> yeah =[  I mean I'd love a command to find out what the heck is special about my system
<SpamapS> delinquentme: there's no such thing, though you can use config management to assert what you want to be true. :)
<delinquentme> sudo apt-get install --reinstall mysql-server  
<SpamapS> delinquentme: note that there is a new LTS .. with more up to date instructions https://help.ubuntu.com/12.04/serverguide/mysql.html
<delinquentme> also it would be awesomely nice if this would reinstall the cruft which went wrong
<SpamapS> delinquentme: it won't because it doesn't want to disturb your data
<SpamapS> delinquentme: which means leaving /etc/* and /var/lib/mysql/* alone
<SpamapS> delinquentme: you can 'apt-get purge mysql-server-5.1' to get rid of your modified config files and (optionally) databases.
<delinquentme> Like i've not modified the dbs at all ... I can copy them back in 
<delinquentme> and then i just chown those DB files to mysql
<SpamapS> delinquentme: if you haven't modified them at all, you can just rm -rf /var/lib/mysql
<SpamapS> delinquentme: and then do the purge, and then reinstall
<SpamapS> delinquentme: the files will be created w/ the right perms, and all will be well
<delinquentme> however I still need to get the databases in for the project
<delinquentme> drop them in and chown them?
<delinquentme> and then all will be good?
<delinquentme> E: Couldn't find package mysql-server-5
<delinquentme> nm
<delinquentme> E: Couldn't find package mysql-server
<delinquentme> i mean it was mysql-server **
<delinquentme> Ok now I've purged ... then re-run "sudo apt-get install mysql-server"  but it didn't prompt me to set a password .. therefore .. not all of the original settings files were removed?
<delinquentme> and what would cause this command: sudo netstat -tap | grep mysql  .... to hang
<delinquentme> ?
<delinquentme> seems to me like thats indications of bigger issues
<SpamapS> delinquentme: mysql-server-5.1 is the actual package to purge
<delinquentme_> in #mysql
<delinquentme_> Ok so trying this again ... I've just run "sudo apt-get install mysql-server"  and I have no /var/lib/mysql dir
<delinquentme_> ok good so now the 5.1 purge worked ^_^
<delinquentme_> " ureadahead will be reprofiled on next reboot "
<delinquentme_> so im going to reboot
 * SpamapS applauds delinquentme_'s energy
<delinquentme> OK did fresh install .. set new password ... still socket weirdness when trying to start
<SpamapS> delinquentme: did mysqld actually start though?
<SpamapS> delinquentme: and can you show me the socket weirdness you're talking about?
<delinquentme> SpamapS, ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (13)
<delinquentme> same thing .. basically Im unable to start mysql
<SpamapS> delinquentme: 13 again
<SpamapS> delinquentme: ls -l /var/lib/mysql/mysql.sock
<SpamapS> delinquentme: also thats not the usual mysql.sock path
<SpamapS> delinquentme: should be /var/run/mysqld/mysqld.sock
<delinquentme> ls: cannot access /var/lib/mysql/mysql.sock: No such file or directory
<SpamapS> delinquentme: because thats the wrong dir
<delinquentme> ls: cannot access /var/run/mysqld/mysqld.sock: No such file or directory
<delinquentme> not there either
<delinquentme> ... I dont think I changed any of the default settings
<SpamapS> delinquentme: that socket is created when mysqld starts, so, it probably failed again
<delinquentme> 120910 16:23:19 [ERROR] Do you already d another mysqld server running on socket: /var/lib/mysql.sock ?
<delinquentme> sudo start mysql never actually resolved
<delinquentme> resolves**
<delinquentme> it just extended hangs
<delinquentme> mysql start/running, process 9839
<delinquentme> nice!
<SpamapS> delinquentme: are you absolutely required to use 10.04 btw? The upstart job is *a lot* better on 12.04
<delinquentme> SpamapS, I think im making progress
<delinquentme> and 10.04 is my go to default OS atm
<delinquentme> so its not just this that I have on it ... its the rest of my dev enviros
<SpamapS> delinquentme: ok. Well as somebody who has been actively trying to fix the mysql upstart job, and quite a few others since 10.04 was released, I have to recommend that you start looking at upgrading....
<SpamapS> delinquentme: plus, MySQL 5.5 is a huge improvement over 5.1
#upstart 2012-09-12
<j4m3s> helloooooo #upstart - n00b here
<j4m3s> i have two erlang processes i want to create upstart scripts for, and they require a few other services to be running (haproxy and rabbitmq)
<j4m3s> what's the better approach to this? creating events in rabbitmq that trigger the erlang processes to start   -or-   checking to see if the rabbitmq process is running?
<SpamapS> j4m3s: better is to not assume those services are on the local box
<SpamapS> j4m3s: start everything in parallel, make sure they poll/retry for services that they need.
<SpamapS> j4m3s: that will make your services more robust and easier to scale out.
<chris|> hi
<chris|> I was wondering, is there a way to have the `kill timeout` stanza accept a variable that needs to be sourced from a defaults file?
<SpamapS> chris|: no not really
<SpamapS> chris|: you could put it into a .override file if you're worried about changing the packaged .conf tho
<j4m3s> SpamapS, so polling for services would be more robust than waiting for an event?
<chris|> SpamapS, not, that's not really the issue, thanks though
<SpamapS> j4m3s: yes, because you can't wait for an event that happens on a different server
<SpamapS> chris|: one of the ideas behind upstart is to be able to parse everything in one directory right at boot time. shelling out to source a default file is just more wasted boot time when you could put the value in the .conf file and have it parsed already
<j4m3s> but you can poll for a process on another server? do you happen to know of a good example?
<SpamapS> j4m3s: you'd be polling for its network service
<SpamapS> j4m3s: apache can be started up before mysql, and still use mod_auth_mysql
<chris|> SpamapS, sure, but whenever you are porting from debian or still might need SysinitV support for some reason, it's always desirable to not maintain multiple configurations
<SpamapS> chris|: Indeed, if you *must* keep the /etc/init.d script alive on the same host, then the /etc/default/* file would be good
<SpamapS> chris|: but that seems like a rare occasion... what service needs both the upstart job and the local init.d ?
<kgraham> hello, using fedora/amazon linux and need to know how i would run a job before the network stops
<kgraham> ie 'the first thing run on shutdown/reboot'
<SpamapS> kgraham: start on deconfiguring-networking
<SpamapS> kgraham: thats on Ubuntu 11.10 and later
<SpamapS> kgraham: for fedora/amzn.. not sure.. maybe 'start on starting rc RUNLEVEL=[06]'
<kgraham> ok thanks!
<kgraham> i'll try that
<j4m3s> SpamapS, so am i correct in saying, pid's are unnecessary with upstart?
<SpamapS> j4m3s: pidfiles are usually not necessary, though there are still cases where they are needed when a program does weird stuff that makes it hard for upstart to track it
<j4m3s> SpamapS, if i have a start on stopped job, if that job respawns does the other job also restart?
<j4m3s> err lemme rephrase a lil more clearly, if i have a script for myapp, and it starts on stopped job foo, will myapp restart whenever foo respawns?
<SpamapS> j4m3s: start on and stop on just change the "goal state" for a job. If its already 'start' , start on does nothing.
<j4m3s> SpamapS, ok thanks
<j4m3s> so i have a job that i've started with upstart and when i do initctl status foo it says foo is stopped/waiting
<j4m3s> *stop/waiting
<SpamapS> j4m3s: indeed, if the main process exits, and you don't have 'respawn' specified, thats the normal behavior
<j4m3s> i have respawn specified
<j4m3s> the process is running 
<j4m3s> but upstart is having trouble tracking it i suppose
<SpamapS> j4m3s: did it exit 0? if it exits with a 0 return code and no signal killing it, then that is also considered a normal exit, not somethign to respawn for
<SpamapS> j4m3s: ah, so it daemonizes, but you haven't specified expect fork/daemon ?
<j4m3s> correct
<SpamapS> j4m3s: two options. 1: use expect fork/daemon, or tell the program not to daemonize
<SpamapS> I prefer the latter
<pulz> i have been testing upstart with some daemons latly, and in most cases upstart will report OK running, even if the process is killed 
<pulz> any reason for this behavoiur ?
<j4m3s> SpamapS, it appears rabbitmq-server created a bash wrapper that calls the erlang bins but doesn't exit
<j4m3s> SpamapS, and the job i'm creating is for a erlang process
<SpamapS> j4m3s: the only problem with the bash wrapper approach is that HUP, TERM, and KILL signals will be sent to the bash process. You want it to exec something that stays running/attached
<j4m3s> SpamapS, so i circumvented the bash script and called erl from the upstart script, but it still daemonizes and exits cleanly
<j4m3s> SpamapS, wait nvm my changes got wiped
#upstart 2012-09-13
<j4m3s> SpamapS, i can't seem to get this to work. the erlang program has to be detached and there's not a fork being done.
<SpamapS> j4m3s: erl can't run in the foreground?
<SpamapS> j4m3s: you can't detach without forking
<SpamapS> its the definition of detaching.. fork.. parent exits
<j4m3s> SpamapS, i put in expect fork but 'stop foo' hangs bc the PID it has is diff than the one running
<j4m3s> SpamapS, http://pastie.org/private/m0emxwvlroxbd0di4nyng 
<j4m3s> the line i commented out is the what the start.sh script executes it also sets the HOME var
<SpamapS> j4m3s: yeah don't use expect fork if you aren't positive that the process forks
<SpamapS> things get ugly tha tway
<SpamapS> j4m3s: in your case, the process forks once to run the shell script, but who knows what start.sh does
<j4m3s_> SpamapS, the line that's commented out i got from the start.sh file
#upstart 2012-09-15
<girasquid> I'm using the Foreman gem to export upstart config files - I exported an upstart config, and when I run `sudo start <my app>` I get told it's running, but..when I do `ps aux | grep <my app>` I see nothing, and /var/log/<my app> seems to be empty. How/where should I be looking to debug why upstart is telling me it started my service, but my processes aren't running?
#upstart 2012-09-16
<Kiall> Heya - I'm wondering if it's possible to have an upstart job wait for a specific filesystem to be mounted before starting a service? 
<Kiall> Specifically, I have some EC2 servers with EBS drives (think USB drives if you're not familiar), these drives may not be connected during boot, so the "nobootwait" flag is used in /etc/fstab to allow the system to continue to boot. I'd like to have services that keep their data on these drives wait until they are connected before starting.. Eg mysql.
<real_ate> hey everyone, i just have a quick question about upstart. I may be completely wrong in asking this but if someone could give me some guidance that would be great! 
<real_ate> how do i access RESULT in a post-stop stanza? I want to do something when the upstart job dies but has stopped respawning 
#upstart 2013-09-09
<ondine> hi, i have a quick question about upstart jobs with instances
<ondine> asuming i have a job 'start-workers' that starts 5 worker instances N=1..5, i know i can restart all workers with service start-workers restart
<ondine> but can i also safely restart only a specific worker, say worker N=3?
<xnox> ondine: yes. $ restart worker N=1
<xnox> e.g. look at $ sudo initctl list
<xnox> ondine: that will show all jobs (instance)
<ondine> xnox: okay, thanks
#upstart 2013-09-10
<ianco> Hello all. Does upstart handle/support logrotate currently?
<ianco> Im seeing differing things around the webs
<jodh> ianco: if you are referring to whether upstart handles logrotate running in /var/log/upstart/, the answer is yes.
<ianco> without a service restart?
<ianco> and is there any docs I can check out about it?
<ianco> my searching is leading me nowhere fast :|
<jodh> ianco: now I am confused. Please can you re-phrase the question?
<ianco> Im just looking for more info on this statement I found
<ianco> "Its default logging, for example, does not support log rotation and is only compatible with logrotate if you use the copytruncate hack"
<ianco> any truth to that?
<jodh> ianco: upstart will detect that logrotate has moved the file. Take a look at /etc/logrotate.d/upstart to see how logroate is configured to rotate upstart logs on a default ubuntu system.
<jodh> ianco: the behaviour of the upstart logger is documented in init(5) ('console' stanza in the "Process environment" section).
<ianco> thanks :) is this new behavior or has upstart always handled logrotate and moved log files ?
<jodh> ianco: I don't know where you are getting this other information. Best to stick with the reference material we provide where possible. The logging has always worked liked that since logging was introduced (upstart 1.4) see  - http://upstart.ubuntu.com/cookbook/#stanzas-by-category.
<ianco> jodh, thanks, I appreciate your help
<ianco> :)
<ianco> dont believe everything you read, i guess :P
#upstart 2013-09-11
<smokes2345> so i have an upstart process that isn't killing all its children when we try to stop, what's the workaround?
<jodh> smokes2345: sounds like a bug with the application upstart is running?
<smokes2345> â¢jodhâ¢ don't think so. if upstart is sending the kill signal to all the child processes, they would quit normally
<smokes2345> perhaps i didn't phrase the issue properly...
<jodh> smokes2345: upstart kills the main job process and all the processes in that process group.
<smokes2345> i have an x11 program i want to run as a service, its started with startx. when we try to stop the service, upstart kills startx, but none of its children are affected
<smokes2345> i've tried using 'expect fork' and 'expect daemon', but upstart seems to end up monitoring a process that doesn't exist or dies soon after starting
<jodh> smokes2345: so, the processes you are talking about must be in a different process group. Sounds like you have a rather custom setup: conventionally, you'd run the apps you care about as jobs in their own right.
<JanC> I think smokes2345 probably doesn't need expect fork/daemon ?
<smokes2345> â¢JanCâ¢ well, i didn't think so, but if i could get it to monitor more than 2 forks i could use 'expect' to solve my problem
<smokes2345> â¢jodhâ¢ we are running an in-house x11 app, but it is started using very conventional methods. the upstart script starts x as a normal user using startx, startx then calls the users .xinitrc script which starts our app
<jodh> smokes2345: what version of upstart are you using? Your life would be significantly easier if you were using a Session Init (http://upstart.ubuntu.com/cookbook/#session-init).
<smokes2345> not quite sure about the version, although i'm sure it's not the latest and greatest. how can i get the version?
<smokes2345> ah, upstart 0.6.4
<smokes2345> thats what initctl reports anyway
<smokes2345> so session init is out of the question
<smokes2345> â¢jodhâ¢ initctl reports upstart 0.6.4
<smokes2345> so i've tried a couple of variations of this http://superuser.com/questions/213416/running-upstart-jobs-as-unprivileged-users
<smokes2345> but no joy
<JanC> smokes2345: maybe you should not use startx to start X11 ?
<smokes2345> â¢JanCâ¢ yeah, i'm trying xinit now
<JanC> or maybe a simplistic DM
<JanC> something like nodm
<smokes2345> no need for a dm, looks like xinit will work fine
#upstart 2013-09-12
<wjb> I have read that `service reload` sends SIGHUP, causing the daemon to reread its configuration. If the configuration has changed, will the daemon die and be replaced by another process that has read the new configuration?
<hikiko> hi
<hikiko> I am trying to find out where the log_daemon_msg messages go to get the output of a startup script I have in /etc/init.d (the lightdm script) but I cant find it.. any ideas? (thanks)
<jodh> wjb: 'reload' simply sends the main job process the appropriate signal. Use 'stop $job && start $job' to make the job restart with latest *upstart* configuration.
<jodh> hikiko: /var/log/boot.log
<hikiko> jodh, it's not there and ideally I'd like to see the messages each time I restart lightdm
<hikiko> not only at boot time
<wjb> jodh: Thanks. Another question if you a moment: the upstart script I'm working on has a script stanza that contains "exec sudo -u www-data my-server-script". So I see that process runningÂ as root, and I also see a process running my-server-script as www-data. Which of these does upstart see as the main job, and therefore send signals to?
<jodh> hikiko: hold on - on an ubuntu system, lightdm is an upstart job, not a SysV one. Any output from lightdm goes to /var/log/upstart/lightdm.log.
<wjb> jodh: I want the signal to go to the www-data process, not the sudo. But I'm not sure that is happening.
<hikiko> jodh, ** (lightdm:2044): WARNING **: Session pid=2164: Error writing to session: Broken pipe
<hikiko> that's the only type of messages I see there
<hikiko> and also I added an echo "my message" >> /tmp/foo
<hikiko> and there's no /tmp/foo created
<jodh> wjb: if you have upstart >= 1.4 ("initctl version") use the 'setuid' and 'setgid' stanzas rather than sudo. See http://upstart.ubuntu.com/cookbook/#run-a-job-as-a-different-user, http://upstart.ubuntu.com/cookbook/#setuid, http://upstart.ubuntu.com/cookbook/#setgid
<jodh> wjb: as for which pid it tracks, it will track the first by default. See http://upstart.ubuntu.com/cookbook/#expect (worth reading atleast twice! :-)
<wjb> jodh: Thanks! I just read that for the first time a few minutes ago and am still digesting it. :)
#upstart 2013-09-15
<allaire> Hi, when I use upstart to run let say a python script which normallly runs in the foreground, how does upstart put it in background? (the python script is not a daemon)
<allaire> I suppose upstart takes the code in the script and performs something similar to nohup script & ?
<allaire> Finally, if our script does not write a pid file (since it's not a daemon), does upstart drops the pid it tracks somewhere? What's the best way to deal with upstart script that exec a script that does not create a pid file by itself (like a plain python or ruby script)
#upstart 2014-09-09
<collinanderson> what's the default value for the "console" option?
<jodh> collinanderson: For modern versions, 'log'. See first sentence of http://upstart.ubuntu.com/cookbook/#console.
<collinanderson> jodh: excellent. i missed that. thanks
<mustmodify> Can someone translate this for me? start: Rejected send message, 1 matched rules; type="method_call", sender=":1.196" (uid=1000 pid=4298 comm="start puma ") interface="com.ubuntu.Upstart0_6.Job" member="Start" error name="(unset)" requested_reply="0" destination="com.ubuntu.Upstart" (uid=0 pid=1 comm="/sbin/init ")
<mustmodify> I pretty much get that it didn't start... but how to I start to troubleshoot?
<mustmodify> I'm following this: https://gist.github.com/madwork/7680478#file-puma-conf
<mustmodify> and when I go to user production and paste lines 14..21, it works.
<mustmodify> ah, I wasn't using sudo. Can I get it to tell me I'm not using sudo rather than giving me this useless error?
#upstart 2014-09-12
<jgornick> Hey guys, is it possible to setup a job to star as the very last job
<mwalsher> hello
#upstart 2015-09-09
<dlundquist> Can I cause upstart to reexec itself? (I'm trying to pivot_root)
<JanC> dlundquist: sending SIGTERM should restart upstart
<dlundquist> JanC: thank you, that did it
<xMopxShell> Hi, is it possible to make a service depend on a service that may or may not exist?
<xMopxShell> i.e., if it exists, wait for it to start, if not, continue
