#upstart 2007-02-12
<AStorm> Hello.
<AStorm> I heard that you're doing ubuntu init scripts too.
<_ion> Welcome back. :-)
<AStorm> I'm especially interested in the fsck - esp. not parallelising checks on things on the same physical drive.
<AStorm> I have some ideas I've already implemented, but I'd like to hear yours.
<AStorm> fsck itself isn't that good at checking multiple fses
<AStorm> I wonder how does fsck check if the deviced are on the same physical device...
<Md> major number?
<_ion> That doesn't help much if you're e.g. comparing /dev/sda1 to /dev/volumegroup/foo which happens to be on software RAID-1, which happens to reside on /dev/sda2 and /dev/sdb2.
<Keybuk> I didn't think fsck minded?
<_ion> I assume fscking /dev/sda1 and /dev/sda2 simultaneously is *very* slow.
<_ion> I don't see a reason for it not to work, but it's gotta be slow.
<AStorm> Ok guys, did I miss something?
<Keybuk> nope
<AStorm> pfsck uses blkid_get_devname
<AStorm> Which will fail with LVM or RAID.
<AStorm> (fail to spot the use of the same device)
<AStorm> My idea is to do some ls in /sys
<AStorm> The dependency info is all exported in sysfs
<Keybuk> what's the problem with checking the same device?
<AStorm> seeking
<AStorm> Means slowdown.
<_ion> *Huge* slowdown, most likely.
<AStorm> Not all that bad, because most fsck checks are small.
<AStorm> I also wrote a tiny util to detect swap partitions.
<AStorm> (4K on glibc)
<AStorm> It could be useful when doing fsck totally automatically.
<AStorm> (e.g. on HAL event when you hotplug your new shiny USB hard disc)
<AStorm> Though some fsck options could probably fix it for us... if they are correctly implemented, that is
<AStorm> not just checking the fstab
<AStorm> Yes, it's broken. Will default to ext2.
<AStorm> We could really use some FS detector.
<Keybuk> AStorm: vol_id can do that
<AStorm> How?
<Keybuk> (detect swap partitions)
<AStorm> Didn't work for me here...
<AStorm> Ah, now it works. Stupid me.
<AStorm> even the FS type (if known to the kernel)
<AStorm> Ok, then it's no problem at all
<AStorm> the devices will be discerned by sysfs data.
<AStorm> ( reading /sys/block/*/* and /sys/block/*/*/slaves if present
<AStorm> Probably there's some tool for that in sysfsutils too.
<_ion> Has there been a lot of progress with the complex-event-config implementation?
<AStorm> It works here :-)
<Keybuk> _ion: I've written a parser
<Keybuk> just need to rewrite that into the actual code and make it nice
<AStorm> is "when x" isomorphic to "start on started x; stop on stopping x"?
<Keybuk> there's no such thing as "when x" ?
<AStorm> Bah.
<AStorm> I meant "with x"
<Keybuk> "with x" only exists in the complex-event-config
<Keybuk> it's somewhat analogous to "starting x until stopped x", except that there's no serialisation of the two
<AStorm> So it's more lax?
<Keybuk> yes
<AStorm> Nice still.
<AStorm> I'm looking forward to complex-event-config full implementation.
<Keybuk> me too
<solemnwarning> Is upstart more stable then initng?
<Keybuk> http://people.ubuntu.com/~scott/complex-event-config.patch
<Keybuk> \o/
<mbiebl> hehe, cool
<mbiebl> Keybuk: Do you know, if the old sysv init sets the environment variables PREVLEVEL and RUNLEVEL on startup to N and S?
<Keybuk> I have no idea
<Keybuk> there's an open bug that the example job doesn't do that
<Keybuk> I put the code in, and rc seemed to not work properly
<Keybuk> so I'm not convinced
<mbiebl> It's quite strange, the Debian udev scripts checks for the presence of these variables (and also splashy)
<Keybuk> (this might just be a problem with /var/run/utmp, and /var/run not existing, of course)
<mbiebl> But they are not set by /etc/init.d/rc
<mbiebl> So, imho they must be set by sysv init.
<mbiebl> I also find it strange, that bootmisc resets /var/run/utmp (to :) in S
<Keybuk> yeah
<Keybuk> sysvinit sets those itself
<Keybuk> I think the bug is that we track the runlevel using /var/run/utmp as the authorative reference
<mbiebl> Hard to N and S?
<Keybuk> whereas sysvinit has an integer in memory that tracks it, and just uses utmp as an update
<mbiebl> yeah, this is quite funny
<mbiebl> if /var/run/utmp persists during reboot
<mbiebl> You'd get 2 S from runlevel during S
<Keybuk> pretty much the problem I had <g>
<mbiebl> Do you know, if the behaviour of runlevel and PREVLEVEL/RUNLEVEL is specified somewhere?
<Keybuk> I doubt it
<Keybuk> I just used the sysvinit source as a reference for how they should behave
<Keybuk> 0.3.5 is happily somewhat closer than 0.2.7 was
<Keybuk> since "shutdown" now only generates runlevel change events
<mbiebl> btw, I modified /etc/even.d/rcS for Debian to set PREVLEVEL/RUNLEVEL unconditionally to N/S
<mbiebl> This is inconsistent to the output of runlevel at this  stage, but it seems to work fine.
<Keybuk> that's a good idea :)
<mbiebl> Yeah, it's then consistent to runlevel/previous defined in /etc/even.d/rc
<AStorm> Keybuk, as to that init..
<AStorm> Gentoo init depends on the fact that the RUNLEVEL is set to S or 1 on start.
<AStorm> Had to fix that.
<mbiebl> AStorm: What about PREVLEVEL?
<mbiebl> Does the Gentoo init system make any assumptions about the value of this var?
<AStorm> Not at all.
<AStorm> (also, the runlevel names are special, like sysinit, single, nonetwork, default, reboot - not just numbers - but that's Gentoo-specific)
<AStorm> the special sysinit runlevel is active only in S and there's explicit check for that
<mbiebl> What's the output of runlevel during boot?
<AStorm> Hmm?
<mbiebl> Or when you have booted your system completely
<AStorm> Completely.
<AStorm> I'm running on upstart right now.
<AStorm> (but totally using Gentoo scripts)
<AStorm> (except the gettys - that also had to be hacked a bit)
<AStorm> To be exact, the gettys were running too early
<AStorm> making the init process look like crap
<AStorm> Couldn't run them on stopped runlevel ...
<AStorm> Because it didn't stop
<AStorm> So I had to hack the last init script to emit that event.
<AStorm> The always last script is /etc/init.d/local.start
<AStorm> (due to after * clause in it)
<AStorm> s/stopped runlevel/started runlevel/
<Keybuk> runlevel is an event
<Keybuk> not a job
<Keybuk> events don't have a started or stopped :p
<Keybuk> (unless you have an /etc/event.d/runlevel, of course)
<_ion> Yay for complex-event-config.patch
<Keybuk> it's still not "official" yet
<Keybuk> it's only gdb-tested, and doesn't have test cases
<Keybuk> and the code is messy at best
<Keybuk> the implementation has some interesting side-effects too
<Keybuk> e.g.
<Keybuk> (foo and bar) until baz
<Keybuk> foo happens, nothing
<Keybuk> bar happens, it gets started
<Keybuk> baz happens, it gets stopped
<Keybuk> now, if foo or bar happen, it will be started again
<Keybuk> since "foo and bar have happened" is satisfied
<Keybuk> the until only "forgets" the AND combination
<Keybuk> I actually quite like this behaviour, since the other intent can be expressed as:
<Keybuk> foo until baz and bar until baz
<Keybuk> (which requires foo and bar to both happen after baz, before it will be started again)
<_ion> Well, that's exactly what i'd have expected to happen on (foo and bar) until baz
<_ion> :-)
<_ion> Agh, sorry. I misread your line as "if foo and bar happen, it will be started again"
<_ion> Perhaps because there's "foo and bar" directly under the "foo or bar" ;-)
<Keybuk> heh
<Keybuk> after baz happens, it doesn't require foo and bar to both happen again
<Keybuk> just one of them
<AStorm> That's a bug.
<Keybuk> is it though?
<AStorm> It is. If you want "once" modifier, add it.
<AStorm> from once foo and bar until baz
<Keybuk> the "and" operator is TRUE if either expression has ever been TRUE
<_ion> From the computer's point of view, that's the logical behavior. But from a human's point of view, it might be nicer if the code somehow reset the value of the (foo and bar) expression.
<AStorm> hmm, or even
<AStorm> from once foo and once bar until baz
<_ion> ...when baz becomes TRUE
<Keybuk> _ion: right, so the plan was that until would reset the tree on one side to FALSE when the other side went TRUE
<Keybuk> and act as a flip-flop
<AStorm> So, if both are satisfied _once_, they don't get reset
<Keybuk> but that would have interesting interactions later with "with"
<AStorm> But then... hmm.
<AStorm> Weird.
<AStorm> foo and once bar
<AStorm> once foo and bar
<AStorm> But both once degenerates to on.
<AStorm> No, it doesn't :-)
<AStorm> That'd be a nice extension.
<AStorm> on once abc
<AStorm> for a single-fire event
<AStorm> (or even make on optional in this case)
<AStorm> Like it?
<Keybuk> not sure what the behaviour would be
<AStorm> With once, or with those ands?
<AStorm> The and should be like logic and
<AStorm> And once should set a permanent flag
<AStorm> It would be transitive too
<AStorm> once (foo and bar) -> once foo and once bar
<AStorm> similar with or...
<Keybuk> I still don't get what "once" does?
<AStorm> thought about adding xor?
<AStorm> Keybuk, acts when the event is fired, but only once
<Keybuk> so how is "once foo" different from "foo" /
<Keybuk> once foo has been emitted, foo is always TRUE
<AStorm> foo will fire multiple times
<AStorm> Hmm.
<Keybuk> sure, it can affect the tree multiple times
<Keybuk> can you give me a use case for "once" ?
<AStorm> Let me think...
<AStorm> once startup-finished... mail me some info
<AStorm> But then, you could create a one-time event
<Keybuk> why not just make it a one-time event in the first place ? :p
<AStorm> It's much better when you change and
<AStorm> from bar and once foo until baz
<Keybuk> change and?
<AStorm> This means that if foo is fired once
<AStorm> and baz is fired, we get launched
<AStorm> unlike from bar and foo until baz
<Keybuk> I still don't understand why this is useful, sorry
<AStorm> which would mean that if both are fired...
<AStorm> This means "foo got executed and finished, then on baz, do something"
<Keybuk> I think you're confusing events and jobs?
<AStorm> Probably.
<AStorm> Due to JobEvents :P
<Keybuk> job events went away
<AStorm> Good.
<AStorm> Well, then and works correctly it seems.
<AStorm> (except that behaviour is not necessarily intuitive and has to be documented)
<Keybuk> documentation is third on my TODO list :p
<AStorm> :>
<AStorm> Could lend you a hand there. Which format do you prefer?
<Keybuk> docbook, I guess
<AStorm> Could be asciidoc generated docbook?
<Keybuk> what's that?
<AStorm> asciidoc is a plaintext system
<AStorm> http://www.methods.co.nz/asciidoc/
<AlexExtreme> hmm
<AlexExtreme> I could also help with docs
<_ion> asciidoc looks quite nice.
<AlexExtreme> hmm
<AlexExtreme> the complex-event-config patch looks good
<_ion> Ooo, nice. http://www.methods.co.nz/asciidoc/music-filter.html
<_ion> Ooo, nice2. http://www.methods.co.nz/asciidoc/source-highlight-filter.html
<AlexExtreme> Keybuk, how are the new jobs going? or haven't you started work on them?
<Keybuk> slowly, but surely
<AlexExtreme> cool
<Keybuk> should be done today
<_ion> Yay
<AStorm> I have a question: How would one do normal logging to both console and system log?
<AStorm> Use logger + echo?
<Keybuk> or something like bootlogd or logd, etc.
<AStorm> Yes, I was thinking about that logd
<AStorm> How do you use that?
<Keybuk> right now it's disabled
<Keybuk> has a few bugs
<AStorm> Ok, not a problem. For a while, I'll stick with plain echo.
<AStorm> The hardest part is writing the network script
<AlexExtreme> we talked about the possibility of integration with syslog-ng a while back
<AStorm> the gentoo one is so universal, that it's quite slow :>
<AStorm> (also, the network devices aren't parallelised)
<_ion> astorm: It might be wise to start network interfaces based on events emitted by udev, too.
<AStorm> Yep.
<AStorm> Gentoo does that, but it has to wait for udev to settle :P
<AlexExtreme> frugalware's networking script is quite simple, but it's not flexible in terms of bootup. everything is done by an external C program which uses profiles
<AStorm> We don't have to.
<AStorm> AlexExtreme, blah
<AlexExtreme> why blah? :p
<AStorm> I'd rather use a bunch of shell scripts :>
<AStorm> and iproute2
<AlexExtreme> :)
<AStorm> and probably wireless-tools
<Md> AStorm: only because currently daemons are started unconditionally when the interface is up. the correct way would be to start daemons when the interface is up, syslogd is running, /var is mounted, etc
<AStorm> Md, yes, unfortunately.
<AStorm> Gentoo init can be fixed, yes.
<AStorm> But is it worth it? It is somewhat cruddy (e.g. /sbin/rc)
* _ion still recommends just starting /sbin/rc as the compatibility layer, and rewriting things gradually as upstart jobs.
<AStorm> _ion, I'm doing it right now, thank you
<AlexExtreme> well
<AStorm> :>
<AlexExtreme> that would be hard
<AlexExtreme> because
<_ion> The result is probably a lot better than converting the gentoo script to upstart jobs automatically.
<_ion> scripts
<AlexExtreme> hmm wait
<Keybuk> that's harder for Gentoo, because of their intricately interlinked deps
<AStorm> No, it's the same as running the gentoo init.
<AlexExtreme> no, forget i said anything
<AlexExtreme> :)
<Keybuk> they can't just remove one script
<AStorm> Exactly. That's why they have to be wrapped.
<Keybuk> because then all of its dependers wouldn't work
<Keybuk> where as we can
<AStorm> I simply wrap them in a wrapper script
<AStorm> for upstart.
<AStorm> And add a "runlevel" with a list of the deps.
<AStorm> Still haven't got to modifying depscan.sh or rc-update to modify the file
<AlexExtreme> hmm
<AlexExtreme> that usplash in feisty atm is nice
<AStorm> Hmm, what about checking a filesystem just before mounting? :>
<AStorm> (except if a flag is given in the fstab, maybe...)
<AStorm> upstart could be turned into an automounter too...
<AStorm> (maybe not for network filesystems, for these there's autofs)
<AStorm> Except that it wouldn't know who inserted the disc. So here, Unix user rights would have to be used.
<AlexExtreme> hmm
<AlexExtreme> does anyone know if kdump is implemented on feisty yet?
<AStorm> Hmm, does initctl emit block until the event has been totally handled?
<AStorm> and what about initctl start?
<AStorm> ping, anyone?
<int0x0c> AStorm, You have upstart running on Gentoo, right?
<AStorm> int0x0c, in emulation currently
<AStorm> writing the basic init and a proper init script emulation layer will take time
<AStorm> I'm doing that currently
<int0x0c> ahhh
<AStorm> s/currently/right now/
<int0x0c> heh
<int0x0c> Have you seen Keybuk around recently?
<int0x0c> I'm looking to port the init system fully to upstart
<AStorm> [19:23:08]  * Keybuk has quit ("")
<int0x0c> He's supposed to release a full upstart jobset soon
<AStorm> I'm doing that too :-)
<int0x0c> bah
<AStorm> I'd appreciate the init scripts, but then, I'm porting it right now too.
<int0x0c> ahhh, alright
<int0x0c> I'm afraid I don't understand the init system well enough to port the base system scripts without some research
<int0x0c> it would be great to get a full upstart init sequenceon gentoo
* int0x0c hates his dying spacebar
<AStorm> It's actually quite simple.
<AStorm> mount proc, dev, remount root ro
<AStorm> start udev
<AStorm> fsck root
<AStorm> fsck other fses
<AStorm> mount them
<AStorm> that's the end of basic setup.
<AStorm> The last step is starting gettys
<AStorm> That's the basic skeleton. You probably want to add there some daemons, network setup, sound setup, such things.
<AStorm> (forgot about remounting root rw :> )
<AStorm> Also mount /dev/pts and /dev/usb optionally
<AStorm> also, mounting sys
<int0x0c> hmm
<int0x0c> Have you implemented those as pure upstart jobs?
<AStorm> Doing it right now.
<AStorm> Actually, mostly done.
<AStorm> Still working on improving the fsck
<AStorm> Then, the init level/profile has to be launched. E.g. a script starting everything in some directory
<AStorm> and stopping
<AStorm> a runlevel simulation, that is.
<AStorm> (or eINIT's mode, rather)
<AStorm> At least until profiles are implemented.
<AStorm> Well, the actual depscan.sh just launches a bunch of computation using awk
<AStorm> I'd rewrite it in Python myself.
<AStorm> (it's on Gentoo anyway)
<AStorm> The emulation requires complex-event patch
<AStorm> (otherwise it's impossible to link deps together)
<AStorm> It's in awk, because the need system is too hard to implement in bash :>
<int0x0c> Do you suppose you could post your base startup job somewhere?
<_ion> Haha, i just now saw the picture Keybuk has at his launchpad page. :-)
<AStorm> Hey guys...
<AStorm> I wonder if this script is correct (with complex-events patch)
<AStorm> http://wklej.org/id/8aba67fafb
<AStorm> (check-fs does what you expect :-) )
<AStorm> Maybe change the from line to
<AStorm> start on started udev
<AStorm> stop on stopped check-fs FS=/
<AStorm> Hmm, or do I need something to listen on initctl events?
<AStorm> Ah, no, I have to add sleep inf probably
<AStorm> so that it's still running? (even with the fix for jobs w/o execs)
<AStorm> I wonder...
<AStorm> my test-event1 doesn't register as a job
<AStorm> Ah, my bad. Braino
<AStorm> Ok, got it :-)
<AStorm> Is there a way to create a one-fire command that will block until the other event happens, or do I have to use that sleep inf hack?
<AStorm> (which leaves nasty KILL messages around)
<AStorm> mbiebl, oh, maybe you know.
<mbiebl> AStorm: hi, what's up
<AStorm> I wish to run a single-fire command, which should block until an event is fired
<AStorm> sleep inf and using the dependencies leaves KILL messages.
<AStorm> (looks yucky)
<mbiebl> AStorm: could you explain a bit more detailed what you want to achieve
<AStorm> I want to launch another event (which is an instance, so can't just wait on something)
<AStorm> and then, block, until it finished
<mbiebl> "launch an event" ?
<AStorm> Actually, start a job
<AStorm> But emitting an event would do the same
<AStorm> Start command x, which returns
<AStorm> and still wait until that one event
<mbiebl> AStorm: Sorry, I think you lost me. I'd have to see the bigger picture to understand what you want to do.
<AStorm> Launch 3 fsck in parallel
<AStorm> and wait until they complete
<AStorm> each fsck is a job.
<AStorm> (because I would like to hook that up to udev events later)
<mbiebl> Why do you have to wait, until all three of them have been completed?
<_ion> "initctl emit foo" blocks automatically
<AStorm> _ion, yep, but what will receive it? :>
<AStorm> mbiebl, because then the mounting may happen
<AStorm> (automounting that is)
<_ion> /etc/event.d/test:
<_ion> start on foo
<_ion> exec sleep 5
<AStorm> And I also want to use it in system startup sequence
<_ion> % sudo initctl emit foo
<AStorm> _ion, hmm, I have to pass parameters too
<mbiebl> Well, I'd say, mount each partition separately when fsck has completed.
<_ion> You probably should make the fsck job emit "filesystem-checked" and the mount job "start on filesystem-checked" or something like that.
<mbiebl> I see no reason to way for / not being mounted because /usr is still checked.
<mbiebl> s/way/wait/
<AStorm> mbiebl, that's not the problem
<AStorm> the problem is /usr/share mounted while /usr isn't
<AStorm> :-)
<mbiebl> Well, then the /usr/share mount job should "start on fsck-checked /usr"
<AStorm> yeah right :>
<AStorm> And I do know that from udev events :P
<_ion> I'm not sure there should be separate mount jobs for separate partitions.
<AStorm> _ion, there can, not a problem at all
<AStorm> If I serialise mounting after the fsck
<AStorm> _all_ of them
<mbiebl> This is actually quite tricky.
<mbiebl> You can't know in advance the partitioning scheme of a user.
<AStorm> I can emit some events, right.
<AStorm> yes, but can make educated guesses :P
<mbiebl> So providing upstart mount jobs for each separate partition won't be possible.
<mbiebl> You could, as _ion suggest, do the mounting in on single job.
<_ion> Better make the mount intelligent: if MOUNTPOINT is /usr/share and there's a /usr entry in /etc/fstab, block until /usr is mounted.
<_ion> Or something like that.
<AStorm> Exactly.
<AStorm> But not on the fstab check
<AStorm> on mtab
<AStorm> or rather, /proc/mounts
<_ion> No, that doesn't contain /usr until /usr is mounted, right?
<AStorm> fstab will be checked anyway, the mounts will be serialised separately by an external program
<AStorm> I've temporarily solved the problem already, thank you.
<AStorm> (I meant they will wait on each other, the scripts launched with an argument)
<AStorm> Hmm, I could wait on the pids
<_ion> % deps() { while read fstab_fs fstab_mountpoint rest; do case "$MOUNTPOINT" in "$fstab_mountpoint"*) echo "$fstab_mountpoint"; esac; done </etc/fstab }; MOUNTPOINT=/usr/share deps
<_ion> If there's /usr in /etc/fstab, that should print it.
<AStorm> More or less.
<_ion> That's not a real implementation, it would print "/usr/loc" for /usr/local, but it demonstrates the idea.
<AStorm> I know what I have to do...
<AStorm> but the parsing will be done by an external program anyway.
<AStorm> I just wonder, how to emit x events and wait on them all
<AStorm> Well, wrong actually
<_ion> for e in foo bar baz; do initctl emit "$e" & done; wait
<mbiebl> AStorm: The basic question is, when can certain jobs start to run
<AStorm> I can use complex-events and stop on (fsck / and fsck /usr and fsck /var and ... )
<AStorm> But that will leave evil messages
<AStorm> I will launch them using events, thank you.
<AStorm> But what will wait w/o args do?
<AStorm> mbiebl, that will be assured by another program and the deps :-)
<_ion> wait for all children. To wait for only those initctls:
<AStorm> Great then. Should've read the manpage better.
<_ion> pids=; for e in foo bar baz; do initctl emit "$e" & pids="$pids $!"; done; wait $pids
<_ion> (i didn't test that, though)
<AStorm> Not a problem with "all children"
<AStorm> The shell is running only these commands.
<AStorm> Ok, so now the fsck problem is solved (thanks to emitting events)
<AStorm> per-disk fsck is done too properly
#upstart 2007-02-13
<mbiebl> My idea would be to distinguish between to different events: one being the root filesystem is availabe (basically everything besides /usr)
<mbiebl> and the second being that the complete filesystem is mounted.
<mbiebl> A small helper programm should figure out from parsing /etc/fstab which mount points have to be mounted for the first event
<mbiebl> and then runs initctl emit root-fs-mounted
<AStorm> mbiebl, exactly
<mbiebl> and when all filesystems are mounted emit the second event.
<AStorm> Yes, that's the plan and that's what I do.
<AStorm> I mount the systems in parallel too
<AStorm> (some time savings?)
<mbiebl> You'd have to benchmark that, if it is really worth it.
<AStorm> That's why just after fsck is done iff the fs is not a subpartition, it is mounted
<AStorm> the fsck is bound to the mount
<AStorm> and the exact sequence is precalculated externally by a bash script
<AStorm> (could embed it and launch on runtime, possibly)
<AStorm> (but it probably requires something from /usr)
<_ion> deps() { grep -v '^[[:space:] ] *#' /etc/fstab | while read fs mp rest; do case "$MOUNTPOINT" in "${mp%/}"/*) echo "$mp"; esac; done }; MOUNTPOINT=/usr/share deps
<_ion> This should work, i think.
<_ion> (for all other mountpoints than / itself)
<mbiebl> _ion: I'd reverse it: everything besides /usr
<mbiebl> (Maybe /opt should be excluded too)
<_ion> Hmm. That function is supposed to give an answer to "i'm supposed to mount $MOUNTPOINT now; what mountpoints do i need to wait for first?"
<mbiebl> Hm, but you couldn't use that expression in a "start on bla" stanza
<mbiebl> Imo jobs should depend on either the root-fs being mount or everything (which includes network mounts) being mounted.
<_ion> This is what i was thinking:
<_ion> /etc/event.d/fsck:
<AStorm> Good, but actually I don't have to ask that
<AStorm> I know the sequence beforehand.
<AStorm> And yes, they do that (at least here)
<_ion> start on block-device-added blahblah
<_ion> script
<AStorm> They dep upon started mount-filesystems
<_ion>     # do the magic
<AStorm> That's one script, yes.
<AStorm> But then, you don't want to mount it yet
<_ion>     if that was successful: 
<AStorm> You don't know what the user wants :>
<_ion>         initctl emit filesystem-checked "$MOUNTPOINT"
<_ion> end script
<_ion> /etc/event.d/mount:
<AStorm> automounting has to have another table anyway
<_ion> start on filesystem-checked
<_ion> script
<_ion>     deps() { ... }
<_ion>     waitfor="$(deps)"
<_ion>     # wait for stuff if they're not already mounted
<_ion>     mount blahblah "$MOUNTPOINT" blahblah
<_ion>     On success, emit filesystem-mounted -eMOUNTPOINT
<_ion> end script
<AStorm> :>
<AStorm> I actually do something like that, but in a script when setting the sequence.
<AStorm> It fscks them in this order too
<_ion> I don't think there should be a predefined order or sequence.
<_ion> Things should be done based on events, starting from when the devices appear.
<AStorm> The sequence is limited to a single disc.
<AStorm> (so as to not thrash and seek excessively)
<AStorm> Let me see...
<AStorm> Ok, doable, just a slight mod. I'll publish the scripts a bit later.
<AStorm> the thing is I launch "do-fsck thatdisc"
<AStorm> Actually, do-mount thatdisc
<AStorm> and this thing orders the mounts properly
<_ion> To avoid thrashing, you should have something like:
<_ion> /etc/event.d/fsck:
<_ion> ...
<_ion> script
<_ion>     physdev=$(get the physical device name from somewhere)
<AStorm> fsck is just before mounting, so no problem
<AStorm> But then, the mount point ordering.. hmm
<_ion>     watershed -i "fsck-$physdev" fsck "$DEVNAME"
<AStorm> I'd have to separate fsck from mounting. Will do that
<_ion>     On success, initctl emit filesystem-checked -eMOUNTPOINT
<_ion> end script
<AStorm> So, I'll have to wait on all fsck's to finish
<_ion> And that's all you need to do, still no need to define a sequence.
<AStorm> which kind of destroys the event baseness
<AStorm> I don't have no watershed :P
<AStorm> s/don't//
<AStorm> but I have my own script checking the devices.
<AStorm> Ok, BBL
<_ion> wait
<_ion> The watershed source is in the diff.gz at https://launchpad.net/ubuntu/+source/udev/103-0ubuntu12
<AStorm> I don't need it
<AStorm> there's sysfs, uh
<AStorm> and udevinfo
<_ion> Are you going to implement your own locking for the fscks to avoid thrashing?
<AStorm> No, I'll use event dependencies for it.
<AStorm> :>
<_ion> So you're going to have multiple fsck jobs, multiple mount jobs etc?
<AStorm> Exactly. Multiple instances of fsck job.
<_ion> I mean multiple fsck files in /etc/event.d
<AStorm> No.
<AStorm> Parameters work ok :P
<_ion> I'm not sure i understand how you're going to implement the thrashing-avoiding functionality then.
<AStorm> simple
<AStorm> launch "fsck-disc thedisc"
<AStorm> (on block-device-added probably, or maybe by hand by another script)
<AStorm> this will detect (or load from a file) what partitions are on that drive
<_ion> And that script implements the locking, no?
<AStorm> What locking? You don't need it, you scan the partitions in random order :>
<_ion> Well, let's say i run:
<AStorm> because, as you can see, no multiple partitions from the same drive can happen
<AStorm> (even counting dm or raid, because I'm a bit smarter when checking for "same device")
<_ion> Ah, i was indeed thinking that the fsck job would be triggered for each partition, but if it's only triggered for each physical device, it should work. True.
<AStorm> I wonder how to do that vgscan/mdscan thing with udev events
<AStorm> and esp. vgchange
<AStorm> These things will probably generate their own events, or none at all.
<AStorm> Well, could do them in another step altogether, that will waste a bit of time maybe...
<AStorm> Ehh, if I could just listen for some "stopped" events...
<AStorm> and I do mean listen. Not run something on them.
<AStorm> e.g. the physical group is on devices x, y and z
<AStorm> we want to wait until fscking on these devices ends and launch our own
<AStorm> and another is on devices a and b
<AStorm> wait on these too before fscking, but independent from x, y or z
<_ion> Personally i'd make something emit an event for each *partition* (also LVM partitions etc.) and make the fsck job start on that, figure out the physical device and implement locking based on that.
<AStorm> Hack, hack I say!
<AStorm> :>
<AStorm> The locking should be based on an event happening
<AStorm> (e.g. stopped fsck-disc somedrive)
<AStorm> But you can't just wait on an event in a script
<AStorm> initctl should be extended to support that
<AStorm> e.g. initctl wait stopped xyzzy
<_ion> I fail to see where the hack is.
<AStorm> I'd have to sleep on a file or something
<AStorm> or still sleep waiting for an event
<_ion> In order to do what?
<AStorm> How would the other partition jobs know that I'm not scanning the device?
<AStorm> Tell them on pipe? :P
<AStorm> also, race condition galore
<_ion> One instance: watershed -i fsck-physdev-sda fsck /dev/sda1
<_ion> Another instance: watershed -i fsck-physdev-sda fsck /dev/volumegroup/foo
<AStorm> Yes, and?
<AStorm> and /dev/vg/x is on sda1
<AStorm> :>
<_ion> Third instance: watershed -i fsck-physdev-sdb fsck /dev/sdb1
<_ion> The first thing and the third thing are run simultaneously and the second one will block until the first is done.
<AStorm> How would the different watershed instances communicate?
<AStorm> Or are they smarter than that, somehow detecting that the device is in use?
<AStorm> or do they poll?
<_ion> They use /var/lib/watershed as a state directory, there are state files based on the ID (fsck-physdev-sda for example).
<AStorm> Or create a lock? (race condition possible...)
<_ion> I haven't looked at the implementation.
<AStorm> Aha, locks.
<AStorm> the process returns when it created the lock, right?
<_ion> I don't see how there could be a race condition.
<_ion> That's the point of locks: to avoid race conditions. :-)
<AStorm> I'd have to wait on it to create the lock, can't run x of them in parallel
<AStorm> Yes, but two things at the same time can try to create the lock :>
<AStorm> Both will fail, or just the one?
<AStorm> (that's fcntl for ya)
<_ion> One will succeed, one will block
<_ion> The point of watershed is that you run x of them in parallel and they handle the locking under the surface.
<AStorm> Good. Unless the kernel is broken, you say?
<AStorm> I can do such locking myself too :P
<AStorm> man flock
<_ion> Well, the kernel could be broken and poop turnips, but that's a kernel bug and not something one should work around in a totally unrelated program.
<AStorm> or course, this requires write access to some dir
<_ion> Naturally you could implement locking yourself, i just mentioned an already available tool.
<AStorm> preferably not /var, as that could get mounted before we remove the lock
<AStorm> some special state directory would be best
<_ion> watershed allows you to change the default with a command line parameter.
<_ion> For non-root users, it's ~/.watershed
<AStorm> Yes yes, that's nice, but the default should be good too.
<AStorm> I mean the root's default.
<AStorm> But then, why watershed when I can go with flock? :>
<_ion> /var/run/watershed is exactly the right default. :-) If it needs to be used before /var is mounted, it's a pretty special use case.
<AStorm> (and a bit of sysfs scanning)
<_ion> Whoops, /var/lib/watershed
<_ion> Argh, brainfart. It's /var/run/watershed of course.
<AStorm> The most convenient way would be to place a symlink on / device
<AStorm> Hmm. flock on a file will it be, then
<AStorm> Though I'd prefer waiting on an event.
<AStorm> (e.g. initctl wait)
<_ion> After actually looking at watershed's source (as it has no man pages ;-) ), that's not its purpose after all.
<_ion> Then there's dotlockfile(1)
<Bourne> hi... how do I turn off scripts using upstart, just like the way Sysvinit command used to do?
<_ion> What do you mean by that?
<_ion> 'initctl stop jobname' stops an upstart job
<_ion> ('stop jobname' works as well)(
<Bourne> you know
<Bourne> in ubuntu breezy
<Bourne> you wanted to disable the scripts
<Bourne> just issued sysvinit
<cables> Breezy uses init, not Upstart
<cables> Edgy uses upstart
<_ion> cables: Yes and no.
<Bourne> and a in that window of sysinit, you just disable that
<cables> _ion, how's that?
<Bourne> how to do it with upstart
<Bourne> coz this is way slow
<Bourne> this is like XP... loading 89 default services
<cables> Bourne, http://en.wikipedia.org/wiki/Upstart
<_ion> Feisty will be the first Ubuntu release that actually contains upstart jobs (other than basic tty ones). And even that hasn't been implemented yet. In any Ubuntu release you're currently using, pretty much everything is still started by sysvrc scripts.
<cables> Bourne,  read it. It's backwards compatible with sysvinit, so it will run your init scripts.
<cables> Bourne, _ion, I believe that Breezy actually just has the regular init, though...
<cables> Bourne, sorry, misunderstood your previous message to mean that you use breezy currently.
<Bourne> no
<Bourne> i used to
<Bourne> i just got back into the linux thing
<Bourne> and noticed that a lot of the thigns aren't there anymore
<cables> Bourne, Upstart is backwards compatible with sysvinit, you can configure it the same way...
<Bourne> switching to xubuntu desktop now
<Bourne> GNOME got way toooo slow
<Bourne> even on this fast computer
<Bourne> cables: i can't find a command here.... 
<Bourne> cables: to call upstart and disable services... is there one?
<Bourne> cables: i can't find a command here.... 
<Bourne> cables: to call upstart and disable services... is there one?
<_ion> You don't have services started by upstart, other then the sysvrc scripts and some ttys.
<Bourne> how do I disable services...
<Bourne> that is all i want to know
<Bourne> just a command...
<cables> _ion, Isn't upstart just basically emulating sysvinit at this point?
<_ion> bourne: You're using xubuntu, right? Applications  System  Services
<_ion> cables: Yes.
<Bourne> ubuntu. yeah, i tried that already and disabled services.... 
<Bourne> but I feel there's a whole bunch in the background
<cables> Bourne, if it's still running slow, then that's clearly not the solution...
<Bourne> i mean... 
<Bourne> the Teardown stuff in ubuntu page
<Bourne> talks about 30 unusable scripts
<Bourne> their not listed in the Applications System Services
<Bourne> http://wiki.ubuntu.com/Teardown
<Bourne> well
<Bourne> I can replace upstart then
<Bourne> thats the only choice left
<_ion> Wtf?
<Bourne> as if I mark sysvinit to install, ubuntu takes out upstart in the software list
<Bourne> it does... if you go there and select sysvinit, it marks, but it reds out upstart, saying it's going to be removed
<cables> Bourne, you don't WANT to install sysvinit.
<_ion> Why the frell would you want to do that?
<Bourne> cables: what's the reason for that?
<Bourne> coz just the thought of upstart be loading a PCMCIA or Server stuff annoys me
<cables> Bourne, Upstart is just running sysvinit scripts now.
<cables> Bourne, so you just need to change those with a tool like "sysv-rc-conf"
<Bourne> oh yeah
<Bourne> tha command still works?
<cables> Upstart isn't actually being used as anything more than sysvinit in Edgy Eft
<cables> Bourne, if you install it... "sudo aptitude install sysv-rc-conf"
<_ion> Programmers: http://johan.kiviniemi.name/blag/2007/02/13/structure-and-interpretation-of-computer-programs/
<AStorm> _ion, great, thanks!
<AStorm> Smell of LISP... double plus then for functional programming.
<cables> Wimpog, have you asked your question?
<Wimpog> preparing it now
<Wimpog> GUYS, I need some help. I just installed Ubuntu server Edgy. AND THERE IS NOT /etc/inittab. GUYS from #ubuntu said it is by default like that.  I want to know if there is some kind of equivalent of inittab in Edgy and if yes, where is it and what is it called?
<_ion> /etc/event.d
<Wimpog> "/etc/event.d" is a directory
<_ion> Indeed.
<Wimpog> I'm looking for some kind of equivalent of inittab
<_ion> Please elaborate on what you're trying to achieve.
<Wimpog> say I want to change the boot level
<Wimpog> normally you just edit /etc/inittab
<_ion> Ok, the compatibility jobs in /etc/event.d currently do parse /etc/inittab to find out the default runlevel. Create /etc/inittab and add the line id:N:initdefault: where N is the default runlevel you want.
<AStorm> Or, modify the default job instead :-)
<AStorm> hmm... what about adding a pipe to upstart?
<_ion> That too, but then you'd need to change the number in four places, i think. :-)
<AStorm> I like those Plan9-ish programs that only require echo :>
<AStorm> But then, initctl is adequate too.
<AStorm> There could be some proc-alike filesystem for the jobs :P
<AStorm> Maybe I'll write something with FUSE. Later.
<_ion> Hi Keybuk
<Keybuk> heyhey
<_ion> I moved the branch to my own server, since i keep having strange problems with launchpad.net (connections stalling until timeout), https://code.launchpad.net/~ion/+branch/libnih/watch-delayed
<_ion> Also: https://code.launchpad.net/~ion/+branch/upstart/watch-delayed
<Keybuk> oh, cool, launchpad lets you "move" branches now?
<_ion> Oh, i'm not saying the fault is with launchpad. I've had such problems with many overseas servers.
<_ion> Well, i renamed the old one to watch-delayed-old and then created a new one that mirrors the branch at my server. :-)
<Keybuk> ahh
<Keybuk> ok
<_ion> Has there already been work to switch things from sysvrc to upstart 0.3.5 jobs?
<_ion> For feisty, that is.
<Keybuk> yeah, need to start publishing that soon
<Keybuk> I'm not sure it'll actually make it for feisty as default now
<Keybuk> keep finding nasty bugs
<_ion> Cool
<Keybuk> so I'm gonna just publish it and let everyone hack on it
<Keybuk> what do you think is the best way to publish?  bzr archive?
<_ion> Yeah, i'd like to test it, perhaps find bugs to fix. Yeah, bzr sounds good.
<_ion> Btw, is upstart going to go to the initramfs for feisty?
<Keybuk> no
<mbiebl> Keybuk: I guess you already noticed and I tell you nothing new:
<mbiebl> The sulogin job file in feisty doesn't parse
<Keybuk> oh?  I didn't notice that
<mbiebl> It still uses start instead pre-start
<Keybuk> d'oh
<mbiebl> Do I need the complex-event-spec patch for the [^x]  syntax?
<Keybuk> no
<Keybuk> that's just using plain upstart
<Keybuk> arguments to events are compared with fnmatch
<mbiebl> ah, ok
<Keybuk> https://code.launchpad.net/~keybuk/+branch/upstart/replacement-initscripts
<_ion> \o/
<Keybuk> it's nowhere near complete, yet
<ben_> Thanks Keybuk!
<Keybuk> been trying to think how to do the "mount and unmount block device" type things
<Keybuk> either they need to be two jobs
<Keybuk> but there's also a certain elegance to them just being one job
<Keybuk> was thinking about adding a kind of syntax:
<Keybuk>   instance
<Keybuk>   start on block-device-added sd*
<Keybuk>   stop on block-device-removed $1
<Keybuk> ie. if the argument to "stop on" is $[0-9] * then it matches the equivalent argument in whatever event started it
<_ion> That would be nice.
<_ion> Will the block-device-added event have 'sda' or 'sda1' as the parameter?
<Keybuk> depends which got added :p
<Keybuk> it'd have sda for the block device, and then sda1 if there was a partition
<_ion> All right. Good.
<_ion> I was thinking something like this, is this sane at all? 0) The fsck job starts on block-device-added and emits filesystem-checked MOUNTPOINT=/foo/bar or whatever. 1) The mount job starts on filesystem-checked, and calculates a list of mountpoints it needs mounted *before* $MOUNTPOINT can be mounted (e.g. /foo/bar needs /foo if such exists in fstab). Then it blocks until the dependencies are mounted, and finally mounts $MOUNTPOINT.
<_ion> deps() { grep -v '^[[:space:] ] *#' /etc/fstab | while read fs mp rest; do case "$MOUNTPOINT" in "$mp"/*) echo "$mp"; esac; done }; MOUNTPOINT=/foo/bar deps
<Keybuk> never grep fstab!
<Keybuk> use getmntent()
<_ion> That was just a quick hack to demonstrate the concept. :-)
<Keybuk> I hadn't considered that; (that /usr has to be mounted before /usr/share)
<Keybuk> it can't really block, because that's racy
<Keybuk> what if /usr gets mounted between it checking for it, and it blocking?
<_ion> Perhaps have something like:
<_ion> initctl wait filesystem-mounted /usr
<Keybuk> racy
<_ion> True. :-\
<Keybuk> what happens if the filesystem is mounted while forking and execing initctl?
<Keybuk> a better way might be to have the "mount from fstab" job run whenever a block device is added *or* another path is mounted
<Keybuk> it'd check whether all necessary paths are mounted first
<Keybuk> if they're not, it'd exit
<_ion> Ah, that sounds good.
<Keybuk> if they are, it'd mount it -- and if it gets "already mounted", fail silently
<_ion> Hm, perhaps i should setup a virtual machine to play with replacement-initscripts in.
<Keybuk> of course, that precludes using any $1 trick, since that'd be the path that was mounted above it
<_ion> Is there something like vmware-player, but free?
<Keybuk> qemu?
<mbiebl> _ion: If you want a nice gui you can also use virtualbox
<mbiebl> http://www.virtualbox.org/
<mbiebl> They even provide packages for Dapper/Edgy
<_ion> Thanks
<mbiebl> I use it myself. It's not as feature rich yet as vmware (it lacks the enterprise features)
<mbiebl> but works great so far on my machine.
* Keybuk just uses vmware
<mbiebl> Do you have a license?
<mbiebl> Or do you run vmware player
<Keybuk> licence
<AlexExtreme> ooh, nice
<AlexExtreme> i'll try to play around with the new replacement scripts on frugalware today :)
<Keybuk> AlexExtreme: note that they're only about quarter done; mounting of disks onwards is still todo
* Keybuk has no particular problem with paying for useful software
<AlexExtreme> yes
<Keybuk> obviously I'd rather have the source, but for vmware we have access to the developers, so that's good too <g>
* AlexExtreme uses vmware
<AlexExtreme> ... the trial version :P
<AlexExtreme> I should probably switch from WS to Server sometime
<Keybuk> I just use WS
<AlexExtreme> I gotta admit that this WS 6 Beta is nice
<Keybuk> not seen 6
<AlexExtreme> I've got a feisty install to play around with :)
<AlexExtreme> hmm
<AlexExtreme> Keybuk, i noticed you use mountpoint in the mount-kernel-filesystems job, isn't that a sysvinit util?
<Keybuk> yes
<AlexExtreme> hmm, shouldn't that be replaced somehow? or were you planning on doing that anyway?
<Keybuk> why replace it?
<Keybuk> the replacement code would be identical, no?
<AlexExtreme> well, i don't mean rewrite it, just add a util that does the same to upstart, as it would be nice to get rid of the sysvutils (or whatever it's called) package sometime
<Keybuk> why?
<Keybuk> the tools in that package are perfectly fine they way they are, no?
<AlexExtreme> hmm
<AlexExtreme> that's a point
<AlexExtreme> forget i said anything
* Keybuk doesn't see a problem with upstart-compat-sysv depending on sysvutils
<Keybuk> :p
<AlexExtreme> now
<Keybuk> (mountpoint is in initscripts, so obviously that one will have to move somewhere)
<AlexExtreme> wait, damn
<AlexExtreme> there goes my plans for working on upstart jobs
<Keybuk> hmm?
<AlexExtreme> damn school work :)
<Keybuk> heh
<AlexExtreme> oh well, bbl
<Keybuk> mine today is "damn management stuff"
<Keybuk> :(
<Keybuk> http://codebrowse.launchpad.net/~keybuk/upstart/main/changes
<Keybuk> sweet
* ..[topic/#upstart:Keybuk] : Upstart 0.3.5 | http://upstart.ubuntu.com/ | http://upstart.ubuntu.com/wiki/ | http://upstart.ubuntu.com/doc/getting-started.html | http://codebrowse.launchpad.net/~keybuk/upstart/main/changes | irc logs: http://people.ubuntu.com/~fabbione/irclogs | http://upstart.ubuntu.com/wiki/UpstartOnGentoo
<mbiebl> sweet, inded
<Md> Keybuk: looks like the "description" and "files modified" sections should be merged
<Keybuk> heh
<Keybuk> only because I'm dilligent at changelogs, mostly
<mbiebl> Keybuk: If I use the emit statement
<mbiebl> How do I distinguish if I have two events or one event with a parameter
<mbiebl> e.g. "emits foo bar"
<Keybuk> emits just takes event names
<Keybuk> it's just a documentation stanza, really
<Keybuk> so other things can output dot for the relationships, etc.
<Keybuk> also thinking of using it for "jobs that get told who's waiting for events they emit" kind of thing
<mbiebl> And these events are triggered after entering the "started" state?
<Keybuk> no
<Keybuk> it's just documentation of what events that job is likely to emit on its own
<Keybuk> e.g.
<Keybuk> /etc/event.d/udev:
<Keybuk>   emits block-device-added block-device-removed
<Keybuk> just says "udev might emit these events on its own"
<mbiebl> ah, ok.
<Keybuk> so one assumes that there's a udev rules.d file that does RUN+="/sbin/initctl emit block-device-added" in that example
<Keybuk> it lets future UIs generate pretty graphs :p
<mbiebl> What about "on".
<mbiebl> I think it's ambiguous.
<mbiebl> We have the "start on event" syntax, and simply the "on event"
<Keybuk> heh
<Keybuk> depends whether you're using complex-event-config.patch or not :p
<mbiebl> Really? "start on" also works with 0.3.5
<Keybuk> right
<Keybuk> "start on" and "stop on" are definitely staying
<Keybuk> they take the name and arguments of an event that causes the job to be immediately started or stopped
<Keybuk> multiple copies of those act as a kind of "or"
<mbiebl> Yeah, my point is, we should get rid of the "on event" syntax.
<Keybuk> well
<Keybuk> so complex-event-config defines a new stanza that permits a much richer language to define the events
<Keybuk> FOO started apache and filesystem-mounted until filesystem-unmounted and network-up until network-down
<Keybuk> the "FOO" is not really pinned down yet
<Keybuk> the experimental patch/branch steals "on" for that
<Keybuk> with the intent that we'd rename it later
<Keybuk> so on is already in the process of going away
<Keybuk> it does happen that "FOO event" is exactly equivalent to "stop on event"
<Keybuk> and that will always be the case
<Keybuk> sadly
<mbiebl> Hm, I'll have to reread the complex-event-config spec again, I guess.
<mbiebl> It's a bit the same with exec and respawn
<mbiebl> I'd prefer if respawn would simply be a flag
<Keybuk> that's just syntactic sugar
<mbiebl> (with an optional rate parameter)
<Keybuk> respawn limit 5
<Keybuk> respawn interval 10
<Keybuk> uh
<Keybuk> sorry
<Keybuk> respawn limit 5 10
<Keybuk> <g>
<mbiebl> Well, the thing is, it makes it harder to generate nice GUIs.
<Keybuk> yeah
<Keybuk> respawn CMD nearly got dropped over the weekend
<Keybuk> but it got a last-minute temporary parole
<Keybuk> I'm edging closer to removing it though, since neither "service" or "daemon" accept commands
<Keybuk> and service ~= respawn
<AStorm> Can't use dotlockfile
<AStorm> it's in mutt package, won't dep on it :P
<AStorm> flock it is.
<AStorm> (and checking /proc/mounts after taking the lock, in case we've waited and somebody mounted a filesystem earlier as a dependency)
<AStorm> I'd still prefer listening on events.
<AStorm> (the locking stuff is mighty complex)
#upstart 2007-02-15
<haylo> hi
<_ion> keybuk: Would something like this be correct for replacement-initscripts/etc/udev/rules.d/85-upstart.rules?
<_ion> SUBSYSTEM=="block", ACTION=="add", RUN+="initctl emit block-device-added %k"
<_ion> SUBSYSTEM=="net", ACTION=="add", RUN+="initctl emit network-device-added $env{INTERFACE}"
<_ion> SUBSYSTEM=="block", ACTION=="remove", RUN+="initctl emit block-device-removed %k"
<_ion> SUBSYSTEM=="net", ACTION=="remove", RUN+="initctl emit network-device-removed $env{INTERFACE}"
<Keybuk> with "/sbin/initctl", yes
<Keybuk> though I'm also trying to work out how to get the fstab in there
<Md> and if you use $env{INTERFACE} maybe it's a good idea to check if it is defined
<Keybuk> %k would suffice instead of env{INTERFACE}
<Md> also, you may want to react on block online/offline events too
<Md> good point
<Keybuk> Md: I thought online/offline went away, and got replaced by change
<haylo> ok, guys, i am sorry but I have to ask how to get to the code of the replacement-initscript. I am trying for over half an hour now
<Md> Keybuk: maybe, anyway one of them may need to be handled
<haylo> I only get some launchpad pages and baz errors
<Keybuk> haylo: what did you try?
<haylo> Keybuk: i searched on the launchpad site linked in the announcement mail,
<haylo> Keybuk: i tried the codebrowser instead of code variant
<Keybuk> you found https://code.launchpad.net/~keybuk/+branch/upstart/replacement-initscripts ?
<haylo> Keybuk: i tried the url on the launchpad site with baz get
<Keybuk> baz?
<_ion> You need to use bzr
<Keybuk> right, wrong tool :p
<Keybuk> bzr branch http://bazaar.launchpad.net/~keybuk/upstart/replacement-initscripts
<Keybuk> would do the trick
<haylo> i just installed 'bazaar'
<haylo> not right?
<Keybuk> install 'bzr'
<haylo> doh
<haylo> why are there two VCS' with the same name?
<haylo> that's just mean :)
<Keybuk> silly, isn't it
<Keybuk> baz was an attempt to change GNU Arch into "Bazaar"
<Keybuk> it was abandoned, and bzr was written from scratch instead
<haylo> which shell does upstart use to execute the script portions of the jobs?
<haylo> is it always /bin/bash or /bin/sh (which could mean it getting mangled to dash at least)
<haylo> ah ok
<_ion> /bin/sh -e, i think
<_ion> It uses -e by default, right?
<haylo> oh
<haylo> so ${KVER%{KVER#*.*[^.] }} is no bashism
<haylo> o k
<Keybuk> yeah /bin/sh
<Keybuk> that's ordinary POSIX :p
<haylo> hm
<haylo> doesn't work though :)
<haylo> hey, this is fun!
<Keybuk> there's a typo in it
<Keybuk> should be ${KVER%${KVER#*.*[^.] }}
<haylo> KMAJ should contain, what? 2.6?
<haylo> ok
<_ion> keybuk: Btw, funny thing, one of the upstart test cases failed, but then i wasn't able to repeat the failure anymore: http://soijabanaani.net/tmp/upstart_util_test_events_failure
<_ion> keybuk: It might be relevant that i was using libnih.watch_delayed and upstart.watch_delayed, but seems like the test case didn't really have anything to do with my modifications.
<_ion> Hi Artanicus
<Artanicus> hello
<Keybuk> _ion: hmm, odd
<_ion> After the failure, i ran 'while util/test_events; do true; done' for quite a while without a new failure.
<Keybuk> _ion: why are there three "Testing emit_action()" lines?
<_ion> Uh. I have no idea.
<_ion> Unless the xfce terminal has a bug when copying a chunk of text while scrolling down...
<Keybuk> in fact, they're all duplicated a lot
<_ion> I tried printing numbers 1..1000 and copying and pasting them, didn't stumble upon such a bug.
<Keybuk> weird then
<Keybuk> I can't reproduce that one
<Keybuk> looking at the test case, it suggests that stdout went away
<Keybuk> maybe you ran out of fds temporarily?
<_ion> That would be quite strange, too, i think.
<AlexExtreme> hmm
<AlexExtreme> feisty is *really* nice
<Keybuk> yeah, it's getting there
<AlexExtreme> although
<AlexExtreme> just got a crash in synaptic
<AlexExtreme> and apport is really good, too :p
#upstart 2007-02-16
<_ion> Sigh, doing anything in a virtual machine is simply too slow on this computer.
<_ion> It probably takes a minute just to finish the initramfs script. :-)
<_ion> I guess i'll have to hack on replacement-initscripts on this computer directly.
<Keybuk> heh
* Keybuk just hacks directly
<_ion> (offtopic: http://johan.kiviniemi.name/blag/2007/02/16/monitorcalc/)
<Keybuk> you really do like ruby, don't you? :p
<_ion> It's not perfect, but i haven't found any better ones yet. :-)
<Keybuk> what for though?
<Keybuk> a language can never be perfect for everything
<Keybuk> only perfect for something
<Keybuk> what is ruby's task?
<AStorm> Theoretically, network programming and Perl's place
<AStorm> Ruby = better Perl
<Keybuk> so ruby has regular expressions as a first class object?
<AStorm> As a language construct, you meant? Yes.
<AStorm> And an object too.
<AStorm> but then, it isn't all that useful :P
<Keybuk> what does a ruby program that returns all lines from a file matching a regular expression look like?
<AStorm> Beats me. I don't program in Ruby.
<Keybuk> it would surprise me if it's as good at parsing as Perl is
<AStorm> Why not?
<Keybuk> since Perl does that so well, at the expense of other langauge niceties
<AStorm> Anything is as good at parsing as perl.
<Keybuk> eh?
<AStorm> Though the syntax might be bulkier a bit.
<Keybuk> the syntax is everything
<Keybuk> and the speed at which it can do it
<AStorm> Nothing at all.
<AStorm> Speed, yes, that matters.
<Keybuk> you could write a program to extract data from files and generate a report in C...
<Keybuk> ...you just wouldn't
<Keybuk> you would always use Perl
<AStorm> No, because it doesn't have any good regexp library.
<AStorm> I would use Python personally.
<Keybuk> of course it does
<Keybuk> Python isn't good at this kind of thing
<AStorm> pcre you mean
<AStorm> It is _very_ good
<Keybuk> no, it's quite bad
<AStorm> if you don't mind re.match("WHATEVER")
<AStorm> Instead of ~=
<Md> which tends to suck, yes
<AStorm> It has splitting and other things too.
<Keybuk> while (<>) {
<Keybuk>         print $1 if /[a-z] ([0-9] +)/gi;
<Keybuk> }
<Keybuk> -- 
<Keybuk> what's the Python equivalent of that?
<AStorm> for line in stdin: ...
<Keybuk> no,
<Keybuk> not stdin
<AStorm> for line in stdin: x = re.match("[a-z] ([0-9+)"); if x: print x.groups(1)
<Keybuk> BZZT
<AStorm> What is that <> thingy?
<AStorm> Some variable?
<Keybuk> doesn't parse filenames given on the command line in preference to stdin
<Keybuk> doesn't print each number on the line
<Keybuk> and you've tried to bunch it up into one line, which is very bad Python
<AStorm> Yep
<AStorm> It's IRCy Python
<AStorm> :P
<AStorm> It would probably take 5 lines to do exact the same thing as in Perl.
<AStorm> Which I don't know.
<Keybuk> I've reached 30 lines so far ...
<AStorm> :P
<AStorm> What does that thing means actually?
<AStorm> *mean
<Keybuk> whatever you want it to mean
<AStorm> That <>
<Keybuk> the point is that Perl is *VERY* good at extracting information out of files and generating data sets from it
<Keybuk> extraordinarly good
<AStorm> Well, yep.
<Keybuk> <> is "each line for each file listed on the command line, or stdin"
<AStorm> Aha.
<AStorm> That'll be 7 lines of Python
<AStorm> :P
<AStorm> One for more
<Keybuk> no, it's still not 7 lines of python
<AStorm> and two imports
<AStorm> Ok, 8, miscalculated.
<AStorm> :P
<Keybuk> two imports?
<AStorm> Yep.
<Keybuk> oh, right, re
<AStorm> Heh, one less even.
<AStorm> http://docs.python.org/lib/module-fileinput.html
<AStorm> :P
<AStorm> That's what Perl does with that <>, isn't it?
<Keybuk> yes
<Keybuk> you still can't get anything legible
<AStorm> You can do even better with Python 2.5
<AStorm> print x.groups(1) if x
<AStorm> s/if/when/
<AStorm> print x.groups(1) when x
<AStorm> or even.. in older Python...
<Keybuk> still not x.groups(1) I'm afraid
<AStorm> It is.
<Keybuk> no
<Keybuk> Perl will print all matches in the line
<Keybuk> because I used /g
<AStorm> x is an instance of a matchobjects.
<AStorm> Aha, one missing flag.
<Keybuk> (python doesn't have this flag <g>)
<AStorm> re.GLOBAL
<Keybuk> AttributeError: 'module' object has no attribute 'GLOBAL'
<AStorm> Oh yes it does :P
<AStorm> re.findall
<Keybuk> no
<AStorm> re.finditer
<Keybuk> right, that's not a flag
<Keybuk> import re
<Keybuk> import fileinput
<Keybuk> RE = re.compile(r'[a-z] ([0-9] +)', re.IGNORECASE)
<Keybuk> if __name__ == "__main__":
<Keybuk>         for line in fileinput.input()
<Keybuk>                 for number in line.findall():
<Keybuk>                         print number
<Keybuk> is the equivalent Python
<AStorm> Yep.
<Keybuk> which is infintely less useful than the Perl
<Keybuk> ergo. at parsing, Perl wins
<AStorm> But you don't have to compile :P
<Keybuk> you do bloody well have to compile
<Keybuk> this file is 2GB!
<AStorm> No, you don't.
<Keybuk> compiling that regexp every time will run slow as a dog
<AStorm> 100 latest matches are buffered.
<AStorm> NO.
<Keybuk> actually
<Keybuk> heh
<AStorm> Python won't compile it every time.
<Keybuk> Python can't parse that file
<Keybuk> it ENOMEM'd :p
<Keybuk> Perl took under a second
<Keybuk> and used only a few 100K
<AStorm> Because you used findall
<AStorm> and not finditer
<AStorm> :P
<Keybuk> ok
<Keybuk> finditer still running
<wasabi> Heh. And here I am, a C# user.
<wasabi> The bane of all of you!
<Keybuk> finditer still running
* Keybuk kills it
<Keybuk> Perl wins
<Keybuk> I'm not saying Python is bad, Python is very good at certain things
<Keybuk> it just does emphatically not beat Perl at being a parsing/extraction language
<AStorm> Why? My 2GB file took actually... like 10s
<AStorm> Probably not :P
<Keybuk> 10s is well over 10 times longer than Perl
<AStorm> It's a general purpose language, hell.
<Keybuk> there's no such thing
<Keybuk> no language is ever general purpose
<AStorm> Eh...
<Keybuk> every single one ends up specialising in being able to do one thing well
<Keybuk> whether deliberately or not
<AStorm> It's a language with no special purpose in mind
<AStorm> other than being easily embedded
<Keybuk> easily embedded => Lua
<Keybuk> so Python loses there
<AStorm> Yep.
<AStorm> actually, I know why it did regexp so slow
<Keybuk> one of my major, deep-seated, proto-religious hates is people who use one language for every single task
<AStorm> Object lookups :P
<Keybuk> especially if that language is PHP :p
<Keybuk> but it applies to any
<AStorm> PHP is evil, yep.
<AStorm> Especially due to unreadability :P
<Keybuk> you do not write applications with user interfaces in Perl
<Keybuk> you do not write data parsing and report generation apps in Python
<AStorm> Oh, right, Python isn't too good at threading too.
<AlexExtreme> well, PHP is kinda sucky. especially at GUIs
<AStorm> Well, I do. But then, anyway :P
<AStorm> But these need a bit of special care in tight loops.
<AlexExtreme> would you beleive that the package manager in KateOS (a linux distro) is written in PHP
<AlexExtreme> with PHP-GTK2
<AStorm> Esp. putting object lookups outside of them.
<AStorm> OMFG!
<Keybuk> you don't write *anything* in Java, C++ or PHP
<AStorm> I'd write it in Haskell, blah :P
<AStorm> C++ is good if you have a requirement on it.
<Keybuk> AStorm: haskell is good for mathmetical or functional data sets
<AlexExtreme> I hate C++ :)
<AStorm> Or have to write low-level code.
<Keybuk> C++ is never good
<Keybuk> C++ is useless for low-level code
<Keybuk> and terrible for object oriented code
<AStorm> No, it isn't :P
<AlexExtreme> Java... ugh, bloat.
<Keybuk> low-level => C
<Keybuk> object-oriented => C# or Python
<AStorm> C++ is good enough for low-level semi-OO code.
<Keybuk> I bet you any low-level C++ code you find is really C mangled by a C++ compiler
<AStorm> Thought C can do still.
<AStorm> *though
<AStorm> Keybuk, uh... tell that to guys liking templates :P
<Keybuk> why the hell would you need templates for low-level code?! :p
<AStorm> C# requires mono runtime, which is ugh.
<Keybuk> one ioport looks much like another
<Keybuk> C++ requires C++ runtime, which is ugh ugh ugh
<AStorm> That's why :P
<AStorm> No, it doesn't.
<AlexExtreme> yes it does
<AStorm> It's statically compilable.
<AlexExtreme> libstdc++
<AStorm> ^
<AlexExtreme> well
<AlexExtreme> it is
<AlexExtreme> but
<Keybuk> you can build the mono runtime into the binary
<AlexExtreme> that makes your application unnecessarily massive
<AStorm> Keybuk, yep? Tell me about it.
<AStorm> AlexExtreme, not that much.
<AStorm> Linkers are quite smart nowadays.
<wasabi> mono is statically compilable too heh
<wasabi> apples to apples
<AStorm> wasabi, yes, it is
<AStorm> But it's a large runtime :P
<Keybuk> Linkers are dump
<Keybuk> uh, dumb
<wasabi> it's not that large...
<AStorm> Not GNU ld :P
<Keybuk> AStorm: have you *looked* at the size of libstdc++ ?! :p
<AStorm> Yep. 3MB
<Keybuk> AStorm: GNU ld is dump; it does exactly what you tell it
<Keybuk> AStorm: and the size of mono? :p
<Keybuk> (tip: it's about the same size)
<AStorm> 36MB runtime
<AStorm> (count libs in!)
<wasabi> It's not a 36MB runtime heh.
<wasabi> You have to only count USED libs heh
<Keybuk> AStorm: no, mono libs include a lot more than libstdc++
<Keybuk> I can include glibc and gtk+, etc. into the stdc++ calculation, if you like
<Keybuk> it wouldn't win :p
<AStorm> glibc can't be
<AStorm> mono uses it too
<AStorm> :P
<wasabi> -rw-r--r-- 1 root root 2.4M 2007-02-07 16:57 /usr/lib/libmono.so.0.0.0
<AStorm> Unless it's implemented in asm
<wasabi> So, basic VM is 2.4 MB.
<AlexExtreme> bbl
<AStorm> Basic VM is nothing.
<wasabi> So, count the classes you use.
<AStorm> wasabi, a lot more
<wasabi> Which I can't do for you since I don't have a program in bind.
* Keybuk wonders how big Python is
<AStorm> all system classes
<wasabi> Not all system classes.
<AStorm> Keybuk, exactly... 2.5MB dll
<wasabi> If you don't use System.Windows.Forms.dll, you don't need it.
<AStorm> wasabi, ok, most of them :P
<Keybuk> AStorm: compared to a 3MB so? :p
<wasabi> Harldy.
<wasabi> System.Web
<Keybuk> Mono is smaller then
<wasabi> System.Runtime.Remoting
<wasabi> System.Security, System.Xml
<wasabi> System.Data, System.Design
<AStorm> wasabi, ok, anyway.
<wasabi> Mono.HalfABillion
<AStorm> the problem is the mono requirement :P
<Keybuk> what's wrong with Mono?
<wasabi> Why, again? :0
<AStorm> That's why Haskell wins :P
* Keybuk likes C# - it's a nice language
<wasabi> haha
<Keybuk> with a good hereditary too
<AStorm> Compilable to straight machine code.
<AStorm> Yes, C# is nice
<Keybuk> you can compile any language straight to machine code
<AStorm> though from .NET I prefer Nemerle.
<Keybuk> admittedly, some of the higher-level ones like Python are damned tricky to do
<AStorm> Keybuk, to _standalone_ code (semi-standalone)
<AStorm> Yep.
<Keybuk> but it's possible
<AStorm> Haskell compiles to C--
<wasabi> But it's *pointless*
<Keybuk> simply because if you couldn't express the language as machine code, you could never run the damned thing
<AStorm> or optionally to C
<AStorm> Keybuk, JIT != full static compilation
<Keybuk> a runtime is just pre-packaged machine code for language constructs, after all
<AStorm> And a JIT :P
<Keybuk> AStorm: I'm not talking about JIT
<wasabi> Every program we write relies on other services.
<AStorm> Mono uses JITting a lot
<wasabi> Mono can pre-JIT.
<AStorm> wasabi, well, some do more, some less
<Keybuk> if a language can be interpreted, on the fly, then it can be compiled to machine code in advance
<wasabi> If you really give a shit.
<AStorm> wasabi, yep, somewhat
<AStorm> But not NET 2.0
<wasabi> Hmm. MS 2.0 can pre-JIT. Don't know about Mono.
<AStorm> Keybuk, of course.
<Keybuk> if a language cannot be compiled, then ergo, it cannot be interpreted
<Keybuk> you could define a language that was literally impossible to compile
<AStorm> But writing such a compiler is another matter.
<Keybuk> but then you would also not be able to interpret it
<AStorm> Heck, you can even compile a subset of special Python
<AStorm> RPython
<Keybuk> simply because the only way to define an un-compilable language is to design a language that requires the machine to do things that it cannot do
<AStorm> with the use of PyPy toolchain
<Keybuk> you can compile all of Python
<Keybuk> Python is written in C
<Keybuk> C is compiled
<wasabi> This argument has no goal anymore.
<AStorm> Keybuk, gah
<Keybuk> ergo all Python constructs can be expressed as machine code
<Keybuk> so all the compiler need to is take the machine code constructs compiled from the C, and chain them together appropriately
<AStorm> Yeah right.
<AStorm> Do that and see the app fail like hell :P
<wasabi> C# is a valuable language. It does work so I don't have to. It makes my software easier to program. It's pretty elegant as it goes for languages.
<Keybuk> why would it fail?
<wasabi> Hence. YAY.
<AStorm> Notice import statement.
<Keybuk> notice that import frequently loads C .so files
<AStorm> C# is ok for me.
<AStorm> Keybuk, some, most not :P
<Keybuk> so?
<AStorm> Anyway, please write the compiler :D
<AStorm> I'd be grateful.
<Keybuk> I don't want to write the compiler
<AStorm> There's no compiler, because it's a "damn hard" problem
<Keybuk> if the speed that machine code provides were a requirement, I probably wouldn't write it in Python to start with
<Keybuk> note that hard != impossible
<AStorm> Yep.
<AStorm> It's already done for restricted python
<Keybuk> actually, I shouldn't imagine it's even that hard
<AStorm> (not too restricted)
<Keybuk> there are one or two python constructs that would require some exotic games in machine code
<Keybuk> but nothing too untoward
<AStorm> wasabi, still, I love some constructs of Nemerle more than C#
<Keybuk> you'd end up with more than a few jumping-point routines
<wasabi> Yeah. I like that you can use them side by side. ;)
<AStorm> Well. RPython compiles to straight nice C :P
<wasabi> I'm really excited by a lot of C# features.
<wasabi> I think LINQ just totally rocks.
<AStorm> wasabi, examples, because I'm not completely sold on it
<wasabi> LINQ: language integrated queries.
<wasabi> For instance:
<wasabi> ISet<string> strings = new ISet<string>(); /* add a bunch of strings to strings */
<Keybuk> Python is just a wannabe-Lisp anyway :p
<AStorm> wasabi, ahha, nice :P
<AStorm> Keybuk, indeed, OO Lisp
<Keybuk> eh?
<wasabi> foreach (string s in (from strings as string where string.StartsWith("f")) { do stuff with s; }
<Keybuk> there's no such thing as OO Lisp or non-OO Lisp
<AStorm> Object-oriented Lisp
<Keybuk> Lisp is Lisp
<wasabi> You can use the syntax "from" to apply constraints to enumerations in code.
<AStorm> I was talking about Python :P
<AStorm> I still prefer Haskell syntax to Scheme one :P
<AStorm> Only thing missing is . for accessing "members" of a class
<AStorm> It's currently used for function composition.
<Keybuk> err?  Lisp is even more insanely "everything is a first class object" than Python is
<AStorm> Haskell' (Prime) will resolve this problem.
<AStorm> Yep, most of the time.
<AStorm> Except (define ...)
<Keybuk> to the degree that any lisp expression can either be lisp code to evalulate that expression, or a value
<AStorm> I know. Been there, done Lisp :P
<Keybuk> (define ...) is just def in Python
<AStorm> Nope.
<AStorm> def is defun
<AStorm> define is more powerful
<AStorm> No equivalent Python construct.
<Keybuk> maybe I'm confused there, it's been a while since I did any serious lisp
<wasabi> Okay, lets say you have a collection of Person objects... and you wanted to find Person objects with a given age and first name:
<wasabi> var q = from persons as p where p.Name.FirstName == "Bob" and p.Age = 123;
<wasabi> foreach (Person p in q) { do stuff with them }
<AStorm> wasabi, nice sugar :P
<wasabi> Yup. It turns into a AST, which a ORM tool can implement.
<AStorm> Not entirely necessary anyway.
<wasabi> Or it becomes a simple loop.
<wasabi> Either way, your code looks clear and concise.
<wasabi> Instead of programatic.
<AStorm> I don't know if it's entirely clear :P
<AStorm> Looks like a SQL query.
<wasabi> Yeah, it is the same idea. YOu are filtering a batch of data.
<wasabi> But it's completely type safe.
<AStorm> Well, I actually prefer Haskell :P
<wasabi> And IDEs can do pop up completion and fun stuff on it. ;)
<AStorm> filter (\x -> x.Name.FirstName == "Bob" and p.Age == 123) thedb
<AStorm> That's assuming H'
<AStorm> not H98
<wasabi> How is that implemented?
<wasabi> Can that be translated at runtime to a SQL statement?
<AStorm> Probably :P
<AStorm> Depends on what x is
<AStorm> if x is an SQL-alike object
<AStorm> Of course, it's no syntactic sugar :P
<AStorm> it depends on the qualities of the p element
<AStorm> Esp. on its class and implementation of == and . operator
<wasabi> Ahh.
<wasabi> So the p class could be a class which completely changes the dynamics of the == and .
<AStorm> Yes.
<wasabi> Inorder to formulate one query that can be turned into one SQL statement.
<AStorm> Another boon is it can be used everywhere :P
<AStorm> You get an OODB gratis.
<wasabi> Yeah.
<AStorm> Ah, wrong
<wasabi> Python has a 'yield' statement, right?
<AStorm> this can't be done in this way
<wasabi> ?
<AStorm> p would have to be a Monad :P
<AStorm> wasabi, yes, it does
<wasabi> Monad?
<wasabi> I like yield. ;)
<wasabi> One of my favorite new C# features.
<AStorm> wasabi, yes, a wrapper around interactions with outside world
<AStorm> Signifying mostly "this is not a function"
<wasabi> hmm
<AStorm> Because it's not :P
<AStorm> So, it'd have to be written more like x <- (p->Something->SomethingElse); x==blah
<AStorm> but that can be also "undone"
<AStorm> Because it's actually a syntactic sugar for some >>= operators
<AStorm> Which work like a pipe in Linux.
<AStorm> Etc. etc.
<wasabi> Yeah. All of the stuff in C# is delayed evaluation too.
<wasabi> It never actually runs until it's used.
<wasabi> var q = from whatever; just returns an instance of some class that could potentially do the filter.
<AStorm> Unfortunately not lazy, just delayed :P
<wasabi> Could be lazy. If it's not SQL.
<AStorm> Nemerle does have some lazy eval constructs.
<wasabi> Default implementation is lazy... it finds the NEXT element, and stops.
<wasabi> SQL implementation could implement that by streaming results.
<AStorm> Aha. Ok.
<AStorm> But generally, C# is strict, right? Is there a method to disable that strictness locally easily?
<wasabi> No. It is always strict.
<AStorm> Generators?
<wasabi> Don't understand the reference.
<wasabi> Explain.
<AStorm> I think it supports them.
<AStorm> You can iterate using .next and .previous, but the objects get accessed only then
<AStorm> not earlier
<wasabi> Oh. Yeah. IEnumerable contains .MoveNext()
<wasabi> yield returns implementations of IEnumerable which access a method statemachine
<wasabi> And the query stuff does too
<AStorm> Aha, just what I though. Like Python iterator.
<wasabi> Yeah.
<AStorm> A bit of laziness in a strict language.
<wasabi> Hmm. By strict you mean? It's still completely typesafe.
<wasabi> What it generates must still be of the declared type.
<wasabi> IEnumerable<T>
<wasabi> Basically, using yield looks like.
<wasabi> public IEnumerable<string> GetString() { yield return "string1"; yield return "string2"; }
<wasabi> foreach (string s in GetString()) { operate on s; }
<AStorm> wasabi, strict == not lazy
<wasabi> The foreach construct basically uses MoveNext() and Current() on the IEnumerable instance.
<AStorm> Not dynamic.
<AStorm> lazy != dynamic
<AStorm> Are there any nice operators/functions to access an Nth object of an IEnumerable?
<wasabi> No. It's not an IEnumerable then.
<wasabi> IList<T> contains an indexor.
<wasabi> IList<T> : IEnumerable<T>
<wasabi> You could certainly implement an IList<T> which worked on a backing IEnumerable by advancing.
<AStorm> Aha, correct :P
<wasabi> But you would prefer not to i'd imagine.
<AStorm> :>
<wasabi> As you'd lose the benefits of the delayed evaluation.
<AStorm> Depends for what. E.g. memoisation
<wasabi> Simply calling List.Count would force teh entire thing to be evaluated.
<wasabi> public IEnumerable<Foo> GetFoos() { for (int i = 0; i < 5; i++) return Foo();  }  <--- generator?
<wasabi> public IEnumerable<Foo> GetFoos() { for (int i = 0; i < 5; i++) yield return Foo();  }  <--- generator?
<wasabi> bah. I meant "new Foo()"
<AStorm> wasabi, sure
<AStorm> Because counting is non-lazy always
<AStorm> You could have CountAlreadyDone
<wasabi> Sure.
#upstart 2007-02-18
<rehcla> Md ?
<shawarma> Does upstart have a mechanism for forcing fsck on boot? (Like "shutdown -F" used to do)
<_ion> I don't see a reason for it not to be possible.
<shawarma> No of course not, but is it already there?
<shawarma> brb
<_ion> Well, the replacement-initscripts aren't ready yet, the fsck job is yet to be implemented AFAIK.
<shawarma> Ok.
<shawarma> Thanks
<int0x0c> Is there presently a mechanism for starting dbus itself under the  replacement-init-scripts
<_ion> The files under etc/event.d in the replacement-initscripts repo is really everything there is so far, unless Keybuk has new stuff not pushed to the repo yet.
<AlexExtreme> nope, nothing new there
<_ion> dbus will probably be one of the easiest ones, though. :-)
<AlexExtreme> yep :)
#upstart 2008-02-11
<sadmac> how can I tell what the current log level of upstart is?
<ion_> keybuk: Please take a look at the refcount branch now. (nih_alloc_reparent, nih_realloc still have to be taken care of.)
<ion_> keybuk: Does that look okay?
<mkelly32> hi, i do "start logd" using the sample logd job, but nothing is ever recorded in /var/log/boot, even when i run a test job with console logged and exec echo "test"
<mkelly32> am i misunderstanding how logd is supposed to work?
<ion_> logd is temporarily not used or working AFAIK.
<mkelly32> okay
<mkelly32> well, i'm using 0.3.9, not bzr
#upstart 2008-02-12
<sadmac> Keybuk: did you look at this? https://bugzilla.redhat.com/show_bug.cgi?id=432384
<Keybuk> sadmac: I hadn't seen that
<Keybuk> for some reason, I'm not getting mails from your bugzilla
<sadmac> hm. weird.
<sadmac> Keybuk: do you have an account? (I don't think it filters on that, but maybe).
<Keybuk> I do, yes
<Keybuk> and I set myself to be cc'd on the meta bug
<sadmac> hmm.
<sadmac> spam filter?
<Keybuk> I've checked that 
<Keybuk> my spam filter only grey lists on spam
<ion_> ma164709 < ion_> keybuk: Please take a look at the refcount branch now. (nih_alloc_reparent, nih_realloc still have  to be taken care of.)
<ion_> ma164728 < ion_> keybuk: Does that look okay?
<sadmac> Keybuk: I'll hit devel-list about it.
<sadmac> Keybuk: or you could (might be nice to introduce yourself to the people :)
<Keybuk> ion_: it's on my TODO to look at :)
<Keybuk> ion_: did you push the branch to LP?
<Keybuk> cause I can't remember the URL :)
<ion_> keybuk: Hm, i probably didnât make LP mirror it. Let me see...
<Keybuk> s'ok, just the original URL would do too :p
<ion_> Yeah, just grepping for it. :-)
<ion_> http://johan.kiviniemi.name/software/bzr/libnih.refcount/
<Keybuk> ion_: why uint32_t for refcount?
<Keybuk> unsigned int seems the proper type?
<ion_> Oh, i forgot to change that. We already discussed this. :-)
<ion_> What i changed now is just the most recent commit.
<Keybuk> looks good though
<Keybuk> can't see any obvious problems
<Keybuk> I'm wondering whether you need the restriction that you can't ref/unref a non-parent object
<Keybuk> it should work if you can
<Keybuk> since the parent holds one ref, and the other object another
<Keybuk> if the parent is freed, the object will stay around because it's still ref'd
<Keybuk> that other object would have to take care of unref'ing in its destructor though
<ion_> OTOH, it would be somewhat confusing if an object is in multiple âcontainersâ, only one of which is the parent. I was thinking that a ârootâ object (parentless) and all its children would be a single refcounted entity.
<ion_> keybuk: Ok, i fixed refcountâs type. Pushed.
<Keybuk> it might be interesting if each of the objects could have an n-way relationship with each other, rather than strict parent/child
<Keybuk> that way unref'ing would be automatic
<ion_> True...
<Keybuk> maybe I'm overcomplicating things as usual there though
<sadmac> Keybuk: nobody else is reporting missing emails. Could it be something in your bugzilla account settings?
#upstart 2008-02-13
<JohnFlux_> Hey all
<JohnFlux> I come in here every month to check on the status of upstart and dbus :)
<JohnFlux> Is it now possible to communicate via dbus to upstart to start/stop/restart/etc services?
<JohnFlux> if not dbus, then some other method
<JohnFlux> I know you guys are busy, but if I can communicate to upstart, then I can add this functionality to the kde task manager
<JohnFlux> then users can easily start and restart services
<atsampson> well, trivially you can communicate with it using initctl stop/start/status/etc...
<JohnFlux> atsampson: well, I'll need to know the pid of the process as well
<JohnFlux> and just running initctl status   won't be very reliable.  presumably the output could change 
<atsampson> initctl'll tell you that; "nmbd (start) running, process 1254"
<JohnFlux> between distros, with versions etc
<atsampson> it looks from the man page as if the output's designed to be parsed
<JohnFlux> also there'd be no notification if the status changes etc
<JohnFlux> I think someone was working on dbus interface
<JohnFlux> dunno how far along that got
<atsampson> you can say "initctl jobs" to get notifications about job state changes
<JohnFlux> how does that work?
<atsampson> well, you run "initctl jobs", and it prints a parseable line to stdout for each state change
<JohnFlux> hmm, it needs to be run as root?
<JohnFlux> is that a bug?
<atsampson> yes, it does, and no, it would be a bug if you could modify system services as a non-root user, surely?
<JohnFlux> why would monitoring them be modifying the system services?
<atsampson> why wouldn't monitoring system services be a privileged operation?
<atsampson> (disclaimer: I'm just a user of upstart, so please feel free to ignore anything I say...)
<JohnFlux> atsampson: hmm
<JohnFlux> atsampson: 'ps' doesn't require root, and you could monitor whether services start/stop through that
<JohnFlux> although not very efficently
<atsampson> if I'm using upstart to manage services on a machine with untrusted users, I probably don't want them seeing the details of all the services I'm running
<JohnFlux> I can see that there might be sensitive information
<JohnFlux> but for just simple status changes, that should be public knowledge
<JohnFlux> a normal user can run 'status apache'  etc
<atsampson> ps does require root on some systems, at least to see processes that aren't yours (there's at least one kernel patch for Linux that provides sensible permissions on the files on /proc, for example)
<JohnFlux> hmm, wait no they can't
<JohnFlux> a normal user can do    /etc/init.d/apache2 status
<JohnFlux> but not   status  apache2
<JohnFlux> that's very strange
<atsampson> start/stop/status are just symlinks to initctl; "start" = "initctl start"
<JohnFlux> atsampson: the vast vast majority of users aren't going to be filternig ps
<JohnFlux> atsampson: hum
<JohnFlux> that's really wrong
<atsampson> erm, no, it's not...
<JohnFlux> checking if apache is running should not require root
<JohnFlux> just check whether the pid is there
<JohnFlux> atsampson: why would   /etc/init.d/apache2 status   not require root, but status apache2  does ?
<atsampson> if you want that behaviour, I think it'd be better to put it somewhere other than upstart
<JohnFlux> s/would/should/
<atsampson> it seems a bit silly to put lots of complicated access control stuff into init, rather than having some other daemon/service that can make requests to upstart on your behalf and do access control
<JohnFlux> atsampson: well, I want to be able to show the list of running services to the user in the kde task manager
<atsampson> so perhaps the solution'd be to have a daemon that spoke the upstart protocol to init on one side, and dbus to other processes, and then you could configure that to do access control?
<JohnFlux> `hmm
<JohnFlux> how does initctl work?
<atsampson> init's got to be as simple, reliable and secure as possible, really
<JohnFlux> how does it check the status of a process?
 * atsampson straces
<atsampson> it talks over a Unix socket to init, it appears
<JohnFlux> ah hmm
<atsampson> there's a library in the upstart package that wraps up the protocol, so it could presumably be used in something else without too much trouble...
<JohnFlux> presumably that requires root privillages as well?
<atsampson> yes
<JohnFlux> hum
 * atsampson decides it's a good time for tea
<JohnFlux> it's always a good time for tea :)
<atsampson> :)
<JohnFlux> I suspect I'm going to have to make a small setuid root program that just calls initctl statu
<JohnFlux> s
<JohnFlux> :-/
<atsampson> or provide a bit of sudo config...
<JohnFlux> well, users aren't going to want to type in a root password just to see what is running
<JohnFlux> i don't think any other OS does that
<Keybuk> JohnFlux: sorry, wasn't around to answer your questions earlier
<Keybuk> is it possible to communicate via dbus?  no, not today
<Keybuk> will it be?  yes, in trunk - there's only one thing higher on the todo list now
<Keybuk> is it possible by any other method today?  yes, released versions use a custom IPC protocol
<Keybuk> you can use that through initctl
<Keybuk> or use libupstart (which initctl is a CLI wrapper for)
<Keybuk> obviously the library is deprecated, and doesn't even exist in trunk
<Keybuk> the initctl output is intended to be parsed by humans and machines
<Keybuk> but obviously may change with the DBus change
<Keybuk> why root only?
<Keybuk> simple: avoids local-root security exploits due to bugs in the initctl protocol
<Keybuk> dbus would solve that since non-root requests can be made through the dbus daemon
<JohnFlux> Keybuk: hey
<JohnFlux> Keybuk: thanks for the reply
<JohnFlux> Keybuk: It would be awesome to get this information via dbus, as a non-root user
<JohnFlux> Keybuk: I'll wait for that :-)
<kwrobel2008> Hi everybody
<kwrobel2008> I'd like to post a question here regarding switching of runlevels:
<kwrobel2008> I'm running Ubuntu Gutsy (Server Edition), and would like to run a script I wrote
<kwrobel2008> that backs up an entire HD to external media
<kwrobel2008> However, I'd like to first switch to a "runlevel" that's similar to single-user mode
<kwrobel2008> Basically, I don't want any crucial background process running while the script executes
<kwrobel2008> And afterwards, I'd like to go back to the "runlevel" I came from (probably 2?)
<kwrobel2008> Does anyone here know how I might accomplish that?
<Keybuk> telinit 1
<Keybuk> that switches you to single-user mode
<kwrobel2008> HI
<kwrobel2008> Right. No I got the general idea
<kwrobel2008> But how do I control the flow of things?
<kwrobel2008> can I run telinit 1, and the next call my script, and then telinit 2?
<Keybuk> I'm not sure I follow, sorry
<kwrobel2008> Right. And I know that upstart is really an event-driven model.
<kwrobel2008> But since it's being used as a so-called SysV Init replacement...
<kwrobel2008> Anyway, let me try to be more specific:
<kwrobel2008> I wrote a script, called bootbak,
<kwrobel2008> That script reformats an external hard USB drive (250GB Maxtor)
<kwrobel2008> and then copies the entire filesystem to that drive, modifies /etc/fstab and grub/menu.lst,
<kwrobel2008> and writes an appropriate MBR to it so I can boot of the USB HD in case of total system failure
<kwrobel2008> Now, the server to be backed up is an email server which receives tons of emails, especially during the night (SPAM, sigh)
<ion_> Why not install multiple HDDs and use redundancy?
<kwrobel2008> That aside, what I was thinking is to bring down all services, like postfix, dovecot, apache a.s.f.
<kwrobel2008> ion_: eventually, we will install multiple USB HD on a hub and run that script each day against a different drive
<kwrobel2008> for rotation
<kwrobel2008> Point is, to make it short: I'd like to do this:
<kwrobel2008> init 1
<ion_> I mean, multiple internal HDDs and RAID-1 or higher.
<kwrobel2008> /usr/local/bin/bootbak
<kwrobel2008> init 2
<Keybuk> does that work in SysVinit?
<kwrobel2008> ion_: Well, we want to have backups in case we mess up our configurations so bad that we can't get the server up and running again
<kwrobel2008> Keybuk: That's exactly it. It doesn't...
<kwrobel2008> init 1, at the end of it, kills ALL processes, including the script itselve
<Keybuk> pretty much, yes
<ion_> I donât really see why youâd need to kill the services to do backups.
<kwrobel2008> First off, inconsistencies. The backup runs for >5minutes
<kwrobel2008> during which time many emails could have come in, etc
<kwrobel2008> MySQL is also running on that server
<kwrobel2008> Anyway, I know I can run the script in runlevel 2 while everything is up and running.
<ion_> The new emails would go to the next backup.
<kwrobel2008> Right now however, I'm the only user on that  server (it's a new box)
<ion_> You probably should take plaintext dumps of the database.
<kwrobel2008> ion_: I understand.
<kwrobel2008> ion_: Let's pretend for a moment
<kwrobel2008> Is it possible to control the system in such a way that you could switch it into a different runlevel temporarily
<kwrobel2008> in order to do whatever maintenance tasks
<kwrobel2008> and then bring it back up?
<kwrobel2008> (All automated of course)
<kwrobel2008> Maybe upstart is just not the right tool for that?
<kwrobel2008> Here's what I had in mind: create a job in /etc/event.d that listens to trigger run-bootbak,
<kwrobel2008> then add a crontab entry for, say, 10pm that runs initctl emit run-bootbak
<atsampson> presumably you could have another runlevel that's like 1, but rather than sulogin runs the backup script then switches back to runlevel 2 when it's finished...
#upstart 2008-02-15
<atsampson> can you get upstart to always try to restart a service, rather than giving up after a few tries?
<atsampson> one of mine is an ssh tunnel to another machine, so if the machine is down for a bit, it goes away and doesn't come back...
<atsampson> (it used to be a daemontools service, which always retries, but with a brief delay between retries)
<atsampson> I should note that other than that, upstart works very nicely for me :)
#upstart 2008-02-17
 * Keybuk finishes cleaning up Launchpad and the Wiki
 * AlexExtreme noticed (mails from launchpad)
<Keybuk> heh
<Keybuk> this is of course distraction against attempting to solve a problem :)
<Keybuk> http://upstart.ubuntu.com/wiki/JobAtomicity
<Keybuk> ^ that problem
<Keybuk> some bits are obvious, some aren't
<Keybuk> e.g.
<Keybuk>   start job FOO=bar BAR=baz
<Keybuk> obviously, that starts the job
<Keybuk>   stop --id 1234
<Keybuk> that sets it stopping again
<Keybuk>   start --id 1234
<Keybuk> that sets it starting again
<Keybuk> BUT does it still have FOO=bar and BAR=baz in its environment?
<Keybuk> *tumbleweed*
<ion_> *grasshoppers*
#upstart 2009-02-09
<sadmac2> Keybuk: pull the async branch one more time for the last round of test cases (/w changelog)
<Keybuk> great, thanks
<ion_> http://heh.fi/tmp/speaking_in_tongues
<sadmac> ion_: looks like procmail
<ion_> Yeah
<sadmac> strcat(strcat(strcpy(...
<sadmac> http://markmail.org/message/t4bgv6c7gr7mnykl
<sadmac> ^^poor bastard
<ion_> :-D
 * ion_ introduces sadmac to U+2B11, â¬
<sadmac> unenter?
 * sadmac really needs to find an IM that works for him
<sadmac> Keybuk: does launchpad not show subcommits anymore, or did you collapse the async branch before you merged it?
<Keybuk> you have to go into more details to see them?
<Keybuk> I didn't collapse it
<sadmac> ah. yeah, it changed.
<Keybuk> it does that
#upstart 2009-02-10
<keesj> LOC == Lines Of Code but also  is Level Of Confidence
<ion_> Lot of Crap
<keesj> :p
<keesj> Keybuk: would it be possible to post the presenation you gave at fosdem somewhere?
<Keybuk> yes, I will do once I have things sorted out
#upstart 2009-02-11
<iamthelostboy> hi.. if i have a custom job, which is for instance just a simple script, does upstart automatically emit started and stopped events for that upstart job to be chained to other upstart jobs?
<sadmac> the BOOLEAN type in DBus has an alignment of 4
<sadmac> \o/
<nteon> hello
#upstart 2009-02-12
<nteon> is it safe to say that the stuff in utils/ directory is depreciated and unmaintained?
<Keybuk> no
<Keybuk> not at all
<sadmac> Keybuk: anything else in libnih/upstart need some code love?
<Keybuk> sadmac: haven't merged my stuff from the last week or two back in yet
<sadmac> Keybuk: what are we keeping for 0.10? RH is going to need init to do SELinux, and I'm wondering if that's worth doing now.
<Keybuk> why do you need init to do it?
<Keybuk> you can set the policy in the initramfs
<sadmac> that's what we're doing now.
<sadmac> there's push to move it back though.
<Keybuk> I'm not going to have a "drop everything" flag day like we did 0.3->0.5
<Keybuk> every thing in 0.5 today will be in 0.10, or will be gradually changed
<sadmac> that's good to know.
<Keybuk> timetable's been pushed up a bit too
<sadmac> oh?
<Keybuk> I'm probably going to be working full-time on Upstart from a week or two's time through to June
<Keybuk> rather than later
<Keybuk> http://people.ubuntu.com/~scott/stig-20090208-0940.png
<Keybuk> (my first attempt at a native Upstart boot)
<sadmac> 18s. Too slow.
<sadmac> :D
<Keybuk> no
<Keybuk> you're mis-reading the chart
<Keybuk> it's 8.5s
<sadmac> you did early login?
<Keybuk> everything after that is me getting a terminal opening, and fumbling to stop the bootchart collector ;)
<sadmac> oh
<Keybuk> no, that's all-up
<Keybuk> X in 3s
<sadmac> If the service management stuff is stable enough (and why wouldn't it be) I'm pretty sure I can write some decent dynamic ionice stuff that'd make early login work well.
<Keybuk> define "early login" ?
<sadmac> popping up gdm before we've loaded everything (in the event driven sense: before the running set has stabilized)
<Keybuk> that's bringing up gdm once its dependencies are met
<Keybuk> maybe that's the same thing?
<ion_> keybuk: CoolÃ2 (the timetable and the native boot) :-)
<Keybuk> if you installed other services like apache, they'd start up about the same time
<sadmac> Keybuk: its that + not making gdm depend on some "the system is done booting" trigger
<Keybuk> oh, there's no such thing
<sadmac> Keybuk: right
<Keybuk> gdm depends on d-bus, and /usr being mounted
<sadmac> Keybuk: doing anything /but/ early login is a pain in the ass now.
<Keybuk> hal is system-bus activated
<Keybuk> and depends on acpid
<Keybuk> (so X blocks on it)
<Keybuk> also depends on d-bus and udev
<sadmac> Keybuk: so the idea of this would be that upstart could be set in a kind of "background mode" such that when a new service's goal changed, it would be ioniced into the "idle" class. It would be ioniced back into the class specified by its configuration (or the default) when it reached the "started" or "stopped" states.
<ion_> And this would be done to ~everything else than the services that lead to gdm getting started?
<sadmac> ion_: it would be turned off until gdm was started
<sadmac> ion_: the idea is "get non-interactive stuff out of the way of the user"
<ion_> Ah
<nteon> sadmac: I've been looking into upstart in Fedora CVS.  the devel branch doesn't seem to be building (missing dbus-devel as a BuildDep), and I was wondering if it was that or some other reason rawhide is using upstart 0.3.9 from F10
<nteon> sadmac: (im looking to get some rawhide builds working on my OLPC XO)
<sadmac> nteon: we aren't going to 0.5.1
<sadmac> nteon: neither is ubuntu.
<sadmac> nteon: next release shipped by any major distro (afaik) will be 0.10
<nteon> sadmac: good to know, thanks :)
<nteon> I suppose I can stop trying to get my rawhide box booting with 0.5.1...
<sadmac> heh
<nteon> around when is 0.10 expected to come out?
<sadmac> nteon: in time for the next Ubuntu release... damned if I can remember all the dates
<sadmac> should be F12 material
<nteon> sadmac: sounds good
<nteon> sadmac: I'm cautiously interested in getting more involved in Fedora development, so if theres anything I can do to help with upstart packaging or patching, let me know (not that I'm intimately familiar with the codebase or anything)
<sadmac_> Keybuk: did you go back to a custom control protocol for initctl?
<sadmac_> Keybuk: there's no reference to dbus in initctl in trunk
<mbiebl> sadmac: SUBDIRS = m4 intl nih nih-dbus dbus init doc po # util 
<mbiebl> util/ in trunk isn't ported (yet) afaics
<keesj> what would be the right place to increase the timeout between initctl and upstart?
<keesj> we lost a lot of functionality while swichting to dbus
<keesj> while adding more code :p
<keesj> and dependencies
<keesj> I hope there will be something in return
<ion_> More secure IPC. Upstart can determine when a service is running based on when it appears on D-Bus.
<keesj> is there kerberdos support in dbus. otherwise i don't why it's more secure
<ion_> More eyes looking at the networking code.
<ion_> Especially relevant when normal users gain access to Upstart.
<keesj> I see. 
<keesj> I did not look at how a user is stopped from sending events yet
<ion_> Also, the D-Bus dependency should be a non-issue in most places. For an embedded system with no need for IPC and extremely limited resources, perhaps the D-Bus code can be even disabled. If not yet, the one needing it is free to share a patch. :-)
<keesj> my kind of patches and ideas generaly don't get accepted in upstart.
<keesj> BADCODE i guess
<Keybuk> sadmac2: 0.5 is the current branch
<Keybuk> I haven't ported the util stuff over yet
<sadmac2> Keybuk: ah
#upstart 2009-02-13
<cwillu> is there a list of event anywhere?
<sadmac> cwillu: any string can be a valid event. but for a list of ones upstart uses internally see man events
<cwillu> no manual entry
<cwillu> my next gripe was going to be the difficulty in finding documentation for upstart-as-used-in-the-wild
<cwillu> :p
<sadmac> cwillu: install fedora :)
<sadmac> then look at the manpage
<cwillu> after the kde debacle, you'll understand if I decline your generous offer :p
<cwillu> but I'd take a link to an online version :p
<sadmac> kde debacle? I think we have a damn good KDE implementation. (digging in koji for that manpage...)
<cwillu> thanks
<cwillu> http://linux.die.net/man/5/events
<sadmac> whoa. didn't know they would pick that up...
<cwillu> which isn't terribly useful
<sadmac> what are you missing?
<cwillu> the justification for using upstart natively rather than the old style rc jobs
<cwillu> except not worded quite so aggressively
<sadmac> cwillu: I'm regrettably too tired for that now.
<sadmac> cwillu: netsplit.com might be a good place to look. somewhere in the archives scott has done that many times.
 * sadmac sleeps
<cwillu> sadmac, I don't need to be sold on upstart over the old format, but I'm not having an easy time of actually using it as such
<cwillu> http://www.netsplit.com/2007/12/07/how-to-and-why-supervise-forking-processes/
<cwillu> does that imply that I don't need to use start-stop-daemon or its ilk anymore?
<suihkulokki> cwillu: pretty much yeah
<cwillu> suihkulokki, there's no way to specify a user to run as is there?
<sadmac2> notting: ping
<notting> what's up?
<sadmac2> notting: do you prefer SELinux policy loading in init or initrd?
<notting> i prefer it not be in init(1) as then the first thing init has to do is policy loading gunk and then re-execing itself
<sadmac2> sold
<notting> whether that's in the initramfs, or in a shell script sitting at /sbin/init...
<sadmac2> notting: I don't see doing it in another piece of code as being better than having it in pid 1.
<sadmac2> the only real added ugly is figuring out whether init is being executed the first time or the second.
<ion_> \o/
<sadmac> ion_: yes?
<ion_> sadmac: Time.at(1234567890) passed.
<sadmac> ion_: sweet
#upstart 2010-02-16
<ensi> hi
<ensi> can someone help me figure out why this application wont start with upstart
<ensi> http://codepad.org/gWC6IZ8J
<ensi> there's the upstart file that is put in /etc/event.d/
<ensi> and the output that i get from initctl event
<ensi> i've spent about 6 hours on this, getting kinda frustrated
<ensi> thanks :)
<plautrba> ensi: try echo 'i'm here!' > /var/log/swapper.log
<ensi> yeah that part works
<ensi> the echo that is
<ensi> it looks like it it starts it and the kills it immediately?
<ensi> like whats up with that?
<blueyed> Is there a way to reset what upstart thinks that a job is running? both start and stop are blocking (need ctrl-C) and the process is not being started properly (unlike during boot)
<blueyed> the conf: http://pastebin.com/f5a4419f0
#upstart 2010-02-17
<Alpha1650> Hi. I have a little question about upstart: As I understood, "console output" means that this task uses the console for input and output. What happens if two tasks need console for in- and output? Does the second have to wait until the first ended? Or can they run both simultaneously? Then what happens if both want to read something from console the same time?
<Alpha1650> Hi. I have a little question about upstart: As I understood, "console output" means that this task uses the console for input and output. What happens if two tasks need console for in- and output? Does the second have to wait until the first ended? Or can they run both simultaneously? Then what happens if both want to read something from console the same time?
<Keybuk> Alpha1650: they interleave and steal characters from each other
<Keybuk> depends who's first to call read()
<Alpha1650> Keybuk: Okay, that doesn't sound very good. I'm having this ubuntu issue here with mountall and cryptsetup interleaving and breaking the splash and making it a gamble whether the machine will boot or not. I just set mountall's console to "none" what seems to make the machine bootable with splash disabled.
<Keybuk> Alpha1650: on lucid?
<Alpha1650> Keybuk: Karmic.
<Keybuk> ah, yeah
<Alpha1650> Keybuk: Are there any ideas/chances/intentions to solve the interleaving issues with upstart or is this what the apps will have to solve themselves?
<Keybuk> I don't think it's up to Upstart to arbitrate the console
<Keybuk> there are already better ways to do that
<Keybuk> e.g. Plymouth
<Alpha1650> Keybuk: Do you know what mountall needs the console input for? As far as I can see it's first duty is to mount the filesystems when their devices come up. What does it need console (especially input, output might be for fsck or so) for?
<coolo> Alpha1650: fsck also needs input on errors
<Keybuk> right, fsck error recovery, etc.
<Alpha1650> Keybuk: coolo: I see, but then this is really an uggly situation where some kind of syncronisation and queuing of console-input/output-events is necessary, no? Maybe something like "interaction_start()" and "interaction_stop()" where scripts can within do input and output that belongs together, for example the password prompt and the following input in cryptsetup. And other tasks reaching a interaction_start() have to wait until the o
<Keybuk> actually it's more than that
<Keybuk> multiple cryptsetup devices might need to ask for the passphrase at the same time
<Keybuk> you need something for everything during boot to talk to that deals with the screen
<Keybuk> which might be a plain text console or a graphical console
<Keybuk> as I said, Plymouth solves this already
<Alpha1650> Is there a project site for Plymouth?
<Alpha1650> I don't understand how Plymouth could solve the multiple tasks interleaving problem without syncronising at least their input
<ion> Kind of. Its API doesnât seem to allow multiple clients to query things at once, with the user being able to switch focus between them. The API also doesnât seem to provide asynchronous queries, so that you ask a question, get an answer handle, go do other stuff and get a message over a file descriptor with the handle and the user reply (which should be handled by the plymouth client library by calling a callback you provided when asking the question).
<Keybuk> ion: huh? it's API allows exactly this
<ion> The latter issue already causes a problem with Upstart: thereâs a user that consistently has his startup hang because Upstart just happens to say âdidnât find the device for a mount, what to do?â and blocks on the answer, although the device does appear very soon.
<ion> keybuk: Ok, iâve misinterpreted it then. I didnât take a good enough look at the plybootclient/whatever headers.
<ion> It didnât seem to provide a way to integrate with an arbitrary main loop, instead it just seemed to provide a way to jump to its main loop until something happens, which isnât good if you want asynchronous queries.
<Keybuk> plymouth's main loop uses epoll
<Keybuk> which is infinitely stackable
<Keybuk> the mountall integration is a bit poor because I wrote it in a hurry
<Keybuk> needed it in before Alpha/Feature Freeze
<Keybuk> can fix it later as "bug fixing" :p
<ion> Yes, it wouldnât be hard to make plymouth provide an API to do that. Just one function to provide plymouthâs epoll fd and another function to call from your main loop when thereâs something on that fd. Perhaps it already does, i might have missed it when skimming through the API.
<Alpha1650> Keybuk: you wrote on your blog that plymouth will not be used because it relies on kernel mode setting.
<ion> alpha1650: Many months ago, in the context of another Ubuntu release?
<Alpha1650> Months ago, yes. Sep 2009.
<ion> Things change. :-)
<Alpha1650> Yea, so this already changed. Good. (I have nvidia hardware that will not work with kernel mode setting I think.)
<Keybuk> Alpha1650: that's not what I wrote
<Alpha1650> Keybuk: Then I misunderstood you. What did you write?
<Alpha1650> Keybuk: Yea, I got causality wrong.
<Keybuk> Alpha1650: we didn't use plymouth in karmic because we already had usplash
<Keybuk> which could already do KMS (it turned out)(
<Keybuk> and there was a strong push to keep with usplash unless there was a truly compelling reason to switch
<Keybuk> especially since plymouth wouldn't support non-KMS hardware
<Keybuk> fortunately it turned out there were some bugs in usplash that needed fixing
<Keybuk> and one of the key people who was pushing for keeping usplash got involved in fixing them
<Keybuk> and remembered how nasty and ugly the code was
<Alpha1650> Keybuk: :D
<Keybuk> so I was able to push plymouth again for lucid with the key feature "it means you don't have to fix usplash again" :p
<ion> :-)
<Keybuk> for lucid, plymouth was accepted with no resistance
<Alpha1650> Okay, so my hope can return, that Ubuntu will succeed this time in what it failed for the last 3 years: Making splash and dm-crypto work together properly.
<Alpha1650> I really had no ubuntu install where dm-crypto and splash worked smooth. Always problems.
<Keybuk> the release manager himself is caring a lot about cryptsetup
<Keybuk> and working actively on it
<Alpha1650> :)
<Alpha1650> So this issue will hopefully be fixed in the future. What about now? How can I make mountall work together properly with my crypto?
<Keybuk> it will be fixed in lucid
<Alpha1650> Yea, but Lucid will be released in months. Shall I not turn my PC off and on again in that time? ;)
<Keybuk> there's no plan for an update to karmic
<Keybuk> or, if you prefer, the update is lucid ;)
<Alpha1650> Keybuk: Well, but until then a workaround for those people like me out there would be nice. You know the issue and know what's happening there. What would happen, if I set the "start on" of mountall from "startup" to the event when my /home-crypt-device was decrypted for example? cryptsetup does in no way depend on what mountall does, right? So why not let mountall depend on what cryptsetup does?
<Alpha1650> This would maybe delay the mount for a short period of time but I'm sure everyone having this issue is willing to pay this "price" for the solution.
<Keybuk> there isn't a workaround
<Keybuk> like you say, it's broken
<Keybuk> a lot of work has gone into karmic to fix it
<Keybuk> new packages
<Keybuk> new integration
<Keybuk> etc.
<Keybuk> none of this is trivial to backport to a stable release
<Keybuk> err, into lucid
<Alpha1650> Yea, sure. I do fully understand that there is no reasonable way to fix this by a package update. But to me (correct me if I got something wrong!) this seems to be a syncronisation issue, right? So my hope is that you can find a workaround that will make you loose some of your parallel execution brought by upstart but will ensure that those commands run in the right order and don't interfere.
<Keybuk> we're not looking for a workaround for karmic at this point
<Keybuk> karmic was released in October
<Keybuk> the whole point of a six-monthly release schedule is that you're always working on the *new* release
<Keybuk> once a release is out of the door, it doesn't tend to get updates
<Keybuk> the update is the next release
<Alpha1650> So if it doesn't work, it doesn't work and people should stop using their computer, go outside, enjoy sun (or snow in this case), return in 6 month and try again?
<Keybuk> updates sometimes break a minority of users
<Keybuk> it's a fact of life
<Keybuk> this is one of the reasons we recommend backing up a computer before upgrading to a new release
<Keybuk> just in case the new release has a critical regression for you
<Keybuk> obviously we will do everything we can to fix the regression
<Keybuk> and, especially if it affects a large number of users, or is security-related, even do an update to the release
<Keybuk> but we can't always do that
<Keybuk> sometimes the fix is so large or fundamental that it has to be "sorry, we'll have that fixed in the next release"
<Alpha1650> Yea, I have my old install around here, so the backup exists. But I like karmic, so ...
<Keybuk> that's another reason we have the LTS releases
<Keybuk> they're a slower, more stable cycle for the kinds of users who can't risk a little downtime
<Keybuk> or a little manual hacking
<Keybuk> and we *do* do big stable updates for them
<Alpha1650> Well, as I said, I see that fixing this cleanly would be a lot of work.
<Keybuk> e.g. we've not long released 8.04.4 - another update to hardy
<Alpha1650> But I'm disappointed that there is absolutely no "hacking workaround" for this issue. That's what I don't understand.
<Keybuk> Alpha1650: I don't know cryptsetup well enough
<Alpha1650> No package update or so, just a "howto live with this bug if you have it".
<Keybuk> I've heard of people hacking their crypttab so that all of the passphrases are asked for in the initramfs
<Alpha1650> Keybuk: Yea, exactly such things!
<Keybuk> but such things tend to be different for every person
<Alpha1650> Already thought about this but couldn't find the options yet.
<Keybuk> it could take a developer a month to research the issue and write a howto
<Keybuk> that's a third of the development time of lucid!
<Alpha1650> :P
<Keybuk> it could even mean the bug never gets fixed for lucid
<Keybuk> because the only developer who knows enough spent their entire time worrying about the bug in karmic
<Keybuk> now, often in the process of fixing a workaround is made obvious
<Keybuk> but in this case, the bug is "we got this completely wrong" and the fix is "redo it all from scratch"
<Keybuk> thus there's no real obvious workaround
<Alpha1650> No, the crypttab-thing you talked about is the kind of workaround I was looking for.
<Alpha1650> Dirty but useful.
<Keybuk> so try that
<Keybuk> the bug reports are probably a useful source of ideasa
<Alpha1650> is initctl emit the way to emit signals within upstart tasks?
<Alpha1650> Or are there other ways?
<Keybuk> initctl emit is the way to emit events
<Alpha1650> Okay, thank you.
<Alpha1650> Keybuk: Is mountall the one who shuts down the splash to do fsck?
<Keybuk> Alpha1650: mountall logs to the splash
<Keybuk> not sure why the splash would vanish
<Alpha1650> Keybuk: Hm, okay. Something is stopping splash and then the fsck output appears ...
<acemo> where should i place a new upstart script(i want it to be executed at start up)?
<Keybuk> acemo: /etc/init
<Alpha1650> Keybuk: I found "my own little workaround" for the ubuntu mountall-cryptsetup issue: - disable splash screen (as this breaking and reenabling it during boot is quite annoying) - I copied the "cryptroot"-script from initramfs, named it "crypthome" and modified it in a way that it runs directly before cryptroot in initramfs and decrypts the /home. This makes booting a fast and reliable thing again. :)
<Alpha1650> Thank you all for your help in answering my questions, especially Keybuk.
<acemo> I have this script http://pastebin.com/m23288b88 at /etc/init/spawn-fcgi-php but it seems to be not getting started. What am i doing wrong?
<plautrba> acemo: only 1 "start on" stanza is accepted and you have to rename your job file to have ".conf" suffix
<acemo> plautrba: thanks, that did the trick =)
<acemo> would using exec sudo -u username echo 'bla' be the right way to let upstart run a program as a different user?
<plautrba> Keybuk: upstart-0.6.5.tar.bz2 is missing. It is intended?
<plautrba> *Is it
<Keybuk> yes
<plautrba> ok
<Keybuk> with libnih out, there's no real difference between the sizes
<Keybuk> and the .tar.gz is more widely accepted
<plautrba> just for sure
<sadmac2> plautrba: why did you update to 0.6.5? the cvsbot hasn't even created a libnih repo yet.
<plautrba> sadmac2: it's prepared in devel branch
<plautrba> f13 was branched out yesterday
<sadmac2> plautrba: we still can't build it anywhere
<plautrba> sadmac2: it's not tagged yet
<sadmac2> plautrba: meh. fair enough :) I think we'll want it for F13 too. Its a minor enough update that we can put it in.
<plautrba> sadmac2: sure, f13 is goal, but you can check it in devel branch now
<plautrba> sadmac2: i've figured out that libnih.spec really needs %define _filter_GLIBC_PRIVATE 1
<plautrba> but libtool-2.2.6a-rpath.patch is not necessary
<sadmac2> plautrba: I didn't pull either out since the package was reviewed. Figured I'd do it as I checked them in.
<sadmac2> plautrba: something about the move from /usr/lib{,64} to /lib{,64} must've made the libtool thing go away.
<plautrba> sadmac2: it's done with first sed in %build
<sadmac2> plautrba: ah yes.
<acemo> is using sudo and exec the correct way to run a service as a different user as root?
<notting> runuser should work
<acemo> okay lets see if this works ^.^
<acemo> hmm nope
<acemo> the part between script and end script.. can i put a normal shell script in there? like this.. http://pastebin.com/m39f4ca91 or is it a whole different sort of script?
<sadmac2> acemo: yep, its just pumped through /bin/sh
<acemo> aweasome!
<sadmac2> acemo: I would use daemonize=false though, so you don't need to mess with expect stanzas
<sadmac2> acemo: and I'd exec it
<sadmac2> acemo: and move the chmod to post-start
<acemo> ohh great, ubuntu doesn't has runuser
<acemo> http://pastebin.com/m4ab0da0 im so far now, i'll just have to find out how to get runuser on my ubuntu machine and am not sure about the post-stop part.. it has to be chmodded before someone gets to be able to see the site.
<notting> huh. it's part of our coreutils package
<notting> (our == fedora)
<acemo> notting: runuser seems to be a stripped down version of su
<mxey> is runuser a tool supposed to be used to switch to another user before starting a daemon?
<mxey> Or what is the way to achieve that?
<acemo> mxey: not sure.. am trying to mess around with it now
<mxey> I think there should be an option for this, like there are options for chdir, chroot and umask.
<mxey> btw, I really like the oom option.
<acemo> oom option?
<mxey> setting the OOM killer "priority". Helpful for setting critical services like SSH to oom never.
<acemo> i c
#upstart 2010-02-18
<stimble_> anyone know a way to list all the emitted events on a system?
<BuBU> Hi
<BuBU> I've a small question about upstart + scripts
<BuBU> how to disable the stop on error stuff ?
<BuBU> I mean ie in the script I'm doing I've things like rmmod mymodule
<BuBU> but if mymodule is not loaded... then the script stops :(
<ion> Itâs just sh, you could do e.g. ârmmod foo || :â
<ion> Also, âmodprobe -r fooâ doesnât fail if foo isnât loaded.
<BuBU> I've same problems with some stuff like PSB_EXISTS=$(lsmod|grep psb)
<BuBU> for some reasons this does not work if psb is not in the list
<BuBU> in fact behaviour in upstart is the same as when using set -e in a shell script
<BuBU> so how to disable the set -e ?
<BuBU> for upstart for my given script ?
<ion> Rather donât disable -e but write code that embraces the error handling you get from -e.
<BuBU> so things like that: PSB_EXISTS=$(lsmod|grep psb)
<BuBU> how can I write clean code ? :)
<ion> For instance, if lsmod | grep -q '^psb '; then ...; else ...; fi. But why are you doing that in the first place?
<BuBU> ok thx
<BuBU> in fact I'm adding poulsbo (intel GMA500) support for an ubuntu live cd
<BuBU> but I want to keep that livecd usable for non-GMA500 box
<BuBU> thx ion
<BuBU> will check soon
<ion> Kernel modules should generally be loaded by udev.
<BuBU> I know
<BuBU> the problem is that poulsbo sucks :)
<BuBU> it use a modified drm.ko
<BuBU> which does not work with regular devices
<BuBU> so needs to rename based on the board
#upstart 2010-02-20
<JohnA> is there any way of getting upstart to start/stop the ttys when a keyboard/mouse/monitor is attached to a headless machine (server)
<JohnA> TTYs is probably the wrong item, maybe a GUI (gnome, xfce ...)
#upstart 2011-02-14
<jab_doa> hi
<jab_doa> is there an easy way to do logging with upstart? eg. redirect all output of a script to a logfile or syslog?
<SpamapS> jab_doa: | logger
<jab_doa> ok
<jab_doa> thanks
<squig> im new to upstart is there a way to get  a script to run before another script  (I need to run a command before networking)
<SpamapS> start on starting X
<SpamapS> squig: so 'start on starting networking'
#upstart 2011-02-15
<RobSpectre> Hey gents - is it possible to run a task from a particular directory?
#upstart 2011-02-16
<Getty> hiho guys... aehm i'm new to upstart and i'm debian user, just to drop out the most important ;) i added a new .conf script (like i did before) but this script doesnt work... if i do initctl stop of it, it just hangs, and the initctl list says its running on a process id, but that one doesnt exist
<Getty> i can exec the command by myself on shell, so that seems ok
<Getty> i always need to reboot then its fixed...
<Getty> that scares me
<JanC> Getty: that sounds like you have a forking process and the current version of upstart can't "follow the forking"
<JanC> so basically it's looking at the wrong process
<Getty> its something that forks, yes, but the startup process stays (php-cgi spawns for fastcgi)
<Getty> expect fork is there
<Getty> my problem is actually that initctl "hangs" on a process that doesnt exist
<Getty> what he do?
<Getty> and what i could do beside reboot to fix that kind of stuff?
<JanC> what do you mean by "the startup process stays"?
<JanC> the process run from the job stays around?
<Getty> it doesnt fork and is gone, it starts and generates forks but still runs
<Getty> its even a foreground process
<JanC> then don't use "expect fork"
<Getty> oh
<Getty> i got that out of the example of someone who use it that way
<Getty> and i mean it works now... after reboot
<ion> Donât use âexpectâ if youâre not 100Â % sure of the forking behavior of the main process. That will confuse Upstart. Hereâs a nasty kluge to force Upstart back into a good state without rebooting: http://heh.fi/tmp/workaround-upstart-snafu
<Getty> ruby code is strange
<JanC> I was just going to find my bookmark for ion's "nast kludge"   ;)
<JanC> Getty: it just creates new processes until the expected PID exists again
<Getty> on stop?!
<Getty> of a non existing process?
<Getty> 8-)
<Getty> on start i'm with you, but not on stop
<JanC> once it exists again, you can "stop" it  ;-)
<Getty> ok ok... back to what happened
<Getty> i made the .conf script, reload-configuration, started it
<Getty> saw that it had a mistake
<Getty> stopped it => hang... ctrl-c worked, check ps list, its gone
<ion> A future release of Upstart will have better code to follow forks. The current implementation can be broken easily by telling Upstart to âexpectâ one thing and having the main process do something else.
<Getty> check initctl list, says stop/killed
<Getty> doing start => hang, ctrl-c worked, check ps list, nothing there
<Getty> initctl says start/running with a process id that doesnt exist
<Getty> initctl stop => hang
<ion> âworkaround-upstart-snafu Nâ where N is the pid from initctl status.
<ion> the nonexistent pid
<Getty> ah! cool ok
<Getty> i can life with workaround if i have some! :)
<Getty> upstart is really nice
<JanC> Getty: not using expect is a better solution if you don't really need it anyway
<Getty> JanC: oh i will check on this anyway, i just wanted that workaround for same situations
<JanC> yeah, it's probably more convenient than rebooting  ;)
<Getty> its a xen so i dont have THAT kind of a feeling to reboot
<Getty> also.... i guess as long as no productive site is running on it and i still "learn" with upstart, i guess its good to reboot anyway some more often
#upstart 2011-02-17
<SeveredCross> Hey folks, is there a way I can diagnose why pre-start isn't being executed?
<SeveredCross> I'm trying to execute an avahi-publish-service command, but I never see it run
<SeveredCross> Also, is there a way to pull in environment variables from an external source?
<SeveredCross> With an old-style init script, I'd just source some file from /etc/defaults or something
<SpamapS> SeveredCross: one trick I've used is to set 'console output' and then in the pre-start do 'set -x' as the first line.. that will show you everything the pre-start is doing.
<SeveredCross> I already figured it out, but thanks, I'll remember that next time
<SeveredCross> I was just being terminally stupid and forgot to quote a variable.
<SpamapS> :)
<SeveredCross> That's what I get for trying to write a job for an init daemon that I've never even looked at at near midnight.
<SeveredCross> Bloody last-minute ideas will be the death of me.
<SpamapS> I call that a "learning opportunity"
#upstart 2011-02-18
<Keybuk> ion: talking of which
<Keybuk> http://upstart.at/git/?p=scott/intendant.git;a=summary
<Keybuk> could do with some testing
<ion> woot
<Keybuk> basically you build it then do
<Keybuk> sudo ./intendant /some/binary
<ion> Iâll try it out after some sleep.
<Keybuk> in theory no binary should be able to escape it's supervision
<Keybuk> if you can try it with a few favorite daemons and mail me the logs, that'd be great
<Keybuk> (since it's also for comparing techniques)
<ion> Alright
<ion> Whatâs the clone URL? I guessed git://upstart.at/scott/intendant.git but it didnât work.
<SeveredCross> Hey folks, how do I set the working directory for an upstart job?
<Keybuk> SeveredCross: "chdir /working/directory" in the .conf file
<SeveredCross> Awesome, thanks.
<Keybuk> ion: oh, does it not say? git://upstart.at/git/scott/intendant.git iirc
<ion> That was my second guess and it didnât work either. :-)
<Keybuk> weird
<ion> When using gitweb, one can add a file named âcloneurlâ to the root of each repository and the UIâll display the contents.
<ion> That is, to the directory where âdescriptionâ goes as well.
<Keybuk> try now
<ion> Works with /git/. (Btw, isnât that redundant? When iâve been running git-daemon i havenât had the need to include that part in the clone URLs.)
<Keybuk> yeah, turns out that is an option to git-daemon
<Keybuk> I added --base-path=/git --base-path-relaxed now
<Keybuk> so both should work
<ion> Yeah, works now.
<twb> My new upstart job needs DNS resolution to be working before it starts
<twb> What even do I "start on" ?
<twb> (This is on lucid, btw)
<SeveredCross> Probably networking.
<SeveredCross> I don't think there's anything more fine-grained that would get you /just/ DNS.
<twb> It didn't like networking
<twb> Based on what vsftpd does, I used start on (runlevel [2345] and net-device-up IFACE!=lo)
<twb> Which assumes there's only one iface, but that's OK for my immediate purposes
<twb> IIUC networking didn't work because the *network* comes up before dhclient finishes setting up the configuration recommended to it by the dhcp server
<twb> UGH, but now it isn't terminating properly
<twb> http://paste.debian.net/108031/
<twb> http://paste.debian.net/108033/
<JanC> you could probably hook into a dhclient exit hook
<JanC> or something like that
<twb> there should already be one...
<twb> at least, i thought there was
<twb> although come to think of it, the host I'm testing is in the dmz and has static networking
<twb> For some reason http://paste.debian.net/108034/ STARTS UP again during the shutdown sequence
<twb> I give up.  I'll just remove the options where collectd needs name resolution, and hard-code IPs into its config file.
<SeveredCross> Ooh, I've been looking at deploying collectd, heh.
<SeveredCross> I want to collect data from my machines, relay it to a central box, and make pretty graphs so I can monitor them.
<twb> That's what I'm doing
<twb> I'll pastebin my build scripts for you
<SeveredCross> Oh, nice, thanks.
<twb> http://paste.debian.net/108035/ is the hub
<twb> http://paste.debian.net/108036/ is the spokes
<twb> (Note that the hub starts out as a spoke, so really hub = spoke+hub)
<twb> And the latter should also have an "apt-get install collectd-core" at the top.
<SeveredCross> Nice.
<SeveredCross> Thanks, I'll save those and work off of them.
<twb> Unless your spokes are all LXC containers, you'll want to add in a bunch more plugins, e.g. disk and network
<SeveredCross> Yeah, disk, network, processes and memory is what I want to monitor.
<twb> Ref. http://collectd.org/wiki/index.php/Table_of_Plugins
<SeveredCross> I'll probably end up graphing the data with rrdtool on the hub side, but I'm not sure how that's going to work
<twb> In my code, you can see I use collection3 to do it
<SeveredCross> Yeah.
<twb> It's at least as nice as munin
<SeveredCross> I'm just unsure if it's going to display all the machines in the same graph for any particular data type, or if it'll do graph per spoke per data type.
<twb> SeveredCross: collection3 can display all hosts for a graph, or all graphs for a host (or subsets of either).
<SeveredCross> Nice.
<twb> Each image is of one datum of one host
<SeveredCross> I'd like to at some point write a frontend for it that I can just connect to a connectd hub and it'll get the data, but that's far off in the future.
<twb> SeveredCross: well, collection3 does that -- you connect by way of HTTP :-P
<SeveredCross> :P
<twb> Or you could rsync the rrd databases over ssh, I guess
<SeveredCross> Yeah, but that's failure-prone, and why do that when I have a fancy VPN connection to the hub. :>
<twb> What, you run rsh instead of ssh on your private networks?
<SeveredCross> Heh, no, but I'm forgetful, and I don't like having lots of files on my home machine that I don't need.
<ion> keybuk: Youâve got mail.
<Keybuk> ion: thx!
#upstart 2011-02-19
<J13R> hi
<J13R> I'm not familiar with upstart, but it'd be cool if dependencies were a graph so that services/events can be processed in parallel. Any plans for that?
<J13R> Also, wouldn't it be better to store declarative descriptions of services how to get from a stopped to a running state and vice versa, rather than repetitive scripts? (DRY)
#upstart 2011-02-20
<micheil> in upstart 0.6.5-6, is the "pid file PATH" stanza supported?
<micheil> taken from: http://upstart.ubuntu.com/wiki/Stanzas?highlight=%28%28CategoryDoc%29%29#pid
<JanC> micheil: all stanzas support by the version you use should be documented in init(5) on your system
<micheil> okay
#upstart 2012-02-14
<bradleyayers> can anyone explain the wait-for-started job to me?
<bradleyayers> or wait-for-state
<MrBusiness> Hello
<RAOF> Aloha
<MrBusiness> Hello RAOF. I guess I should make another stab at explaining my issue?
<RAOF> Yeah.  What do you have, and what behaviour are you required to implement :)
<MrBusiness> Well, let's see.
<MrBusiness> My difficulty is that I have three java programs that need to run in parallel on separate JVMs. I've been asked to set these up to run as services, and I have been given some old scripts to adapt to the purpose. However, I am also aware that these Java programs may end up having multiple instances in simultaneous execution on a given system, with only minor configuration variations between them.
<MrBusiness> Hm
<MrBusiness> Something ate my text.
<MrBusiness> My difficulty is that I have three java programs that need to run in parallel on separate JVMs. I've been asked to set these up to run as services, 
<MrBusiness> that was the beginning that got eaten there
<RAOF> Ok.
<MrBusiness> Now, what makes this tricky is this:
<MrBusiness> Most services I see, from the looks of them, only ever have a single instance of a given binary running at any point in tiem
<MrBusiness> whereas this system might have a very large number of very similar processes running at the same time, but I presume, from the description of the requirements that I was provided with, that each instance needs to be an individual service that can be interacted with on a standalone basis.
<MrBusiness> The question is, where and how do I get that differentiation?
<MrBusiness> Would I need to concoct a scheme to register each service with a slightly different name from all the others?
<MrBusiness> I would need some kind of reliable method of managing all those PIDs
<RAOF> So, I guess one question is: what interaction requirements do you have?
<RAOF> I still don't have a good idea of what you need the end result to look like.
<MrBusiness> I need to be able to stop, start, restart, and possibly status the services.
<MrBusiness> Those seem, from my own admittedly limited conception, to be about the only things these processes are capable of anyway. 
<RAOF> Ok.  And this is logically more than one service?  ie: would you ever want to stop component A without stopping component B?
<MrBusiness> The only interactive elements to any of them are in defining config-files for them to load at startup. Beyond that, they run on automatic. 
<MrBusiness> I believe so, yes.
<MrBusiness> Let me see if I can muster a useful description
<MrBusiness> There are three main applications that form this little pipeline of programs. If all three of these "parts" are running, then we have what could be viewed as a complete "pipeline" (though not in the '|' shell metacharacter sense). 
<MrBusiness> But, we might have more than one of these "pipelines" executing at the same time, and we would potentially want to be able to stop any one of the three constituent programs in any of our pipelines using the service interface.
<RAOF> This is starting to sound like you've been asked to square the circle; this doesn't sound like a service interface.
<MrBusiness> That could very well be
<MrBusiness> I wasn't kidding when I said that updating my resume might be more productive. The deadlines for this project are absurd. Still, figured I'd give it the good ol' college try. 
<MrBusiness> If nothing else, at least now I know a bit more about the varieties and variations on different approaches to GNU/Linux services. That counts for something.
<MrBusiness> Anyway, I appreciate your help RAOF.
<MrBusiness> Thanks much!
<RAOF> Sorry for not being *actually* helpful :)
<MrBusiness> Ah, no, you've given me plenty to chew on. If nothing else, I now see that the path ahead is not quite so clear-cut as I had been led to believe. This is my first time dealing with the definition of services, and I have a suspicion that my tech lead is imagining this services business as being a lot simpler in concept than it is in reality. 
<RAOF> It's just not clear to me *why* you've been asked to do this; if I knew that it might be possible for me to point you in the right direction, or at least at people who'd be able to point you in the right direction.
<MrBusiness> Well, I gotta say, after all this chatting, it's not entirely clear to me why we're doing it that way either. I guess my lead wants us to have granular control over the processes without having to explicitly log in and start them from some user name. I guess we want init to start them, but to permit human interaction with them when and where necessary.
<RAOF> So, it's (also) not clear to me how the many instances happen.  Do you have three primary processes (let's call them A, B, and C) which may or may not spawn a bunch of subprocesses, or do have a bunch of independent (Aâ, Bâ, Câ), (Aâ, Bâ, Câ) sets?
<MrBusiness> the latter, they're all independent processes
<RAOF> How do they get started?
<MrBusiness> a call to the JVM, providing it with the name of a specific .jar file and feeding it a specific configuration file
<RAOF> You could servicify each set there.  Do you always want all of the sets running?
<MrBusiness> Well, not sure, but I don't think so. I think I actually want to be able to stop or restart each piece of a given set.
<MrBusiness> i.e. A1 stop
<MrBusiness> not sure how to do those fancy subscript numbers you did.
<RAOF> Heh :)
<RAOF> <compose>_1 :)
<RAOF> So, I guess a final question is: is there a deterministic set of processes you might possibly want to be servicifying?
<MrBusiness> Deterministic in what sense?
<RAOF> ie: is there a set 1, set 2, set 3, â¦ set n, with a fixed n.
<MrBusiness> n would be contingent on the needs of the people deploying it. 
<RAOF> Then they need to write the service scripts themselves.
<MrBusiness> ah
<RAOF> Because, as far as I'm aware, in *all* implementations of unix-y services, the services are singletons.
<MrBusiness> Ok.
<MrBusiness> Makes sense to me.
<MrBusiness> Hey, "singleton"! Excellent word, and almost certainly the piece of nomenclature I've been missing in my descriptions.
<RAOF> They don't have to be single processes, just logically be a single entity.
<RAOF> So either each A_n, B_n, C_n could be a service, or each set (A_n, B_n, C_n) could be a service, but there's no way to address "Component A, 3rd instance".
<MrBusiness> Yeah. There would have to be some kind of trick to the way each service gets registered in the system, and even then, it would fall to the user to be able to distinguish which service was the one he or she wanted to interact with.
<RAOF> Right.
<MrBusiness> Guess I'll just have to go chat with The Man tomorrow to see what he thinks. I'm just the disposable hatchet man here, and if he wants to make assumptions about the thing, I'll swing that hatchet far, wide, and fast. 
<RAOF> And because services are registered as configuration files, that would mean you'd need to be dynamically generating configuration files and messing with them.
<MrBusiness> Yeah
<RAOF> I guess you *could* have a single meta-service with a conf-file consisting basically of "the services are A_n, B_n, C_n", but that doesn't support "Stop Aâ"
<MrBusiness> Yeah, it's a thought, though probably not feasible within the present time constraints. 
<MrBusiness> I'll figure something out. 
<jyfl987> http://pastebin.com/m0JeStkP   anyone can tell me why this configure dont work?? http://pastebin.com/m0JeStkP
<bradleyayers> jyfl987: remove script and end script
<bradleyayers> setuid and setgid are supported as job commands
<bradleyayers> unless you're using upstart < 1.4
<jyfl987> bradleyayers: then how can i achive my goal? i need to use postgres:postgres to run the command to start postgresql
<bradleyayers> just use su
<jyfl987> bradleyayers: can you show me an full example ?
<jyfl987> i am just a newbie to this, and have lots of question to ask you guys here
<ion> Iâve had more success sudo. âexec sudo -H -u user -- command [args]â. su seems to like to leave an extra process hanging around.
<jyfl987> 1, which shell interpreter does the commands in script sections use?
<ion> /bin/sh
<ion> With set -e
<jyfl987> 2, will cd xxx works? cause i see chdir command in the upstart's cookbook
<ion> It does.
<jyfl987> 3, is it need update-initramfs or other updating job after i have changed some job's configure , i mean in ubuntu (i use ubuntu lucid )
<jyfl987> 4, how to simplely let some job dont start at the bootup but i still can start it by manual via initctl start job_xx
<jyfl987> 5, i see 'emits event-name' and 'initctl emit event-name' in my ubuntu's /etc/init/ directory, is there any differences between them?
<jyfl987> 6, if i dont use 'exec somecommand' to run some job but just 'somecommand', will the job freeze at that point?
<jyfl987> 7, how to got the job log when starting or ending ?
<SpamapS> ion: start-stop-daemon is even better than su
<SpamapS> ion: or sudo
<SpamapS> ion: mostly because it is made for this purpose. su keeps a pam session open, thats where you see the extra process.
<SpamapS> jyfl987: have you read the upstart cookbook yet?
<jyfl987> am reading it
<ion> Iâve found sudo to be better than start-stop-daemon, too.
<ion> In my Upstart jobs, that is.
<SpamapS> ion: could you explain why? It might be worth noting your findings in the cookbook
<SpamapS> jyfl987: emits is just a form of documentation, which helps init-checkconf and the upstart visualization tool determine how events affect the system.
<SpamapS> jyfl987: exec is recommended, otherwise you'll leave behind the shell process .. which you probably don't want or need.
<SpamapS> jyfl987: job output logging is a new feature in Upstart 1.4, which is in Ubuntu precise (soon to be 12.04). If you want to log a job's output,    exec foo 2>&1 | logger -t foo 
<SpamapS> jyfl987: oh and no, you don't need to re-run update-initramfs. upstart is started after the initramfs is done mounting the root filesystem.
<jyfl987> SpamapS: so precise will keep on using upstart?
<SpamapS> jyfl987: yes. as yet, nobody has successfully argues that "everythingd" a.k.a. systemd should be the pid 1.
<SpamapS> s/argues/argued/
 * SpamapS wishes irc worked like skype and would fix those mistakes. ;)
<jyfl987> SpamapS: ok this encourage me to learn upstart
<jyfl987> i dont want to found that they change to a new system after i finally learnt the old one
<jyfl987> SpamapS: so the problem is i change the setuid and setgid command to su postgres and start the job again, still faild
<ion> spamaps: Iâm not sure you can turn off start-stop-daemonâs functionality that checks for the existence of the process based on certain matchers (such as the /proc/*/exe). That overlaps (and perhaps interferes) considerably with what Upstart is supposed to do, and certainly implements it in a worse way.
<ion> For instance,
<ion> # start-stop-daemon --exec /bin/sleep --start -- 5 &
<ion> # start-stop-daemon --exec /bin/sleep --start -- 5 
<ion> /bin/sleep already running.
<SpamapS> ion: perhaps we need --startas instead of --exec ...
<ion> start-stop-daemon: need at least one of --exec, --pidfile, --user or --name
<SpamapS> $ start-stop-daemon --startas /bin/sleep --name first --start -- 5 & start-stop-daemon --startas /bin/sleep --name second --start -- 5
<SpamapS> that works
<jyfl987> hi guys can you just came back to my question?
<ion> That feels like quite a bit of a kluge when you could just use sudo. :-)
<jyfl987> my blog are waiting for my configure file :]
<SpamapS> ion: perhaps.. either way, the cookbook needs a note about multi-instance jobs
<SpamapS> jyfl987: first off, why are you converting postgres to an upstart job?
<jyfl987> SpamapS: because i am using a vps with debian5 installed and my target version of pgsql seems not in the repo
<jyfl987> so i need to install it by manual
<SpamapS> debian5 huh?
<jyfl987> yep
<jyfl987> sorry its ubuntu 10.04
<jyfl987> lts
<SpamapS> ah
<SpamapS> ok well you could just backport the packages from precise
<jyfl987> i want to learn the knowledge because my blog service is written by me using python, it wont be in repo, so i still need to configure a job by myself
<SpamapS> jyfl987: ok cool. well in this case, perhaps you need to tell postgresql to run in the foreground
<SpamapS> jyfl987: it probably becomes a daemon by default
<jyfl987> SpamapS: i run the command by mannual which will run in foreground 
<SpamapS> jyfl987: | logger or > logfile and you will get some idea of why it is failing.
<jyfl987> but when i put the command in postgresql.conf at /etc/init and then trying initctl start postgresql , it told me the service has been start/running and told me the thread id, but it seems be killed after that
<jyfl987> i dont know why
<jyfl987> and no log output 
<jyfl987> wait i will show you the current configure file
<jyfl987> http://pastebin.com/y8RtTKSZ  SpamapS here it is
<ion> âsu postgresâ doesnât do anything without a terminal.
<ion> The cookbook tells you how to log the output from the job. Add something like the following just after âscriptâ: 
<ion>   exec >/tmp/myjob.out 2>&1
<ion>   set -x
<jyfl987> ion: then you are recommend me to use sudo -u postgres exec xxx?
<ion> No. exec sudo -H -u postgres -- xxx
<jyfl987> ion: will try that
<jyfl987> ion: why i need to add set -x?
<ion> Itâs useful for debugging.
<jyfl987> after sudo exec .. ?
<ion> Please read what we say more carefully. :-P I said âjust after âscriptââ.
<ion> Where did this âsudo execâ come from?
<jyfl987> its in the scritp ... end script section
<jyfl987> ok i got the error output
<jyfl987> sudo: exec: command not found
<ion> Please read the command i wrote again.
<jyfl987> oops
<jyfl987> ion: finally it works
<jyfl987> ion: so how to add a stop script for it?
<ion> If the postgresql binary handles SIGTERM properly, you shouldnât need to.
<jyfl987> current i can only use initctl start postgresql to start the service but when i using initctl stop postgresql it told me unknow instance
<ion> Then it has probably died already. Youâll need to investigate why.
<jyfl987> got it, the original thread seems died, and a new thread started, i can see this from ps aux
<ion> See if you can disable that behavior.
<SpamapS> probably daemonized itself
<SpamapS> you *might* be able to use expect fork
<SpamapS> but make sure it actually forks first. :)
<jyfl987> hmm, i think i dont need that this time, so after my postgresql service started there should be a started-postgresql events?
<SpamapS> jyfl987: started JOB=postgresql  ... 
<jyfl987> ok
<jyfl987> will change the configure file
<jyfl987> SpamapS: export fork works , thanks so much and to you ion
<jyfl987> SpamapS: but chdir seems not works
<jyfl987> ion: is it that sudo wont use the currrent work directory ?
<ion> # cd /var; sudo -H -u ion pwd
<ion> /var
<jyfl987> ok
<evinrude> does upstart take care of stopping daemons? As is in, I do not have to track a PID "echo $$" so that I can kill it later on?
<SpamapS> evinrude: it can do that yes
<SpamapS> evinrude: Some daemons it has a hard time keeping track of
<evinrude> great, thanks. Off to the wiki I go
<SpamapS> evinrude: I'd recommend http://upstart.ubuntu.com/cookbook/
<evinrude> SpamapS: thanks!
<frem> I wrote this upstart job which sources a python virtualenv and starts a webserver. http://paste.pocoo.org/show/551041/
<frem> but when I run it, it's shown with the stop/waiting status?
<njin> just installed 1.4, no problem. thanks
<cande> hi, i'm trying to make a upstart .conf script, but when running "sudo start myservice" from commandline, it just freezes
<RAOF> cande: You probably want to (at least) pastebin your conf file so people can inspect it.
<cande> ok, http://pastebin.com/qWAY15eh
<cande> RAOF, make any sense?
<RAOF> Hm.  That looks to me like it should work, but I'm not particularly expert.
<cande> ok thx
#upstart 2012-02-15
<cande> do i need  both files in /etc/init/job.conf  and  /etc/init.d/job ?
<RAOF> No
<Guest89327> when i ask upstart stop the process, does it send a signal to process or process group?
<ks07> Hey, is it possible to have upstart run a service as an underprivileged user?
<Guest89327> ks07: http://upstart.ubuntu.com/cookbook/#setuid ?
<ks07> Ah thanks xaka
#upstart 2012-02-16
<KettleCooked> I'm following tutorials on daemonizing my node.js webserver for my Debian distro - I'm at the point where I'm suggested to install upstart. I'm now guessing that would replace sysvinit and it says it'll be "potentially harmful". Is it safe to do an apt-get install upstart on a Debian Squeeze distro or should I expect hell to break loose?
<JanC> KettleCooked: I have never tried upstart in Debian (you could try in a virtual machine if you want)
<JanC> but I suppose that warning is mostly there because it's not well-tested in Debian yet
<KettleCooked> gonna try virtual
#upstart 2012-02-17
<kklimonda> does upstart provide support for something similar to systemd instances?
#upstart 2012-02-18
<habermann24> i want to use user jobs. i have one in ~/.init/ ...
<habermann24> but start hello_world does not find the job
<habermann24> using ubuntu 11.10 .. any ideas?
<JanC> kklimonda: does http://upstart.ubuntu.com/cookbook/#instance answer your question?
#upstart 2013-02-11
<xnox> stgraber: =) i should sleep ;-)
<stgraber> xnox: yeah because it soon won't be worth it anymore ;)
<phroddon> Hi
<phroddon> Does anyone know if it's possible to interrupt properly a job which is still in a pre-start section ?
<phroddon> example: My pre-start script spawns a little script S which might take a long time to returns. If during this time, I issue a "stop" command on my job, this stop command is handled only once the pre-start has finished.
<phroddon> like this: http://www.pastebin.ca/2312825
<phroddon> (a stop command occuring during the sleep command will not stop the job immediately)
<SpamapS> phroddon: that may be a corner case that could be handled better.. but I'd say that pre-starts should probably be small and predictable
<phroddon> SpamapS: thx for your answer, yes the cookbook gives the same advice "...but any heavy logic is discouraged," :)
<stgraber> not again... my build of the current trunk is hanging on a test but isn't on my laptop... back to figuring out what's different between the buildd and my build environment
<stgraber> oh, at least it's not just me, the PPA daily builds are also hanging ;)
<stgraber> gah, I reverted commit by commit and don't see what causes the regression, so re-uploading the last known working to see if it hangs too...
<stgraber> if it does, then I'll need to do my usual 8.04 VM + lp buildd chroot and try to figure out what's going on here...
<stgraber> (or just ask for a non-virt PPA and avoid that madness)
<stgraber> slangasek: thanks for the blueprint update ;)
<slangasek> stgraber: sure ;)  I wasn't sure of the status of the other WIs, if you consider them done or not?
<stgraber> slangasek: I guess I can mark them done, they'll need some work once we get the environment bug fixed (where you can't do "start on started <something>" where <something> changes the environment)
 * slangasek nods
<stgraber> but that can go in the "get that stuff in the archive" work item :)
<stgraber> already, there goes any hope I had left to get a working upstart today... re-building the same upload I did on Thursday also hangs on the buildds so it's not something we broke in trunk since...
<stgraber> it's either some kind of race or something changed on the buildds (I must have tried around 15 builds this afternoon, so if it's a race, I was terribly lucky on Thursday)
<stgraber> now to setup my 8.04 VM and see if I can reproduce in a buildd chroot...
#upstart 2013-02-12
<derEremit> is this the right channel for upstart script questions or is this just for devel?
<xnox> derEremit: any upstart related questions and init discussions are welcome here =)
<derEremit> i've written an upstart script for a node.js app and my question is how do i start it when mongodb is up
<derEremit> thats the mongodb script
<derEremit> http://pastebin.com/Yj6W4UV0
<derEremit> somehow did not find an answer online
<xnox> in nodejs upstart job: start on started mongodb
<xnox> derEremit: did you see the cookbook http://upstart.ubuntu.com/cookbook/ ?
<derEremit> i just wasnt shure if the start on "name" should just match the "name.conf"
<derEremit> would this also work if mongodb didn't provide an upstart script of its own but a basic init.d ?
<xnox> derEremit: man 7 upstart-events
<xnox> derEremit: no, it would currently not work if mongodb was just a basic init.d script, but we are working on generating events from init.d scripts.
<derEremit> ok, thats all i need ( for now )
 * xnox should just convert mongodb to an upstart job ;-)
<derEremit> no, mongodb is already ok, had this question for some other project i cant remember atm
<derEremit> but thanks anyway
<xnox> no problem =)
<derEremit> It was redis-server which doesn't yet provide an upstart script
<stgraber> slangasek: so the reason why the upstart test hangs is because of some kind of change in the latest gcc upload. Reverting gcc makes the test pass.
<slangasek> phooey
<slangasek> stgraber: go harrass doko? :)
<stgraber> yeah, I poked him in #ubuntu-devel, hopefully he'll show up soon
<slangasek> ok
<stgraber> until then I'll go do something else because that's going to block my next PPA upload for user sessions :)
<stgraber> oh and I'll keep an eye on the buildds and cancel any upstart build I see on there. We already wasted around 40 hours of PPA buildd time since gcc was uploaded...
#upstart 2013-02-13
<aschmitz> Is it possible to change the permissions on a given job's log at /var/log/upstart/jobname.log?
<aschmitz> (Obviously, I can change it manually, but will that stay after rotation?)
<afournier> is it possible to disable a job by adding a file (ie /etc/init/myjob.disable) ?
<marrusl> afournier, create a file called /etc/init/myjob.override that contains the word "manual" on a line by itself.
<marrusl> afournier, see: http://upstart.ubuntu.com/cookbook/#disabling-a-job-from-automatically-starting 
<marrusl> (especially section 11.44.1)
<afournier> i knew i saw something like this before, just could not find it again
<afournier> thanks a lot
<stgraber> jodh: ping
<jodh> stgraber: hi
<stgraber> jodh: do you have some time to spend with doko to figure out the gcc problem?
<stgraber> jodh: he identified the broken patch and reverted it, but we need to get a smaller reproducer
<stgraber> jodh: https://bugs.launchpad.net/gcc/+bug/1123588
<jodh> stgraber: I'll see what I can do. Currently having problems with dbus and the test suite.
<stgraber> hey doko 
<doko> hanging around like upstart after failed tests? ;p
<jodh> doko: is the broken gcc on one of the porter boxes?
<stgraber> jodh: it was in the archive up until 3 hours ago, so you probably still have it on your machine (or you can grab it from Launchpad)
<doko> jodh, no, grap it from lp. I *think* that the cpp-4.7 and gcc-4.7 debs (cc1) should be enough to install
<doko> jodh, is upstart supposed to be built with -Os?
<jodh> doko: hmm. I thought that -Os was historically only used for libnih.
<jodh> doko: interestingly, I don't get the failure with that gcc version on i386 with ' -O0 -g -ggdb3 -fno-inline'. I'll try with -Os/-O2...
<doko> jodh, fails with -O2 too
<doko> jodh, stgraber said something about a dump probably failing. which code/file is this?
<stgraber> test_job is the failing test, but it's itself testing job.c and in this case some of the stateful re-exec bits
<stgraber> it looks like what's wrong is the saved state as the json reports the job as stopped while it's actually running
<stgraber> so my guess is that the bug affects one of the functions responsible for saving the state of the job
<doko> jodh, fails for me too with -fno-inline
<jodh> right - I can recreate it now.
<doko> jodh, stgraber: so, building all the test code with -O0 and all other files in init with -O2 lets the tests succeed
<stgraber> right, though that probably means that an actual stateful re-exec of init would fail similarly to what the test shows...
<doko> removing everything in this file except the two *deserialize* functions still shows the issue
<jodh> doko: I've got a minimal C program that uses all the macros and functions in that deserialise code and I cannot make it fail.
<doko> :-(
<jodh> doko: just thought I'd try a clang build but that explodes in flames with an internal compiler error too :(
<doko> jodh, well, -O0 works, but maybe that's not what you want for production
<jodh> doko: yeah, not ideal. Can we get some more context on the original problem from the gcc devs? I've looked at the gcc bug, but I don't understand much of it (the change was primarily for C++ right?)
<doko> jodh, I don't suspect so
<doko> jodh, so your reduced test case doesn't show the error at all?
<jodh> doko: sorry, no. It's here if you want to try it: http://paste.ubuntu.com/1645562/. I'll take another look tomorrow.
#upstart 2013-02-14
<afournier> nobody has problem updating busybox with opkg ?
<afournier> could be related http://trac.freesmartphone.org/ticket/43
<afournier> 5 years ago :p
<jodh> afournier: wrong channel?
<afournier> yes
<afournier> indeed
 * afournier needs more coffee
<afournier> thanks jodh
<afournier> btw, i am using upstart with busybox, and ntpd for instance, so i do something like "exec ntpd -l -p pool.ntp.org", ntpd then output to stdout, and everything ends in /var/log/upstart/myjob.log as expected. But is there a stanza to output that in syslog automaticaly, or do i have to append "| logger" ?
<afournier> i'd like every single output to go to the syslog...
<afournier> exec >/dev/kmsg 2>&1
<xnox> we don't have: "console log syslog" yet
<xnox> (e.g. log to both /var/log/upstart/myjob.log and syslog
<xnox> )
<tom0815> hello, I have one question: is it possible with upstart to check, if a network service is running on an other host and only a local server, if the other network service is available?
<xnox> tom0815: one can do anything in the pre-start script, see cookbook.
<xnox> tom0815: are you after failover? in that case look up hearbeat that can handover networked services for you like that
<tom0815> thank you, I have a two servers. on one a mysql db is running and i want to start a second service on the other service, but only, if mysql db is up and running on the first server
<tom0815> so it is no failover scenario
<xnox> tom0815: in pre-start script try to ping mysql port on the mysql host.
<xnox> tom0815: and stop if not available.
<xnox> tom0815: your job should start on when network is up
<xnox> tom0815: and you probably want a loop & timeout to try connecting to mysql later.
<xnox> and eventually give up
<xnox> tom0815: see cookbook for (linked in the topic) about pre-start scripts, and starting on when network is up.
<tom0815> thanx alot!  I will have a look to the cookbook
<afournier> that's funny i am working on the same thing (almost)
<afournier> and it's a pain :)
<afournier> i did something like mysqladmin --wait=10 ping
<afournier> and then i want to "initctl emit --no-wait database-up"
<afournier> is it ok to call "restart" from the post-start script in case something failed ?
<afournier> i know it could lead to infinite loop, but i don't know if it's possible
#upstart 2013-02-15
<afournier> hi! i am doing something like this "start on (postinsts-done or database-up DB=mydb)" is it possible to know which signal (postinsts-done or database-up) triggered the start ?
<xnox> afournier: well DB environment variable will be set in the later case.
<xnox> afournier: and in the former case UPSTART_EVENTS
<xnox> afournier: see http://upstart.ubuntu.com/cookbook/#standard-environment-variables
<afournier> ok
<xnox> afournier: just check upstart_events and see what it has in it.
<xnox> (all caps)
<afournier> yes
<afournier> thanks xnox
<afournier> i love upstart
<xnox> =)
<afournier> i don't understand the shutdown / reboot process
<afournier> i have a a bug in a job so a chain of process does not stop
<afournier> but then why upstart does not end up killing everything
<xnox> on ubuntu upstart does kill all processes it tracked
<xnox> if the pid of the job does not match the pid of processes it tracks upstart will not kill it
<xnox> later /etc/init.d/sendsigs kills everything
<afournier> so it means upstart does cannot track some of my jobs
<afournier> could be because of a wrong expect stanza...
<xnox> yes
<tgm4883> We've got an upstart job where occasionally the process doesn't respond to SIGTERM and continues to run. This for some reason allows multiple processes to start when starting the job again (like the upstart job lost track of the previous process). Is there something we can do with the upstart job (or something to troubleshoot) to see why it's allowing this? For reference, here is the upstart job  https://github.com/MythTV/packaging/blob/master/de
<tgm4883> b/debian/mythtv-backend.upstart
<tgm4883> https://github.com/MythTV/packaging/blob/master/deb/debian/mythtv-backend.upstart
<tgm4883> hmm, reading thought the expect parts of the cookbook. This is probably what I need. Is there a way to see what PID(s) upstart is currently tracking for a job? 
<xnox> tgm4883: the mythtvbackend should be prepended with exec
<xnox> e.g. see /etc/init/hwclock.conf
<tgm4883> xnox, and that will make it so upstart stops losing track of it?
<xnox> tgm4883: next please start it and check that the pid from $ status mythtv-backend matches pgrep mythbackend
<xnox> when only one mythbackend is running
<xnox> if they don't match then expect/fork stanza is not right
<xnox> if the pid is what you expect then
<xnox> we shall wait for jodh to wake up tomorrow to poke at it a little more
<tgm4883> xnox, what does the exec do?
<tgm4883> looking at the status PID (without using exec), it's not right 
<xnox> http://upstart.ubuntu.com/cookbook/#exec
<xnox> tgm4883: yeah, you are probably tracking the pid of "test -f" call ;-0
<tgm4883> root      1738  0.0  0.0   4400   612 ?        Ss   Feb08   0:00 /bin/sh -e /proc/self/fd/9
<tgm4883> that is what it's tracking
<tgm4883> actual PID is 1740
<tgm4883> I'm assuming because the upstart job calls a wrapper script
<tgm4883> and that expect isn't right
<xnox> tgm4883: kill everything, add expect, start up compare the pids
<tgm4883> xnox, can I just edit /etc/init/mythtv-backend.conf directly and restart?
<tgm4883> xnox, or do I need to notify upstart that some changes happened
<xnox> tgm4883: no just edit and it will pick it up (it uses inotify)
<xnox> if you are on overlayfs then you could use: initctl reload-configuration
<xnox> to be sure.
<xnox> tgm4883: if the pids are still 2 digits apart you need expect daemon
<xnox> if one, than it's a fork
<xnox> expect fork
<tgm4883> with exec, looks like it's tracking the right PID
<tgm4883> with using expect at all
<xnox> if the pids are correct
<xnox> than it all should just work ;-)
<tgm4883> :)
<tgm4883> xnox, have time for one more question?
<xnox> and sigterm is now send to the correct process =)
<xnox> tgm4883: go on.
<tgm4883> xnox, does upstart allow for conditional "start-on" yet? eg. we'd like to do "start on mysql started" but only if mysql is installed
<tgm4883> I know it was on the roadmap at one point, but I haven't heard anything about it
<xnox> start on mysql or (other)
<xnox> in pre-start:
<xnox> check for status mysql and if job is unknown continue, if known and not started yet abort starting just yet.
<tgm4883> ok, so we can do that in the pre-start script?
<tgm4883> sorry, trying to find this in the cookbook
<xnox> start on mysql or (other-events)
<xnox> pre-start script
<xnox> ! status mysql || { stop; exit 0; }
<xnox> end script
<xnox> start on started mysql or (other-events) that is
<tgm4883> ok, and that will still work even if mysql isn't installed? 
<tgm4883> I misunderstood the cookbook then
<tgm4883> (or it changed since we first made the upstart job)
<SpamapS> tgm4883: start on started mysql will not give any errors
<SpamapS> tgm4883: if nothing ever emits 'started mysql' ...
<SpamapS> tgm4883: then the job never starts
<tgm4883> SpamapS, well that's not really what we want
<tgm4883> SpamapS, we'd like to start on mysql started, but account for instances where mysql isn't installed (eg. it's installed on another machine)
<SpamapS> tgm4883: hang on...
<SpamapS> tgm4883: how is upstart supposed to know about other hosts?
<tgm4883> SpamapS, it's not. It's just suppose to assume that "If mysql is install, wait for mysql to start. Otherwise continue starting on other deps"
<SpamapS> tgm4883: if you can handle that case in your app, just start whenever
<SpamapS> no guarantees the other host will be up ever
<tgm4883> SpamapS, it's not my app. we just package it
<tgm4883> SpamapS, I'm just trying to make this the easiest on users without causing loads of work for them to do
<SpamapS> tgm4883: perhaps your pre-start should spin polling for the mysql server
<tgm4883> SpamapS, currently it does. We look for /usr/sbin/mysqld
<tgm4883> SpamapS, but that is a really crappy way to do it
<SpamapS> no no no
<SpamapS> the port
<SpamapS> wherever it is
<SpamapS> while ! mysqladmin ping ...
#upstart 2013-02-16
<xnox> stgraber: so I'm running upstart based session now and there are job conflicts.
<xnox> so there is upstart job to start gnome-settings-daemon but also gnome-session is starting an ubuntu session, which in turn also start gnome-settings-daemon
<xnox> thus it means that if upstart job is quicker all is good, or in my case
<xnox> where gnome-session was quicker, the upstart job was stuck respawning because the upstart's g-s-d could claim the bus name.
<stgraber> xnox: right, in that case you need to kill the one from gnome-session
<xnox> yeah. I see how you did not want to use a new session name (since nothing would work out of the box)
<stgraber> xnox: I had that race happen less than 10% of the time here and there's little we can do in upstart until the desktop team changes gnome-session to stop spawning it
<stgraber> yeah, I actually use a different name to trigger the upstart support, but then change the name half-way through xsession to avoid everything blowing up
<stgraber> so yeah, no easy way not to have g-s-d spawned
<stgraber> and g-s-d doesn't have a --replace or --kill option so I can get the upstart job to cleanup any existing one (as I'm doing with pulseaudio IIRC)
<xnox> stgraber: well... =) we can prepend a new XDG_CONFIG_DIRS pointing to /usr/share/upstart/xdg/ where we drop the xdg overrides which disable things that are spawned by gnome-session
<xnox> and then gnome-session will notice the override and not spawn g-s-d, yet we keep the same session name
 * xnox takes a task to implement this
<stgraber> ah, I guess that'd be a reasonable workaround until we switch to upstart for good
<xnox> and thus not requiring any changes in the packages.
<xnox> stgraber: where is your integration branch?
<xnox> also thanks to dbus logs I have a bug to file....
<stgraber> xnox: lp:~stgraber/ubuntu/raring/upstart/user-session-ppa
<xnox> ok.
<xnox> hmm..... ssh agent didn't work...
 * xnox tries to find what sets XDG_CONFIG_DIRS and if gnome-session sets & overrides it, I may have a problem in my cunning plan
<xnox> /etc/X11/Xsession.d/ awesome =)
<xnox> well I'm currently abusing /usr/share/upstart/xdg for disabling. But I guess going in the future we will want /etc/xdg/xdg-ubuntu-upstart/ to be added to the XDG_CONFIG_DIRS such that one can ship both
<xnox> an upstart job and a .desktop autostart file
<xnox> and then only the upstart job will be used during upstart session.
<xnox> not sure how/if/when other DE will switch to upstart managed sessions.
<xnox> compiz is being respun by gnome-session
<xnox> but i'm not sure where/how we can override /usr/share/gnome/wm-properties/compiz.desktop
<xnox> also /me is very confused about hidding jobs it needs type,name,exec to validate plus hidden to hide.
<xnox> yet xdg autostart spec says only hidden key + matching file name is needed
<xnox> *sigh
<xnox> *
<stgraber> I think we'll really need to spend a bit more time and just get rid of gnome-session earlier than first planned. That's the only way to get all the benefits and avoid the weird issues we're seeing currently
<stgraber> I tried that the other day but for some reason I wasn't getting any of the indicators
<stgraber> maybe the recent fixes for dbus and cwd improved that though. I'd have to try it again
<stgraber> oh, and we'd need to implement the gnome-session dbus APIs then (for session logout at least)
<xnox> hmm... and what about the rest of the /etc/xdg/autostart/*.desktop files shipped by the rest of the packages?
<xnox> (i'm hoping not that many)
<xnox> and other apps that self-drop ~/.config/autostart/ e.g. google-chrome, shutter and others/
<xnox> ?
<xnox> upstart-xdg-autostart-bridge?!
<stgraber> shouldn't be too hard to have an upstart job spawn those once and forget about them
<xnox> or crazy hacks to make gnome-session check up with upstart and not start stuff spawned by upstart or make it ask upstart for a list of things it knowns
<xnox> and only spawn the set that upstart didn't know about
<xnox> that assumes, of course, matching and stable job names.
#upstart 2014-02-10
<Mariano9> hi xnox, jodh I need to export an SSL cert before respawning the service..with a pre-start stanza it's not working
<Mariano9> like: pre-start script
<Mariano9> export SSL_CERT=/path
<Mariano9> echo 'text'
<Mariano9> end script
<xnox> Mariano9: things you export in pre-start, don't propagate elsewhere.
<xnox> Mariano9: you can use the global "export SSL_CERT=/path" upstart stanza, then it will be available during all stages (pre-start, script/exec, post-start, pre-stop, post-stop, etc.)
<Mariano9> ok..let's see
<Mariano9> is "env" stanza what you mean?
<Mariano9> xnox: 
<xnox> Mariano9: yeah, that. =) see $ man 5 init
<Mariano9> ok, tried with env and export stanzas, none worked for my example
<Mariano9> which user is the one who runs upstart scripts?
<Mariano9> xnox: could you take a look at my redis-commander upstart script? I'm trying to export an SSL cert path to allow twitter sensu hander to pick it up
<Mariano9> https://gist.github.com/Mariano-gon/8916561
<xnox> Mariano9: what's running on port 3030?
<xnox> Mariano9: redis-commander itself or something else?
<Mariano9> sensu client
<xnox> Mariano9: ok. looks all correct. But the environmet variable only affect redis-commander.
<Mariano9> i think you're right..(thinking here)..
<xnox> Mariano9: since nc, doesn't do ssl =)
<xnox> Mariano9: http://serverfault.com/questions/102032/connecting-to-https-with-netcat-nc
<xnox> Mariano9: if sensu client is expecting an ssl connection, use openssl s_client -connect instead.
<xnox> what's the certificate for? and who should be validating requests from where using it?
<Mariano9> the client-server sensu uses twitter.rb (a handler script) to connect with twitter API and send a direct message to X account
<Mariano9> this connection to the API needs SSL certification
<xnox> Mariano9: thus the certificate should be in the environment of the sensu process, and has nothing to do with the above job.
<Mariano9> yes, now I realize that
<Mariano9> :)
<Mariano9> sorry
<xnox> =)))
<Mariano9> other one
<Mariano9> any ideas about rackspace plugin script?
<Mariano9> that one does have an env issue
<xnox> re:bundler? didn't play with that yet.
<Mariano9> yeap
<Mariano9> ok
<Mariano9> let me know if there's something I can help you with
<TJ-> I have an upstart job stuck in "stop/killed" state (ureadhead-other), with init (pid 1) still holding an open log-file for the job, which I need closing (the log-file is inside a chroot that needs umount-ing). The job process no longer exists. How do I proceed?
<smallfoot-> Ubuntu and Canonical are missing the opportunity for upstart in Debian, much thanks to the CLA
<smallfoot-> The CLA is harming adoption of Upstart
<smallfoot-> Debian could have went with Upstart, but now they are going with systemd instead
#upstart 2014-02-11
<Mariano9> guys, anyone willing to check out my upstart script and see why it's respawning for ever? https://gist.github.com/Mariano-gon/8941403
<Mariano9> I think this is a env issue, as If I cd into newrelic plugin directory and run bundle exec .. it'll start without any issues; but it won't from anywhere else
<Mariano9> I tried with env and export stanzas, nothing seems to work
<ion> Do you see its output in /var/log/upstart?
<Mariano9> in /var/log/messages it's mine. https://gist.github.com/Mariano-gon/8941660
<Mariano9> and this same will repeat ten times and then stop
<ion> But do you see the *output* of the process in /var/log/upstart?
<Mariano9> there's no upstart folder in /var/log..
<ion> Ok. In that case you can add manual logging to the job.
<ion> script
<ion>   exec >>/tmp/blah.out 2>&1
<ion>   set -x
<ion>   exec sudo -u root â¦
<ion> end script
<Mariano9> but doesn't the first exec (or the second) shut off the other one?
<smallfoot-> Debian went with systemd
<smallfoot-> they could have went with upstart
<smallfoot-> and probably would have if it weren't for upstart being under CLA
<smallfoot-> you guys lost Debian thanks to your CLA
<smallfoot-> now if there were anyone using upstart they will migrate to systemd
<smallfoot-> if there was anyone who considered upstart, they wont anymore, now everyone will be adopting systemd
#upstart 2014-02-12
<annodomini> While trying to develop an upstart job, I seem to have gotten it in a wedged state. Is there any way to get out of this state?
<annodomini> $ initctl list | grep reprepro
<annodomini> reprepro-gpg start/killed, process 13361
<annodomini> But if I try to run start reprepro-gpg or stop reprepro gpg, it just hangs.
<annodomini> Here's my configuration: https://gist.github.com/lambda/8961767
<annodomini> This is upstart 1.5 on Ubuntu 12.04.4
#upstart 2014-02-13
<dhananjay_> Can anyone point me to a document which explain how process states are managed in upstart? Things like how do assert process is stoppped properly, it has started etc..
<dhananjay_> I read somewhere that systemd uses cgroups, but here http://upstart.ubuntu.com/wiki/Cgroup?highlight=%28%28CategorySpec%29%29 It states upstart does it diffrent.
<Mariano9> xnox: 
<marianogg9> xnox: 
#upstart 2014-02-14
<cantstanya> -_________________________________________________-
<marianogg9> hi guys ,xnox: jodh , can you help me out with this please? https://gist.github.com/marianogg9/9002995
#upstart 2015-02-09
<disappeared> so if i drag a config file into /etc/initâ¦.shouldnât i just be able to do service myservicename 
<disappeared> it says its unrecognizable
<disappeared> not drag, i meant copy
<disappeared> ok nm it does find it its just failing 
<switchflip> Has anyone on here, ever had to deal with upstart and unicorn.... I've encountering a perplexing error that I can't seem to get to the bottom of.
#upstart 2015-02-10
<hitlin37> it is not preferred choice to have grep cmd in pre-start? i don't want to put too much computation in pre-start, so a bit cautious.
#upstart 2015-02-12
<hitlin37> hi
<hitlin37> i havce a job which chehcks in pre-start for configuration.
<hitlin37> and then it starts on as  : start on started main_engine
<hitlin37> now main_engine sometimes gets killed and respwan. that makes my job to skip pre-start
<hitlin37> how to make a job to alwayys chehck pre-start
<hitlin37> Hi, would could be the reason when you reasnpn a message
 * hitlin37 killed by TERM signal
<hitlin37>  killed by TERM signal
<kline> hi, whats the best way to start an upstart job after one sysv service (postgres) but before another (server application). im under the impression that upstart itself for sysv services only emits per-run level events
<kline> really i want to do some database maintenance after postgres has started but before the application server starts
<hitlin37> kline: i use the cookbook for the help
<hitlin37> that's the only thing available.
<kline> hitlin37: i couldnt see anything clear in that
<kline> but thanks for the suggestion
<kline> for changing sysv scripts to upstart, is it as easy as wrapping them in script/end script and prepending their metadata?
<hitlin37> no idea on that
<hitlin37> i'm new to upstart as well
<hitlin37> how many suchscripts you have?
<hitlin37> if its not too many, may be you can re-wrtie
<kline> i have two scripts (postgres and appserver), but i dont understand them well enough to rewrite from scratch
<kline> i know systemd far better than sysv and upstart, but at work we use ubuntu for some of our servers, including this one
<kline> (testing machine that needs the database wiped and repopulated on reboot, hence the task im trying to insert)
#upstart 2015-02-13
<thor77> im running crazy... can someone give me a simple script to cd into a directory and then execute some command there that should run until "stop .." is run?
<dehrenberg> my boot seems to fail with init exiting with EAGAIN. Is there a way to enable extra logging from upstart or something like that?
<dehrenberg> never mind, got it, --debug
#upstart 2015-02-15
<pc__> is there a reason why upstart doesnt let me start a process that reads from console?
<pc__> i dont see the difference between that and serverSocket.listen()
#upstart 2016-02-15
<N3sh108> hello, I am trying to use upstart for my nodejs server but I just get 'stop/waiting'
<N3sh108> the upstart script should be correct though
<N3sh108> is that a normal behaviour?
<JanC> you get stop/waiting but your nodejs server is still running?
<JanC> in that case the upstart job config is not correct
<N3sh108> JanC: I went through it and fixed some problems. Now I get: start/running, process XXX
<N3sh108> but the server doesn't work
<N3sh108> any idea JanC?
<JanC> is the nodejs server actually running?
<N3sh108> nope
<JanC> seems like the nodejs server doesn't start then?
<JanC> or starts and immediately exits or crashes?
<N3sh108> let me check
<JanC> (I've never used nodejs, so no idea what might go wrong)
<N3sh108> ok
<N3sh108> if I run my exec command myself
<N3sh108> it works
<N3sh108> but now when I either start it manually (start <program>) or when I reboot the machine
<N3sh108> I don't get why
<N3sh108> arggg
<N3sh108> wasting so much time 
<JanC> a web search gave me this: https://github.com/cvee/node-upstart  (not sure if it's useful for you?)
<JanC> there are some other articles to be found too, which do things somewhat differently
<N3sh108> still nothing
<N3sh108> fuck this shit
<N3sh108> 1 hour wasted for nothing
<JanC> BTW: when it works when you run it from the command line, but not from an upstart job, then usually the problem is something in the "environment" that is different (e.g. environment variables, working directory, application expects a tty, etc.)
<JanC> also, permissions
<N3sh108> thanks, it magically fixed itself :D
<JanC> possibly it was still confused by a previous test without rebooting?
#upstart 2016-02-17
<supergonkas> Hey is there any opition to change like the working dir?
<supergonkas> like when executing a script set the place where is beeing executed
#upstart 2016-02-18
<cristian_c> hello
<cristian_c> I've to migrate a daemon from upstart/sysv to systemd
<cristian_c> I've read the migration documentation
<cristian_c> but I can't figure out how to fill the Type= and After= fields
<cristian_c> How can I get forking and dependency information?
<cristian_c> Any ideas?
#upstart 2016-02-19
<cristian_c> hi
<cristian_c> I've to migrate a daemon from upstart/sysv to systemd
<cristian_c> I've read the migration documentation but I can't figure out how to fill the Type= and After= fields
<cristian_c> How can I get forking and dependency information?
<cristian_c> Any ideas?
#upstart 2016-02-21
<skylite> why is this init script starts bitlbee in the name of bitlbee user? http://pastebin.com/nZX6Ucx3
#upstart 2017-02-14
<gaghiel> hey all 
<gaghiel> got a small prob with upstart
<gaghiel> have a weird pre-start script that wants to implement sth like a retry logik 
<gaghiel> much like: pre-start script: while(tries<max_tries) do; some command; done
<gaghiel> it pretty much works but the moment the command fails it bails out of the script completely
<gaghiel> is there a way to do sth like set +/-e ?
<hallyn> gaghiel: i don't think so, but you can do some version of "some_command || true"
<gaghiel> hallyn: :) nice one. I'll try that. 
<gaghiel> many thanks!
<hallyn> np
#upstart 2017-02-15
<Joel> I have SSL certs in /etc/ssl/private, which are owned by root:root, any way for logstash to read them before it drops privileges to the logstash user? 
<Joel> I'm wondering if there's someway I can get upstart to open the file handle before it drops privs?
<Joel> probably not, seemed worth a try though
<hallyn> not that i know of right now, but that seems like a useful extension (for when i fork upstart :)  akin to UPSTART_FDS
