[14:40] <crankharder> any way I can tell uptstart to load init files in a given path?
[14:40] <crankharder> include directive of sorts
[14:43] <jodh> crankharder: what are you trying to achieve?
[14:44] <crankharder> I want puppet to set up and configure a directory full of init scripts
[14:44] <crankharder> I can't tell puppet to monitor /etc/init since it'll delete all the scripts that puppet doesn't own
[14:44] <crankharder> so if upstart monitored /etc/init/puppet just like /etc/init we'd be good to go
[14:45] <jodh> crankharder: it does
[14:45] <crankharder> heh, hmm. ok
[14:47] <crankharder> jodh: https://gist.github.com/crankharder/4fa79577333ca89bd546
[14:47] <crankharder> seems to not like that job
[14:52] <jodh> crankharder: What does 'init-checkconf <your_job>.conf' say?
[14:52] <crankharder> $ init-checkconf resque_worker_50.conf
[14:52] <crankharder> ERROR: file resque_worker_50.conf does not exist
[14:53] <jodh> crankharder: init-checkconf '/full/path/to/file.conf'
[14:56] <jodh> crankharder: your 'start on' should probably be 'start on runlevel [2345]' and your 'stop on' is currently wrong (unless you are emitting shutdown somewhere?) - it should be 'stop on runlevel [016]'.
[14:57] <jodh> crankharder: also, you don't need to do your own redirection/logging - upstart does it for you, logging all job output to /var/log/upstart/.
[14:57] <crankharder> had some perms issue, but now getting this:
[14:57] <crankharder> $ init-checkconf /etc/init/puppet/resque_worker_50.conf
[14:57] <crankharder> ERROR: failed to ask Upstart to check conf file
[14:57] <crankharder> /etc/init/puppet is 755 and the job is 744
[14:57] <crankharder> jodh: okay, those points may be fair -- but I have the same job (named differently) in /etc/init and it works
[14:58] <jodh> crankharder: what distro version is this?
[14:58] <crankharder> ubuntu 12.04
[14:59] <crankharder> I get this too:
[14:59] <crankharder> $ init-checkconf /etc/init/resque_worker_5.conf
[14:59] <crankharder> ERROR: failed to ask Upstart to check conf file
[14:59] <crankharder> for someone not in my nested path
[14:59] <crankharder> something
[15:01] <jodh> crankharder: copy the file to /tmp, ensure it is readable, then 'init-checkconf -d /tmp/foo.conf'
[15:01] <jodh> crankharder: if that still fails, I'm guessing you have a dbus issue - is dbus-launch available?
[15:04] <jodh> crankharder: your job can also use use 'setuid', 'setgid' and 'chdir' to simplify the main job to be just 'exec rake environment resque:work QUEUE=high,medium RAILS_ENV=production PIDFILE=/tmp/resque_worker_50.pid VERBOSE=1' I think.
[15:04] <jodh> crankharder: putting pid files in /tmp looks like a bad idea btw.
[15:06] <crankharder> yea, looks like dbus issues.  no, i dont have dbus-launch
[15:08] <crankharder> output here: https://gist.github.com/crankharder/1ee1e4cb50cc8353a3e4
[15:09] <crankharder> it's a bit weird init-checkconf doesn't work on scripts in /etc/init/ ? -- maybe this isn't the issue?
[15:20] <jodh> crankharder: maybe copy your job to a desktop system with a full dbus setup and run init-checkconf on it there. I wonder if you have any odd whitespace chars in that job - copied from windows maybe?
[15:21] <crankharder> so, testing on a more barebones ubuntu 12.04 isntance I have w/o my jobs in it... installing dbus-launch now
[15:21] <jodh> crankharder: another way to check the job without using init-checkconf is: sudo initctl log-priority debug;sudo touch /etc/init/foo.conf;sudo initctl log-priority message' and look at dmesg/syslog to see what messages upstart logged about foo.
[15:22] <crankharder> $ init-checkconf /etc/init/acpid.conf                                                                                                         │
[15:22] <crankharder> ERROR: failed to ask Upstart to check conf file
[15:22] <crankharder> -d shows samee waiting for dbus issues
[15:23] <jodh> crankharder: init-checkconf won't work unless dbus-launch works.
[15:24] <crankharder> not really familliar with dbus launch.  define works?  https://gist.github.com/crankharder/53351450db600a15252e
[15:26] <jodh> crankharder: let's forget init-checkconf - I've run it locally and your job as pasted is syntactically valid. I've given you another option above to syntax check anyway. I suspect the job on disk is not the same as pasted - it might contain '\r's or similar if you copied it from a windows machine maybe?
[15:27] <jodh> crankharder: try the technique above and see what upstart logs to the system log - it will tell you where it is failing to parse the job.
[15:29] <crankharder> okay, so i'm on a new system w/o my custom jobs and I still see the same issue.  It's not related to my custom job.
[15:29] <jodh> crankharder: which issue? the job or init-checkconf?
[15:29] <crankharder> https://gist.github.com/crankharder/cd04bcb4d80259ba58d3
[15:29] <jodh> crankharder: 'sudo status puppet/tty6'
[15:30] <crankharder> okay, that works
[15:32] <crankharder> unfortunately I see nothing in syslog or dmesg from this:  sudo initctl log-priority debug ; sudo touch  /etc/init/cron.conf ; sudo initctl log-priority message
[15:33] <jodh> crankharder: are you by chance running in a cloud environment that uses overlayfs?
[15:33] <crankharder> i'm running ubuntu in virtualbox... but i dont know what overlayfs is, so certainly didn't set tha tup
[15:34] <jodh> crankharder: mount|grep overlay
[15:34] <crankharder> nothing
[15:38] <crankharder> okay, I removed all my custom init scripts and moved tty6 into this puppet path.  giving my unknown job puppet/tty6 on this system (but it worked on the more barebones one)
[15:38] <jodh> crankharder: this makes no sense - your system sounds broken.
[15:39] <crankharder> /etc/init/puppet is 755 on both /etc/init/puppet/tty6.conf is 744 on both
[15:41] <crankharder> I can test this at different points as puppet sets up the system to see if that helps
[15:43] <crankharder> jodh: thanks for the help -- i'll post here if I can track down what change makes this break
[15:44] <jodh> crankharder: you said this job runs fine in /etc/init/ but not in /etc/init/puppet/ - is that now resolved now you know how to start a job in a sub-directory?
[15:45] <crankharder> the subdirectory thing works on a box that hasn't had all my puppet configs (lots of them) applied.  same version of ubuntu tho
[15:45] <crankharder> so I assume it's something puppet is configuring that's causing this
[15:48] <jodh> crankharder: I'd certainly make sure you have a pristine server environment available (ie no puppet) for testing the basic upstart jobs.
[17:04] <CharlieSu> I've got an UpStart task that I'd like to print a message to the console once run..  'start job' should produce something on the users screen..  How can I do this?  I've tried 'console output' 
[17:07] <xnox> CharlieSu: at boot "console output" will print to the active console / boot messages. If run interractively it will print where-ever it's run.
[17:31] <CharlieSu> xnox: doesn't seem to be doing that for me..  I just use "logger 'message'" right?
[17:38] <xnox> CharlieSu: no, just do "echo"
[17:38] <xnox> logger will go whever logger logs, e.g. syslog.
[17:39] <xnox> CharlieSu: "console " stanza controls where job's stdout and stderr is redirected, default is "log" which is under /var/log/upstart/, "output" is to normal stdout and etc.
=== JanC_ is now known as JanC