=== Guest47292 is now known as jrib === marcoceppi_ is now known as marcoceppi [13:56] I'm having trouble with session upstart in Ubuntu. I've forked gnome-settings-daemon into unity-settings-daemon and copied the upstart scripts (http://paste.ubuntu.com/6885290/, http://paste.ubuntu.com/6885292/). However, if I try and run a session gnome-session doesn't start. If I disable the gnome-settings-daemon script it does. So it appears the gnome-settings-daemon script is confusing the startup somehow [13:56] Any ideas? [13:57] http://paste.ubuntu.com/6885304/ is the output of upstart on a failed case [13:57] It doesn't mention gnome-settings-daemon at all [14:03] robert_ancell: so unity-settings-daemon starts, but gnome-session does not. How are you disabling gnome-settings-daemon? Also, is there anything relevant in ~/.cache/upstart/*.log ? [14:04] jodh, I'm disabling by moving the .conf to .conf.disabled [14:04] jodh, I looked in the logs but couldn't see anything obvious [14:04] I can bundle them up [14:07] http://people.canonical.com/~bob/upstart-logs.tgz [14:08] robert_ancell: unity7.conf specifies 'start on xsession SESSION=ubuntu and started gnome-settings-daemon' so if you've disabled gnome-settings-daemon that won't start (unless you've changed other .confs?) [14:08] jodh, I have also changed that to do "or unity-settings-daemon" [14:08] no other changes [14:09] I notice there is a gnome-session log there, might be from a working boot though [14:09] I'll clear the logs and retry [14:09] robert_ancell: can you share the whole config for unity7 as that is likely the issue [14:10] http://paste.ubuntu.com/6885356/ [14:11] http://people.canonical.com/~bob/upstart-logs.tgz for the logs on a fresh run after clearing [14:11] http://people.canonical.com/~bob/upstart-logs2.tgz [14:11] I mean [14:11] robert_ancell: surely that won't work with gnome-settings-daemon though since gnome-settings-daemon specifies 'xsession SESSION!=ubuntu' but unity7 states 'xsession SESSION=ubuntu' [14:12] we are replacing gnome-settings-daemon with unity-settings-daemon in Unity [14:12] gnome-settings-daemon should only run for GNOME etc [14:17] robert_ancell: according to the logs, the unity7 job is stopping in pre-start and not starting compiz. Could that explain it? [14:17] jodh, I think compiz is started by gnome-session currently [14:18] so it's behaving correctly [14:19] my upstart logs in a normal session confirm that (compiz is in the gnome-session.log, unity7.log says "GNOME Session is starting Compiz") [14:21] fancy coming into the office? ;) [14:26] jodh, Is there any more information I can collect? [14:27] robert_ancell: jodh: the typical way to resolve this is to introduce an arbitrary sync event. [14:27] robert_ancell: jodh: e.g. both "unity-settings-daemon" and "gnome-settings-daemon" should do "initctl emit settings-daemon" [14:27] hi guys [14:27] robert_ancell: then unity7 would start on "xsession SESSION=ubuntu and settings-daemon" [14:28] ok, I can try that [14:28] robert_ancell: ditto e.g. gnome/other desktop environments can do something else. [14:28] is there a command to check if an upstart job is correctly formated? like init-chkconfig or some? [14:28] robert_ancell: that way one would also be free to use one of the alternative settings-daemons. [14:29] Mariano9: yes - I put it in the mail I sent you - init-checkconf /path/to/job.conf [14:29] robert_ancell: one would do "initctl emit settings-daemon" in e.g. post-start [14:29] hey jodh [14:29] cannot find the command [14:29] it isn't there, init-chkconfig [14:29] robert_ancell: this is similar to how "indicators-*" start events work, it's simply emitted when indicators should start loading. And e.g. unity, gtk3-old-style-applet and others emit it to indicate they are ready to receive indicators. [14:30] Mariano9: it's *init-checkconf* not chkconfig [14:30] sorry, init-chkconf [14:31] "init-checkconf /etc/init/newrelic_rs_dfw_1.conf, -bash: init-checkconf: command not found" [14:31] Mariano9: what version of upstart are you using? [14:32] Mariano9: run "initctl version" to establish that [14:32] 0.6.5 [14:33] Mariano9: excellent =) so pre-historic. Well, we can attempt to valide that job for you by reading it, if you can pass it on to us. e.g. paste.ubuntu.com [14:34] haha, just a sec [14:36] xnox, same problem after adding the settings-daemon event. Still works if disabling the gnome-settings-daemon config [14:37] https://gist.github.com/Mariano-gon/8845357 [14:37] xnox: there you go [14:37] (notice that I've commented some lines as I'm trying to get it goint) [14:37] Mariano9: you have 2 execs - only the last will be honoured. [14:37] robert_ancell: hm. Is there a ppa i can try, or is just by co-installing gnome & ubuntu settings daemon on my desktop? [14:37] *going [14:38] xnox, you should come to the office to help us debug those g-s-d u-s-d issues ;-) [14:38] seb128: you want me to comute into the office _now_ ?! =) [14:38] xnox, tomorrow morning would work as well ;-) [14:38] strike should be over then? [14:38] xnox, it's not in a PPA at the moment, but I could add it [14:39] Mariano9: assuming you are running ubuntu lucid, procenv won't be available unless you built it from source? [14:39] nope, centOS 6.x here [14:39] seb128: i use trains anyway, so i'm unaffected. Tube is mostly non-existent south of the river. [14:39] Mariano9: what output do you get if you run "sudo start newrelic"? [14:39] Mariano9: ah, ok :) [14:40] xnox, if you could come in tomorrow that would be great, we don't need to solve this immediately [14:40] "newrelic_rs_dfw_1 start/running, process 9251" [14:40] upstart sees it as running normal, but it isn't [14:40] Mariano9: that gist has two exec lines, i think last one wins. [14:40] Mariano9: the problem is the "bundle exec" was that that strange thing for ruby? [14:41] xnox: yes, it's a ruby gem [14:41] Mariano9: that will not work nicely as bundle forks many times and relies on environment variables and HOME, neither of which are set from within upstart [14:41] Mariano9: so you need probably need to do: [14:41] script [14:41] Mariano9: comment out the last line (exec bundle) so that upstart runs procenv first if you have that installed, then diff /tmp/procenv-job.log with a procenv log you take in an environment it does start in. [14:41] source /path/to/bundle.sh [14:42] Mariano9: as mentioned, it may well be env vars. I have no idea what newrelic is but does it need $HOME? [14:42] bundle exec /path/to/newrelic_rs [14:42] end script [14:42] Mariano9: and you probably want "expect daemon" [14:42] Mariano9: how is your bundle installed? user level or system-wide/root ? [14:43] xnox: jodh ok, does it matter the order of the stanzas? [14:43] Mariano9: you can't have duplicates, and at the moment I see two "exec" stanzas. [14:43] Mariano9: no, but as mentioned, if you specify a particular stanza multiple times, only the last is honoured. [14:43] Mariano9: it's declarative, not a top-down execution. [14:43] xnox: we've already covered that - see above [14:43] ah, sorry =) [14:43] right, i meant different stanzas order [14:44] but, it's declarative, so it doesn't matter the order [14:44] Mariano9: order of different stanzas does not matter, so you can have "start on" at the very end, etc. [14:44] great, let's try those [14:46] Mariano9: you should source bundle environment, it usually drops a script in /etc/profile.d or the users ~/.bashrc and the like. Not sure where it would live on a CentOS system however. [14:46] Mariano9: without that, bundle is mostly non-operational. [14:46] robert_ancell: ok, i'll be in tomorrow. [14:46] robert_ancell: and we can debug it. [14:46] xnox, thanks [14:46] xnox, thanks [14:50] xnox: i think it's the same folder as newrelic_rs [14:51] Mariano9: right, so try: [14:51] script [14:51] source that-file [14:51] bundle exec [14:51] end script [14:51] because if I cd into it and run bundle exec ./bin/newrelic_rs, it will start great [14:51] Mariano9: due to chdir stanza you should be in the right directory already. [14:52] Mariano9: right, but bundle probably installed a script somewhere for your shell to find and source whenever you enter that directory. [14:52] like a link? [14:52] Mariano9: so one needs that "magic" snippet in the same way. [14:52] Mariano9: it's been many years since I last used bundle =) [14:55] source stanza will give the right path to execute the bundle command right? [14:55] source is not a stanza, source is a sh built in command [14:56] things between script... end script are just normal shell lines executed. [14:56] and that would be replacing exec. [14:56] ok [14:56] so, again, bundle path is the same as newrelic_rs then [14:56] Mariano9: oh there appears to be an easy solution for bundler. [14:57] Mariano9: --deployement --binstubs options [14:57] Mariano9: http://askubuntu.com/questions/105761/how-to-get-a-bundler-enabled-rails-app-running-as-a-service-with-upstart [14:58] Mariano9: apart from changing start on/stop on to what you had, it should just work on centos. [14:58] let's see [15:01] two questions xnox: 1- what if the stop job hangs? [15:02] how do I actually stop the initctl job? [15:02] "init: Sending TERM signal to newrelic_rs_dfw_1 main process (11127)" and will stay there for ever [15:04] ctrl+c, now it is shown as stop/killed, process 11127 [15:05] Mariano9: after starting a job establish if the reality matches upstart expectations. [15:06] Mariano9: which is: process id in `ps' output matches, initctl status. [15:06] Mariano9: if they don't, the expect stanzas are wrong. more recent upstart handles such situations better. [15:06] Mariano9: but in your case you probably will need to add "expect fork", or "expect daemon". [15:07] Mariano9: and to clear the state, you'd need to re-exec upstart -> which actually for such old upstart means reboot. [15:07] Mariano9: i hope you are not developing this job on a production server. [15:08] host reboot? [15:10] there's no process in ps from newrelic, I've added the expect daemon before executing the start [15:10] is there a way to upgrade this version? [15:12] in fact there is [15:12] would it be good to upgrade it? [15:42] Mariano9: i don't know. I wonder if we should explore pushing a more up to date upstart into EPEL repositories. [15:56] xnox: so in order to clear the state I need to reboot the server? [15:59] Mariano9: or rename the job ;-) then the state is clear for the "new job name". [16:01] xnox: right, eventually I'll have tons of jobs :) [16:02] xnox: is there a way to reload configuration once a job script is edited? [16:04] Mariano9: the configuration is automatically reloaded, but it will only take affect on newly start instance. So e.g. after a proper $ stop foo, the next $ start foo will use new configuration. [16:04] ok [16:04] Mariano9: reload/restart is not enough, those keep on using the in-memory configuration. [19:39] Is it advisable to have an exec command that "blocks"…ie, that does not daemonize or return, but is instead long-running? [20:05] if you can, sure [23:04] mc_bluebeard: you can do that, in that case "your command is ~ running in foreground" and then you shouldn't specify any "expect ..." stanza.