[00:01] <flaf> Hi, I have a little problem with a upstart job in ubuntu trusty 1.12.1.
[00:02] <flaf> When I try to start a job (start ceph-osd id=x) I have an curious error, like if upstart tried to use an old version of the job.
[00:03] <AnrDaemon> pastebin the job file please.
[00:03] <flaf> ok.
[00:04] <flaf> AnrDaemon: http://paste.alacon.org/41419
[00:04] <flaf> AnrDaemon: the point is in /var/log/upstart/ceph-osd-ceph_13.log where I have “/proc/self/fd/9: 8: /proc/self/fd/9: /usr/libexec/ceph/ceph-osd-prestart.sh: not found”.
[00:05] <AnrDaemon> I'd remove "respawn" stanzas until you 101% sure it works as intended. Before bad things happen.
[00:05] <AnrDaemon> That's not your job log.
[00:05] <flaf> And the file “/usr/libexec/ceph/ceph-osd-prestart.sh” is not mentioned in the job file _but_ it was present in the previous version of the job start. Because I have updated the ceph package.
[00:05] <AnrDaemon> You don't even log anything in your job, to begin with.
[00:07] <AnrDaemon> upstart job logs are named after the job, with optional instance name added after it.
[00:09] <flaf> AnrDaemon: I'm not sure to understand. I have to edit the job file and remove the 2 "respawn" stanzas, correct?
[00:09] <flaf> And I retry to start my job?
[00:09] <AnrDaemon> And i strongly suggest moving "instance" and "export" stanzas up.
[00:09] <AnrDaemon> Comment them out.
[00:09] <AnrDaemon> And add "console log" while you're at it.
[00:11] <flaf> If I add "console log", where the log will be created?
[00:11] <AnrDaemon> Where you expect them.
[00:12] <flaf> in /var/log/upstart/ ?
[00:12] <AnrDaemon> Of course :D
[00:13] <AnrDaemon> Also, I'm not exactly sure, what upstart is going to do with "instance ${cluster:-ceph}/$id"
[00:14] <flaf> I have tried. After “start ceph-osd id=13”, I have the error “start: Job failed to start” and no log is created in /var/log/upstart/.
[00:14] <AnrDaemon> It is supposed to be one variable, passed to the job from parent. Do make a check.
[00:15] <flaf> The only file updated in /var/log/upstart/ after my “start” command is /var/log/upstart/ceph-osd-ceph_13.log
[00:16] <flaf> and during my “start” command the line “/proc/self/fd/9: 8: /proc/self/fd/9: /usr/libexec/ceph/ceph-osd-prestart.sh: not found” is added in /var/log/upstart/ceph-osd-ceph_13.log
[00:16] <AnrDaemon> init-checkconf your job file.
[00:18] <flaf> As I said the ceph package has been updated (apt-get) and the new version of /etc/init/ceph-osd.conf doesn't contain /usr/libexec/ceph/ceph-osd-prestart.sh. But the previous version of /etc/init/ceph-osd.conf (before the “apt-get upgrade”) contains this string. Is it possible that the “start” command still use the previous and deleted version of /etc/init/ceph-osd.conf ?
[00:19] <flaf> init-checkconf /etc/init/ceph-osd.conf => “File /etc/init/ceph-osd.conf: syntax ok”
[00:21] <flaf> In the “pre-start” stanza, I have tried to add “echo test >/tmp/log” etc. but after the “start” command no /tmp/log was created.
[00:22] <flaf> AnrDaemon: another curios behavior: “mv /etc/init/ceph-osd.conf /root ; start ceph-osd id=13” => “start: Job failed to start”
[00:23] <flaf> AnrDaemon: normally the error should be “start: Unknown job: ceph-osd”.
[00:23] <AnrDaemon> That's why am I not using these shortcuts. Only initctl, only hardcore.
[00:23] <flaf> (because the file is moved to /root)
[00:24] <flaf> initctl start ceph-osd id=13 => initctl: Job failed to start
[00:24] <flaf> same message.
[00:26] <flaf> In fact, I have the feeling that a reboot could solve my problem. But I'm not sure and this a server in production.
[00:27] <AnrDaemon> You can force upstart to reconsider the jobs.
[00:30] <flaf> ah, how?
[00:47] <AnrDaemon> man initctl
=== m1dnight1 is now known as m1dnight_