bradleyayerscan anyone explain the wait-for-started job to me?02:48
bradleyayersor wait-for-state02:50
MrBusinessHello RAOF. I guess I should make another stab at explaining my issue?05:14
RAOFYeah.  What do you have, and what behaviour are you required to implement :)05:14
MrBusinessWell, let's see.05:15
MrBusinessMy difficulty is that I have three java programs that need to run in parallel on separate JVMs. I've been asked to set these up to run as services, and I have been given some old scripts to adapt to the purpose. However, I am also aware that these Java programs may end up having multiple instances in simultaneous execution on a given system, with only minor configuration variations between them.05:16
MrBusinessSomething ate my text.05:16
MrBusinessMy difficulty is that I have three java programs that need to run in parallel on separate JVMs. I've been asked to set these up to run as services, 05:17
MrBusinessthat was the beginning that got eaten there05:17
MrBusinessNow, what makes this tricky is this:05:18
MrBusinessMost services I see, from the looks of them, only ever have a single instance of a given binary running at any point in tiem05:18
MrBusinesswhereas this system might have a very large number of very similar processes running at the same time, but I presume, from the description of the requirements that I was provided with, that each instance needs to be an individual service that can be interacted with on a standalone basis.05:20
MrBusinessThe question is, where and how do I get that differentiation?05:20
MrBusinessWould I need to concoct a scheme to register each service with a slightly different name from all the others?05:20
MrBusinessI would need some kind of reliable method of managing all those PIDs05:21
RAOFSo, I guess one question is: what interaction requirements do you have?05:21
RAOFI still don't have a good idea of what you need the end result to look like.05:21
MrBusinessI need to be able to stop, start, restart, and possibly status the services.05:22
MrBusinessThose seem, from my own admittedly limited conception, to be about the only things these processes are capable of anyway. 05:22
RAOFOk.  And this is logically more than one service?  ie: would you ever want to stop component A without stopping component B?05:23
MrBusinessThe only interactive elements to any of them are in defining config-files for them to load at startup. Beyond that, they run on automatic. 05:23
MrBusinessI believe so, yes.05:23
MrBusinessLet me see if I can muster a useful description05:24
MrBusinessThere are three main applications that form this little pipeline of programs. If all three of these "parts" are running, then we have what could be viewed as a complete "pipeline" (though not in the '|' shell metacharacter sense). 05:25
MrBusinessBut, we might have more than one of these "pipelines" executing at the same time, and we would potentially want to be able to stop any one of the three constituent programs in any of our pipelines using the service interface.05:25
RAOFThis is starting to sound like you've been asked to square the circle; this doesn't sound like a service interface.05:28
MrBusinessThat could very well be05:28
MrBusinessI wasn't kidding when I said that updating my resume might be more productive. The deadlines for this project are absurd. Still, figured I'd give it the good ol' college try. 05:28
MrBusinessIf nothing else, at least now I know a bit more about the varieties and variations on different approaches to GNU/Linux services. That counts for something.05:29
MrBusinessAnyway, I appreciate your help RAOF.05:30
MrBusinessThanks much!05:30
RAOFSorry for not being *actually* helpful :)05:30
MrBusinessAh, no, you've given me plenty to chew on. If nothing else, I now see that the path ahead is not quite so clear-cut as I had been led to believe. This is my first time dealing with the definition of services, and I have a suspicion that my tech lead is imagining this services business as being a lot simpler in concept than it is in reality. 05:32
RAOFIt's just not clear to me *why* you've been asked to do this; if I knew that it might be possible for me to point you in the right direction, or at least at people who'd be able to point you in the right direction.05:34
MrBusinessWell, I gotta say, after all this chatting, it's not entirely clear to me why we're doing it that way either. I guess my lead wants us to have granular control over the processes without having to explicitly log in and start them from some user name. I guess we want init to start them, but to permit human interaction with them when and where necessary.05:35
RAOFSo, it's (also) not clear to me how the many instances happen.  Do you have three primary processes (let's call them A, B, and C) which may or may not spawn a bunch of subprocesses, or do have a bunch of independent (A₁, B₁, C₁), (A₂, B₂, C₂) sets?05:40
MrBusinessthe latter, they're all independent processes05:41
RAOFHow do they get started?05:41
MrBusinessa call to the JVM, providing it with the name of a specific .jar file and feeding it a specific configuration file05:42
RAOFYou could servicify each set there.  Do you always want all of the sets running?05:44
MrBusinessWell, not sure, but I don't think so. I think I actually want to be able to stop or restart each piece of a given set.05:45
MrBusinessi.e. A1 stop05:45
MrBusinessnot sure how to do those fancy subscript numbers you did.05:46
RAOFHeh :)05:46
RAOF<compose>_1 :)05:46
RAOFSo, I guess a final question is: is there a deterministic set of processes you might possibly want to be servicifying?05:47
MrBusinessDeterministic in what sense?05:47
RAOFie: is there a set 1, set 2, set 3, … set n, with a fixed n.05:48
MrBusinessn would be contingent on the needs of the people deploying it. 05:48
RAOFThen they need to write the service scripts themselves.05:48
RAOFBecause, as far as I'm aware, in *all* implementations of unix-y services, the services are singletons.05:49
MrBusinessMakes sense to me.05:49
MrBusinessHey, "singleton"! Excellent word, and almost certainly the piece of nomenclature I've been missing in my descriptions.05:50
RAOFThey don't have to be single processes, just logically be a single entity.05:50
RAOFSo either each A_n, B_n, C_n could be a service, or each set (A_n, B_n, C_n) could be a service, but there's no way to address "Component A, 3rd instance".05:52
MrBusinessYeah. There would have to be some kind of trick to the way each service gets registered in the system, and even then, it would fall to the user to be able to distinguish which service was the one he or she wanted to interact with.05:53
MrBusinessGuess I'll just have to go chat with The Man tomorrow to see what he thinks. I'm just the disposable hatchet man here, and if he wants to make assumptions about the thing, I'll swing that hatchet far, wide, and fast. 05:53
RAOFAnd because services are registered as configuration files, that would mean you'd need to be dynamically generating configuration files and messing with them.05:53
RAOFI guess you *could* have a single meta-service with a conf-file consisting basically of "the services are A_n, B_n, C_n", but that doesn't support "Stop A₃"05:54
MrBusinessYeah, it's a thought, though probably not feasible within the present time constraints. 05:55
MrBusinessI'll figure something out. 05:55
jyfl987http://pastebin.com/m0JeStkP   anyone can tell me why this configure dont work?? http://pastebin.com/m0JeStkP07:00
bradleyayersjyfl987: remove script and end script07:02
bradleyayerssetuid and setgid are supported as job commands07:03
bradleyayersunless you're using upstart < 1.407:03
jyfl987bradleyayers: then how can i achive my goal? i need to use postgres:postgres to run the command to start postgresql07:05
bradleyayersjust use su07:06
jyfl987bradleyayers: can you show me an full example ?07:06
jyfl987i am just a newbie to this, and have lots of question to ask you guys here07:06
ionI’ve had more success sudo. “exec sudo -H -u user -- command [args]”. su seems to like to leave an extra process hanging around.07:07
jyfl9871, which shell interpreter does the commands in script sections use?07:07
ionWith set -e07:07
jyfl9872, will cd xxx works? cause i see chdir command in the upstart's cookbook07:08
ionIt does.07:08
jyfl9873, is it need update-initramfs or other updating job after i have changed some job's configure , i mean in ubuntu (i use ubuntu lucid )07:09
jyfl9874, how to simplely let some job dont start at the bootup but i still can start it by manual via initctl start job_xx07:10
jyfl9875, i see 'emits event-name' and 'initctl emit event-name' in my ubuntu's /etc/init/ directory, is there any differences between them?07:11
jyfl9876, if i dont use 'exec somecommand' to run some job but just 'somecommand', will the job freeze at that point?07:12
jyfl9877, how to got the job log when starting or ending ?07:13
SpamapSion: start-stop-daemon is even better than su07:14
SpamapSion: or sudo07:14
SpamapSion: mostly because it is made for this purpose. su keeps a pam session open, thats where you see the extra process.07:15
SpamapSjyfl987: have you read the upstart cookbook yet?07:15
jyfl987am reading it07:15
ionI’ve found sudo to be better than start-stop-daemon, too.07:16
ionIn my Upstart jobs, that is.07:16
SpamapSion: could you explain why? It might be worth noting your findings in the cookbook07:17
SpamapSjyfl987: emits is just a form of documentation, which helps init-checkconf and the upstart visualization tool determine how events affect the system.07:18
SpamapSjyfl987: exec is recommended, otherwise you'll leave behind the shell process .. which you probably don't want or need.07:19
SpamapSjyfl987: job output logging is a new feature in Upstart 1.4, which is in Ubuntu precise (soon to be 12.04). If you want to log a job's output,    exec foo 2>&1 | logger -t foo 07:20
SpamapSjyfl987: oh and no, you don't need to re-run update-initramfs. upstart is started after the initramfs is done mounting the root filesystem.07:20
jyfl987SpamapS: so precise will keep on using upstart?07:20
SpamapSjyfl987: yes. as yet, nobody has successfully argues that "everythingd" a.k.a. systemd should be the pid 1.07:21
* SpamapS wishes irc worked like skype and would fix those mistakes. ;)07:21
jyfl987SpamapS: ok this encourage me to learn upstart07:22
jyfl987i dont want to found that they change to a new system after i finally learnt the old one07:22
jyfl987SpamapS: so the problem is i change the setuid and setgid command to su postgres and start the job again, still faild07:23
ionspamaps: I’m not sure you can turn off start-stop-daemon’s functionality that checks for the existence of the process based on certain matchers (such as the /proc/*/exe). That overlaps (and perhaps interferes) considerably with what Upstart is supposed to do, and certainly implements it in a worse way.07:24
ionFor instance,07:24
ion# start-stop-daemon --exec /bin/sleep --start -- 5 &07:24
ion# start-stop-daemon --exec /bin/sleep --start -- 5 07:24
ion/bin/sleep already running.07:24
SpamapSion: perhaps we need --startas instead of --exec ...07:26
ionstart-stop-daemon: need at least one of --exec, --pidfile, --user or --name07:26
SpamapS$ start-stop-daemon --startas /bin/sleep --name first --start -- 5 & start-stop-daemon --startas /bin/sleep --name second --start -- 507:27
SpamapSthat works07:27
jyfl987hi guys can you just came back to my question?07:27
ionThat feels like quite a bit of a kluge when you could just use sudo. :-)07:27
jyfl987my blog are waiting for my configure file :]07:27
SpamapSion: perhaps.. either way, the cookbook needs a note about multi-instance jobs07:29
SpamapSjyfl987: first off, why are you converting postgres to an upstart job?07:29
jyfl987SpamapS: because i am using a vps with debian5 installed and my target version of pgsql seems not in the repo07:30
jyfl987so i need to install it by manual07:30
SpamapSdebian5 huh?07:31
jyfl987sorry its ubuntu 10.0407:32
SpamapSok well you could just backport the packages from precise07:33
jyfl987i want to learn the knowledge because my blog service is written by me using python, it wont be in repo, so i still need to configure a job by myself07:34
SpamapSjyfl987: ok cool. well in this case, perhaps you need to tell postgresql to run in the foreground07:36
SpamapSjyfl987: it probably becomes a daemon by default07:36
jyfl987SpamapS: i run the command by mannual which will run in foreground 07:37
SpamapSjyfl987: | logger or > logfile and you will get some idea of why it is failing.07:38
jyfl987but when i put the command in postgresql.conf at /etc/init and then trying initctl start postgresql , it told me the service has been start/running and told me the thread id, but it seems be killed after that07:38
jyfl987i dont know why07:38
jyfl987and no log output 07:39
jyfl987wait i will show you the current configure file07:39
jyfl987http://pastebin.com/y8RtTKSZ  SpamapS here it is07:39
ion“su postgres” doesn’t do anything without a terminal.07:40
ionThe cookbook tells you how to log the output from the job. Add something like the following just after “script”: 07:40
ion  exec >/tmp/myjob.out 2>&107:40
ion  set -x07:40
jyfl987ion: then you are recommend me to use sudo -u postgres exec xxx?07:43
ionNo. exec sudo -H -u postgres -- xxx07:43
jyfl987ion: will try that07:44
jyfl987ion: why i need to add set -x?07:45
ionIt’s useful for debugging.07:45
jyfl987after sudo exec .. ?07:46
ionPlease read what we say more carefully. :-P I said ‘just after “script”’.07:46
ionWhere did this “sudo exec” come from?07:47
jyfl987its in the scritp ... end script section07:47
jyfl987ok i got the error output07:48
jyfl987sudo: exec: command not found07:48
ionPlease read the command i wrote again.07:49
jyfl987ion: finally it works07:52
jyfl987ion: so how to add a stop script for it?07:52
ionIf the postgresql binary handles SIGTERM properly, you shouldn’t need to.07:53
jyfl987current i can only use initctl start postgresql to start the service but when i using initctl stop postgresql it told me unknow instance07:53
ionThen it has probably died already. You’ll need to investigate why.07:54
jyfl987got it, the original thread seems died, and a new thread started, i can see this from ps aux07:55
ionSee if you can disable that behavior.07:56
SpamapSprobably daemonized itself07:56
SpamapSyou *might* be able to use expect fork07:56
SpamapSbut make sure it actually forks first. :)07:57
jyfl987hmm, i think i dont need that this time, so after my postgresql service started there should be a started-postgresql events?07:57
SpamapSjyfl987: started JOB=postgresql  ... 07:57
jyfl987will change the configure file07:58
jyfl987SpamapS: export fork works , thanks so much and to you ion08:23
jyfl987SpamapS: but chdir seems not works09:17
jyfl987ion: is it that sudo wont use the currrent work directory ?09:33
ion# cd /var; sudo -H -u ion pwd09:36
evinrudedoes upstart take care of stopping daemons? As is in, I do not have to track a PID "echo $$" so that I can kill it later on?19:09
SpamapSevinrude: it can do that yes19:10
SpamapSevinrude: Some daemons it has a hard time keeping track of19:11
evinrudegreat, thanks. Off to the wiki I go19:11
SpamapSevinrude: I'd recommend http://upstart.ubuntu.com/cookbook/19:11
evinrudeSpamapS: thanks!19:12
fremI wrote this upstart job which sources a python virtualenv and starts a webserver. http://paste.pocoo.org/show/551041/20:29
frembut when I run it, it's shown with the stop/waiting status?20:30
njinjust installed 1.4, no problem. thanks20:40
candehi, i'm trying to make a upstart .conf script, but when running "sudo start myservice" from commandline, it just freezes23:21
RAOFcande: You probably want to (at least) pastebin your conf file so people can inspect it.23:32
candeok, http://pastebin.com/qWAY15eh23:33
candeRAOF, make any sense?23:37
RAOFHm.  That looks to me like it should work, but I'm not particularly expert.23:43
candeok thx23:57

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!