| bradleyayers | can anyone explain the wait-for-started job to me? | 02:48 |
|---|---|---|
| bradleyayers | or wait-for-state | 02:50 |
| MrBusiness | Hello | 05:12 |
| RAOF | Aloha | 05:14 |
| MrBusiness | Hello RAOF. I guess I should make another stab at explaining my issue? | 05:14 |
| RAOF | Yeah. What do you have, and what behaviour are you required to implement :) | 05:14 |
| MrBusiness | Well, let's see. | 05:15 |
| MrBusiness | My difficulty is that I have three java programs that need to run in parallel on separate JVMs. I've been asked to set these up to run as services, and I have been given some old scripts to adapt to the purpose. However, I am also aware that these Java programs may end up having multiple instances in simultaneous execution on a given system, with only minor configuration variations between them. | 05:16 |
| MrBusiness | Hm | 05:16 |
| MrBusiness | Something ate my text. | 05:16 |
| MrBusiness | My difficulty is that I have three java programs that need to run in parallel on separate JVMs. I've been asked to set these up to run as services, | 05:17 |
| MrBusiness | that was the beginning that got eaten there | 05:17 |
| RAOF | Ok. | 05:18 |
| MrBusiness | Now, what makes this tricky is this: | 05:18 |
| MrBusiness | Most services I see, from the looks of them, only ever have a single instance of a given binary running at any point in tiem | 05:18 |
| MrBusiness | whereas this system might have a very large number of very similar processes running at the same time, but I presume, from the description of the requirements that I was provided with, that each instance needs to be an individual service that can be interacted with on a standalone basis. | 05:20 |
| MrBusiness | The question is, where and how do I get that differentiation? | 05:20 |
| MrBusiness | Would I need to concoct a scheme to register each service with a slightly different name from all the others? | 05:20 |
| MrBusiness | I would need some kind of reliable method of managing all those PIDs | 05:21 |
| RAOF | So, I guess one question is: what interaction requirements do you have? | 05:21 |
| RAOF | I still don't have a good idea of what you need the end result to look like. | 05:21 |
| MrBusiness | I need to be able to stop, start, restart, and possibly status the services. | 05:22 |
| MrBusiness | Those seem, from my own admittedly limited conception, to be about the only things these processes are capable of anyway. | 05:22 |
| RAOF | Ok. And this is logically more than one service? ie: would you ever want to stop component A without stopping component B? | 05:23 |
| MrBusiness | The only interactive elements to any of them are in defining config-files for them to load at startup. Beyond that, they run on automatic. | 05:23 |
| MrBusiness | I believe so, yes. | 05:23 |
| MrBusiness | Let me see if I can muster a useful description | 05:24 |
| MrBusiness | There are three main applications that form this little pipeline of programs. If all three of these "parts" are running, then we have what could be viewed as a complete "pipeline" (though not in the '|' shell metacharacter sense). | 05:25 |
| MrBusiness | But, we might have more than one of these "pipelines" executing at the same time, and we would potentially want to be able to stop any one of the three constituent programs in any of our pipelines using the service interface. | 05:25 |
| RAOF | This is starting to sound like you've been asked to square the circle; this doesn't sound like a service interface. | 05:28 |
| MrBusiness | That could very well be | 05:28 |
| MrBusiness | I wasn't kidding when I said that updating my resume might be more productive. The deadlines for this project are absurd. Still, figured I'd give it the good ol' college try. | 05:28 |
| MrBusiness | If nothing else, at least now I know a bit more about the varieties and variations on different approaches to GNU/Linux services. That counts for something. | 05:29 |
| MrBusiness | Anyway, I appreciate your help RAOF. | 05:30 |
| MrBusiness | Thanks much! | 05:30 |
| RAOF | Sorry for not being *actually* helpful :) | 05:30 |
| MrBusiness | Ah, no, you've given me plenty to chew on. If nothing else, I now see that the path ahead is not quite so clear-cut as I had been led to believe. This is my first time dealing with the definition of services, and I have a suspicion that my tech lead is imagining this services business as being a lot simpler in concept than it is in reality. | 05:32 |
| RAOF | It's just not clear to me *why* you've been asked to do this; if I knew that it might be possible for me to point you in the right direction, or at least at people who'd be able to point you in the right direction. | 05:34 |
| MrBusiness | Well, I gotta say, after all this chatting, it's not entirely clear to me why we're doing it that way either. I guess my lead wants us to have granular control over the processes without having to explicitly log in and start them from some user name. I guess we want init to start them, but to permit human interaction with them when and where necessary. | 05:35 |
| RAOF | So, it's (also) not clear to me how the many instances happen. Do you have three primary processes (let's call them A, B, and C) which may or may not spawn a bunch of subprocesses, or do have a bunch of independent (A₁, B₁, C₁), (A₂, B₂, C₂) sets? | 05:40 |
| MrBusiness | the latter, they're all independent processes | 05:41 |
| RAOF | How do they get started? | 05:41 |
| MrBusiness | a call to the JVM, providing it with the name of a specific .jar file and feeding it a specific configuration file | 05:42 |
| RAOF | You could servicify each set there. Do you always want all of the sets running? | 05:44 |
| MrBusiness | Well, not sure, but I don't think so. I think I actually want to be able to stop or restart each piece of a given set. | 05:45 |
| MrBusiness | i.e. A1 stop | 05:45 |
| MrBusiness | not sure how to do those fancy subscript numbers you did. | 05:46 |
| RAOF | Heh :) | 05:46 |
| RAOF | <compose>_1 :) | 05:46 |
| RAOF | So, I guess a final question is: is there a deterministic set of processes you might possibly want to be servicifying? | 05:47 |
| MrBusiness | Deterministic in what sense? | 05:47 |
| RAOF | ie: is there a set 1, set 2, set 3, … set n, with a fixed n. | 05:48 |
| MrBusiness | n would be contingent on the needs of the people deploying it. | 05:48 |
| RAOF | Then they need to write the service scripts themselves. | 05:48 |
| MrBusiness | ah | 05:48 |
| RAOF | Because, as far as I'm aware, in *all* implementations of unix-y services, the services are singletons. | 05:49 |
| MrBusiness | Ok. | 05:49 |
| MrBusiness | Makes sense to me. | 05:49 |
| MrBusiness | Hey, "singleton"! Excellent word, and almost certainly the piece of nomenclature I've been missing in my descriptions. | 05:50 |
| RAOF | They don't have to be single processes, just logically be a single entity. | 05:50 |
| RAOF | So either each A_n, B_n, C_n could be a service, or each set (A_n, B_n, C_n) could be a service, but there's no way to address "Component A, 3rd instance". | 05:52 |
| MrBusiness | Yeah. There would have to be some kind of trick to the way each service gets registered in the system, and even then, it would fall to the user to be able to distinguish which service was the one he or she wanted to interact with. | 05:53 |
| RAOF | Right. | 05:53 |
| MrBusiness | Guess I'll just have to go chat with The Man tomorrow to see what he thinks. I'm just the disposable hatchet man here, and if he wants to make assumptions about the thing, I'll swing that hatchet far, wide, and fast. | 05:53 |
| RAOF | And because services are registered as configuration files, that would mean you'd need to be dynamically generating configuration files and messing with them. | 05:53 |
| MrBusiness | Yeah | 05:53 |
| RAOF | I guess you *could* have a single meta-service with a conf-file consisting basically of "the services are A_n, B_n, C_n", but that doesn't support "Stop A₃" | 05:54 |
| MrBusiness | Yeah, it's a thought, though probably not feasible within the present time constraints. | 05:55 |
| MrBusiness | I'll figure something out. | 05:55 |
| jyfl987 | http://pastebin.com/m0JeStkP anyone can tell me why this configure dont work?? http://pastebin.com/m0JeStkP | 07:00 |
| bradleyayers | jyfl987: remove script and end script | 07:02 |
| bradleyayers | setuid and setgid are supported as job commands | 07:03 |
| bradleyayers | unless you're using upstart < 1.4 | 07:03 |
| jyfl987 | bradleyayers: then how can i achive my goal? i need to use postgres:postgres to run the command to start postgresql | 07:05 |
| bradleyayers | just use su | 07:06 |
| jyfl987 | bradleyayers: can you show me an full example ? | 07:06 |
| jyfl987 | i am just a newbie to this, and have lots of question to ask you guys here | 07:06 |
| ion | I’ve had more success sudo. “exec sudo -H -u user -- command [args]”. su seems to like to leave an extra process hanging around. | 07:07 |
| jyfl987 | 1, which shell interpreter does the commands in script sections use? | 07:07 |
| ion | /bin/sh | 07:07 |
| ion | With set -e | 07:07 |
| jyfl987 | 2, will cd xxx works? cause i see chdir command in the upstart's cookbook | 07:08 |
| ion | It does. | 07:08 |
| jyfl987 | 3, is it need update-initramfs or other updating job after i have changed some job's configure , i mean in ubuntu (i use ubuntu lucid ) | 07:09 |
| jyfl987 | 4, how to simplely let some job dont start at the bootup but i still can start it by manual via initctl start job_xx | 07:10 |
| jyfl987 | 5, i see 'emits event-name' and 'initctl emit event-name' in my ubuntu's /etc/init/ directory, is there any differences between them? | 07:11 |
| jyfl987 | 6, if i dont use 'exec somecommand' to run some job but just 'somecommand', will the job freeze at that point? | 07:12 |
| jyfl987 | 7, how to got the job log when starting or ending ? | 07:13 |
| SpamapS | ion: start-stop-daemon is even better than su | 07:14 |
| SpamapS | ion: or sudo | 07:14 |
| SpamapS | ion: mostly because it is made for this purpose. su keeps a pam session open, thats where you see the extra process. | 07:15 |
| SpamapS | jyfl987: have you read the upstart cookbook yet? | 07:15 |
| jyfl987 | am reading it | 07:15 |
| ion | I’ve found sudo to be better than start-stop-daemon, too. | 07:16 |
| ion | In my Upstart jobs, that is. | 07:16 |
| SpamapS | ion: could you explain why? It might be worth noting your findings in the cookbook | 07:17 |
| SpamapS | jyfl987: emits is just a form of documentation, which helps init-checkconf and the upstart visualization tool determine how events affect the system. | 07:18 |
| SpamapS | jyfl987: exec is recommended, otherwise you'll leave behind the shell process .. which you probably don't want or need. | 07:19 |
| SpamapS | jyfl987: job output logging is a new feature in Upstart 1.4, which is in Ubuntu precise (soon to be 12.04). If you want to log a job's output, exec foo 2>&1 | logger -t foo | 07:20 |
| SpamapS | jyfl987: oh and no, you don't need to re-run update-initramfs. upstart is started after the initramfs is done mounting the root filesystem. | 07:20 |
| jyfl987 | SpamapS: so precise will keep on using upstart? | 07:20 |
| SpamapS | jyfl987: yes. as yet, nobody has successfully argues that "everythingd" a.k.a. systemd should be the pid 1. | 07:21 |
| SpamapS | s/argues/argued/ | 07:21 |
| * SpamapS wishes irc worked like skype and would fix those mistakes. ;) | 07:21 | |
| jyfl987 | SpamapS: ok this encourage me to learn upstart | 07:22 |
| jyfl987 | i dont want to found that they change to a new system after i finally learnt the old one | 07:22 |
| jyfl987 | SpamapS: so the problem is i change the setuid and setgid command to su postgres and start the job again, still faild | 07:23 |
| ion | spamaps: I’m not sure you can turn off start-stop-daemon’s functionality that checks for the existence of the process based on certain matchers (such as the /proc/*/exe). That overlaps (and perhaps interferes) considerably with what Upstart is supposed to do, and certainly implements it in a worse way. | 07:24 |
| ion | For instance, | 07:24 |
| ion | # start-stop-daemon --exec /bin/sleep --start -- 5 & | 07:24 |
| ion | # start-stop-daemon --exec /bin/sleep --start -- 5 | 07:24 |
| ion | /bin/sleep already running. | 07:24 |
| SpamapS | ion: perhaps we need --startas instead of --exec ... | 07:26 |
| ion | start-stop-daemon: need at least one of --exec, --pidfile, --user or --name | 07:26 |
| SpamapS | $ start-stop-daemon --startas /bin/sleep --name first --start -- 5 & start-stop-daemon --startas /bin/sleep --name second --start -- 5 | 07:27 |
| SpamapS | that works | 07:27 |
| jyfl987 | hi guys can you just came back to my question? | 07:27 |
| ion | That feels like quite a bit of a kluge when you could just use sudo. :-) | 07:27 |
| jyfl987 | my blog are waiting for my configure file :] | 07:27 |
| SpamapS | ion: perhaps.. either way, the cookbook needs a note about multi-instance jobs | 07:29 |
| SpamapS | jyfl987: first off, why are you converting postgres to an upstart job? | 07:29 |
| jyfl987 | SpamapS: because i am using a vps with debian5 installed and my target version of pgsql seems not in the repo | 07:30 |
| jyfl987 | so i need to install it by manual | 07:30 |
| SpamapS | debian5 huh? | 07:31 |
| jyfl987 | yep | 07:32 |
| jyfl987 | sorry its ubuntu 10.04 | 07:32 |
| jyfl987 | lts | 07:32 |
| SpamapS | ah | 07:33 |
| SpamapS | ok well you could just backport the packages from precise | 07:33 |
| jyfl987 | i want to learn the knowledge because my blog service is written by me using python, it wont be in repo, so i still need to configure a job by myself | 07:34 |
| SpamapS | jyfl987: ok cool. well in this case, perhaps you need to tell postgresql to run in the foreground | 07:36 |
| SpamapS | jyfl987: it probably becomes a daemon by default | 07:36 |
| jyfl987 | SpamapS: i run the command by mannual which will run in foreground | 07:37 |
| SpamapS | jyfl987: | logger or > logfile and you will get some idea of why it is failing. | 07:38 |
| jyfl987 | but when i put the command in postgresql.conf at /etc/init and then trying initctl start postgresql , it told me the service has been start/running and told me the thread id, but it seems be killed after that | 07:38 |
| jyfl987 | i dont know why | 07:38 |
| jyfl987 | and no log output | 07:39 |
| jyfl987 | wait i will show you the current configure file | 07:39 |
| jyfl987 | http://pastebin.com/y8RtTKSZ SpamapS here it is | 07:39 |
| ion | “su postgres” doesn’t do anything without a terminal. | 07:40 |
| ion | The cookbook tells you how to log the output from the job. Add something like the following just after “script”: | 07:40 |
| ion | exec >/tmp/myjob.out 2>&1 | 07:40 |
| ion | set -x | 07:40 |
| jyfl987 | ion: then you are recommend me to use sudo -u postgres exec xxx? | 07:43 |
| ion | No. exec sudo -H -u postgres -- xxx | 07:43 |
| jyfl987 | ion: will try that | 07:44 |
| jyfl987 | ion: why i need to add set -x? | 07:45 |
| ion | It’s useful for debugging. | 07:45 |
| jyfl987 | after sudo exec .. ? | 07:46 |
| ion | Please read what we say more carefully. :-P I said ‘just after “script”’. | 07:46 |
| ion | Where did this “sudo exec” come from? | 07:47 |
| jyfl987 | its in the scritp ... end script section | 07:47 |
| jyfl987 | ok i got the error output | 07:48 |
| jyfl987 | sudo: exec: command not found | 07:48 |
| ion | Please read the command i wrote again. | 07:49 |
| jyfl987 | oops | 07:49 |
| jyfl987 | ion: finally it works | 07:52 |
| jyfl987 | ion: so how to add a stop script for it? | 07:52 |
| ion | If the postgresql binary handles SIGTERM properly, you shouldn’t need to. | 07:53 |
| jyfl987 | current i can only use initctl start postgresql to start the service but when i using initctl stop postgresql it told me unknow instance | 07:53 |
| ion | Then it has probably died already. You’ll need to investigate why. | 07:54 |
| jyfl987 | got it, the original thread seems died, and a new thread started, i can see this from ps aux | 07:55 |
| ion | See if you can disable that behavior. | 07:56 |
| SpamapS | probably daemonized itself | 07:56 |
| SpamapS | you *might* be able to use expect fork | 07:56 |
| SpamapS | but make sure it actually forks first. :) | 07:57 |
| jyfl987 | hmm, i think i dont need that this time, so after my postgresql service started there should be a started-postgresql events? | 07:57 |
| SpamapS | jyfl987: started JOB=postgresql ... | 07:57 |
| jyfl987 | ok | 07:58 |
| jyfl987 | will change the configure file | 07:58 |
| jyfl987 | SpamapS: export fork works , thanks so much and to you ion | 08:23 |
| jyfl987 | SpamapS: but chdir seems not works | 09:17 |
| jyfl987 | ion: is it that sudo wont use the currrent work directory ? | 09:33 |
| ion | # cd /var; sudo -H -u ion pwd | 09:36 |
| ion | /var | 09:36 |
| jyfl987 | ok | 09:40 |
| evinrude | does upstart take care of stopping daemons? As is in, I do not have to track a PID "echo $$" so that I can kill it later on? | 19:09 |
| SpamapS | evinrude: it can do that yes | 19:10 |
| SpamapS | evinrude: Some daemons it has a hard time keeping track of | 19:11 |
| evinrude | great, thanks. Off to the wiki I go | 19:11 |
| SpamapS | evinrude: I'd recommend http://upstart.ubuntu.com/cookbook/ | 19:11 |
| evinrude | SpamapS: thanks! | 19:12 |
| frem | I wrote this upstart job which sources a python virtualenv and starts a webserver. http://paste.pocoo.org/show/551041/ | 20:29 |
| frem | but when I run it, it's shown with the stop/waiting status? | 20:30 |
| njin | just installed 1.4, no problem. thanks | 20:40 |
| cande | hi, i'm trying to make a upstart .conf script, but when running "sudo start myservice" from commandline, it just freezes | 23:21 |
| RAOF | cande: You probably want to (at least) pastebin your conf file so people can inspect it. | 23:32 |
| cande | ok, http://pastebin.com/qWAY15eh | 23:33 |
| cande | RAOF, make any sense? | 23:37 |
| RAOF | Hm. That looks to me like it should work, but I'm not particularly expert. | 23:43 |
| cande | ok thx | 23:57 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!