=== Md_ is now known as Md [15:54] i just wrote my first upstart script for uwsgi, how cna i test it? [16:01] fission6: put it in /etc/init/uwsgi.conf and run 'sudo start uwsgi' [16:02] fission6: you can also do 'init-checkconf uwsgi' [16:02] thanks SpamapS [16:02] 'init-checkconf uwsgi' complains there is no file [16:03] do i need to do 'init-checkconf uwsgi.conf' [16:05] fission6: yeah, may even want to do /etc/init/uwsgi.conf [16:06] yea i just tried that, ERROR: failed to ask Upstart to check conf file [16:10] SpamapS: http://dpaste.org/NuSvR/ [16:10] fission6: I've not used init-checkconf much.. so forgive me for not knowing how to address that :-P [16:18] hmm maybe someone else knows, seems like it would be fundemental issue [16:23] SpamapS: how do you check if your upstart script works then? [16:27] fission6: start it [16:27] fission6: errors will show up in syslog [16:29] nice thnaks [18:16] am i going to get burnt if i upgrade from 1.3 to 1.4 [19:32] is symlinking from etc/init/ to a folder iwth a conf a bad idea? === chris|| is now known as chris| [20:04] can you symblink to a upstart from /etc/init [20:19] fission6: no [20:19] fission6: for a number of reasons, the files need to be there, on the root disk, in /etc/init === jMCg_ is now known as jMCg [20:33] ok [20:55] any ideas why restart for my job doesn;t work but start and stop do [21:09] fission6: the upstart restart command is.. a little weird [21:10] fission6: if you use 'service foo restart', that should do a stop/start [21:18] really? [21:18] whats up with that, let me try [21:20] did not work [21:20] i find htis weird, would have though it just does a stop start, which both work [21:26] fission6: it does, but it does it really fast, and without running post-stop [21:26] or pre-stop, I forget [21:26] fission6: anyway, its possible you need a post-stop which makes sure your service is dead [21:34] ? [21:34] seems odd [21:34] and convoluted [22:29] SpamapS: can you start multiple processes with a single conf? [22:52] fission6: yes, through use of the instance keyword [22:56] i want to be clear SpamapS i want to start two seperate daemons, one to start a queueing peice and one for uwsgi [22:58] fission6: ah, make that two .confs then [22:58] fission6: one can be 'start on started x' and 'stop on stopping x' so they'll act as one. [22:59] fission6: though IMO, its better to have things de-coupled unless they absolutely must be coupled [22:59] hm [22:59] i'd prefer a master script to start up the queying celeryd process then uwsgi web app process [23:00] i'd prefer one conf [23:01] how are they related? sounds like they aren't. [23:01] SpamapS: ^ [23:01] welll one wouldn't run without the other [23:01] celery does all the tasks associated with the web app uwsgi, such as email sending [23:02] you can have 1 conf, but you'll have to use pidfiles and daemonize things yourself [23:03] so you can't have multiple 'exec' or multiple script stanzas? [23:03] fission6: 2 confs has some real advantages. For instance, when the uwsgi segfaults, respawn can start it back up again.. it doesn't have to then stop celery, then start them both... which is more complicated and error prone [23:05] so maybe 2 scripts and 1 maste script to start each on of them? [23:07] upstart .conf files aren't scripts, they are configuration files ☺ [23:08] if one of the two processes depends on the other, then you define such a dependency in the configuration of those jobs, otherwise you just start them in parallel... [23:09] so it sounds like what i am doing shouldn't be done in one sript [23:09] err conf file [23:10] most likely not [23:10] JanC: do you work with web apps by chance and Upstart confs, whats the nature of most of the confs [23:10] Hi, I'm having a problem where this script is hanging after "start myapp": http://pastebin.com/nXSd79yP [23:11] when I run the command (minus "exec" of course) manually, it starts up just fine, so I know the command is at least correct [23:11] I've tried expect in all 3 modes, and it doesn't affect it in any way [23:11] suggestions? [23:11] fission6: I currently don't really work with webapps that need their own upstart jobs, no [23:14] is there a way to mayb debug that it's even getting to that point? [23:14] or see what's going on at all? [23:14] because "start myapp" just goes to a new line and hangs [23:15] maybe look into syslog [23:15] you can enable logging (in recent upstart versions) [23:16] per-job logging actually [23:16] hmm [23:16] logging for upstart itself can also be enabled with a kernel parameter or using initctl [23:18] oh, and I think when using start-stop-daemon, yous certainly don't want to use expect fork/daemon [23:18] I tried commenting it out too [23:20] well, you probably don't want to use the --background option for start-stop-daemon [23:20] ok let me try that [23:21] same thing [23:21] as the manpage says, you should use that only in a last resort ☺ [23:21] yeah :-/ [23:22] why is exec used in script stanza? [23:22] instead of script stanza, you mean? [23:24] the 1st respawn line is also obsolete of course [23:25] ahh, thanks [23:26] you might also want to check the logs of the application [23:27] the application isn't even being started... I looked, but there isn't even a process for the app in the process list [23:27] why do they use exec with two differnt meanings [23:27] fission6: eh? [23:27] exec in a script stanza and exec outside ofi t or does it do the same thing [23:28] I think it does the same thing [23:28] exec inside a script does not have anything to do with upstart [23:28] right. why do "we" use it then to kick off the main daemon your are upstarting [23:28] it does whatever your shell does with the exec command [23:29] right, so do i need it in an upstart script, seems its used as a standard but i dont see what it does [23:29] also still unclear why i cant run multiple things in a conf [23:31] you can run multiple things from one conf [23:31] but it won't supervise them separately then [23:32] JanC: one other issue i have is i cn cleanly start and stop my service but i can not restart it, it stops but doesn't restart [23:33] as SpamapS said, that probably means it tries to restart it too quickly [23:33] ok. so what does that mean in terms of resolving it, seems like an odd issue [23:34] if you use start-stop-daemon, it offers some ways to deal with that [23:34] this whole thing is becoming such a tangled mess just to get uwsgi to restart when my ec2 reboots [23:34] fission6: I feel your pain [23:35] JanC: do you have an example of a start-stop-daemon implmentation [23:35] or otherwise you might need some custom pre-stop or post-stop script or so [23:36] yea sort of odd was hoping upstart was a bit cleaner, looks like supervisord is a bit easier [23:43] fission6: I find upstart rather easy to understand, it's usually more work to understand how the applications that it needs to start/stop/etc. work ;) [23:44] well when it starts before something stops its a bit retarded, especially when its managing the process [23:45] yes/no [23:46] it's difficult to tell when an application starts & stops, or to be more exact: when it stopped "enough" that you can (re)start it ☺ [23:46] fission6: IMO, you are coupling things that don't need to be coupled [23:47] fission6: just start celery and the webapp service on their own. They'll talk if they can. [23:47] fission6: this will make it easier to split out the two into dedicated services later on anyway [23:47] well, unless the webapp really needs celery, then you make it wait on celery... [23:48] in both cases, you want 2 jobs [23:49] thats fine, i agree on tht front [23:49] i dont like how i can not currently restart uwsgi when before i got into upstart i woudl jsut stop adn start the process via control - c in a screen window [23:50] can we focus on why restart isn't working, maybe getting thatin place will make me happy [23:51] this is my uwsgi.conf to be clear http://dpaste.org/w1FBw/ [23:53] SpamapS: JanC could this be why http://projects.unbit.it/uwsgi/wiki/Upstart read the --die-on-term section [23:54] fission6: is your uwsgi service printing errors? I bet its trying to LISTEN on a taken port because the OS hasn't given back the socket yet. [23:55] nah [23:55] By default sending SIGTERM to uWSGI would mean "brutally-reload-it", while normally apps tend to shutdown on SIGTERM. To shutdown uWSGI use SIGINT or SIGQUIT. If you cannot live with such setup you can use --die-on-term option. [23:55] this could be it [23:56] that might indeed be the reason [23:56] wow, what an abuse of SIGTERM [23:57] ☺ [23:57] TERM is for *TERMINATING* not reloading [23:57] well, depends on how uwsgi works [23:57] my guess is its problem the reason since i only see --die-on-term mentioned in the upstart docs http://projects.unbit.it/uwsgi/wiki/Upstart [23:57] a process that doesn't exit very soon after SIGTERM is going to get a SIGKILL soon after from some overzealous BOFH [23:57] maybe it has a master process that automatically restarts child processes that get terminated or the like [23:57] fission6: yes you want --die-on-term [23:58] and you should send a really nasty email to the uwsgi authors explaining how dumb this is [23:58] we have HUP for reloads... anything further, use USR1 [23:58] fission6: you can also use 'kill signal QUIT' I think [23:59] sort of 6 and 1/2 dozen at that point [23:59] ugh, and it also doesn't realize that Upstart has had socket activation since 10.04 [23:59] when you say "we have"