| j4m3s | i'm trying to set the priority of a process in my upstart script with nice and it doesn't appear to be taking | 04:16 |
|---|---|---|
| j4m3s | where does one typically put the 'nice <priority>' setting in an upstart script? | 04:26 |
| === ion_ is now known as ion | ||
| === zz_Kiall is now known as Kiall | ||
| === Kiall is now known as zz_Kiall | ||
| === zz_Kiall is now known as Kiall | ||
| SpamapS | j4m3s: its a root level keyword, so just put it anywhere outside a script stanza. | 14:50 |
| MFen | i'm trying to set up an upstart job for bip | 15:18 |
| MFen | i've carefully studied http://upstart.ubuntu.com/cookbook/#implications-of-misspecifying-expect and even did the test described here.. http://upstart.ubuntu.com/cookbook/#how-to-establish-fork-count which demonstrated that bip forks twice (is a daemon) | 15:18 |
| SpamapS | MFen: ^5 for reading the cookbook. :) | 15:19 |
| MFen | so i'm using expect daemon. according to that chart, i shouldn't be getting a hang (even if i'm wrong), but i am | 15:19 |
| MFen | any double-secret troubleshooting tips available? :) | 15:19 |
| MFen | SpamapS: ltns :) ... i think we used to chat in #linux | 15:20 |
| MFen | loooong ago | 15:20 |
| SpamapS | or #linuxhelp | 15:20 |
| MFen | yeah | 15:20 |
| SpamapS | which I still lurk in, just in case somebody wants to reminisce ;) | 15:21 |
| SpamapS | MFen: do you have the output of your fork test handy? | 15:21 |
| MFen | no but i'll run it again | 15:21 |
| MFen | well, yeah. the output was "2" :) | 15:21 |
| SpamapS | I mean the strace.log | 15:22 |
| MFen | sure | 15:22 |
| SpamapS | the grep for fork/clone may be wrong | 15:22 |
| MFen | ok | 15:22 |
| SpamapS | Have seen at least one program that does clone that is actually doing threads | 15:22 |
| MFen | oh, heh. gotta log in to irc directly if i'm going to do this. brb | 15:23 |
| SpamapS | MFen: also, about 90% of daemons fork way too soon anyway, and so the 'expect fork|daemon' is actually not really accurate.. does bip have a "run without daemonizing" mode? | 15:23 |
| SpamapS | MFen: just use pastebinit ? | 15:23 |
| MFen | bip has a -n | 15:23 |
| MFen | i'll pastebin it, but bip is an irc multiplexer that i'm using right now | 15:24 |
| SpamapS | Oh hah ok | 15:24 |
| SpamapS | MFen: hang on tho | 15:24 |
| MFen | k | 15:24 |
| SpamapS | MFen: consider just using -n | 15:24 |
| MFen | tried that too (with no expect) | 15:24 |
| MFen | iirc what happens in that case is it hangs on stop | 15:24 |
| SpamapS | MFen: the only drawback to that over expect fork/daemon is that the job will be considered "started" as soon as it forks/execs bip | 15:24 |
| MFen | it was doing a weird thing where it would hang, but stop anyway | 15:24 |
| SpamapS | MFen: does it not respect SIGTERM ? | 15:25 |
| MFen | so i ^C then stop again and it says it's already stopped | 15:25 |
| MFen | it does when i run it straight from a command line | 15:25 |
| SpamapS | ^C is SIGINT , which is often handled differently than SIGTERM, which is what upstart sends before SIGKILL | 15:25 |
| MFen | oh i see. ok, i can find out what SIGTERM does | 15:26 |
| MFen | brb | 15:26 |
| === moonfallen is now known as mfen | ||
| mfen | alll right | 15:29 |
| mfen | kill -15 works fine on bip -n | 15:30 |
| SpamapS | alright | 15:31 |
| SpamapS | mfen: then the only other thing to consider is that things that want to 'start on started bip' may not actually be able to contact bip's network port... so if you want to support that type of behavior, you have to add a post-start which polls bip until its actually available. | 15:31 |
| SpamapS | mfen: note that even sshd suffers from that race.. so don't feel like you have to add a post-start | 15:32 |
| SpamapS | mfen: also test SIGHUP | 15:32 |
| SpamapS | mfen: as that is what reload sends | 15:32 |
| mfen | the strace.log is 22k lines | 15:33 |
| SpamapS | mfen: pastebinit < strace.log ;) | 15:33 |
| mfen | did not know that | 15:34 |
| SpamapS | its a beautiful tool | 15:35 |
| SpamapS | wish we had it 10 years ago in #linuxhelp | 15:35 |
| mfen | kill -HUP has no effect | 15:36 |
| mfen | hmm. i got http://paste.ubuntu.com/ as the output of pastebinit | 15:37 |
| mfen | i'm guessing that was supposed to have a unique identifier in it | 15:37 |
| mfen | i'll just put it in dropbox | 15:37 |
| mfen | https://dl.dropbox.com/u/19400544/goonmill.org/strace.log | 15:38 |
| mfen | bah, that can't be right | 15:38 |
| SpamapS | mfen: its possible the pastebin didn't like 22k lines ;) | 15:39 |
| mfen | yeah | 15:39 |
| mfen | dropbox doesn't either afaict | 15:40 |
| SpamapS | 21730 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xf72f2968) = 21731 | 15:40 |
| SpamapS | Thats a thread | 15:40 |
| mfen | ok | 15:40 |
| mfen | so is this actually a fork? | 15:40 |
| SpamapS | Probably need to update the cookbook | 15:40 |
| SpamapS | mfen: no | 15:40 |
| mfen | the thing is, i've tried all three | 15:41 |
| SpamapS | well it might be | 15:41 |
| SpamapS | I have never understood how clone turns into fork | 15:41 |
| SpamapS | it does.. sometimes.. but ... whenever I see it.. expect doesn't work. I think ptrace gets confused | 15:41 |
| SpamapS | mfen: you're better off with -n | 15:42 |
| mfen | all right, here's the whole bip.conf now: | 15:42 |
| mfen | setuid cdodt | 15:42 |
| mfen | chdir /home/cdodt | 15:42 |
| mfen | exec bip -n | 15:42 |
| mfen | that hangs when i start | 15:43 |
| mfen | even simpler version. just this: exec sudo -u cdodt -i bip -n ... also hangs when i start | 15:46 |
| mfen | i tried strace -fF -p 1 ...... i'm seeing read(3, ...) = -1 EAGAIN | 16:00 |
| mfen | lots of those repeated | 16:00 |
| mfen | that's when i try to start bip | 16:01 |
| mfen | custom upstart jobs seem to work fine, e.g. "exec sleep 10000" works as i expect it to work | 16:18 |
| mfen | this is a virtual server, i've seen some suggestions that virtual servers have issues, but nothing else is having an issue but bip | 16:18 |
| mfen | i'm going to try wrapping it in a script, perhaps that script will behave better | 16:18 |
| mfen | all right, here's a possible clue.. every time i run bip status, it gives me the status for process 4667 | 16:21 |
| mfen | i think it started with that pid at some point when i was working on this yesterday | 16:21 |
| mfen | in all the attempts since then, status bip hasn't looked at a different pid | 16:21 |
| mfen | i'm rebooting to see if i can clear it | 16:21 |
| mfen | ahhhhhhhhhhhhhhh it works now | 16:24 |
| mfen | ok so here's what i think happened: i started bip with no expect yesterday. it did its thready-fork thing, and upstart lost track of it, but remembered the pid of 4667 | 16:25 |
| mfen | after that point, upstart was trying to operate on that pid and getting confused | 16:25 |
| mfen | this looks like a bug in upstart to me.. i don't see any way to truly clear the status of a job | 16:25 |
| MFen | so there ya go. | 16:28 |
| SpamapS | MFen: thats a well known, very old bug | 17:18 |
| SpamapS | :( | 17:18 |
| SpamapS | MFen: if you get expect wrong, you get a mis-remembered pid | 17:19 |
| SpamapS | MFen: and as yet, there's no way to tell upstart to forget said pid | 17:19 |
| * jMCg subscribes to upstart-devel@ | 22:21 | |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!