[00:32] <_mup_> ensemble/robust-hook-exit r288 committed by jim.baker@canonical.com [00:32] <_mup_> Comments and better MATCH for mocking [00:37] <_mup_> ensemble/robust-hook-exit r289 committed by jim.baker@canonical.com [00:37] <_mup_> PEP8 and better test name [00:45] adam_g, did the robust-hook-exit branch work for you? [00:45] in any event, i have pushed up a version of this branch that i'm going to now submit for review, now that it has comprehensive testing [00:46] (unless of course it doesn't actually work ;) ) [00:47] adam_g, i did find one bug in the reaping mechanism, but i suspect it would not have actually impacted your use, just a resource exhaustion problem for long-term running in the machine agent [00:47] and again fixed in this most recent push [01:17] jimbaker: sorry. yes, i did test it on friday and it worked just fine. [01:33] adam_g, cool. if you could please try the latest version of the branch, which i just pushed for review, that would be very helpful [01:34] i believe in a nutshell we can model such bad hooks as something like this: sleep 0.05 && echo "Slept for 50ms" && sleep 1 && echo "Slept for 1s" && sleep 1000000 & [01:35] the & at the end be the operative part to fork, but not to close file descriptors in the child [05:23] hmm.. toying with a Venn diagram explaining why service orchestration is not config management, but encompasses some of it.. [05:23] http://spamaps.org/files/venn-config-service.png [05:24] The suggestion there is that the blue stuff on the right is what is being bolted on to config management.. [05:30] SpamapS: what do you mean 'bolted on'? [05:41] adam_g: meaning they're being added as afterthoughts. [05:42] adam_g: I don't even think, actually, that any of the free cfg mgmt solutions have a built in coordination method.. they just fake it by running themselves over and over. [05:42] adam_g: and the elasticity comes from that paradigm too [05:44] SpamapS: i dont know about chef, but the way puppet traditionally worked was, you install a central puppet master that hosts the configuration, and an agent sleeps on client machines, awakens at interval and to see if they need updates [05:45] err, checks in to see.. [05:45] Right, and there's no telling who will see the changes first unless you force a run, right? [05:45] right [05:45] well [05:45] using mcollective, now, though [05:45] you essentially "turn off" puppet on the clients [05:45] right, that new bolt on that is, IMO, a healthy competitor to ensemble. :) [05:46] and dispatch puppet runs via mcollective, which is running on the clients [05:46] because a lot of the time you need to upgrade clients first, then update the server to use some new feature.. [05:47] zright [05:47] I've felt for a while that puppet is nothing like Ensemble, and mcollective is a little like ensemble. :P [05:47] what puppet realy lacks IMHO is coordination between systems, especially for encforcing ordering dependencies with regards to state changes [05:47] * adam_g agrees 100% [05:47] :) [05:48] Ensemble ends up having that by the nature of doing things in a service oriented fashion.. but the upgrade-formula hook is, IMO, a bit weak for ongoing coordination. [05:49] I really need to be able to enumerate and act on relations in upgrade-formula.. so I can wake up the related pairs and tell them about any new necessary info. [05:50] This is where mcollective's simpler approach wins out.. because all it has to do is dispatch puppet. [06:49] <_mup_> Bug #816264 was filed: PYTHONPATH causing corrupt environment < https://launchpad.net/bugs/816264 > === daker_ is now known as daker [10:56] oh we now have config-set \o/ yaay === wrtp_ is now known as wrtp [11:28] hey all [11:28] is there a way to give a merge proposal more than one prerequisite branch? [11:35] fwereade, no.. [11:35] hazmat: does that imply I'm Doing It Wrong? ;) [11:35] fwereade, logically no.. typically i use a straight line of deps for merges.. but that's tool limitation imo. [11:36] hazmat: I resisted the urge to just keep everything in one straight line [11:36] hazmat: the thing that's bugging me is it's only going to get worse [11:36] fwereade, :-) bzr-pipeline is a nice bzr plugin that can automate some of the tedium of managing a straight-line of dependent branches [11:37] hazmat: the next thing I want to do wants to build on the results of 2 independent pipelines [11:37] hazmat: mreging trunk updates through the whole pipeline and suchlike? [11:37] fwereade, i'm using it for the security work right now, which is like 7 branches atm, pipeline helps me automate pushing changes through, as well keeping a single checkout workspace for the work on all the branches [11:37] fwereade, yup [11:37] hazmat: cool, I'll check it out [11:38] hazmat: can I retrofit it onto an existing pipeline or does it need to remember its own state? [11:39] hazmat: actually, now I think of it, I can suspend work on that feature and go back to the last one I suspended because it was breaking my brain (I think my brain has recovered a bit now ;)) [11:39] fwereade, you can retrofit an existing *branch* with just bzr reconfigure-pipeline, you can retrofit an existing pipeline but its effectively just adding a new branch and merging the existing branch to it.. [11:40] i also appreciate the single workspace model for working with multiple branches [11:40] heh, I think I'd go even more insane without separate workspaces [11:41] * hazmat looks around for pre-review coffee [11:48] fwereade, out of curiosity what editor do you favor? [11:49] hazmat: vim [11:49] hazmat: and yourself? [11:49] fwereade, right on.. i'm an emacs guy.. i believe we now have achieved 50/50 parity on the dev team ;-) [11:49] hazmat: haha, nice [11:50] hazmat: I had a bad early impression of emacs... a colleague used it and we seemed to spend about 50% of our pairing time fiddling with the settings [11:51] hazmat: not really a fair test ofc ;) [11:52] fwereade, yeah.. i think i try to keep my settings fiddling to once a year... i know incredibly little about mucking with emacs actually. [11:53] hazmat: the bad early impression of vim came from trying to type in command mode, ofc... that was exciting, but at least it wastes minimal time [11:58] fwereade, yeah.. my first vim experience was modifying some oracle setup on an ancient solaris, i ended up tracking down a sysadmin for a cheat sheet... i'm not sure that i ever fully recovered ;-) [11:59] although i've grown to like solaris since [11:59] hazmat: heh [12:00] hazmat: I think having someone who knows the tool to hold your hand is probably the critical factor [12:16] GOod mornings! [12:25] What a beautiful review queue [12:28] niemeyer: heh :) [12:28] niemeyer: good morning [12:29] fwereade: No irony.. it's awesome to come back to a review queue like that.. it'd be better to not have it because everything was already merged, but it's much more pleasant than coming back to nothing [12:33] niemeyer: cool :) [12:35] moin. [12:35] heh, editor discussions. [12:35] Hey Aram! [12:36] anyone else using acme or sam? :-). [12:36] hi! [12:36] Aram: Was just checking out your PDP11 emulator.. crazy stuff :-) [12:37] Aram: I think you'll find mostly emacs/vim users around here [12:38] yeah, I imagine :-). [12:39] I actually read all the PDP-11/40 manual and really liked it. very concise and sane compared to 4833 pages of intel manual. [12:45] Aram: How are the Intel manuals when compared to ARM's? [12:46] never read the ARM manuals. I assume they are at least smaller, as ARM is a very small arch and doesn't have 30+ years of legacy cruft. [12:49] Aram: Yeah, they feel quite pleasant overall [12:49] Aram: I know ARM itself (not the SOC makers) actually sells exactly that kind of thing, so it's somewhat expected that they'd do a good job on it [12:49] Aram: I'm curious if Intel had similar material [12:50] I would add ARM support in my emulator at some point to test the power of Go inferred interfaces. [12:50] yeah, Intel has tons of material. very very very very detailed. [12:50] also, guides for kernel developers and compiler writers. [12:50] guides for optimizing etc. [12:50] the only problem is that x86 itself is insane. [12:51] the manuals are good. very sterile, but good. [12:51] (though I like the docs coming from AMD more). [12:51] Aram: Nice/not nice :) [12:52] Aram: I note you don't have many tests on the emulator yet (hint!) [12:53] yeah... that's true. [12:53] I'm reading about the Go test framework now. [12:54] Aram: It's quite neat, even if a bit slim to my taste [12:54] Aram: I came up with a slightly more comprehensive version when I started playing with it, following more well-known xunit styles (http://labix.org/gocheck) [12:55] Aram: It builds on the standard one, though, rather than replacing it entirely [12:55] aha, nice. I'll take a look into it. [13:10] Aram: i use acme [13:11] :-). [13:11] smiley face every time i use it indeed :-) [13:14] niemeyer, g'morning! re review queue, it looks like kanban generation has gone stale again [13:14] hazmat: Hmm, indeede [13:15] PS hi gustavo! [13:17] wrtp: Rog! [13:17] wrtp: How're things going there? [13:19] niemeyer: pretty good thanks; just hacking up a website for our wedding guests and realising how terrible i am at html/css... [13:19] so many rules, so little clarity :-( [13:20] still, it's fun doing it in Go [13:21] wrtp: Hah, yeah.. coming back to HTML is always tricky after getting used to the level of cleanliness/power of a CLI [13:22] niemeyer: more like the first time. CSS didn't exist the last time i wrote a web site :-) [13:23] and some people [ahem, looks over shoulder] don't like the "vanilla" look... [13:25] wrtp: Wow, ok, that's been a _while_ :-) [13:25] yeah, probably coming up for 15 years :-) [14:20] Kanban is unblocked [14:22] jimbaker: ping [14:22] fwereade: ping [14:26] * hazmat checks out the new golang course [14:28] robbiew: ok I am relocated and back 100% now. Went over the slides with bacon and he gave me some stuff to fix. [14:28] cool [14:34] niemeyer: heyhey [14:35] fwereade: yo [14:35] fwereade: Would you mind to review this branch: https://code.launchpad.net/~jimbaker/ensemble/expose-provider-ec2/+merge/68478 [14:35] niemeyer: a pleasure :) [14:35] fwereade: This is touching the EC2 provider API, so might be nice to have your view on it since you've been so close to it [14:35] fwereade: Thanks! [14:39] <_mup_> ensemble/security-agents-with-identity r297 committed by kapil.thangavelu@canonical.com [14:39] <_mup_> hierarchy initialization includes otp container. [14:40] any sexy new formulas that people are tackling this week? (already got cloud foundry) [14:42] niemeyer, hi [14:42] fwereade, thanks for looking at the expose-provider-ec2 [14:43] jimbaker: np at all [14:43] jimbaker: I'm happy to see the ugliest code in ec2 is gone now :) [14:44] fwereade, you mean the use of all of those callbacks + authz? [14:44] jimbaker: I think we're talking about the same thing, yeah [14:45] jimbaker: for some reason it really stuck out ;) [14:45] fwereade, i couldn't really read that code, so i just had to refactor it first. it does feel cleaner [14:48] jimbaker: Hello [14:48] niemeyer, hi [14:48] jimbaker: I have one feedback on this too [14:49] niemeyer, ? [14:49] jimbaker: It doesn't feel like there's a reason to be using classes and inheritance there [14:50] niemeyer, you mean the use of EC2PortOperation for get_machine_group_name? [14:51] jimbaker: Yeah, and also the classes themselves [14:51] jimbaker: Foo(x).do_bar() can easily be do_bar() [14:52] niemeyer, oh sure, that's just following the existing convention. but i agree about it being unnecessary. if i were to do it again, i would not group it this way for the provider ops as a whole [14:52] jimbaker: Conventions are nice until they are not [14:53] niemeyer, absolutely [14:53] jimbaker: I think this should be fixed.. let's not grow up classes which are really just functions [14:53] niemeyer, so in my case, i'm reading through this operations (hey! command pattern. or something like that) trying to think, what the heck is the reason for them [14:53] because generally in python if we have classes they mean something [14:54] but in this case, even the fact that some params go in __init__, others in run, made no sense [14:54] jimbaker: Overengineering in some cases, caused by us trying to figure a good way to organize code there [14:54] jimbaker: Would you mind to reorganize it as functions and see how it goes? [14:54] so it was definitely unhelpful noise because it made it harder to start working with the code [14:55] niemeyer, sounds good, i can definitely pilot what it could look like [14:55] niemeyer, in general i think the way it's done in the dummy provider is nicer [14:55] niemeyer, its not in go, but https://gist.github.com/1100458 is a "get the right ubuntu ami" tool [14:56] the other thing that i found annoying was the use of __init__.py when we don't use that convention elsewhere in the code [14:57] smoser: Hah, sweet! [14:57] smoser: I'll dump that in my bin and start playing with it [14:57] smoser: any chance of an entry in the contest for that? ;-) [14:57] maybe [14:58] is there an sh2go converter ;-) [14:58] or maybe i'll just write that instead [15:04] smoser: it's in awk, you mean, not shell [15:04] ;) [15:05] well, yeah, sh2go would have to include implementations of sed and awk and cat and perl and wget.... [15:05] i leave that as an exercise to you [15:06] jimbaker: I'm a bit confused about the machine_id thing: I understand we can't use the instance ID, but is there really no way to figure out the machine ID giver a ProviderMachine? [15:06] given a ^^^ [15:07] fwereade, perhaps the better api would be to have the ProviderMachine know its logical machine id [15:08] fwereade: I would personally strongly favour that, but not enough to actually disapprove of the branch ;) [15:08] er, I have no idea where I got this habit of talking to myself :/ [15:09] fwereade, it looks like i'm going to do more refactoring anyway in light of the conversation i just had w/ niemeyer [15:09] jimbaker: I saw that [15:09] it would definitely clean things up, so that's a good thing [15:10] jimbaker: I'm not sure the scope we're talking about though [15:10] fwereade, i don't believe the scope is very much at all, fortunately [15:11] jimbaker: cool, as long as we're not planning to change every single operation from a class to a function [15:11] because that feels a bit drastic to me [15:11] fwereade, i'm not going to do a general refactoring of the code, but a small one to try out just using functions for the securitygroup instead of java-like commands [15:11] jimbaker: cool [15:12] fwereade, if that looks good, the next step is to have a refactoring branch that can apply it to the rest of the code in ec2 provider [15:12] jimbaker: sounds good to me [15:13] fwereade, maybe that branch can remove having both MachineProvider and ProviderMachine ;) [15:14] jimbaker: <3, that gives me a headache [15:14] fwereade, indeed. i know why that came about, but it must die ;) [15:15] jimbaker: anyway, I'l like the machine_id thing to go, but it feels separate to me -- we'll need some independent code to make machine_id gettable given a machine [15:15] jimbaker: so I don't think it's a reason to disapprove [15:16] jimbaker: but I'm not sure how to track my concern -- just file another bug, and point out that it dirties up the port interface? [15:17] fwereade, i think in general we would use "approve" for a branch at this point if the remaining changes are obvious and agreed upon. but it probably is best to just say "needs fixing" [15:17] jimbaker: cool, thanks [15:17] fwereade, so no more bug, just point it out in the review [15:26] fwereade: i. [15:26] fwereade: o/ [15:32] RoAkSoAx: er, what? [15:32] fwereade: was just saying hi (o/) [15:32] RoAkSoAx: heyhey :) [15:33] fwereade: heh how's it going [15:33] RoAkSoAx: I ought to recognise translated gestures ;) [15:33] RoAkSoAx: not too shabby; yourself? [15:34] fwereade: good good [15:34] fwereade: bug squashing day for me [15:35] fwereade: any updates on the key related stuff? === andreas__ is now known as ahasenack [15:38] RoAkSoAx: hey again :) [15:38] fwereade_: hehe troubles with xchat? [15:39] RoAkSoAx: yeah, today seems to be a stuff-breaking-randomly day [15:39] RoAkSoAx: seem to have VMs actually installing though which is reassuring [15:41] fwereade_: yeah either installer or mirror/proxy issue [15:42] fwereade_: any luck on the key thingy? [15:42] RoAkSoAx: niemeyer explained how config stuff gets to other machines, if I think a mo I might find it again ;) [15:43] fwereade_: cool [15:44] fwereade_: no I think that by the time of the sprint, we should have everything merged in trunk, or, at least have a common branch [15:44] RoAkSoAx: yep, at least there's a way we can do it ;) [15:47] fwereade_: cool, let's coordinate that tomorrow as today I'll be working on unrelated stuff [15:59] * hazmat heads out to lunch [15:59] TeTeT: glad to see you o/ :) [15:59] m_3: hey, how's that NFS formula looking? [15:59] kim0: the class starts in one hour, right? [16:00] TeTeT: yes [16:00] thanks man [16:00] RoAkSoAx: hey, how did the cloud days presentation go? I missed it. [16:00] SpamapS: i think it went well [16:00] m_3: please ping me when u're back :) [16:01] SpamapS: we couldn't demo anything as there were installation/installer issues [16:01] RoAkSoAx: thats pretty understandable given the number of moving parts involved in bare metal deployment. [16:02] SpamapS: indeed [16:05] jimbaker: i used your branch pretty extensively last night while working on some formulas, works well! [16:05] adam_g, glad to hear that! [16:09] Lunch time here [16:09] Back for more reviews in a moment [16:25] Hrm.. so I am running into an interesting situation that I think we need an answer to. [16:26] Basically, I have a service config that affects the relationships a service has. I want to be able to set the database that a service uses. [16:27] adam_g: did you see that service configs have landed? [16:29] SpamapS: i haven't but thats cool, show me how to use it over beer :) [16:30] adam_g: Indeed. :) It should let you get rid of any hard codes you have in the openstack stuff [16:30] adam_g: can I say that there are formulas for openstack in my talk? [16:33] SpamapS: cool, maintaining that has bitten me a bunch [16:33] SpamapS: yah, definitely! i wanted to actually "release" those formulas out o fmy +junk branches but a bunch of openstack bugs were introduced last week that broke everything [16:33] * SpamapS is almost done w/ slides.. just need to add a few more lolcatz [16:33] adam_g: son of a! ;) [16:51] hrmph.. 13 slides.. 40 minute presentation. I don't know if 3 minutes per slide is a good average [16:51] SpamapS: sorry, been head-down on a stack for ubuntu-classroom [16:52] m_3: no worries. :) [16:53] ec2 picks now to be dog-slow [17:02] m_3: if that demo is broken, feel free to replace with any formula that works :) [17:03] kim0: thanks [17:03] kim0: still testing [17:03] rock n roll [17:03] kim0: you know how to get the screen bigger in byobu classroom? [17:03] kim0: it's maxing out at half the screen [17:04] m_3: the host session needs to be a bigger terminal [17:04] m_3: resize the terminal running the host session, and press F5 in byobu [17:04] kim0: perfect... thanks! [17:08] m_3: are you practicing or is this going now? [17:08] kim0: scratch that [17:08] kirkland: perfect... thanks! [17:08] SpamapS: at 1800UTC [17:09] think I'll just set up the stack and talk about it rather than wait for it [17:10] m_3: which reminds me, i need to test your updates to byobu-classroom and propose it for principia [17:10] m_3: i'll try to do that today [17:11] kirkland: you're a composer, just push :) [17:11] kirkland: simple update... oneliner [17:11] SpamapS: k [17:11] m_3: kewl, just want to retest === andreas__ is now known as ahasenack [17:22] must dash, later all [17:40] http://spamaps.org/files/ensemble-oscon-2011.odp [17:40] Note that it doesn't make much sense w/o the notes [17:42] SpamapS: Is it done, or is it to-do? [17:42] niemeyer: Thursday afternoon. [17:42] to-do [17:43] http://www.oscon.com/oscon2011/public/schedule/detail/18367 [17:43] SpamapS: Awesome! [17:43] SpamapS: Re. config vs. relations, yeah, we'll need dynamic relations at some point [17:43] niemeyer: I think its a corner case that can be dodged right now [17:44] niemeyer: but one that will come up as diagrams get bigger and bigger [17:45] I wonder if there couldn't just be a command... refresh-relations .. that just tells the local unit agent to re-run all the hooks as if the relation data had changed. [17:45] SpamapS: Absolutely.. I don't have a good design in mind for it yet, to be honest, but I think we can do something good [17:46] SpamapS: It's a bit tricky because we have to take into account dependency resolution too [17:46] SpamapS: So in terms of ordering, we should get dependency resolution in first, so that we can implement this in light of the understanding that dep-resolv will provide [17:47] Yeah.. I'm less excited about dependency resolution than you guys are. :) [17:48] I can't really articulate why tho [18:05] SpamapS: Well, it's not that I'm terribly excited either. I'm much more excited about stacks [18:05] SpamapS: But feels like a very nice convenience we want to get in place [18:08] Yeah, stacks! [18:11] i think dynamic relations i think config based dependencies for generic formulas, that run apps, and have app spec'ified deps [18:13] hazmat: parsing failure :) [18:14] I think hazmat was just using lossy compression [18:14] up the bitrate a bit [18:15] connection loss [18:17] but just as a common example a rails or wsgi app that checkouts from vcs, and deploys the app.. but it has to assume currently a set of optional/required deps representing app infrastructure choices, does it use memcached or postgres v. mysql v. nosql. [18:17] really their runtime dynamic deps, and what we capture are static ones [18:17] hazmat: Right.. that's exactly the context [18:19] niemeyer, effectively we're another cli api.. for declaring additional deps.. although we lack an execution context that's service local rather than unit local. [18:20] hazmat: We don't declare additional deps.. the dependencies can be pre-defined, since they are always fixed [18:20] hazmat: Unless there's some genetic voodoo within the formula :-) [18:22] hazmat: E.g. the hooks will be pre-defined [18:23] There's very cool stuff we can do around this.. [18:23] Hopefully before 2017 [18:23] :-) [18:32] <_mup_> Bug #816581 was filed: Ensemble needs easy end user initial configuration < https://launchpad.net/bugs/816581 > [18:36] <_mup_> ensemble/expose-hooks r282 committed by jim.baker@canonical.com [18:36] <_mup_> Merged trunk === niemeyer_ is now known as niemeyer [18:49] jimbaker: robus-hook-exit looks nice in general, but I'm missing some of the context about why it's needed [18:49] jimbaker: I've posted some comments there === daker is now known as daker_ [18:50] niemeyer, ok. i will take a look at your comments [18:50] jimbaker: Feel free to bring any of them up for debate here, please [18:51] niemeyer, yes, i assume you read the corresponding bug? [18:51] jimbaker: I've read your merge proposal description, the code, and the test [18:52] niemeyer, re the refactoring, sounds good to me, again i was following the convention in this code [18:52] jimbaker: Which convention? [18:52] niemeyer, so does it make sense the scenario being described, that a hook script may exit, but w/o its file descriptors being closed? [18:52] niemeyer, the convention of the existing file [18:53] jimbaker: Which convention, specifically? [18:54] niemeyer, the usage of the nested functions cleanup_process, which i followed in doing cleanup_ended [18:54] jimbaker: Ok, I see [18:54] jimbaker: When introducing new logic, some reevaluation is needed to ensure the style used still makes sense [18:55] niemeyer, sure, we just had that discussion earlier ;) [18:55] jimbaker: It might not be a big deal with a small closure.. when you have more than a full screen of closure which doesn't really use the fact it's a closure in a good way, something isn't right [18:56] niemeyer, correct neither nested function closes over variables [18:56] Right, anyway.. mionr [18:56] mionr [18:56] miNNOr [18:56] :-0 [18:56] jimbaker: So [18:56] jimbaker: On the actual meaning of the branch [18:57] niemeyer, i think it is captured succinctly in a hook like this one: [18:57] jimbaker: I don't understand the scenario.. a hook exiting without fds being closed? How's that a problem? [18:57] sleep 0.05 && echo "Slept for 50ms" && sleep 1 && echo "Slept for 1s" && sleep 1000000 & [18:57] some computation, some output [18:58] and of course the very relevant & at the end to fork it in the background [18:58] jimbaker, but how is that an exceptional condition outside of its a taking a long time [18:58] :-) [18:58] jimbaker: Ok.. sorry, I'm not trying to be difficult in any way. How's that an issue? [18:58] hazmat, niemeyer - this arises in certain formulas where we have badly behaving code [18:59] jimbaker: This feels pretty normal? [18:59] rabbitmq specifically per the bug [18:59] so in particular adam_g brought it up [18:59] niemeyer, it shouldn't be normal but it exists [18:59] What exists? [18:59] i worked on the branch because i made the original change from exited to ended semantics [18:59] hazmat: It's a problem, and it exists.. ookaaay :-) [18:59] and this adds robustness to how we handle hooks, we had switched in the past to get better testing output [18:59] thomas herve diagnosed it [19:00] jimbaker: What was the problem he diagnosed? [19:00] jimbaker: and how is the branch fixing it? [19:00] indeed.. that was very helpful.. he pointed out the difference between processEnded vs processExited on the process protocol [19:01] that child processes were not properly closing file descriptors because how they were redirecting things [19:01] in the context of a rabbitmq, which had some odd behavior per its start scripts [19:01] hazmat, correct [19:01] and apparently this not so uncommon and there's an easy fix for us [19:02] jimbaker: How not closing file descriptors affect the way process ending happens? [19:02] niemeyer, its waiting for the file descriptors to be closed [19:02] vs waiting for the process to exit [19:02] niemeyer, waiting on process ending means waiting for the FDs to be closed [19:02] niemeyer, so this is generally the right thing to do because it means we fully capture useful stdout/stderr for logging [19:03] niemeyer, i had changed the semantics from exited to ended when i standardized the logging [19:03] niemeyer, as you may recall, we were doing some odd things there that ultimately was just making the tests to *usually* wait long enough for the logs to be complete [19:04] jimbaker: No, it doesn't mean that in general.. if you fork a process, and wait for it to exit with normal system calls, you don't wait for FDs to be closed [19:04] niemeyer, correct [19:04] niemeyer, so that's the distinction between waiting on process exited vs process ended [19:05] niemeyer, the twisted process protocol supports both, since they both have some nice qualities [19:05] m_3: is your mongo formula in principia yet? [19:05] jimbaker: Which fd is processEnded holding out on? stdin? [19:05] niemeyer, the FD is going to be stdout or stderr [19:05] or both [19:06] niemeyer, in this particular test hook that i just pasted here, it's stdout [19:06] SpamapS: nope, I just got hazmat's and stripped it of replication [19:06] SpamapS: negronjl has one inbound that I expect will be nice [19:06] niemeyer, so because we capture these streams for logging, ideally we work with them [19:07] jimbaker: There's some detail missing still.. if a process exits, you get a SIGPIPE [19:07] niemeyer, in general this doesn't impact hook code - the process exits, it's done for the invoker. however if there's still more output, these are still written to logs [19:07] m_3: I'll push it soon .. [19:07] jimbaker: and the process ended [19:08] jimbaker: Your example shows a sleep going into background, and never returning [19:08] niemeyer, correct - a child process just hanging around just like what we see in real cases [19:09] niemeyer, who knows when it returns? [19:09] jimbaker: Hmm, ok, I think I start to see what you mean [19:09] niemeyer, in any event, in that case we reap them because we don't want processes like that around indefinitely [19:09] jimbaker: There's a child process, started by the hook, and we want it to continue running [19:09] jimbaker: Is that the case? [19:10] niemeyer, i chose 5 seconds because it's much larger than reasonable for good children, but doesn't matter in practice even w/ lots of bad children [19:10] jimbaker: Uh.. lost again [19:10] niemeyer, only for a little bit, just to let its logs settle [19:10] the actual reaper time probably would be fine w/ something like 200 ms [19:10] jimbaker: there are daemons that take a long time to start up .. but usually they don't let the parent die until they're ready for it to die. [19:11] SpamapS, correct - this doesn't impact such "good" children [19:11] niemeyer, so we are not trying to solve a larger problem of having time limits on hooks [19:11] In rabbit's case, it seems like we should report a bug in the init script that it needs to run w/ nohup [19:12] SpamapS, correct, that would be the normal simple way to get the desired behavior [19:12] which turns bad children into good children by first redirecting its own fds to dev null or a log [19:12] SpamapS, :) [19:12] like long-term care insurance [19:12] the analogies just pour in [19:12] m_3: Great example node.js app btw.. [19:13] m_3: you may not have gotten much uptake on that... today is node.js day at OSCON ;) [19:13] cool thanks man [19:13] jimbaker, we are and there is a bug open for that, but that has other things it needs to take into account like respecting debug hooks which will break time limits.. a standard hook timeout though needs to be quite large (large installs come to mind). [19:13] didn't know if I was going into too much detail [19:13] lots of zzzz's I think [19:13] hazmat, correct, a much larger problem [19:13] m_3: The log of the session may prove useful to someone. :) [19:14] wish there was a way to get the ajaxterm session too [19:14]