_mup_ensemble/robust-hook-exit r288 committed by jim.baker@canonical.com00:32
_mup_Comments and better MATCH for mocking00:32
_mup_ensemble/robust-hook-exit r289 committed by jim.baker@canonical.com00:37
_mup_PEP8 and better test name00:37
jimbakeradam_g, did the robust-hook-exit branch work for you?00:45
jimbakerin any event, i have pushed up a version of this branch that i'm going to now submit for review, now that it has comprehensive testing00:45
jimbaker(unless of course it doesn't actually work ;) )00:46
jimbakeradam_g, i did find one bug in the reaping mechanism, but i suspect it would not have actually impacted your use, just a resource exhaustion problem for long-term running in the machine agent00:47
jimbakerand again fixed in this most recent push00:47
adam_gjimbaker: sorry. yes, i did test it on friday and it worked just fine. 01:17
jimbakeradam_g, cool. if you could please try the latest version of the branch, which i just pushed for review, that would be very helpful01:33
jimbakeri believe in a nutshell we can model such bad hooks as something like this: sleep 0.05 && echo "Slept for 50ms" && sleep 1 && echo "Slept for 1s" && sleep 1000000 &01:34
jimbakerthe & at the end be the operative part to fork, but not to close file descriptors in the child01:35
SpamapShmm.. toying with a Venn diagram explaining why service orchestration is not config management, but encompasses some of it..05:23
SpamapSThe suggestion there is that the blue stuff on the right is what is being bolted on to config management..05:24
adam_gSpamapS: what do you mean 'bolted on'?05:30
SpamapSadam_g: meaning they're being added as afterthoughts.05:41
SpamapSadam_g: I don't even think, actually, that any of the free cfg mgmt solutions have a built in coordination method.. they just fake it by running themselves over and over.05:42
SpamapSadam_g: and the elasticity comes from that paradigm too05:42
adam_gSpamapS: i dont know about chef, but the way puppet traditionally worked was, you install a central puppet master that hosts the configuration, and an agent sleeps on client machines, awakens at interval and to see if they need updates05:44
adam_gerr, checks in to see..05:45
SpamapSRight, and there's no telling who will see the changes first unless you force a run, right?05:45
adam_gusing mcollective, now, though05:45
adam_gyou essentially "turn off" puppet on the clients05:45
SpamapSright, that new bolt on that is, IMO, a healthy competitor to ensemble. :)05:45
adam_gand dispatch puppet runs via mcollective, which is running on the clients05:46
SpamapSbecause a lot of the time you need to upgrade clients first, then update the server to use some new feature..05:46
SpamapSI've felt for a while that puppet is nothing like Ensemble, and mcollective is a little like ensemble. :P05:47
adam_gwhat puppet realy lacks IMHO is coordination between systems, especially for encforcing ordering dependencies with regards to state changes05:47
* adam_g agrees 100% 05:47
SpamapSEnsemble ends up having that by the nature of doing things in a service oriented fashion.. but the upgrade-formula hook is, IMO, a bit weak for ongoing coordination.05:48
SpamapSI really need to be able to enumerate and act on relations in upgrade-formula.. so I can wake up the related pairs and tell them about any new necessary info.05:49
SpamapSThis is where mcollective's simpler approach wins out.. because all it has to do is dispatch puppet.05:50
_mup_Bug #816264 was filed: PYTHONPATH causing corrupt environment <Ensemble:New> < https://launchpad.net/bugs/816264 >06:49
=== daker_ is now known as daker
kim0oh we now have config-set \o/ yaay10:56
=== wrtp_ is now known as wrtp
fwereadehey all11:28
fwereadeis there a way to give a merge proposal more than one prerequisite branch?11:28
hazmatfwereade, no..11:35
fwereadehazmat: does that imply I'm Doing It Wrong? ;)11:35
hazmatfwereade, logically no.. typically i use a straight line of deps for merges.. but that's tool limitation imo.11:35
fwereadehazmat: I resisted the urge to just keep everything in one straight line11:36
fwereadehazmat: the thing that's bugging me is it's only going to get worse11:36
hazmatfwereade, :-)  bzr-pipeline is a nice bzr plugin that can automate some of the tedium of managing a straight-line of dependent branches11:36
fwereadehazmat: the next thing I want to do wants to build on the results of 2 independent pipelines11:37
fwereadehazmat: mreging trunk updates through the whole pipeline and suchlike?11:37
hazmatfwereade, i'm using it for the security work right now, which is like 7 branches atm, pipeline helps me automate pushing changes through, as well keeping a single checkout workspace for the work on all the branches 11:37
hazmatfwereade, yup11:37
fwereadehazmat: cool, I'll check it out11:37
fwereadehazmat: can I retrofit it onto an existing pipeline or does it need to remember its own state?11:38
fwereadehazmat: actually, now I think of it, I can suspend work on that feature and go back to the last one I suspended because it was breaking my brain (I think my brain has recovered a bit now ;))11:39
hazmatfwereade, you can retrofit an existing *branch* with just bzr reconfigure-pipeline, you can retrofit an existing pipeline but its effectively just adding a new branch and merging the existing branch to it..11:39
hazmati also appreciate the single workspace model for working with multiple branches11:40
fwereadeheh, I think I'd go even more insane without separate workspaces11:40
* hazmat looks around for pre-review coffee11:41
hazmatfwereade, out of curiosity what editor do you favor?11:48
fwereadehazmat: vim11:49
fwereadehazmat: and yourself?11:49
hazmatfwereade, right on.. i'm an emacs guy.. i believe we now have achieved 50/50 parity on the dev team ;-)11:49
fwereadehazmat: haha, nice11:49
fwereadehazmat: I had a bad early impression of emacs... a colleague used it and we seemed to spend about 50% of our pairing time fiddling with the settings11:50
fwereadehazmat: not really a fair test ofc ;)11:51
hazmatfwereade, yeah.. i think i try to keep my settings fiddling to once a year... i know incredibly little about mucking with emacs actually.11:52
fwereadehazmat: the bad early impression of vim came from trying to type in command mode, ofc... that was exciting, but at least it wastes minimal time11:53
hazmatfwereade, yeah.. my first vim experience was modifying some oracle setup on an ancient solaris, i ended up tracking down a sysadmin for a cheat sheet... i'm not sure that i ever fully recovered ;-)11:58
hazmatalthough i've grown to like solaris since11:59
fwereadehazmat: heh11:59
fwereadehazmat: I think having someone who knows the tool to hold your hand is probably the critical factor12:00
niemeyerGOod mornings!12:16
niemeyerWhat a beautiful review queue12:25
fwereadeniemeyer: heh :)12:28
fwereadeniemeyer: good morning12:28
niemeyerfwereade: No irony.. it's awesome to come back to a review queue like that.. it'd be better to not have it because everything was already merged, but it's much more pleasant than coming back to nothing12:29
fwereadeniemeyer: cool :)12:33
Aramheh, editor discussions.12:35
niemeyerHey Aram!12:35
Aramanyone else using acme or sam? :-).12:36
niemeyerAram: Was just checking out your PDP11 emulator.. crazy stuff :-)12:36
niemeyerAram: I think you'll find mostly emacs/vim users around here12:37
Aramyeah, I imagine :-).12:38
AramI actually read all the PDP-11/40 manual and really liked it. very concise and sane compared to 4833 pages of intel manual.12:39
niemeyerAram: How are the Intel manuals when compared to ARM's?12:45
Aramnever read the ARM manuals. I assume they are at least smaller, as ARM is a very small arch and doesn't have 30+ years of legacy cruft.12:46
niemeyerAram: Yeah, they feel quite pleasant overall12:49
niemeyerAram: I know ARM itself (not the SOC makers) actually sells exactly that kind of thing, so it's somewhat expected that they'd do a good job on it12:49
niemeyerAram: I'm curious if Intel had similar material12:49
AramI would add ARM support in my emulator at some point to test the power of Go inferred interfaces.12:50
Aramyeah, Intel has tons of material. very very very very detailed.12:50
Aramalso, guides for kernel developers and compiler writers.12:50
Aramguides for optimizing etc.12:50
Aramthe only problem is that x86 itself is insane.12:50
Aramthe manuals are good. very sterile, but good.12:51
Aram(though I like the docs coming from AMD more).12:51
niemeyerAram: Nice/not nice :)12:51
niemeyerAram: I note you don't have many tests on the emulator yet (hint!)12:52
Aramyeah... that's true.12:53
AramI'm reading about the Go test framework now.12:53
niemeyerAram: It's quite neat, even if a bit slim to my taste12:54
niemeyerAram: I came up with a slightly more comprehensive version when I started playing with it, following more well-known xunit styles (http://labix.org/gocheck)12:54
niemeyerAram: It builds on the standard one, though, rather than replacing it entirely12:55
Aramaha, nice. I'll take a look into it.12:55
wrtpAram: i use acme13:10
wrtpsmiley face every time i use it indeed :-)13:11
hazmatniemeyer, g'morning! re review queue, it looks like  kanban generation has gone stale again13:14
niemeyerhazmat: Hmm, indeede13:14
wrtpPS hi gustavo!13:15
niemeyerwrtp: Rog!13:17
niemeyerwrtp: How're things going there?13:17
wrtpniemeyer: pretty good thanks; just hacking up a website for our wedding guests and realising how terrible i am at html/css...13:19
wrtpso many rules, so little clarity :-(13:19
wrtpstill, it's fun doing it in Go13:20
niemeyerwrtp: Hah, yeah.. coming back to HTML is always tricky after getting used to the level of cleanliness/power of  a CLI13:21
wrtpniemeyer: more like the first time. CSS didn't exist the last time i wrote a web site :-)13:22
wrtpand some people [ahem, looks over shoulder] don't like the "vanilla" look...13:23
niemeyerwrtp: Wow, ok, that's been a _while_ :-)13:25
wrtpyeah, probably coming up for 15 years :-)13:25
niemeyerKanban is unblocked14:20
niemeyerjimbaker: ping14:22
niemeyerfwereade: ping14:22
* hazmat checks out the new golang course14:26
jcastrorobbiew: ok I am relocated and back 100% now. Went over the slides with bacon and he gave me some stuff to fix.14:28
fwereadeniemeyer: heyhey14:34
niemeyerfwereade: yo14:35
niemeyerfwereade: Would you mind to review this branch: https://code.launchpad.net/~jimbaker/ensemble/expose-provider-ec2/+merge/6847814:35
fwereadeniemeyer: a pleasure :)14:35
niemeyerfwereade: This is touching the EC2 provider API, so might be nice to have your view on it since you've been so close to it14:35
niemeyerfwereade: Thanks!14:35
_mup_ensemble/security-agents-with-identity r297 committed by kapil.thangavelu@canonical.com14:39
_mup_hierarchy initialization includes otp container.14:39
jcastroany sexy new formulas that people are tackling this week? (already got cloud foundry)14:40
jimbakerniemeyer, hi14:42
jimbakerfwereade, thanks for looking at the expose-provider-ec214:42
fwereadejimbaker: np at all14:43
fwereadejimbaker: I'm happy to see the ugliest code in ec2 is gone now :)14:43
jimbakerfwereade, you mean the use of all of those callbacks + authz?14:44
fwereadejimbaker: I think we're talking about the same thing, yeah14:44
fwereadejimbaker: for some reason it really stuck out ;)14:45
jimbakerfwereade, i couldn't really read that code, so i just had to refactor it first. it does feel cleaner14:45
niemeyerjimbaker: Hello14:48
jimbakerniemeyer, hi14:48
niemeyerjimbaker: I have one feedback on this too14:48
jimbakerniemeyer, ?14:49
niemeyerjimbaker: It doesn't feel like there's a reason to be using classes and inheritance there14:49
jimbakerniemeyer, you mean the use of EC2PortOperation for get_machine_group_name?14:50
niemeyerjimbaker: Yeah, and also the classes themselves14:51
niemeyerjimbaker: Foo(x).do_bar() can easily be do_bar()14:51
jimbakerniemeyer, oh sure, that's just following the existing convention. but i agree about it being unnecessary. if i were to do it again, i would not group it this way for the provider ops as a whole14:52
niemeyerjimbaker: Conventions are nice until they are not14:52
jimbakerniemeyer, absolutely14:53
niemeyerjimbaker: I think this should be fixed.. let's not grow up classes which are really just functions14:53
jimbakerniemeyer, so in my case, i'm reading through this operations (hey! command pattern. or something like that) trying to think, what the heck is the reason for them14:53
jimbakerbecause generally in python if we have classes they mean something14:53
jimbakerbut in this case, even the fact that some params go in __init__, others in run, made no sense14:54
niemeyerjimbaker: Overengineering in some cases, caused by us trying to figure a good way to organize code there14:54
niemeyerjimbaker: Would you mind to reorganize it as functions and see how it goes?14:54
jimbakerso it was definitely unhelpful noise because it made it harder to start working with the code14:54
jimbakerniemeyer, sounds good, i can definitely pilot what it could look like14:55
jimbakerniemeyer, in general i think the way it's done in the dummy provider is nicer14:55
smoserniemeyer, its not in go, but https://gist.github.com/1100458 is a "get the right ubuntu ami" tool14:55
jimbakerthe other thing that i found annoying was the use of __init__.py when we don't use that convention elsewhere in the code14:56
niemeyersmoser: Hah, sweet!14:57
niemeyersmoser: I'll dump that in my bin and start playing with it14:57
niemeyersmoser: any chance of an entry in the contest for that? ;-)14:57
smoseris there an sh2go converter ;-)14:58
smoseror maybe i'll just write that instead14:58
ahasenacksmoser: it's in awk, you mean, not shell15:04
smoserwell, yeah, sh2go would have to include implementations of sed and awk and cat and perl and wget....15:05
smoseri leave that as an exercise to you15:05
fwereadejimbaker: I'm a bit confused about the machine_id thing: I understand we can't use the instance ID, but is there really no way to figure out the machine ID giver a ProviderMachine?15:06
fwereadegiven a ^^^15:06
jimbakerfwereade, perhaps the better api would be to have the ProviderMachine know its logical machine id15:07
fwereadefwereade: I would personally strongly favour that, but not enough to actually disapprove of the branch ;)15:08
fwereadeer, I have no idea where I got this habit of talking to myself :/15:08
jimbakerfwereade, it looks like i'm going to do more refactoring anyway in light of the conversation i just had w/ niemeyer 15:09
fwereadejimbaker: I saw that15:09
jimbakerit would definitely clean things up, so that's a good thing15:09
fwereadejimbaker: I'm not sure the scope we're talking about though15:10
jimbakerfwereade, i don't believe the scope is very much at all, fortunately15:10
fwereadejimbaker: cool, as long as we're not planning to change every single operation from a class to a function15:11
fwereadebecause that feels a bit drastic to me15:11
jimbakerfwereade, i'm not going to do a general refactoring of the code, but a small one to try out just using functions for the securitygroup instead of java-like commands15:11
fwereadejimbaker: cool15:11
jimbakerfwereade, if that looks good, the next step is to have a refactoring branch that can apply it to the rest of the code in ec2 provider15:12
fwereadejimbaker: sounds good to me15:12
jimbakerfwereade, maybe that branch can remove having both MachineProvider and ProviderMachine ;)15:13
fwereadejimbaker: <3, that gives me a headache15:14
jimbakerfwereade, indeed. i know why that came about, but it must die ;)15:14
fwereadejimbaker: anyway, I'l like the machine_id thing to go, but it feels separate to me -- we'll need some independent code to make machine_id gettable given a machine15:15
fwereadejimbaker: so I don't think it's a reason to disapprove15:15
fwereadejimbaker: but I'm not sure how to track my concern -- just file another bug, and point out that it dirties up the port interface?15:16
jimbakerfwereade, i think in general we would use "approve" for a branch at this point if the remaining changes are obvious and agreed upon. but it probably is best to just say "needs fixing"15:17
fwereadejimbaker: cool, thanks15:17
jimbakerfwereade, so no more bug, just point it out in the review15:17
RoAkSoAxfwereade: i.15:26
RoAkSoAxfwereade: o/15:26
fwereadeRoAkSoAx: er, what?15:32
RoAkSoAxfwereade: was just saying hi (o/)15:32
fwereadeRoAkSoAx: heyhey :)15:32
RoAkSoAxfwereade: heh how's it going15:33
fwereadeRoAkSoAx: I ought to recognise translated gestures ;)15:33
fwereadeRoAkSoAx: not too shabby; yourself?15:33
RoAkSoAxfwereade: good good15:34
RoAkSoAxfwereade: bug squashing day for me15:34
RoAkSoAxfwereade: any updates on the key related stuff?15:35
=== andreas__ is now known as ahasenack
fwereade_RoAkSoAx: hey again :)15:38
RoAkSoAxfwereade_: hehe troubles with xchat?15:38
fwereade_RoAkSoAx: yeah, today seems to be a stuff-breaking-randomly day15:39
fwereade_RoAkSoAx: seem to have VMs actually installing though which is reassuring15:39
RoAkSoAxfwereade_: yeah either installer or mirror/proxy issue15:41
RoAkSoAxfwereade_: any luck on the key thingy?15:42
fwereade_RoAkSoAx: niemeyer explained how config stuff gets to other machines, if I think a mo I might find it again ;)15:42
RoAkSoAxfwereade_: cool15:43
RoAkSoAxfwereade_: no I think that by the time of the sprint, we should have everything merged in trunk, or, at least have a common branch15:44
fwereade_RoAkSoAx: yep, at least there's a way we can do it ;)15:44
RoAkSoAxfwereade_: cool, let's coordinate that tomorrow as today I'll be working on unrelated stuff15:47
* hazmat heads out to lunch15:59
kim0TeTeT: glad to see you o/ :)15:59
SpamapSm_3: hey, how's that NFS formula looking?15:59
TeTeTkim0: the class starts in one hour, right?15:59
kim0TeTeT: yes16:00
kim0thanks man16:00
SpamapSRoAkSoAx: hey, how did the cloud days presentation go? I missed it.16:00
RoAkSoAxSpamapS: i think it went well16:00
kim0m_3: please ping me when u're back :)16:00
RoAkSoAxSpamapS: we couldn't demo anything as there were installation/installer issues16:01
SpamapSRoAkSoAx: thats pretty understandable given the number of moving parts involved in bare metal deployment.16:01
RoAkSoAxSpamapS: indeed16:02
adam_gjimbaker: i used your branch pretty extensively last night while working on some formulas, works well!16:05
jimbakeradam_g, glad to hear that!16:05
niemeyerLunch time here16:09
niemeyerBack for more reviews in a moment16:09
SpamapSHrm.. so I am running into an interesting situation that I think we need an answer to.16:25
SpamapSBasically, I have a service config that affects the relationships a service has. I want to be able to set the database that a service uses.16:26
SpamapSadam_g: did you see that service configs have landed?16:27
adam_gSpamapS: i haven't but thats cool, show me how to use it over beer :)16:29
SpamapSadam_g: Indeed. :) It should let you get rid of any hard codes you have in the openstack stuff16:30
SpamapSadam_g: can I say that there are formulas for openstack in my talk?16:30
adam_gSpamapS: cool, maintaining that has bitten me a bunch 16:33
adam_gSpamapS: yah, definitely! i wanted to actually "release" those formulas out o fmy +junk branches but a bunch of openstack bugs were introduced last week that broke everything16:33
* SpamapS is almost done w/ slides.. just need to add a few more lolcatz16:33
SpamapSadam_g: son of a! ;)16:33
SpamapShrmph.. 13 slides.. 40 minute presentation. I don't know if 3 minutes per slide is a good average16:51
m_3SpamapS: sorry, been head-down on a stack for ubuntu-classroom16:51
SpamapSm_3: no worries. :)16:52
m_3ec2 picks now to be dog-slow16:53
kim0m_3: if that demo is broken, feel free to replace with any formula that works :)17:02
m_3kim0: thanks17:03
m_3kim0: still testing17:03
kim0rock n roll17:03
m_3kim0: you know how to get the screen bigger in byobu classroom?17:03
m_3kim0: it's maxing out at half the screen17:03
kirklandm_3: the host session needs to be a bigger terminal17:04
kirklandm_3: resize the terminal running the host session, and press F5 in byobu17:04
m_3kim0: perfect... thanks!17:04
SpamapSm_3: are you practicing or is this going now?17:08
m_3kim0: scratch that17:08
m_3kirkland: perfect... thanks!17:08
m_3SpamapS: at 1800UTC17:08
m_3think I'll just set up the stack and talk about it rather than wait for it17:09
kirklandm_3: which reminds me, i need to test your updates to byobu-classroom and propose it for principia17:10
kirklandm_3: i'll try to do that today17:10
SpamapSkirkland: you're a composer, just push :)17:11
m_3kirkland: simple update... oneliner17:11
kirklandSpamapS: k17:11
kirklandm_3: kewl, just want to retest17:11
=== andreas__ is now known as ahasenack
fwereade_must dash, later all17:22
SpamapSNote that it doesn't make much sense w/o the notes17:40
niemeyerSpamapS: Is it done, or is it to-do?17:42
SpamapSniemeyer: Thursday afternoon.17:42
niemeyerSpamapS: Awesome!17:43
niemeyerSpamapS: Re. config vs. relations, yeah, we'll need dynamic relations at some point17:43
SpamapSniemeyer: I think its a corner case that can be dodged right now17:43
SpamapSniemeyer: but one that will come up as diagrams get bigger and bigger17:44
SpamapSI wonder if there couldn't just be a command... refresh-relations .. that just tells the local unit agent to re-run all the hooks as if the relation data had changed.17:45
niemeyerSpamapS: Absolutely.. I don't have a good design in mind for it yet, to be honest, but I think we can do something good17:45
niemeyerSpamapS: It's a bit tricky because we have to take into account dependency resolution too17:46
niemeyerSpamapS: So in terms of ordering, we should get dependency resolution in first, so that we can implement this in light of the understanding that dep-resolv will provide17:46
SpamapSYeah.. I'm less excited about dependency resolution than you guys are. :)17:47
SpamapSI can't really articulate why tho17:48
niemeyerSpamapS: Well, it's not that I'm terribly excited either.  I'm much more excited about stacks18:05
niemeyerSpamapS: But feels like a very nice convenience we want to get in place18:05
SpamapSYeah, stacks!18:08
hazmati think dynamic relations i think config based dependencies for generic formulas, that run apps, and have app spec'ified deps18:11
niemeyerhazmat: parsing failure :)18:13
SpamapSI think hazmat was just using lossy compression18:14
SpamapSup the bitrate a bit18:14
hazmatconnection loss18:15
hazmatbut just as a common example a rails or wsgi app that checkouts from vcs, and deploys the app.. but it has to assume currently a set of optional/required deps representing app infrastructure choices, does it use memcached or postgres v. mysql v. nosql.18:17
hazmatreally their runtime dynamic deps, and what we capture are static ones18:17
niemeyerhazmat: Right.. that's exactly the context18:17
hazmatniemeyer, effectively we're another cli api.. for declaring additional deps.. although we lack an execution context that's service local rather than unit local.18:19
niemeyerhazmat: We don't declare additional deps.. the dependencies can be pre-defined, since they are always fixed18:20
niemeyerhazmat: Unless there's some genetic voodoo within the formula :-)18:20
niemeyerhazmat: E.g. the hooks will be pre-defined18:22
niemeyerThere's very cool stuff we can do around this..18:23
niemeyerHopefully before 201718:23
_mup_Bug #816581 was filed: Ensemble needs easy end user initial configuration <Ensemble:New> < https://launchpad.net/bugs/816581 >18:32
_mup_ensemble/expose-hooks r282 committed by jim.baker@canonical.com18:36
_mup_Merged trunk18:36
=== niemeyer_ is now known as niemeyer
niemeyerjimbaker: robus-hook-exit looks nice in general, but I'm missing some of the context about why it's needed18:49
niemeyerjimbaker: I've posted some comments there18:49
=== daker is now known as daker_
jimbakerniemeyer, ok. i will take a look at your comments18:50
niemeyerjimbaker: Feel free to bring any of them up for debate here, please18:50
jimbakerniemeyer, yes, i assume you read the corresponding bug?18:51
niemeyerjimbaker: I've read your merge proposal description, the code, and the test18:51
jimbakerniemeyer, re the refactoring, sounds good to me, again i was following the convention in this code18:52
niemeyerjimbaker: Which convention?18:52
jimbakerniemeyer, so does it make sense the scenario being described, that a hook script may exit, but w/o its file descriptors being closed?18:52
jimbakerniemeyer, the convention of the existing file18:52
niemeyerjimbaker: Which convention, specifically?18:53
jimbakerniemeyer, the usage of the nested functions cleanup_process, which i followed in doing cleanup_ended18:54
niemeyerjimbaker: Ok, I see18:54
niemeyerjimbaker: When introducing new logic, some reevaluation is needed to ensure the style used still makes sense18:54
jimbakerniemeyer, sure, we just had that discussion earlier ;)18:55
niemeyerjimbaker: It might not be a big deal with a small closure.. when you have more than a full screen of closure which doesn't really use the fact it's a closure in a good way, something isn't right18:55
jimbakerniemeyer, correct neither nested function closes over variables18:56
niemeyerRight, anyway.. mionr18:56
niemeyerjimbaker: So18:56
niemeyerjimbaker: On the actual meaning of the branch18:56
jimbakerniemeyer, i think it is captured succinctly in a hook like this one:18:57
niemeyerjimbaker: I don't understand the scenario.. a hook exiting without fds being closed?  How's that a problem?18:57
jimbakersleep 0.05 && echo "Slept for 50ms" && sleep 1 && echo "Slept for 1s" && sleep 1000000 &18:57
jimbakersome computation, some output18:57
jimbakerand of course the very relevant & at the end to fork it in the background18:58
hazmatjimbaker, but how is that an exceptional condition outside of its a taking a long time18:58
niemeyerjimbaker: Ok.. sorry, I'm not trying to be difficult in any way.  How's that an issue?18:58
jimbakerhazmat, niemeyer - this arises in certain formulas where we have badly behaving code18:58
niemeyerjimbaker: This feels pretty normal?18:59
hazmatrabbitmq specifically per the bug18:59
jimbakerso in particular adam_g  brought it up18:59
hazmatniemeyer, it shouldn't be normal but it exists18:59
niemeyerWhat exists?18:59
jimbakeri worked on the branch because i made the original change from exited to ended semantics18:59
niemeyerhazmat: It's a problem, and it exists.. ookaaay :-)18:59
hazmatand this adds robustness to how we handle hooks, we had switched in the past to get better testing output18:59
jimbakerthomas herve diagnosed it18:59
niemeyerjimbaker: What was the problem he diagnosed?19:00
niemeyerjimbaker: and how is the branch fixing it?19:00
hazmatindeed.. that was very helpful.. he pointed out the difference between processEnded vs processExited on the process protocol19:00
jimbakerthat child processes were not properly closing file descriptors because how they were redirecting things19:01
hazmatin the context of a rabbitmq, which had some odd behavior per its start scripts19:01
jimbakerhazmat, correct19:01
jimbakerand apparently this not so uncommon and there's an easy fix for us19:01
niemeyerjimbaker: How not closing file descriptors affect the way process ending happens?19:02
hazmatniemeyer, its waiting for the file descriptors to be closed19:02
hazmatvs waiting for the process to exit19:02
jimbakerniemeyer, waiting on process ending means waiting for the FDs to be closed19:02
jimbakerniemeyer, so this is generally the right thing to do because it means we fully capture useful stdout/stderr for logging19:02
jimbakerniemeyer, i had changed the semantics from exited to ended when i standardized the logging19:03
jimbakerniemeyer, as you may recall, we were doing some odd things there that ultimately was just making the tests to *usually* wait long enough for the logs to be complete19:03
niemeyerjimbaker: No, it doesn't mean that in general.. if you fork a process, and wait for it to exit with normal system calls, you don't wait for FDs to be closed19:04
jimbakerniemeyer, correct19:04
jimbakerniemeyer, so that's the distinction between waiting on process exited vs process ended19:04
jimbakerniemeyer, the twisted process protocol supports both, since they both have some nice qualities19:05
SpamapSm_3: is your mongo formula in principia yet?19:05
niemeyerjimbaker: Which fd is processEnded holding out on?  stdin?19:05
jimbakerniemeyer, the FD is going to be stdout or stderr19:05
jimbakeror both19:05
jimbakerniemeyer, in this particular test hook that i just pasted here, it's stdout19:06
m_3SpamapS: nope, I just got hazmat's and stripped it of replication19:06
m_3SpamapS: negronjl has one inbound that I expect will be nice19:06
jimbakerniemeyer, so because we capture these streams for logging, ideally we work with them19:06
niemeyerjimbaker: There's some detail missing still.. if a process exits, you get a SIGPIPE19:07
jimbakerniemeyer, in general this doesn't impact hook code - the process exits, it's done for the invoker. however if there's still more output, these are still written to logs19:07
negronjlm_3:  I'll push it soon .. 19:07
niemeyerjimbaker: and the process ended19:07
niemeyerjimbaker: Your example shows a sleep going into background, and never returning19:08
jimbakerniemeyer, correct - a child process just hanging around just like what we see in real cases19:08
jimbakerniemeyer, who knows when it returns? 19:09
niemeyerjimbaker: Hmm, ok, I think I start to see what you mean19:09
jimbakerniemeyer, in any event, in that case we reap them because we don't want processes like that around indefinitely19:09
niemeyerjimbaker: There's a child process, started by the hook, and we want it to continue running19:09
niemeyerjimbaker: Is that the case?19:09
jimbakerniemeyer, i chose 5 seconds because it's much larger than reasonable for good children, but doesn't matter in practice even w/ lots of bad children19:10
niemeyerjimbaker: Uh.. lost again19:10
jimbakerniemeyer, only for a little bit, just to let its logs settle19:10
jimbakerthe actual reaper time probably would be fine w/ something like 200 ms19:10
SpamapSjimbaker: there are daemons that take a long time to start up .. but usually they don't let the parent die until they're ready for it to die.19:10
jimbakerSpamapS, correct - this doesn't impact such "good" children19:11
jimbakerniemeyer, so we are not trying to solve a larger problem of having time limits on hooks19:11
SpamapSIn rabbit's case, it seems like we should report a bug in the init script that it needs to run w/ nohup19:11
jimbakerSpamapS, correct, that would be the normal simple way to get the desired behavior19:12
SpamapSwhich turns bad children into good children by first redirecting its own fds to dev null or a log19:12
jimbakerSpamapS, :)19:12
m_3like long-term care insurance19:12
jimbakerthe analogies just pour in19:12
SpamapSm_3: Great example node.js app btw..19:12
SpamapSm_3: you may not have gotten much uptake on that... today is node.js day at OSCON ;)19:13
m_3cool thanks man19:13
hazmatjimbaker, we are and there is a bug open for that, but that has other things it needs to take into account like respecting debug hooks which will break time limits.. a standard hook timeout though needs to be quite large (large installs come to mind).19:13
m_3didn't know if I was going into too much detail19:13
m_3lots of zzzz's I think19:13
jimbakerhazmat, correct, a much larger problem19:13
SpamapSm_3: The log of the session may prove useful to someone. :)19:13
m_3wish there was a way to get the ajaxterm session too19:14
niemeyerjimbaker: Ok, I think I understand, and the approach looks good19:14
m_3should create a screencast I guess19:14
niemeyerjimbaker: It just needs proper documentation19:14
jimbakerhazmat, i think such a solution should allow for measuring *useful* progress back to the agent19:14
niemeyerjimbaker: I'll read the handling of processEnded, just to make sure I'm indeed getting what happens there19:14
jimbakerniemeyer, cool, i will work on that19:14
niemeyerjimbaker: Thanks for the patience explaining it19:14
jimbakerniemeyer, np19:14
jimbakerSpamapS, speaking of node.js, seeing any golang interest at oscon?19:15
SpamapSjimbaker: I'm not there yet. :-/19:15
SpamapStrying to keep up with keynotes and goings on so I won't be lost when I arrive on Thursday19:16
jimbakerSpamapS, cool, makes sense19:16
niemeyerjimbaker: Are you able to reproduce the problem easily?19:16
adam_gjimbaker: google people led a 3.5 hour go tutorial this morning19:19
adam_gniemeyer: the problem is easily reproduced deploying lp:principia/rabbitmq-server19:19
SpamapSHey guys, I could use feedback on this.. please view w/ the notes .. http://spamaps.org/files/ensemble-oscon-2011.odp19:19
niemeyeradam_g: Yeah, by easily I was wondering about a trivial test case19:20
niemeyerI suspect the real issue here is that Python is eating SIGCHLD19:20
niemeyerwhich is the standard way by which a parent gets to know it child has died19:20
jimbakerniemeyer, pretty certain that the ProcessProtocol stuff is quite robust with respect to that19:24
jcastroSpamapS: too many lolcats, that's like 2 years ago19:24
jimbakerniemeyer, i can readily see the child process not completing its writes to output before process exit, simply by looping test_invoker19:25
jimbakerniemeyer, on this laptop maybe occurs 50% of time (don't recall specifics!) if i don't have the yields on process ended19:26
jimbakerniemeyer, so that's often enough that one really doesn't need to loop ;)19:27
niemeyerjimbaker: I think I get it.. I'm wondering mostly about this sentence from therve now:19:29
niemeyer"""I don't think that bug is related to what's done during the install hook, fwiw. It happened to me with an empty install script."""19:29
niemeyerjimbaker: There's no "sleep" in this case, as per your example19:30
niemeyerjimbaker: So why would it remain un-ended19:30
ahs3probably a dumb n00b question...is there a way to have ensemble install a service on one or more existing machines (real ones, not EC2 instances)?  i'm not finding it in the docs, if there is...19:31
jimbakerniemeyer, i'm reading that a bit differently - there's some actual hook being executed, just not an install one19:31
jimbakeror at least just a trivial "empty" install one19:32
jimbakerniemeyer, so in particular i'm looking at reply #9 where we start to move into actual diagnosis19:32
niemeyerjimbaker: Ah, interesting, could be19:32
niemeyerjimbaker: That'd make more sense19:32
hazmatahs3, that's a good question.. not at the moment re a standalone machine, there is  work being done today for physical machine integration  is in the context of using something like orchestra (cobbler)19:32
jimbakerniemeyer, yeah, i'd be very concerned if a empty hook would act like an infinite sleep ;)19:33
hazmatfor setting up ensemble deployment with a physical machine data center19:33
niemeyerjimbaker: Exactly19:33
jimbakerniemeyer, but again we would fix that in the other one, bug 70543319:34
_mup_Bug #705433: Hooks need to have an enforceable timeout. <hooks> <Ensemble:New> < https://launchpad.net/bugs/705433 >19:34
niemeyer    def maybeCallProcessEnded(self):19:34
niemeyer        # we don't call ProcessProtocol.processEnded until:19:34
niemeyer        #  the child has terminated, AND19:34
niemeyer        #  all writers have indicated an error status, AND19:34
niemeyer        #  all readers have indicated EOF19:34
niemeyer        # This insures that we've gathered all output from the process.19:34
hazmatjimbaker, well an empty hook and a timeout are different, the question is really do want to use processExited instead19:35
ahs3hazmat: hrm.  thx.  i thought that might be the case.  what i would especially like is something that assumes a machine is already up, running and available, and then just puts the service on it.19:35
jimbakerhazmat, to clarify: this branch now uses processExited19:35
hazmatjimbaker, indeed it does both19:35
niemeyerjimbaker: Beautiful.. all looks great19:36
jimbakerhazmat, the only thing it uses processEnded for is to do the loseConnection handshake. unless the time between processExited and processEnded is larger than a ludicrous amount (5 s), in which case it simply kills the process with loseConnection19:36
hazmatjimbaker, yeah.. the branch looked good to me19:36
niemeyerjimbaker: Just a sentence or two highlighting the distinction between processEnded and Exited, given that these names are entirely unhelpful19:36
jimbakeri expect closing < 50 ms. 5 seconds ensures the probability of delays in normal closing to be exceedingly low, which seems a reasonable cost19:37
niemeyerjimbaker: This, specifically, will bite us when we read it again:19:37
niemeyer+        # The corresponding process has ended, so output streams have19:37
niemeyerjimbaker: The process has already ended  by the time processExited was called19:37
niemeyerjimbaker: (in unix/non-twisted lingo)19:38
hazmatjimbaker, the long inline functions seem a little strange stylistically, but thats minor19:38
jimbakerhazmat, yeah, no worries, they are moving to our established convention of _ methods since they are not actual closures (in the sense of actually closing over a non-empty set of variables)19:39
jimbakerhazmat, i was simply following the convention in that particular file, but it19:39
jimbakerrepresents somewhat earlier code for us19:40
niemeyerjimbaker: The convention is still the same.. we often put short local snippets in-context, but for 40+ lines the convention changes19:40
hazmatindeed that was so last year ;-)19:40
niemeyeras hazmat says, no big deal.. that's what reviews are for19:41
jimbakerniemeyer, oh sure, i think we all understand what's what19:41
niemeyerjimbaker: Cool, thanks again for the explanation19:42
niemeyerjimbaker: Had no idea about the processEnded concept used by Twisted19:42
jimbakerniemeyer, until this came up, neither did i, that's for certain19:42
niemeyerhazmat: have you looked over the branch?19:43
jimbakerbut it makes perfect sense in the overall context of daemonization19:43
hazmatniemeyer, i have19:43
niemeyerjimbaker: Not sure it does.. the only use case I've seen so far was a bug. ;-D19:43
hazmati think i looked at it when i implemented it, and used exited for the initial implementation, but i definitely understood and agreed why it was switched, and this additional robustness around usage looks good to me19:44
niemeyerhazmat: Cool, would you mind to put your concerns in the MP if you have any, or +1 it?19:44
hazmatniemeyer, sure19:45
niemeyerhazmat: Thanks19:45
niemeyerhazmat: This MP is empty: https://code.launchpad.net/~hazmat/ensemble/security-connection/+merge/6862119:50
niemeyerhazmat: Is it missing a push?19:50
hazmatperhaps.. let me check19:53
hazmati think that was one of the borked branches19:54
hazmatniemeyer, it says its up to date revno 28519:54
hazmatniemeyer, to answer the mp question lp is on crack for that branch19:55
_mup_Bug #816621 was filed: Ensemble doesn't appear to set up a complete environment while running the installation scripts and hooks <Ensemble:New> < https://launchpad.net/bugs/816621 >19:55
niemeyerhazmat: Cool19:57
niemeyerI'll step out for a coffee break.. 19:57
hazmatnegronjl, re cloud foundry bug above, did you  have ruby's eventmachine installed via package locally?20:01
hazmatnegronjl, i'm wondering if the issue is dep not specified in packaging20:02
SpamapSjcastro: I need a new meme. :)20:03
=== koolhead17 is now known as koolhead17|afk
hazmatnegronjl, the package software is depending on eventmachine, and its not being installed via package per the hook logs, unless there's a private copy to cloudfoundry, it definitely seems like a missing dep issue.20:05
SpamapSjcastro: should I go with fuuuuuuuuuuu instead?20:11
jcastroSpamapS: you started with a terminator-skynet meme20:12
jcastroSpamapS: "ensemble with me if you want to live."20:12
jcastroSpamapS: deployment day = judgement day. I mean, there's so many places to go with this20:13
SpamapSjcastro: my reason for lolcats was to say that ensemble was invented primarily to help process all of the pictures of cats incoming from smart phone users.. but it never materialized20:13
SpamapSjcastro: This is true.. and open source can be John Connor's rebels. ;)20:14
jcastroSpamapS: use the fuuuuuu-maker thing from reddit20:14
jcastrowait, no, that's a bad idea, that will just lead to hilarity20:15
niemeyerhazmat: The contributor's page is up-to-date already20:23
hazmatniemeyer, url?20:26
niemeyerhazmat: http://www.canonical.com/contributors20:26
hazmatniemeyer, cool, thanks20:29
SpamapSjcastro: I want the last 20 minutes back... distractions... http://spamaps.org/files/hadoop-rage.png20:33
jcastroI new you wouldn't let me down20:35
jcastroknew even20:35
SpamapSjcastro: Not sure thats hilarity.. its a little too commercial for me. ;)20:45
negronjlhazmat:  All the deps are satisfied.  If you run the script manually, everything works just fine.20:54
negronjlhazmat:  It only breaks when run via ensemble20:54
hazmatSpamapS, slides look nice21:06
niemeyerhazmat: lp is seriously wedged on that branch 21:13
niemeyer% bzr branch lp:~hazmat/ensemble/security-connection21:13
niemeyerbzr: ERROR: Not a branch: "/home/kapil/canonical/ensemble/mine/security/.bzr/pipes/security-connection/".21:13
niemeyerhazmat: !!!21:13
niemeyerSpamapS: Good stuff in the slides indeed21:24
hazmatniemeyer, ugh21:34
hazmatniemeyer, locally it looks fine.. trying with push --overwrite21:36
hazmatyeah.. still nothing21:36
niemeyerhazmat: Trying branching to the side, maybe?21:37
niemeyerhazmat: Worth saving the whole tree somewhere else first!21:37
niemeyerhazmat: To avoid losing any work21:37
hazmatniemeyer, its the middle branch of a pipeline already, additional branches have pushed succesfully21:37
_mup_ensemble/trunk-merge r273 committed by kapil.thangavelu@canonical.com21:38
_mup_merge trunk21:38
hazmatniemeyer, redid the merge proposal with a copy of the branch21:44
niemeyerhazmat: Phew, glad it worked21:44
niemeyerThat was weird21:45
kirklandm_3: what's your LP id?21:45
kirklandm_3: https://launchpad.net/~mark-mims ?21:46
kirklandm_3: okay, I'm sufficiently convinced that's you :-P21:48
m_3kirkland: yes, ~mark-mims21:52
* RoAkSoAx once again remembers the pains of having a nick name different for a LP id... :)21:54
kirklandRoAkSoAx: for the love, yes, standards!21:55
RoAkSoAxkirkland: tried to change my nick to lp id but it was simply better to keep it as is cause nobody knew me with the new nicknam,e :)21:56
niemeyerhazmat is the review queue lord21:57
hazmatniemeyer, slave? ;-)21:57
niemeyerhazmat: Nah, I'm the slave21:57
lynxmanhey guys, just wanted to share the news22:02
lynxmanEnsemble is already in Macports22:02
lynxmanlynxman@kreuzberg:~ $ sudo port search ensemble22:02
lynxmanensemble @0.5 (python, net) Ensemble is a next generation service orchestration framework.22:02
lynxmanerr false alarm, I'm too sleepy :)22:03
SpamapSwell, cool anyway ;)22:05
niemeyerlynxman: neat :-)22:10
lynxmanthought the package was just added to the repo, as soon as it's there I'll let you know :)22:11
niemeyerjimbaker: Finally got to review your debug-log-relations branch22:38
niemeyerjimbaker: Some comments, but good stuff overall22:38
niemeyerjimbaker: Sorry for the delay22:38
dakerhazmat, sorry to disturb you, on this page http://www.canonical.com/contributors there is no mention of "ensemble"22:49
niemeyerdaker: Indeed.. feel free to send it to kapil himself22:51
niemeyerdaker: In addition to the email there22:51
dakerniemeyer, i send what ? don't tell i need to download the pdf, fill it with my name, then scan it.22:53
niemeyerdaker: Ok, I won't tell it.. :-)22:54
hazmatdaker, afaik, that's what needs to be done for contribution to any canonical sponsored software project, i guess a photo would suffice in lieu of a scan22:55
dakerthat sucks :/22:56
niemeyerdaker: Sorry for the boilerplate.. note this is somewhat normal on contributions 22:57
SpamapSHeh.. so.. I'm preparing to upload trunk to Ubuntu .. and I've started the process of running the test suite when the package builds.23:02
SpamapSIt doesn't fail the build..23:02
SpamapSBut it will record the results in /usr/share/doc/ensemble/test-results.txt ... so we can start the process of fixing bugs where tests fail on the buildd.23:03
_mup_Bug #816736 was filed: debian dir gets in the way of packaging and is no longer necessary <Ensemble:New> < https://launchpad.net/bugs/816736 >23:55

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!