[00:19] <_mup_> ensemble/unit-agent-formula-upgrade r213 committed by kapil.thangavelu@canonical.com [00:19] <_mup_> make lifecycle.start deferred fire only after the initial service relation watch has completed processing, this gives a much better interaction within tests, as the unit relations will also be initialized, and zk communication done after the end of the start lifecylce method for test environments with existing relations (and also works fine with none) [00:21] <_mup_> ensemble/unit-agent-formula-upgrade r214 committed by kapil.thangavelu@canonical.com [00:21] <_mup_> additional upgrade workflow tests. [01:48] Yo [03:01] how does ensemble differ from CloudFormation? [03:14] crazed: CloudFormation is a service to coordinate AWS specific functionality [03:14] crazed: So I'd say it differs entirely :-) [03:14] crazed: Ensemble takes care of the actual service deployment and maintenance [03:14] crazed: and configuration, etc [03:14] does it also handle launching aws resources? [03:15] crazed: It does, yeah [03:15] crazed: and shutting them down too [03:15] i've been dropping stuff into userdata currently to launch chef-solo [03:15] does ensemble make life easier? [03:16] crazed: Yeah, absolutely.. that's the primary reason why we're working on ti [03:16] it [03:16] i'm reading through the documentation, don't have much experience with zookeeper unfortunately [03:16] crazed: No other tool at the moment is really focused on the integration of services the way Ensemble does [03:17] crazed: You can pretty much ignore it [03:17] crazed: it's an implementation detail, not necessary for working with it [03:18] how does ensemble handle service specific configurations? for example, i want to use some specific nginx configuration [03:18] where do you store that config, and can it be populated with data based on the running environment [03:21] crazed: Yes, to both, if I understand what you mean [03:21] crazed: Formulas are the basic unit of deployment [03:21] crazed: You can think about it as a package [03:22] crazed: Formulas can communicate with each other through simple interfaces [03:22] crazed: So you can create a frontend app and make it communicate with a postgres app [03:23] i understand the higher level parts with the metadata, relations, etc [03:23] i'm just trying to wrap my head around the config management part [03:24] crazed: Ah, ok [03:24] crazed: This part is one we're actively working on [03:24] crazed: I think we have a published draft of the specification for you, hold on [03:25] i like the relations/formulas a lot, i think that's an aspect missing from CloudFormation [03:25] seems ensemble takes care of that and config management [03:25] which nothing else currenlty does, you have to string together a bunch of pieces to get that full package [03:26] crazed: RIght, you get the exact spirit of why we're building it [03:27] crazed: Sorry about the delay.. the document is misplaced [03:27] definitely something i'm interested in, i do a lot of ec2 deployments [03:27] crazed: Here is a useful link, for the moment: [03:27] http://people.canonical.com/~niemeyer/ensemble/service-config.html [03:28] crazed: Please note this is a specification which is being developed still, so it's not yet working [03:28] crazed: But that's exactly how it will work [03:28] crazed: Well, unless we get feedback which makes us change it [03:31] i'm going to have to read this over, i'll hang out here though and help out where possible [03:32] crazed: That's awesome, your EC2 experience on deployments is most welcome. Thanks a lot. [04:38] <_mup_> ensemble/yaml-state-node r185 committed by bcsaller@gmail.com [04:38] <_mup_> Provides a dict like interface around a Zookeeper node [04:38] <_mup_> containing serialised YAML data. The dict provided represents the [04:38] <_mup_> local view of all node data. [04:38] <_mup_> `flush` pushes this information into the Zookeeper node. [04:38] <_mup_> YAMLState(client, path) [04:38] <_mup_> `client`: a Zookeeper client [04:38] <_mup_> `path`: the path of the Zookeeper node to manage [04:38] <_mup_> The state of this object always represents the product of the [04:38] <_mup_> pristine settings (from Zookeeper) and the pending writes. [04:38] <_mup_> All mutation to the dict expects the use of inlineCallbacks and a [04:38] <_mup_> yield. This includes set and update. [05:00] <_mup_> ensemble/yaml-state-node r186 committed by bcsaller@gmail.com [05:00] <_mup_> added revert to return to pristine state [05:00] <_mup_> added sync (as a published name for _combine) [05:07] <_mup_> ensemble/config-state-manager r189 committed by bcsaller@gmail.com [05:07] <_mup_> service.config_get using new YAMLState [09:03] Morning folks [13:39] unity multi touch is pretty slick [13:39] was there a 'getting started' tutorial somewhere? [13:42] reading docs/getting-started.rst ... [13:46] do I need to get txaws and txzookeeper from source, or are there any packages available? [13:51] TeTeT, source atm [13:52] TeTeT, txzookeeper is in pypi though.. [13:52] hazmat: just built the txzookeeper package, as it had a debian dir in the branch [13:53] TeTeT, yeah.. i put together some packaging for a ppa a while ago, but we have setup any lp source build recipes to automate it [13:53] ^don't [13:54] hazmat: how do I install txaws? python setup.py ? What would be the argument? [13:54] TeTeT, i typically do it in a virtualenv.. [13:54] TeTeT, actually the txaws python package should be fine for us.. unless your trying to use it with eucalyptus [13:55] hazmat: yep, I do it in a natty vm at the moment [13:55] hazmat: and unfortunately I would love to try it with UEC ;) [13:56] ah.. well in that case.. python setup.py develop or install will work.. 'develop' will link that directory to your python installation, install will copy it over.. 'virtualenv' is a python technique to isolate python packages from the system python. [13:56] one step further: $ bin/ensemble [13:56] error: No environments configured. Please edit: /home/ubuntu/.ensemble/environments.yaml [13:56] hazmat: didn't know about virtualenv [13:56] TeTeT, so far so good. it created a sample config for you [13:56] hazmat: so I dumped it in /usr/local ... [13:58] TeTeT, that's fine.. you'll just want keep track in the future in case your upgrading it.. [13:59] hazmat: ok [13:59] now I need to find my amazon credentials first ... somewhere under account they are [14:01] TeTeT, fwiw, i do something like this https://pastebin.canonical.com/46144/ [14:01] that allows me to just update the source directories with bzr and the latest versions are installed [14:02] alternatived you can manipulate PYTHONPATH environment variable to include the source of the each package [14:02] instead of messing with setup.py [14:04] hazmat: not sure if I see through that yet - I'll have a look. I now added the aws credentials. is there an easy 'print hello world' like example? [14:14] TeTeT, ./bin/ensemble bootstrap will startup a machine.. you can do ./bin/ensemble status and ./bin/ensemble deploy --repository=examples mysql after that [14:14] TeTeT: there's a "examples/" folder in there [14:15] TeTeT, i stuck a minimal walkthrough in examples/readme.txt [14:15] kim0 + hazmat : thanks, trying that - how do I inform ensemble on which ssh key to use? I downloaded ensemble-key.pem to the vm, but where to place it? [14:16] hazmat: do you know which branch uses the new hooks (joined, departed) ? [14:16] kim0, trunk [14:16] hazmat: trunk has docs for that too [14:16] ? [14:16] kim0, not yet.. that's in the review queue [14:17] kim0, https://code.launchpad.net/~jimbaker/ensemble/new-hook-semantics-5-docs/+merge/57536 [14:17] awesome thanks [14:19] hazmat: n00b question, why doesn't that branch (new-hook-semantics-5-docs) show in https://code.launchpad.net/ensemble [14:19] TeTeT, you can put the ssh key into the environments.yaml and reference it by path or putting the contents of the public key there. by default it sniffs for a public key, and barfs it can't find one. the config setting for it in the environment is 'authorized-keys' for the content, or 'authorized-keys-path' to reference it by file path. [14:20] kim0, it does for me [14:20] 5th one down after trunk [14:22] oh duh .. old page hehe [14:24] I am not sure why "provides" must be satisfied [14:28] so I copied my ensemble-key.pem to id_rsa.pub and I get this error: http://pastebin.ubuntu.com/594014/ [14:30] sounds like the aws signature is wrong [14:31] crazed: does that mean access and secret key? [14:36] TeTeT, yes.. it sounds like your access/secret key for aws might be wrong [14:36] s/sounds/looks [14:39] hazmat: hmm, I use them from s3fox just fine [14:40] hazmat: wouldn't I need to put my account id somewhere as well? [14:40] TeTeT, your access key / secret key effectively are your account id [14:42] TeTeT, do the ec2 cli tools work for you? [14:43] hazmat: I'll give it a try [14:45] TeTeT, we don't use the private key / cert mechanism which is used for the ec2 soap api. [14:45] and the ec2 cli tools [14:45] hazmat: works and apparently ensemble did bootstrap something: http://pastebin.ubuntu.com/594021/ [14:45] TeTeT, cool! ./bin/ensemble status will tell you what it did :-) [14:46] hazmat: unfortunately not: [14:46] $ bin/ensemble status [14:46] No Ensemble machines found. Is the environment bootstrapped? [14:46] 2011-04-14 15:46:03,116 ERROR No Ensemble machines found. Is the environment bootstrapped? [14:54] TeTeT, hmm.. that's very strange, [14:55] it looks like you launched machines, and now their not visible to ensemble, did the settings change in between bootstrap and status? [14:55] hazmat: yeah, is there a way to get more debugging output I might make available? [14:55] ensemble -v subcommand turns up the cli logging [14:55] hazmat: I changed the credentials one time, but I started another bootstrap [14:56] there's also distributed logging, but it more about observing system operation.. this problem sounds more fundamental [14:56] here's bootstrap -v : http://pastebin.ubuntu.com/594030/ [14:57] TeTeT, thanks thats helpful [14:57] and that's status -v immediately thereafter: http://pastebin.ubuntu.com/594032/ [14:58] TeTeT, so the problem is that we launch the machine via ec2.. that works fine.. but we get an error saving our 'map' to s3, we use the map to find out where things are in the environment. [14:58] hazmat: the status is saved in the map? [14:58] TeTeT, the underlying problem though looks like an issue with txaws s3 [14:59] TeTeT, the instance-id to the bootstrap machine [14:59] or more specifically, the instance-ids of the ensemble zookeeper cluster machines [15:00] TeTeT, its not clear what version of txaws is being used, the traceback shows the package version being used [15:00] * hazmat checks for differences [15:01] hazmat: I used rev no 79 from bazaar and did the $ sudo python setup.py install [15:02] TeTeT, cool, yeah. its in /usr/local [15:05] <_mup_> Bug #760719 was filed: ensemble should verify s3 access before launching bootstrap nodes < https://launchpad.net/bugs/760719 > [15:06] he he, that was quick [15:07] TeTeT, yeah.. i've been using revno 68 of txaws, i'm updating now to try it out. [15:07] still not clear what the issue with s3 write is [15:10] hazmat: confirm that reverting to 68 and doing an install over it seems to work - still launching a new instance [15:11] TeTeT, confirmed that the latest (rev 80) seems to have issues [15:12] still can't connect due to ssh problems - seems my key gets not injected correctly [15:12] hmm.. looks like python2.6 compatibility issues with the latest rev 80 of txaws [15:13] TeTeT, are you using a key from ssh-keygen ? [15:13] hazmat: nah, I generated it via the web console - should I use euca-add-keypair for a change? [15:13] hazmat: I would think natty has 2.7, so not sure if 2.6 is an issue [15:14] TeTeT, its not a provider key/cert.. its just an ssh-keygen key.. preferrably one for which you have the private side in .ssh [15:15] then things like `ensemble ssh` just work [15:23] niemeyer, it looks like your merge of s3-bucket-paths to txaws broke ensemble bootstrap [15:23] and jamu's following yours broke python 2.6 compatibility [15:24] hazmat: Ugh, seriously? [15:24] hazmat: How did it break it? [15:25] niemeyer, yup [15:25] niemeyer, re s3.. http://pastebin.ubuntu.com/594030/ [15:25] hazmat: I actually tested the changes in practice against S3 with the command line tools within txaws [15:25] niemeyer, cool, it could just be something we have to update in our usage. [15:26] hazmat: Hmm.. wait.. I think the changes after the review were not tested [15:26] hazmat: Let me have another look at that [15:26] hazmat: I'll just finish the review I'm doing and will check it out [15:26] niemeyer, great, thanks [15:56] folks .. I've just written my understanding of a trace of how hooks would trigger for the sample wordpress formula .. It's at http://j.mp/fUOgbF .. Would someone please check/correct it ? [16:05] hazmat: I won't be able to look at this right now, sorry.. the review took some time, and I have to get lunch to attend a meeting in less than an hour [16:06] hazmat: If you don't get to it first, I can check it out after the meeting, though [16:06] niemeyer, sounds good, i'm trying to get back into the upgrade stuff [16:06] kim0, looking === niemeyer is now known as niemeyer_lunch [16:23] hazmat: the relation is called "db" in this case [16:23] I'm tracing this particular example [16:23] kim0, i can put it back. [16:23] not too important [16:23] what's the green highlight [16:23] kim0, if you run ensemble debug-log .. before the deploy you'll see it all [16:24] hazmat: great! [16:24] kim0, you can filter to just hook execution as well [16:24] --include hook [16:25] what's the green highlight [16:25] kim0, those hooks don't exist [16:26] hazmat: yeah .. so the sequence is correct .. but the hooks dont exist in this case, right [16:26] yup [16:26] great .. are you done with it [16:27] yup. looks good [16:27] hazmat: thanks :) [16:28] I feel much more confident now hehe ;) [16:28] understanding hook firing sequence is pretty important [16:34] hazmat: I think it'd be useful to merge that trace to the readme file .. to me that was about the most confusing thing about ensemble [16:34] kim0, definitely, sounds good to me. [16:35] cool .. branching .. proposing [16:35] the examples can be improved a bit, jim's got some bashified version which may change the sequence [16:36] hazmat: I was actually reading from the bashified [16:36] kim0, cool [16:36] yeah [16:47] <_mup_> Bug #760818 was filed: debug log is initialized too late in the unit agent startup, to catch install/start hooks < https://launchpad.net/bugs/760818 > [17:15] kim0, i think it would be good to expand the formula example section with this information [17:16] hazmat, i don't think the sequence changes in the bashified version === niemeyer_lunch is now known as niemeyer_meeting [17:16] jimbaker: I just proposed it for merging into your bashified branch [17:16] kim0, ahh, sounds good [17:18] kim0, i'm unable to see your branch proposal [17:18] woot [17:18] but i did get the email about it [17:19] jimbaker: it's the next one [17:19] jimbaker: https://code.launchpad.net/~kim0/ensemble/adding-hook-traces-to-readme-for-bashified-example/+merge/57716 [17:19] kim0, i think for your branch proposal you may need to explicitly mention my branch as a dependency, it was hard for to me read because my branch changes were mixed with yours since it was a diff against trunk [17:19] because I initially proposed it into lp:ensemble [17:19] then deleted that [17:19] and proposed into your branch [17:19] kim0, got it, now that is much clearer [17:20] yeah :) first time to propose merging into other branch :) [17:20] kim0, i still got it messed up a couple of weeks ago with a bunch of stacked branches [17:20] add to redo the proposals [17:20] s/add/had [17:21] jimbaker: I was actually surprised this always failed: bzr push lp:~kim0/ensemble/bashified-wordpress-mysql-examples/adding-hook-traces-to-readme [17:21] launchpad is powerful, but nonobvious some times [17:21] I concluded it wasn't possible to push 2 level stacks or something ?! [17:21] kim0, never tried that style, but it looks like something unsupported for sure :) [17:22] felt natural to me hehe :) [17:22] kim0, it would be nice if the merge proposal did catch that you had branched something besides trunk, and so therefore needed that branch dependency in place [17:22] i think that would be more natural [17:23] yeha [17:26] hazmat, i want to establish something, so to speak; the -relation-established hook is not implemented, right? it was discussed in the docs (In formula.rst), which i updated in my docs branch proposal [17:27] but unless it's hiding really well, it's not in the code [17:27] jimbaker, we did away with it [17:27] hazmat, good. because i noticed that kim0 mentions this hook in his trace analysis [17:27] but apparently such phantom hooks can leave the impression of being there :) [17:29] hazmat, i had removed it in my branch proposal, we just need to update the mentioning text in kim0's proposal, and it will be gone forever i hope. (unless we need to actually implement it) [17:29] jimbaker, sounds good [17:32] there where challenges around implementing it, and utility became a question, since the first hook always executed is relation-join [17:32] s/where/were [17:33] hazmat, agreed, it seems like -relation-joined is sufficient for this case [17:47] <_mup_> ensemble/unit-agent-formula-upgrade r215 committed by kapil.thangavelu@canonical.com [17:47] <_mup_> revert r213, having start execute return after initial watch callback === niemeyer_meeting is now known as niemeyer [18:20] <_mup_> ensemble/unit-agent-formula-upgrade r216 committed by kapil.thangavelu@canonical.com [18:20] <_mup_> upgrade formula error handlers that shutdown hook execution. [19:00] jimbaker: That's the danger of documenting stuff without a notice about whether it's implemented or not. [19:01] niemeyer, indeed [19:04] <_mup_> ensemble/unit-agent-formula-upgrade r217 committed by kapil.thangavelu@canonical.com [19:04] <_mup_> redo hook executor prioritize, instead as run immediate hook execution while stopped, bypassing the queue, and allowing us better control of hook executor start/stop status from lifecycle methods [19:08] hazmat: Interestingly, I can't reproduce the signature failure with a hand-made test case [19:12] hazmat: Nevermind, reproduced a bug when listing the bucket [19:13] niemeyer, cool [19:14] <_mup_> ensemble/unit-agent-formula-upgrade r218 committed by kapil.thangavelu@canonical.com [19:14] <_mup_> update executor tests with new renamed run priority hook method [19:17] we're almost at 1k unit tests [19:19] Wow [19:21] bcsaller, hazmat, niemeyer - standup? [19:21] soudns good [19:21] jimbaker: If there are no pressing issues, I'd be happy to skip it today [19:22] niemeyer, i'm curious about the repository discussion [19:22] niemeyer, sounds good, i can proceed immediately to lunching then :) [19:22] I've been involved in a bunch of things and don't feel like I've really produced much yet.. I'd like to get this fixed and the rest of branches reviewed [19:22] Well, cool [19:22] hazmat, jimbaker: let's do it [19:22] We can do an *actual* standup :) [19:22] bcsaller, ping? [19:22] niemeyer, sounds good, keep us focused [19:23] yeah, here [19:24] bcsaller, skype? [19:24] hazmat: with all the desktop crashes with natty I sometimes forget to log back in :( === bcsaller1 is now known as bcsaller [19:38] hazmat: dropped, back now [19:43] hazmat: https://code.launchpad.net/~niemeyer/txaws/fix-path-with-bucket/+merge/57753 [19:53] <_mup_> Bug #761015 was filed: FORMULA_DIRECTORY should be available to hooks < https://launchpad.net/bugs/761015 > [20:28] <_mup_> ensemble/unit-agent-formula-upgrade r219 committed by kapil.thangavelu@canonical.com [20:28] <_mup_> extract formula_download from machine agent into utility function [20:35] <_mup_> Bug #761053 was filed: Formula state should carry a referenced to a download url < https://launchpad.net/bugs/761053 > [20:35] <_mup_> ensemble/unit-agent-formula-upgrade r220 committed by kapil.thangavelu@canonical.com [20:35] <_mup_> note the relevant bug url by the implementation [20:55] jimbaker: ping [21:19] bcsaller, put some comments on the yaml-state proposal [21:19] hazmat: still working on it but its pretty close to whats up there, thank you for looking at it :) [21:29] niemeyer, hi [21:30] Yo [21:40] <_mup_> ensemble/status-workflow r201 committed by jim.baker@canonical.com [21:40] <_mup_> Collect workflow state for service units and their relations [22:00] <_mup_> ensemble/status-workflow r202 committed by jim.baker@canonical.com [22:00] <_mup_> Cleanup [22:12] <_mup_> ensemble/unit-agent-formula-upgrade r221 committed by kapil.thangavelu@canonical.com [22:12] <_mup_> additional agent upgrade tests. [22:18] * niemeyer => execrising [22:18] * niemeyer => exercising even === niemeyer is now known as niemeyer_away [22:25] <_mup_> ensemble/unit-agent-formula-upgrade r222 committed by kapil.thangavelu@canonical.com [22:25] <_mup_> minor prereview cleanups