[00:03] <_mup_> ensemble/ensemble-resolved r247 committed by kapil.thangavelu@canonical.com [00:03] <_mup_> more resolved cli tests [01:13] <_mup_> ensemble/resolved-state-api r194 committed by kapil.thangavelu@canonical.com [01:13] <_mup_> use new dict merging utility func, and allow non conflicting updates to relation resolved settings. [01:34] <_mup_> ensemble/ensemble-resolved r249 committed by kapil.thangavelu@canonical.com [01:34] <_mup_> complete additional tests for resolved cli [01:36] <_mup_> Bug #767948 was filed: ensemble resolved command is needed to recover from errors < https://launchpad.net/bugs/767948 > [01:50] <_mup_> Bug #767955 was filed: cli needs beautifcation < https://launchpad.net/bugs/767955 > [02:02] <_mup_> ensemble/trunk-merge r187 committed by kapil.thangavelu@canonical.com [02:02] <_mup_> add a resolved spec [02:04] <_mup_> ensemble/resolved-spec r187 committed by kapil.thangavelu@canonical.com [02:04] <_mup_> add resolved spec [02:05] <_mup_> Bug #767961 was filed: unit agent needs support for resolving units and unit relations < https://launchpad.net/bugs/767961 > [02:05] <_mup_> Bug #767964 was filed: ensemble resolved spec/user doc for working with errors < https://launchpad.net/bugs/767964 > [04:40] <_mup_> ensemble/refactor-to-yamlstate r196 committed by bcsaller@gmail.com [04:40] <_mup_> Hook context's relation settings -> YAMLState [06:57] <_mup_> ensemble/config-get r200 committed by bcsaller@gmail.com [06:57] <_mup_> YAMLState changes [12:50] <_mup_> ensemble/trunk-merge r187 committed by kapil.thangavelu@canonical.com [12:50] <_mup_> merge trunk [12:53] <_mup_> ensemble/resolved-spec r189 committed by kapil.thangavelu@canonical.com [12:53] <_mup_> additional doc text regarding --retry [12:54] hey folks o/ [12:54] I'm trying to launch ensemble now, and every time the ec2 instance fails to boot (becomes terminated) ?! [12:55] I think a new natty image was pushed yesterday right ? it seems to have problems [13:06] hazmat: you mentioned the natty image, right [13:51] kim0, sorry its not committed [13:52] i got caught up in working on ensemble resolved [13:52] looking at it now [13:52] kim0, which makes your problems more interesting [13:52] kim0, actually there is an ec2 problem in the default region [13:52] kim0, they where having problems both with ebs and ec2 [13:53] kim0, for http://www.reddit.com/r/golang/comments/gdn3d/express_go_jit_implementation_of_go/ [13:53] as an example [13:53] http://status.aws.amazon.com/ [13:53] you can change the default region in the environments.yaml under the provider [13:55] hazmat: oh thanks [13:56] <_mup_> Bug #768311 was filed: txaws deprecation warnings from elementree with ensemble usage < https://launchpad.net/bugs/768311 > [13:58] <_mup_> ensemble/unit-agent-resolved r251 committed by kapil.thangavelu@canonical.com [13:58] <_mup_> unit and unit relation resolved watch callbacks. [14:02] hmm.. lame the txaws only supports two regions [14:02] EU and US [14:04] yeah [14:04] still failing though [14:04] wonder if us-east = North vigina [14:07] kim0, it does [14:07] kim0, hmm. the other problem is we only have images published in that region.. [14:07] ec2-uri: https://ec2.us-west-1.amazonaws.com [14:07] that works as a syntax to talk to other regions but since we're not multi publishing them yet.. its kinda of moot [14:07] yeah .. so I'll wait a bit I guess [14:07] hmmm [14:08] so much for cloud :) [14:08] i can publish an image in another region, but we don't select internally based on region atm [14:08] ie. it would need a specification of the image id as well [14:10] <_mup_> Bug #768320 was filed: ensemble should support more ec2 regions < https://launchpad.net/bugs/768320 > [14:26] kim0, nice article about the other sites down as a result ... http://eu.techcrunch.com/2011/04/21/amazon-ec2-goes-down-taking-with-it-reddit-foursquare-and-quora/ [14:34] 4 hours and the amazon folks are still scrambling .. it's a shame [15:08] hmmm.. can't cut new images most of the daily amis aren't resolvable correctly. === deryck is now known as deryck[lunch] [16:35] Have we considered taking over txaws? [16:35] Or at least ... inserting ensemble as a giant influence in its development? [16:38] SpamapS, we haven't made the time for it, its really there for anyone who wants it [16:38] jamu's been doing some work on it recently and going through old bug reports [16:39] Thats good [16:39] maybe a release soon? [16:39] * SpamapS feels like a broken record [16:39] SpamapS, agreed that having a txaws release would be a good thing [16:39] In Canonical, if you don't have a cadence, you just don't release. ;) [16:40] :) [16:40] i've got ensemble images in a few more regions, almost done getting region support working in ensemble [16:44] at this point txaws is a blocker for a) working on natty w/o warnings (my bad for silently fixing that on my box and not patching the package.. oops!), b) working w/ eucalyptus and openstack ... anything else? [16:51] SpamapS, b) has been fixed in txaws [16:51] the warning thing is being worked on [16:52] did anyone test ensemble against openstack yet [16:52] kim0, just uec afaik [16:52] hazmat: is that fix released? [16:52] hazmat: and uec works [16:52] ? [16:52] chuck and I tried it against openstack [16:52] same fail as eucalyptus [16:53] ah so both fail [16:53] s3 [16:53] It sounds like there's a patch [16:53] how hard is it to fix that [16:53] SpamapS, no release, its committed to trunk [16:53] hazmat: good enough for me, lets get that into the package. :) [16:54] kim0, the issues you where having with txaws are also fixed. [16:54] hazmat: the logging ? [16:54] kim0, no.. the initial setup, where we had to do a non head revision of txaws to get things working. [16:55] ah ok [16:55] kim0, there are branches in review from gustavo on both the logging issues [16:55] * kim0 nods [16:55] so there will be a packaged release of ensemble soon? [16:55] hazmat: just to be clear, you're saying with txaws trunk, we can deploy ensemble versus uec/openstack sucessfully ? [16:56] TeTeT, i think we're going to have a daily ppa. if i remember the plan correctly, and then universe hopefully in 11.10 [17:00] <_mup_> Bug #768424 was filed: Setup a launchpad daily ppa recipe for ensemble and some deps < https://launchpad.net/bugs/768424 > [17:00] TeTeT: ppa:ensemble/ppa exists now :) [17:00] TeTeT: for lucid and natty [17:01] SpamapS: wow, it works on lucid too? [17:05] great, just installed on a vm, now trying to connect it to the training UEC :) [17:06] how would I configure environment.yaml for my UEC cloud? [17:06] TeTeT, ec2_uri and s3_uri [17:07] TeTeT: that won't work until we update txaws in the PPA [17:07] but i don't think txaws is in that ppa, of which something recent from trunk is needed. [17:07] maybe we should make that happen.. :) [17:07] SpamapS, is that a daily build recipe ppa? [17:07] hazmat: definitely not .. its the one I've tested. [17:07] SpamapS: ah, ok, will postpone the test then [17:08] hazmat: I think an ensemble/daily would be the place for daily builds [17:08] hazmat: basically the one in the PPA is the demo-able version [17:09] SpamapS, sounds good [17:30] SpamapS, i'll take a look at the bzr builder stuff after i get the region portability working === deryck[lunch] is now known as deryck [17:52] hmm. i can launch ensemble in another zone.. but deploying formulas seems to be more problematic. [17:56] ah.. the provisioning agent [18:03] hi all [18:07] <_mup_> ensemble/ensemble-alternate-regions r205 committed by kapil.thangavelu@canonical.com [18:07] <_mup_> region portability [18:08] hi koolhead17 [18:08] koolhead17: o/ [18:08] how are things. hazmat amazon had some major issue today [18:09] hi kim0 [18:09] they're still down :/ [18:09] yes [18:09] :) [18:09] koolhead17, we've been working on a portability.. we had some code assumptions/dep issues with using alternate regions. [18:09] i'm still working on it, i've got it bootstrapping in europe and us-west-1 [18:10] but the provisioning agent still has thing to work out [18:10] k [18:11] kim0: point me to your blog which you suggested me to start reading from please [18:11] internally ensemble is composed of a set of agents, running on a set of machines in a cloud environment. the provisioning agent is what's responsible for launching new machines for deployed services to live on.. bootstrap means getting a single machine in the environment running at least some of the core agents [18:12] ok [18:13] hazmat: i will focus today and read up on the documentation. i have got a amazon linux machine access from my friend [18:13] i hope that is what you guys meant by EC2 [18:14] koolhead17, ensemble really needs it own user provided ec2 account, its launching new machines for deployed services [18:14] koolhead17: you might enjoy http://foss-boss.blogspot.com/2010/10/pointnclick-guide-to-running-ubuntu-in.html [18:14] to get a basic understanding of ec2 [18:14] ok nice [18:14] we'll be looking at lxc support for 11.10 to allow people to run ensemble just on their own laptop/desktop hardware [18:14] you'll also find other cool stuff there :) [18:15] :) [18:15] hazmat: that would rock :) [18:15] hazmat: actually openstack now has lxc support .. [18:15] hazmat: okey [18:15] so hopefully it won't be too much changes for ensemble [18:15] kim0, it does, i'm not sure how much its been tested/used though [18:16] hopefully its good.. there's a lot of libvirt lxc work the last few months [18:16] hazmat: kim0 apology but i need to know about the lxe full form [18:17] koolhead17, what of it? we'll have some attendance at a sprint on lxc in ubuntu with some others in august i think, we'd like to see lxc both within ec2 machines for isolation, and as a machine provider, the latter for desktop support. [18:18] hazmat: what is lxe ? [18:18] we've been looking at using libvirt integration, but at the moment raw lxc is a bit easier. openstack which uses libvirt recently added lxc support so that's worth investigating. [18:18] koolhead17, http://lxc.sf.net [18:18] thanks [18:19] kim0: we have used libvert only on openstack i suppose [18:19] koolhead17, its like a bsd jail or solaris zone.. its an os level containerization of a set of processes such that isolation and resource limits can be put on the process set. [18:20] like Cgroups if am not wrong? [18:20] koolhead17, libvirt sits on top of a number drivers for virtualization as an abstraction api, lxc is one of those drivers. [18:20] koolhead17, its based on cgroups yes [18:20] okey cool. [18:21] cgroups and clone syscall for the namespace isolation [18:22] waoo. [18:26] SpamapS, jamu fixed the txaws warnings on trunk fwiw [18:27] hazmat: yeah seems like txaws trunk is *t3h awesomeness* [18:27] :) [18:28] any easy way to upgrade to that yet [18:29] * kim0 imagining ec2 engineers pulling out their hair now [18:29] kim0, not outside bzr pull | bzr up atm, we can address as part of bug 768424 [18:29] <_mup_> Bug #768424: Setup a launchpad daily ppa recipe for ensemble and some deps < https://launchpad.net/bugs/768424 > [18:35] kim0: does moinmoin gives you permission to import raw html? Am trying to be little optimist :P [18:36] I suppose not [18:39] kim0: that angle i would love to explore latex2html will be our key :P [18:44] koolhead17: a solution might be to write the docs in reST, from which sphinx can generated html for you, PDFs and moinmoin (it seems) can read reST directly (needs testing though) [18:45] generally however, writing in reST sounds like a good plan to me rather than anything else [18:47] support for custom branches on deploy seems to be borked [18:48] kim0: let me think of another angle as well [18:48] :) [18:49] hazmat: oh I was planning on using gustavo's log fixes branch :) [18:49] moinmoin works well with rst, the only challenge is using sphinx directives not supported by it. but that's usually pretty minor [18:49] jimbaker: do you know if the version of wiki.ubuntu.com does [18:49] kim0, it looks like bzr is requiring an lp username and key for the remote machines. [18:49] not sure why.. its a public branch [18:50] kim0, that's in process of being updated [18:50] from ancient 1.6 to 1.9 i think [18:50] kim0, i have only used python.org... but it's a simple plugin for moinmoin and a magic header to specify [18:50] great [18:50] probably plugin is installed on moinmoin [18:51] jimbaker: koolhead17 is writing some openstack docs, that needs to be available in html/PDF/moinmoin [18:51] were not sure what's the best source format [18:51] http://moinmo.in/HelpOnParsers/ReStructuredText [18:51] it seems reST is a good choice for now [18:51] just hope wiki.u.c is updated soonish :) [18:51] yeah, i definitely will always recommend rst [18:52] i write everything in it technical that's not code [18:52] koolhead17: ^ me too :) [18:53] and sometimes not even technical ;-) [18:53] kim0, see jythonbook.com, which i coauthored [18:54] we actually produced microsoft word docs (ok open office, but for our publisher, it was ms word) and html from the sames source rst [18:54] ftw [18:55] the other thing i like is rst2pdf. it doesn't support the sphinx directives either, but it's great for precise production of pdfs. i use it for making presentations [18:55] rst2pdf is well enough [19:00] bcsaller, it seems to me that i need to augment HookContext to support open-port/close-port hook commands. in particular, this seems the right place to do it so that flush will properly commit changes [19:01] (in the future, we can take advantage of ZK multinode support too, so that the flush writes all or none of the nodes - https://issues.apache.org/jira/browse/ZOOKEEPER-965 [19:01] jimbaker, you'll need to use a separate yaml state, and incorporate it into flush [19:01] your probably going to end up conflicting with bcsaller's work on service config [19:01] there both exposing new hook apis [19:01] I'm thinking that like status HookContext is growing deps on too many interfaces and we'll have to refactor how that works in the future [19:02] hazmat, absolutely because it's on the service unit. not relation [19:02] bcsaller, agreed [19:02] HookContext is becoming a kind of dumping ground [19:02] bcsaller, well its going to become a facade over many different facilities [19:02] hazmat, that's one reason i'm pinging bcsaller, it seems that his service config stuff must :) [19:02] as long as it can manage them all isomerically then hopefully its not too bad. [19:02] else we can refactor more [19:03] maybe we can have some sort of plugin mechanism with hookcontext? [19:03] whatever that means [19:03] might be a better way to compose that object though, right now each will have to change flush for example [19:03] we do want one thing to call flush on [19:03] and have an associated client id [19:04] where as we could have a set of strategy objects which get iterated over in said calls [19:04] the client id has been a constant string for a while ;-) [19:04] bcsaller, yes, that's what i wanted to call it :) [19:04] hazmat, i did not know that [19:04] i think it can wait, but yeah.. it would be simple to do a get/set/del across method names of an abstraction object [19:04] here i am thinking it's supposed to do something [19:04] hazmat: if the debug stuff didn't need it to connect to existing sessions then its kind of a dead horse [19:05] er. strategy plugins.. but at them moment it seems like more trouble than its worth.. somehow its going to conflict whereever it gets configured [19:05] hazmat, but these are touching independent pieces [19:05] we just want to make sure that they can be accessed together, flushed together [19:05] jimbaker, to the hook context, are they all just a set of yaml states and methods manipulating them? [19:06] ^aren't [19:06] hazmat, i think they are indeed [19:06] hazmat: agreed for now, but I think the change is simple in the future and it will help keep the various interfaces different but composable [19:06] sounds good, you guys up for a standup bcsaller, jimbaker ? [19:06] yeah [19:07] hazmat, sounds good [19:24] { "port/proto" : True, "port/proto": ...} [19:25] **/units//ports** [19:28] { "open": [{"port": 80, "proto": "tcp"}, ...]} [19:40] port isn't required right? there are portless protocols.. ;) [19:44] cool working in other regions now [19:44] SpamapS, it can be a port range [19:45] SpamapS, there's some critical mass of args for modifying accesss on a security group ;-) [19:45] which are different depending on your intent/args [19:46] <_mup_> ensemble/ensemble-alternate-regions r206 committed by kapil.thangavelu@canonical.com [19:46] <_mup_> remove some of the debug helpers [20:03] <_mup_> ensemble/ensemble-alternate-regions r207 committed by kapil.thangavelu@canonical.com [20:03] <_mup_> verify new ec2 environment schema for regions. [20:05] <_mup_> ensemble/ensemble-alternate-regions r208 committed by kapil.thangavelu@canonical.com [20:05] <_mup_> remove region alias compatibility with txaws (EU/US) in favor of partial domain names (us-east-1, ap-northeast-1, etc) [20:06] TeTeT: hey [20:07] <_mup_> ensemble/ensemble-alternate-regions r209 committed by kapil.thangavelu@canonical.com [20:07] <_mup_> failure to find an ssh-key is not fatal to an ec2 environment serialization... [20:08] hi koolhead17 [20:08] TeTeT: Working on the Ubuntu Enterprise Cloud Course :) [20:08] saw your tweet. am curious about it ^^ [20:09] koolhead17: right now on a sprint and will present uec persistence in a few minutes [20:10] koolhead17: what are you working on cloud wise these days? [20:10] openstack and trying to understand ensemble [20:11] :) [20:14] kim0: announcing-project-reddwarf-database-as-a-service intersting :) [20:14] bcsaller, jimbaker if you guys have time would you mind looking at the ensemble-alternate-region branch.. [20:14] koolhead17: yep [20:14] i'd like to have ensemble working before ec2 is ;-) [20:15] hazmat: can I play with the branch now on a different ec2 region :) [20:15] hazmat: :P [20:15] kim0, yes [20:15] kim0, i had forget something basic i needed to do to make that work [20:15] like push the branch to lp [20:15] hehee :) [20:15] well actually i had a name typo.. but same difference ;-) [20:16] hazmat: what prompted the project name ensemble [20:17] koolhead17: you can direct easy questions at me :) [20:17] koolhead17, roots are in group communication systems and paxos, where ensemble is term used to refer to the working set of intercommunicating servers [20:17] but that definiteion may change latter ;-) [20:18] ok that's a sophisticated answer [20:18] hazmat: I think that should go into a faq [20:18] i think gustavo would be a better person to ask if you want a good origin name story [20:18] kim0: hazmat i met a guy this every sweet fellow asking me do u work on Ubunduu [20:18] kim0, true [20:18] he pronounced it so differently [20:18] :P [20:18] Can anyone think of two other questions to put in a faq :) [20:20] hazmat: so once you push the branch let me know how to play with it (I hope that doesn't conflict with me wanting the instances running ensemble-log fixes branch) [20:21] kim0, you mean regarding the alternate region branch? [20:21] kim0: what's the difference between ensemble and [20:21] yep [20:21] kim0: when will it be ready for production [20:21] kim0: how to contribute [20:21] kim0, if you branch gustavo's and merge mine you should be able to run in a different region, i don't have images up for asia yet, but europe and us (x2) are covered [20:21] shouldn't be any conflicts [20:21] TeTeT: man you're good :) [20:22] kim0: what system do I need to control ensemble from? [20:22] kim0: he he, doing my best, no matter where [20:22] * kim0 starting a faq doc [21:25] kim0: there [21:38] hazmat, i need a txaws branch for your branch to work, right? [21:39] presumably, since it's failing in txaws trunk... [21:47] bcsaller, are you seeing this problem too? http://pastebin.ubuntu.com/597140/ [21:48] jimbaker: I haven't tried to run on ec2 in a while [21:48] and haven't seen that [21:49] bcsaller, just trying out hazmat's new branch for alternative regions and it's not working for me [21:49] jimbaker you also need to specify it as the remote branch [21:49] ensemble-branch in the provider [21:49] hazmat, yes, of course :) [21:49] the provisioning agent also relies on fixes in the branch [21:50] jimbaker, it works with txaws trunk [21:50] all makes sense now... except for the mysterious failure part [21:50] jimbaker, there's all kind of wierdness without the provisioning agent support.. we get partially initialized machine states, that status tends to barf on [21:51] which arguably it should do better with [21:51] * jimbaker needs to do annual checklist next time... was going to alter environments.yaml, but got left out [21:51] still good experience for yet another bug report :) [21:51] manual, manual [21:52] rebooting really did a number on my mind it seems [21:52] <_mup_> Bug #768595 was filed: Status should handle partially initialized machine states, ie. before they been handled by the provisioning agent < https://launchpad.net/bugs/768595 > [21:54] hazmat, where should we report version discrepancies. might be nice if ensemble status did the check, since bootstrap seems too early [21:55] jimbaker, we've got an open bug that to record bzr revno/branch info into zk [21:55] probably could add it to some sort of common cli check under verbose mode [21:55] hazmat, yeah. so if we reported that in ensemble status, that could be a good place [21:55] jimbaker, i'm getting a little concerned about how much back and forth traffic ensemble status is doing.. [21:55] it be nice if we could avoid pulling down the entire zk tree [21:56] to do a status listing [21:56] and feels like we're going in that direction, i'd rather keep the default and let be do filtering and additional display options to target something than have the default be slow [21:56] sure, but it does seem something that can be filtered on. but dev time is useful too [21:57] its a very useful introspection tool [21:57] and it would be nice to expand to do targeted display of relations values, ensemble versions, etc. [21:57] a message highlighting "are you really sure" when running two different branches would be valuable [21:57] jimbaker, how often do you use the ensemble-branch feature [21:58] hazmat, what's worse is when i leave it turned on :) [21:58] most of the time we're i'm just running trunk remotely [21:58] jimbaker, true that [21:58] that's bit me more than once [21:59] we can play with it [21:59] i guess its only once to look at the bootstrap node [22:00] which is required by all entry points anyway [22:05] hazmat, your branch looks good. i can't currently run the examples formulas as-is, but that's waiting on the logging changes [22:05] i will try it one more time w/ the older versions of the example formulas [22:06] what's currently on http://ec2-184-72-28-229.us-west-1.compute.amazonaws.com/ demonstrates why the open-port setting needs to be done in the flush, ideally w/ all-or-nothing semantics [22:07] for future ref: http://pastebin.ubuntu.com/597147/ [22:08] "it works!" yeah, maybe it's fine, maybe it's really in a bad and unsecured state [23:20] jimbaker, thats not the default incidentally, one of the formulas installed apache though [23:21] server comes with nothing running except ssh by default as far services [23:21] fwiw [23:42] hazmat, sure, we just see a partial configuration [23:42] hazmat, i retried with the example formulas in r200 of trunk (the last time i had run it), and i'm still not able to deploy against us-west-1 [23:51] hazmat, i have seen this recur in a couple of attempted deploys - http://pastebin.ubuntu.com/597178/ [23:53] it looks like the service relation is not being added, or something related to its workflow, based on what's missing from ensemble status [23:57] i really should look at this more directly in ZK, not certain what's going on here