[00:31] kim0 sladen you guys wont be needing anything more on the logos right? oxygen atom idea is confirmed? [00:32] let me know if it's not confirmed and if you need further more. [00:32] later [00:40] <_mup_> ensemble/expose-provider-ec2 r289 committed by jim.baker@canonical.com [00:40] <_mup_> More logging [00:56] <_mup_> ensemble/expose-provider-ec2 r290 committed by jim.baker@canonical.com [00:56] <_mup_> Some more logging in the provisioning agent [01:15] <_mup_> ensemble/expose-provider-ec2 r291 committed by jim.baker@canonical.com [01:15] <_mup_> Guard against EC2 errors from txaws === franciscosouza_ is now known as franciscosouza === jimbaker` is now known as jimbaker [03:40] <_mup_> ensemble/expose-provider-ec2 r292 committed by jim.baker@canonical.com [03:40] <_mup_> Fixed incorrect CIDR format for filtering IP permissions in security group === koolhead17| is now known as koolhead17 [07:32] Morning all === daker_ is now known as daker [12:33] moning kim0 [12:34] *morning [12:38] koolhead17: Morning o/ [12:49] * SpamapS shakes off sleep and braces for the morning deluge of email [13:03] g'morning [13:09] Yo all! [13:10] morning and email [13:11] howdy [13:11] * SpamapS wanders downstairs to make a massive omlette [13:13] howdy [13:13] having some trouble here with a formula [13:13] http://paste.ubuntu.com/644085/ [13:13] that's the log [13:13] the formula install hook is: [13:13] http://paste.ubuntu.com/644087/ [13:14] any ideas? [13:14] looks like maybe pipes in the install hook are throwing things off [13:14] 2011-07-14 13:00:32,673: hook.output@DEBUG: hook install ended, exit code 0. [13:15] Kind of looks like it worked fine. [13:15] what is the fail? [13:16] * SpamapS will re-join the discussion in 20 min after his omlette has been devoured [13:17] SpamapS: um, there's hook.output@ERROR all over the place [13:17] SpamapS: starting with 2011-07-14 13:00:11,320: hook.output@ERROR: dpkg-preconfigure: unable to re-open stdin: [13:24] SpamapS, massive omlette :) [13:24] he kirkland [13:25] *hey [13:25] koolhead17: howdy [13:26] kim0, find byoubu very heavy :) [13:34] hmm [13:35] hello [13:36] in order to use specific AMI images for an environment am I going to need to do anything other than set the default-ami parameter in the environment configuration? [13:36] kirkland, does it work though despite the error output (which looks non fatal to me) [13:36] can't seem to make ensemble use anything other than the default natty one ATM [13:36] hazmat: no, sorry [13:36] hazmat: those error commands didn't actually take effect [13:37] jamespage, default-image-id is the magic parameter to be set in environments.yaml per the docs here https://ensemble.ubuntu.com/docs/provider-configuration-ec2.html [13:37] hazmat: for instance, we're trying to write to the file /usr/share/byobu/profiles/classroom [13:37] hazmat: and that just doesn't happen (that file doesn't exist) [13:37] hazmat: ta - thanks for the pointer [13:41] kirkland, is there an escaping issue around that escape sequence?, i see the hook output has the file test and the echo to create the file, and then tries to start the session right after that [13:41] hazmat: hmm, is that script just executed by bash? [13:41] hazmat: or is it processed in some more obscure way? [13:42] hazmat: because if you just run it with bash, it works fine [13:42] kirkland, its executed by whatever shell is specified with the #! at the top [13:42] kirkland, in this case #!/bin/sh [13:42] hazmat: it's just when run in ensemble mode that it doesn't work right [13:42] hazmat: right, sh, i mean (dash) [13:44] kirkland, perhaps changing the install hook to using #!/bin/bash? [13:44] hazmat: hmm, okay, i can try that i suppose [13:44] FYI I requested a sync of zookeeper for oneiric from Debian this morning - this will push the version up to 3.3.3 and will superceded what's currently in the ensemble PPA for oneiric only [13:45] jamespage: Uhmm [13:45] jamespage: I'm not sure the Debian one has the needed patches [13:45] jamespage: Or the needed packaging changes, to be more precise [13:45] (our patches ahve been integrated upstream, IIRC) [13:45] jamespage: Is this going to kill any local changes we have? [13:46] niemeyer: hmm - lemme take a look [13:46] jamespage: Regarding the AMI, please note we have as a goal using stock AMIs, and using formulas/etc to do the tweaking [13:47] jamespage: We still lack some features in that area, like formulas being able to specify which release they're targeting, but that's the goal [13:47] kirkland: You can use debug-hooks as well, to debug hooks in general [13:48] niemeyer: how are you managing your additional patches in the PPA version of zookeeper - I can't see anything in debian/patches that I don't have in the debian version [13:48] jamespage: Does it have the same subpackages et al? [13:49] jamespage: Maybe it's all good and it makes no difference [13:49] jamespage: I recall we've had to do changes in the package originally [13:49] jamespage: But I suppose these were synced upstream by someone more careful than we are :) [13:49] niemeyer: i see the debug hooks are still using tmux :-/ [13:49] kirkland: Hmm.. yeah, it will continue using it until someones change? [13:50] someone hanges [13:50] changes! [13:50] kirkland: For the reasons we discussed [13:50] kirkland: As you know, I don't personally care about whether we use screen or tmux [13:50] niemeyer: if lp:~ensemble/ensemble/zookeeper-package is a good place to look the package structure is the same [13:51] kirkland: But I care about introducing further hacks to use screen [13:51] jamespage: Sounds fantastic.. I bet doko did the syncing work [13:51] so I picked up the upgrade to 3.3.3 and fixed a load of issues that mean't it was going to get dropped in Debian [13:53] jamespage: Oh, you mean you uploaded these fixes to Debian, and then synced back? [13:54] niemeyer: broadley yes - https://bugs.launchpad.net/ubuntu/+source/zookeeper/+bug/810320 contains the changelog since the last release in Ubuntu [13:54] niemeyer: yours truly has now adopted the package in Debian (although SpamapS will be helping me - won't you :-)) [13:54] jamespage: Man.. what can I say.. that's why you're called James Page I guess [13:56] niemeyer: I gave you a one-liner to use screen, that wraps it with the 'run-one' command, which is in Natty, Oneric and beyond, and generally solves the problem of ensuring one, single, unique flocked copy of a given process running on the system at a time [13:57] niemeyer: for some reason (can think why) I thought it might be important for Ubuntu :-) [13:57] kirkland: Yeah, we went over that already.. I'm not introducing further locking in that already messy logic to fix an internal bug of screen [13:57] kirkland: What we have now works well for the purposes we have, and once screen fixes that we can easily switch back [13:59] kirkland: Last we talked you were going to fix screen.. any progress there? [14:00] niemeyer: i have not looked at fixing screen, other than reproducing and confirming the limitation with the upstream maintainer in #screen [14:04] niemeyer: I suspect that even if I fix that bug in screen, you're going to give me another hoop to jump through; then another; then another ;-) [14:04] Jun 17 14:35:49 niemeyer: would you prefer that I patched screen? [14:04] Jun 17 14:37:17 kirkland: Yes, I'd certainly wish that screen implemented this logic by itself in a reliable way [14:04] kirkland: You're being silly.. [14:05] niemeyer: heh, maybe a little, but I've seen your branch reviews :-P [14:05] kirkland: We've been discussing this for a long time, and now you bring back as if I had said I'd switch back to screen, and that somehow I'm trying to prevent you from using screen. [14:05] kirkland: I'm using tmux because screen has a bug, and tmux works. [14:07] niemeyer: imagine if I said "niemeyer: I'm using puppet/chef/cfengine because ensemble has a bug, and puppet/chef/cfengine works." [14:07] kirkland: I'd have no issues with that.. I'd politely disagree, and continue pushing Ensemble forward because I believe in what we are doing. [14:08] niemeyer: i believe in what you're doing too; and i'm a strong proponent of ensemble [14:09] kirkland: Doesn't look like so.. you're creating a major issue out of something I specifically attempted to help you with. [14:09] niemeyer: it's hardly a major issue [14:09] niemeyer: and I specifically gave you a simple workaround, that you rejected [14:10] niemeyer: a temporary workaround [14:10] kirkland: You suggested an approach for me to implement, that involves further locking in an already messy area that was full of bugs as you noticed in the review I pointed you to. [14:25] fwereade, just to clarify a point on your merge proposal " Made a start on rearranging tests as well, but IMO that will be an ongoing process (especially over the next day or two) and so shouldn't hold up a merge. [14:25] ".. The existing tests are all passing? [14:26] hazmat: yes, everything is passing and I'm pretty certain that everything that was originally covered is still covered [14:26] fwereade, cool [14:26] hazmat: however, there are a number of EC2 tests that are really only hitting the common stuff [14:26] hazmat: and I think it would be sensible to gradually transition them [14:27] sounds good [14:27] hazmat: but I think the appropriate boundaries will become clearer as we get a cobbler bootstrap integrated [14:27] hazmat: ofc, don't trust me, run them yourself too :p [14:28] always.. but i wanted context for the comments, thanks [14:29] hazmat: a pleasure :) [14:36] for anyone who was watching the niemeyer/kirkland paintball match, we took it private and hugged it out ;-) [14:36] I have a tmux profile for byobu that will land in the archive very soon ;-) [14:36] * kim0 bugs both kirkland and niemeyer :) [14:36] hugs* even :) [14:37] \o/ woohoo for byobu on tmux [14:44] kirkland: is your formula for setting up a shared cloud byobu ready for public consumption .. I'd love to use it for a session today [14:45] kim0: damn close, has a bug somewhere in the formula [14:47] kim0: the formula is at lp:~kirkland/+junk/byobu-classroom if you want to try it [14:47] kim0: something's wrong with the shell script processing [14:47] kirkland: is there some manual workaround ? [14:48] kim0: yeah; launch and ec2 instance; and run the install hook as root by hand [14:48] kim0: works just fine [14:48] kim0: does not work when ensemble calls it [14:48] got it [14:48] ok I can do that [14:48] kirkland: thanks a lot [14:49] kim0: i'm trying a small change from /bin/sh to /bin/bash now [14:49] * kim0 nods [15:01] hazmat: Not sure if you're already on it, but I think fwereade's branch is now ready for other reviews: [15:01] hmm, i just updated to the latest ensemble, and i'm not able to get status on my newly bootstrapped environment: [15:01] http://paste.ubuntu.com/644173/ [15:01] hazmat: lp:~fwereade/ensemble/tweak-launch-machine [15:02] SpamapS: This might be interesting for you as well, and you may have some feedback to William there [15:02] SpamapS: It'll be more clear why we didn't just merge the logic [15:02] be brutal, it's the only way I'll learn ;) [15:03] niemeyer: I've been following the branch and I think William's direction, however different from my hackery, is far more maintainable long term. [15:03] I was hacking in the things I needed to get going.. not paying any real debt back. ;) [15:04] SpamapS: He just pushed some changes today, not sure if you've seen those [15:04] SpamapS: Yeah, as we discussed before, your hacks are greatly appreciated for real [15:05] SpamapS: andreas has been hacking on top of them to enable the cloud-init support pretty much the whole time [15:05] w00t [15:05] SpamapS: and we'll be basing the XML-RPC work on them [15:05] SpamapS: Sorry, that was Andres [15:05] or RoAkSoAx for the friends [15:05] While I was doing the XML-RPC stuff I kept wondering if there was already a twisted xmlrpc plugin. [15:06] shallow googling produced nothing [15:06] SpamapS: There is XML-RPC support in twisted, not specifically to cobbler though [15:06] Ah, so I was looking externally where I should have probably just RTFM'd [15:07] if you look at OrchestraControl .. it just uses the python xmlrpclib .. there's nothing specific to cobbler itself. [15:08] actually one thing that is broken in the OrchestraControl is that it only logs in at __init__ time ... needs to re-login whenever the token times out. [15:08] Important since the provisioning agent keeps the provider object around forever. [15:09] SpamapS: Hmm, interesting [15:10] SpamapS: That's a good hint, thanks [15:30] Good mornings [15:31] niemeyer: o/ [15:32] m_3: its funny that you asked yesterday how long I had been up, because it was about 4:00pm and I took a little nap... had been up since 03:30 [15:32] m_3: Hey Mark [15:33] m_3: hey o/ [15:33] SpamapS: :) [15:33] * SpamapS is almost back on PDT .. slept till 05:30 today [15:33] SpamapS: yeah, I saw traffic from horribly early hours [15:54] hazmat, SpamapS: any further thoughts on my branch? === koolhead17 is now known as koolhead17|afk [16:11] fwereade: I haven't had time to take another look, but I did late yesterday and I was happy to see things shaping up in a more sane way. :) [16:12] SpamapS: thanks :) [16:12] SpamapS: the really awful bit is gone now :) [16:13] Getting ambiguous endpoint error → http://paste.ubuntu.com/644210/ [16:13] can someone please help ? [16:13] I for one was happy to find some awful code in Ensemble.. no successful project is ever made up of 100% pure cane sugar. There's always some bitterness buried in the pressure to be timely. ;) [16:14] kim0, you're going to get that if there is more than one possible way to relate the services [16:14] jimbaker: I don't think there is [16:14] kim0, use the relation name to further specify [16:15] jimbaker: this is the very simple mysql from examples with one interface only [16:15] SpamapS: what I've seen looks pretty solid to me :) [16:15] with my drupal formula .. also one interface [16:15] jimbaker: and I'm demo'ing this within 45mins :) [16:15] kim0, sorry about that [16:16] kim0, one thing that never made it in the codebase was more guidance in the error message [16:16] jimbaker: I'm mostly sure there's only one relation on both sides [16:16] I tried specifying it as well [16:17] still not happy [16:17] fwereade: right, but some of the stuff in the ec2 provider was clearly thrown together in haste. :) [16:18] kim0, on mydb, there are two relation names (db-admin, db) for relation role of server [16:18] SpamapS: sure... but I've seen far, far worse ;) [16:18] I have been working on an outline for a book I plan to write .. "Your Code Must Suck - How to succeed in software by writing crappy code." [16:18] haha [16:18] kim0, just specify which one you are adding. if you want to add both, you need to do so explicitly [16:20] jimbaker: hm, I specified 'db' and it worked .. but looking at http://paste.ubuntu.com/644222/ I cannot see this other db-admin relation ?! [16:20] kim0, you need to break it up [16:21] fwereade, looking it over, its a lot of churn to digest [16:21] kim0, the error message is definitely unfriendly - we need to fix that [16:21] hazmat: yep, sorry about that [16:21] hazmat: it's *mostly* moving stuff around [16:22] jimbaker: can you please point me where to see this db-admin relation ? [16:22] hazmat: the significant interface changes are where I fixed the machine/instance confusion [16:22] hazmat: ...and that extended its own independent set of tentacles through the codebase :( [16:23] kim0, http://pastebin.ubuntu.com/644225/ [16:24] kim0, i just reformatted the specific text from one of the error lines [16:24] jimbaker: I understand the error says it exists .. however looking at metadata.yaml .. it's not there right ? [16:24] jimbaker: as in http://paste.ubuntu.com/644222/ [16:25] kim0, there's no relation named db-admin as you mentioned in this specific metadata.yaml [16:25] jimbaker: so this is indeed confusing ? [16:26] kim0, indeed. i think you got a version out of whack here [16:26] ok .. glad it works now though [16:26] once done .. I'll see if I can reproduce it [16:26] jimbaker: thanks a ton :) [16:27] kim0, sounds good about reproducing *later* and good luck with your demo *soon* ;) [16:27] kim0, i will file a bug report on this ambiguous endpoint stuff [16:29] kim0, also in looking at the docs, some of the details on add-relation seemed to have disappeared with respect to ambiguous endpoints and using relation names [16:29] kim0, that was in an earlier version of the docs, we should presumably restore some aspects of it [16:37] kim0, this is the original spec for add-relation/remove-relation, https://code.launchpad.net/~jimbaker/ensemble/rel-mgmt-cli - a quick reading indicates that it's still a correct description of the behavior of these commands [16:38] I'll read that soonish [16:40] fwereade: BTW, that is fantastic (fixing the machine/instance confusion). I spent probably a whole day screwing that up when playing with lxc. [16:51] SpamapS: thanks :D [16:55] <_mup_> Bug #810600 was filed: Ambiguous endpoints error in ensemble add-relation or remove-relation is confusing < https://launchpad.net/bugs/810600 > [16:57] SpamapS, fwereade - i should look at that, it's definitely was a pain point in adding support of ec2 firewall mgmt (which almost works, just need to fix a bug in the provisioning agent itself for re-exposing a service) [17:08] jimbaker: cool [17:09] jimbaker: a lot of it is just renaming variables to make it clear when we have a machine or an instance [17:10] fwereade, i think my favorite example in that chunk of code is MachineProvider vs ProviderMachine... [17:10] fwereade, i suppose that means something. but symmetry like that is generally not good ;) [17:42] negronjl: hey, there's already a memcached formula in principia! [17:42] SpamapS: duh!!! sorry about that :) [17:43] * SpamapS reads it anyway .. maybe its better. ;) [17:43] * negronjl thinks it isn't :) [17:43] err.. nope. :) [17:43] negronjl: its part of the whole mediawiki demo that I do. [17:43] SpamapS: cool...I'll be using soon :) [17:44] https://code.launchpad.net/~ensemble-composers/principia/oneiric/memcached/trunk [17:48] arses [17:48] okay, my bad [17:48] i forgot to bump the version of the formula [17:49] so i've been deploying the same code for 4+ hours now [17:49] :-P === daker is now known as daker_ [17:56] \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ [17:56] that formula version bumping got me again [17:56] kirkland: its high time we opened that bug... [17:56] * SpamapS does it [17:57] jamespage: working byobu-classroom formula at bzr+ssh://bazaar.launchpad.net/~kirkland/%2Bjunk/byobu-classroom/ [17:57] jamespage: m_3 is about to remove the 443 https dependency [17:58] jamespage: rather, i think he's going to put ajaxterm on both 80 and 443 [17:58] m_3: ^ that's what I'd like to see anyway [17:58] m_3: so that the user can choose if they want ssh or telnet :-P [17:58] sounds good [17:59] kirkland: thanks for byobu-classroom :) [18:00] worked great for me [18:01] <_mup_> Bug #810648 was filed: The revision number in formula metadata is not very useful to users < https://launchpad.net/bugs/810648 > === daker is now known as daker_ [18:02] kim0: yeah, caught first part of the class... awesome man [18:03] * ahs3 ^5 kim0 -- well done [18:03] <_mup_> Bug #810649 was filed: Revision number should be optional in metadata < https://launchpad.net/bugs/810649 > [18:03] ahs3: o/ Thanks :) [18:03] Ok, two bugs filed for that. [18:03] kirkland: might I suggest you review those, comment on your experiences, and +1/Confirm them? Thx. :) [18:04] Got a good question as I demo'ed ensemble [18:04] QUESTION: I didn't see a formula for apache in the list. Is that implicit? Can one swap out a different http server? [18:04] like can I deploy wordpress, on nginx not apache [18:04] kim0: *programs* are handled by packaging. Formulas are for configuring programs. [18:05] kim0: yes, that might be called the 'wordpress-nginx' formula though [18:05] SpamapS: sure [18:05] a separate copy .. cant it be a config option ?! [18:05] kim0: it could be yes [18:05] cool that's better :) [18:05] but I haven't seen config settings actually available yet... which is a pity. ;) [18:05] I really can't wait for options to land [18:05] yeah :) [18:06] distractions.. :-P [18:08] SpamapS: commented [18:10] kirkland: danke [18:13] SpamapS: marked confirmed [18:13] SpamapS: do i send this mysql merge proposal to lp:ensemble or some other? [18:14] adam_g: if you branched from lp:principia/mysql then you just push to launchpad and do 'bzr lp-propose' .. it will propose the merge against the original source branch. [18:15] ok, ill rebranch and do it that way [18:16] adam_g: how did you do it? [18:17] adam_g: you shouldn't have to re-branch, you can target a merge proposal at anything [18:17] i have no idea, i branched from lp:ensemble a while back it looks like [18:17] adam_g: but it typically is easier/smoother if you're proposing to merge with the branch you started from. [18:17] wait [18:17] you did this on the examples formula? [18:17] :( [18:18] principia has a much richer mysql formula that includes one-way replication. [18:18] ill just rebranch and merge, its not a big deal. the share-db stuff doesn't touch anything else [18:18] Did you branch the principia mysql formula or the examples one though? [18:19] they're totally different [18:20] principia, according to branch.conf [18:20] ok good [18:20] bzr info you mean? [18:20] yeah, that too [18:20] Ok you don't need to re-branch then [18:20] just push it to somewhere on launchpad and propose it for merging into lp:principia/mysql [18:20] 10-4 [18:21] SpamapS: is it possible to use principia's mysql formula, without replication or munin ..etc. i.e. just like the simple examples/ mysql formula ? [18:22] kim0: they're interchangable in the examples yes. The only reason its different is that the example one is trying to be the simple clear example.. whereas the principia one is trying to be highly useful. [18:23] cool! [18:23] so when it says require: munin, it's lying :) [18:25] require is a lie yes [18:25] mediawiki works fine w/o memcached [18:56] SpamapS, probably should have optional: true re mysq/munin [18:56] hazmat: we talked about this.. required things that are then made optional is.. silly ;) [18:57] There should be a new section, Consumes [18:59] the docs make it sound like required will change behavior at some point [19:00] SpamapS: That article was a good reading, btw [19:00] niemeyer: yeah really makes a strong case for exactly what coroutines give you. :) [19:01] light weight simplified thread-like behavior. [19:01] SpamapS: Yeah [19:01] SpamapS: It was also nice to see such arguments about callback-based logic written down [19:10] niemeyer, what article was that? [19:11] hazmat: http://www.usenix.org/events/hotos03/tech/full_papers/vonbehren/vonbehren_html/ [20:28] bcsaller: ping [20:30] niemeyer: pong [20:30] client locked up there [20:31] bcsaller: Hey [20:32] hey [20:32] bcsaller: Would you mind to review this branch from Kapil: https://code.launchpad.net/~hazmat/ensemble/security-node-policy-def/+merge/67972 [20:32] will do [20:32] bcsaller: Thanks [21:06] SpamapS: around? [21:06] # object has changed due to parent rename, get a new handle [21:06] sid = server.get_system_handle("system1", token) [21:07] fwereade, RoAkSoAx: ^^^ [21:07] Persistent object ids are such a novel idea :-( [21:07] niemeyer: gaaaaah [21:47] robbiew: yeah, sup? [21:51] kirkland, was that install hook problem just a shell issue (dash v. bash)? and did your environment status work? [21:57] SpamapS: ping [21:58] negronjl: pong, gotta run in 2 minutes, sup? [21:58] SpamapS: issues deploying mysql formula: http://pastebin.ubuntu.com/644396/ [21:58] SpamapS: whenever you get a chance [22:00] negronjl: thats quite strange... maybe a regression of some kind? [22:00] I haven't deployed it in a while [22:00] anyway, I'm late [22:00] SpamapS: later [22:02] <_mup_> Bug #810765 was filed: cannot deploy mysql < https://launchpad.net/bugs/810765 > [22:07] negronjl, that looks like an invalid metadata.yaml file that doesn't conform to yaml spec [22:08] hazmat: Did anything change on the yaml modules within ensemble? I don't think the mysql formula has been changed in a while. I'm currently playing with it nonetheless. [22:08] negronjl, no.. i kinda of doubt its an ensemble issue given the traceback [22:09] hazmat: I'm checking right now. I'll keep you posted on my findings. [22:15] hazmat: this is happening on all formulas. It does appear to be an Ensemble issue [22:15] ugh.. [22:16] hazmat: I just updated to the latest and greatest ensemble too.... [22:16] * hazmat updates as well [22:22] hazmat: well, the spurious @ERRORS was because I had set -x in my script [22:22] hazmat: (regardless of dash/bash) [22:22] hazmat: set -x prints each command that gets executed on stderr [22:22] hazmat: ensemble was slurping those up and labeling them "@ERRORs" [22:22] m_3 helped me figure that out [22:32] kirkland, but what was the issue with the script not putting the echo'd config file on disk? [22:36] hazmat: Where you able to reproduce the issue? [22:37] negronjl, not using the examples.. i'm trying with principia atm [22:37] negronjl, also works with principia [22:38] negronjl, i've got python-yaml_3.09-5ubuntu2 installed [22:39] hazmat: me too [22:41] hazmat: a bit of background. I was working on a formulat and it was working just fine ( i was doing some last minute checks on it ) when I noticed that ensemble had an upgrade. I shutdown the environment, upgraded it and bootstrapped it. Now I cannot deploy any of the formulas that were working 10 minutes befoe the update [22:45] negronjl, the issue is strange, from the traceback you posted, it looks like python-yaml is saying the metadata.yaml file is invalid... [22:46] negronjl, can you download this paste http://pastebin.ubuntu.com/644418/ and run python downloaded.py abs_path_to_mysql_metadata.yaml [22:46] hazmat ( or anyone else interested ). Can you reproduce ? [22:46] negronjl, i'm not able to reproduce [22:47] if parses it should just print it stdout, if it fails, the metadata.yaml file is indeed invalid [22:47] hazmat: http://pastebin.ubuntu.com/644420/ [22:48] hmm [22:48] negronjl, your using a ppa ensemble installation? [22:48] hazmat: i am [22:52] <_mup_> ensemble/expose-provision-machines r291 committed by jim.baker@canonical.com [22:52] <_mup_> Properly supporting re-exposing a service [22:52] hazmat: I opened a bug (https://bugs.launchpad.net/ensemble/+bug/810765) [22:52] <_mup_> Bug #810765: cannot deploy any formulas < https://launchpad.net/bugs/810765 > [22:52] hazmat: I'll be back in a minute. [23:03] ive got an install hook that succeeds and exits 0, but ensemble.executor never logs it as complete and does not then fire the start hook. [23:04] if i destroy the service and re-deploy it to the same machine, it re-installs and fires 'start' [23:04] negronjl: I ran into a problem with one formula in the repository directory screwing up others... you might check metadata of all or clean up your repo [23:05] if i destroy the service, remove the packages from the system, and redeploy, ensemble gets stuck in 'install' again.. any ideas? [23:05] adam_g, when you say its stuck in install.. what does ensemble status show? [23:06] m_3, good point.. any broken formula metadata in the repo can hose the deploy [23:06] * hazmat updates the bug [23:06] hazmat: null [23:06] I had an accidentally named config.yaml file in one recipe... it took a _while_ to figure that out [23:06] adam_g: crickets all the way man [23:07] s/recipe/formula/g [23:07] hazmat: "hook.executor@DEBUG: Hook complete: ...." is never logged the first time around [23:08] (For install) [23:08] adam_g, if you log into the machine the first time a round, is it still running? [23:08] ie. are we sure the hook has exited? [23:09] yes, positive [23:09] and sure that has exitted successfully (logging in the script to make sure) [23:10] lp:~gandelman-a/+junk/rabbitmq if you'd like to test, its 100% reproducable [23:11] adam_g, can you file a bug.. i'm about to head out for dinner, but i'll have a look latter [23:11] definitely [23:14] m_3, negronjl: In the near future "ensemble publish" will catch those issues [23:14] niemeyer: haven't seen that one... [23:15] m_3: It doesn't exist yet.. it's part of the repo work [23:15] m_3: We'll validate formulas before publishing them [23:15] awesome [23:27] niemeyer, i found a bug in expose-provision-machines with respect to re-exposing a service. the solution is simple: do the firewall checks per service unit with open_close_ports regardless of whether the service is exposed [23:27] niemeyer, however to fix this, i needed to do the guard against the connection being closed in _watch_topology, much like other watches [23:28] niemeyer, should i put that 2-line change in a separate branch and test it? my only concern is that the one example i have seen that tests connections is in ensemble.state.tests.test_agent.AgentDomainTest.test_connect_agent, which is not reliable when looped [23:31] jimbaker: Hmm [23:32] niemeyer, with these changes, and a downstream branch, we should have port exposing working as desired on ec2. the only bit left is 1. change the example formula (that's also 2 lines, including comment); 2. support exposed/unexposed hooks [23:33] jimbaker: Not sure I understand [23:33] jimbaker: You found a bug in expose-provision-machines, and that also needs a guard [23:33] #2 is a bit more work, but it also seems more of a specialized usage (eg, maybe a formula would respond to exposing a service by starting a web-based admin) [23:34] jimbaker: Sounds like we need a branch which addresses these two issues? [23:34] niemeyer, sounds like a good plan [23:34] niemeyer, i will create a branch to address [23:35] jimbaker: Cool, thanks! [23:59] Alright, I'm stepping out to the room.. see you all later