[00:46] <_mup_> Bug #804120 was filed: add-relation between add-unit instances gives instances out of range <Ensemble:New> < https://launchpad.net/bugs/804120 >
[01:06] <_mup_> Bug #803855 was filed: peer hooks are not invoked <Ensemble:Confirmed> < https://launchpad.net/bugs/803855 >
[01:24] <hazmat> SpamapS, negronjl my first comment on the bug shows the output of the peer hook..
[01:24] <negronjl> hazmat:  checking the bug now...
[01:25] <hazmat> negronjl,  relations aren't added between units but between services
[01:25] <hazmat> a peer relation is self-referential.. i gave the example in the bug of adding it for the tomcat service
[01:25] <negronjl> hazmat:  my last post on the bug shows the error that ensemble returns when I try to add-relation between services
[01:26] <hazmat> negronjl, it shows adding a relation between units which isn't valid
[01:26] <hazmat> oh.. nm
[01:26] <hazmat> negronjl, i see now
[01:26] <SpamapS> hazmat: it looks like theres a relation with no services listed in zk
[01:26] <hazmat> negronjl, but that command does add the relation and the hooks do fire
[01:27] <hazmat> negronjl, i've succesfully run debug-hooks against the peer relation hooks in your formula, and watched them execute after i got gustavo's first email re
[01:27] <hazmat> negronjl, that error in status is a bug though
[01:27] <negronjl> hazmat:  I really don't understand your position on this.  I just showed you the command + output.  What am I doing wrong here?
[01:28] <negronjl> hazmat:  Are you running the latest version from PPA or something newer ?
[01:28] <hazmat> negronjl, i'm saying the relation is added by the ensemble add-relation tomcat6 tomcat6.. and that the hooks do fire
[01:28] <hazmat> negronjl, i agree the status command appears to have issues displaying peer relations
[01:28] <hazmat> negronjl, i also agree that peer relations should probably be auto added by deploy
[01:28] <hazmat> but my position is that peer hooks do work now
[01:29] <SpamapS> [zk: localhost:2181(CONNECTED) 7] get /relations/relation-0000000000/peer    
[01:29] <SpamapS> {name: cluster, role: peer}
[01:29] <hazmat> SpamapS, what do you expect to see there?
[01:29] <hazmat> SpamapS, ls on that same node
[01:30] <negronjl> hazmat:  I'll try again with the debug-hooks running but, the output of ensemble status is still a bug even if I see hooks running...I'll get back to you in a minute so I can try again
[01:30] <hazmat> negronjl, agreed re status
[01:30] <SpamapS> hazmat: two units
[01:30] <hazmat> SpamapS, as expected
[01:31] <SpamapS> hazmat: this code sees no  "rel_services" in status...
[01:31] <hazmat> SpamapS, i think we've established there's a bug in status
[01:31] <SpamapS>             rel_service = rel_services[0]
[01:32] <hazmat> ill fix the status bug now
[01:32] <SpamapS>     # filter out 'self'
[01:32] <SpamapS>             rel_services = [rsn for rsn in rel_services if rsn.service_name !=
[01:32] <SpamapS>                             service.service_name]
[01:33] <hazmat> yeah.. its unfortunate that the status doesn't have any peer tests
[01:33] <hazmat> SpamapS, that code is from status i assume
[01:33] <SpamapS> yes, line 220
[01:33] <SpamapS> removing it fixes it
[01:35] <SpamapS> But probably will cause bad output on non peer relations
[01:38] <hazmat> negronjl, did that work?
[01:38] <negronjl> hazmat:  still spinning up.  one more minute
[01:39] <hazmat> sounds like its been an exciting week, wish i could have been sprinting with you guys
[01:39]  * hazmat looks at status
[01:39] <hazmat> SpamapS, thanks for the pointer
[01:45] <SpamapS> hazmat: its been a little nuts yeah
[01:49] <negronjl> hazmat:  just checked it and it works but, status is busted
[01:50] <hazmat> negronjl, cool, i'm working on status right now, it will be in the review queue in an hr or two, but it probably won't get enough reviews to go in b4 next week
[01:50] <hazmat> the PWD for hooks to be the formula directory has been in review for a week as well..
[01:50] <hazmat> negronjl, you could probably bug bcsaller and jimbaker  to do a review and we can get it pushed tomorrow if you need to demo it
[01:51] <_mup_> Bug #804135 was filed: peer relations should be automatic <Ensemble:New> < https://launchpad.net/bugs/804135 >
[01:51] <hazmat> i'll see if i can get to that one as well
[01:51] <hazmat> tonight
[01:51] <negronjl> hazmat:  I will.  Thanks for the help hazmat
[01:52] <hazmat> negronjl, np... sorry if i seemed a little short... its just that i saw it working, i didn't understand what the issue was
[01:52] <negronjl> hazmat:  no worries
[01:53] <hazmat> negronjl, SpamapS btw if you guys are doing a demo, its a little bit of a cheat, but in the bzr history, there's an auto-deploy rev, that will auto deploy dependent formulas and add required relations automatically when deploying
[01:54] <hazmat> very magical ;-)
[01:58] <SpamapS> hazmat: heh.. demo time is in about 7 hours .. I'm just going to demo my LXC branch. ;)
[01:58] <SpamapS> if I wake up in time
[02:00] <hazmat> SpamapS, nice
[02:00] <hazmat> SpamapS, get some sleep.. its late there i assume
[02:01] <SpamapS> hazmat: hacking on interfacing w/ cobbler. ;)
[02:01] <SpamapS> and yeah its about time to hit it
[02:01] <hazmat> SpamapS, via xml-rpc of cobbler?
[02:01] <SpamapS> hazmat: yep
[02:02] <SpamapS> lp:~clint-fewbar/ensemble/orchestra-provider
[02:02] <SpamapS> VERY VERY raw
[02:03] <hazmat> SpamapS, nice
[10:01] <_mup_> Bug #804199 was filed: Ensemble needs a way to pin a formula to a specific package version <Ensemble:New> < https://launchpad.net/bugs/804199 >
[10:02] <_mup_> Bug #804202 was filed: Ensemble needs to enforce configuration sanity <Ensemble:New> < https://launchpad.net/bugs/804202 >
[10:04] <_mup_> Bug #804203 was filed: Ensemble needs to communicate securely with Zookeeper <Ensemble:New> < https://launchpad.net/bugs/804203 >
=== daker_ is now known as daker
[10:53] <m_3> http://paste.ubuntu.com/636296/
[10:53] <m_3> negronjl: http://paste.ubuntu.com/636296/
[10:54] <m_3> negronjl: http://paste.ubuntu.com/636296/
[10:54] <m_3> sorry... irssi retard
[10:57] <_mup_> Bug #804226 was filed: Relation Get should show all elements <Ensemble:New> < https://launchpad.net/bugs/804226 >
[11:14] <_mup_> ensemble/expose-provision-machines r269 committed by jim.baker@canonical.com
[11:14] <_mup_> Watch service unit assignment on machines
[11:44] <_mup_> ensemble/expose-provision-machines r270 committed by jim.baker@canonical.com
[11:44] <_mup_> More guards
[12:27] <_mup_> ensemble/expose-provision-machines r271 committed by jim.baker@canonical.com
[12:27] <_mup_> Refactored tests
[13:37] <hazmat> zookeeper multi-node functionality lands
[13:41] <_mup_> Bug #804284 was filed: REST API for managing ensemble environments, aka expose cli as ensemble daemon <Ensemble:New> < https://launchpad.net/bugs/804284 >
[14:38] <_mup_> Bug #804327 was filed: Implement formula include <Ensemble:New> < https://launchpad.net/bugs/804327 >
[14:50] <_mup_> ensemble/expose-provision-machines r272 committed by jim.baker@canonical.com
[14:50] <_mup_> Robust testing without sleep on working with machines
[15:03] <hazmat> argh.. includes
[15:18] <jimbaker> hazmat, yeah, we need stacks :)
[15:20] <bcsaller> hazmat:  bare metal deployments changes the timeline on the multi availability zone stuff. I am writing up the conversation I just had, we need to chat about this soon
[15:20] <hazmat> jimbaker, indeed, noted the same on the bug comment
[15:21] <jimbaker> bcsaller, so they want this to work in a high-latency env?
[15:21] <hazmat> bcsaller, not sure i see the relation between the two.. multi-availability zone is basically rack level awarness in physical terms.. a --location param seems to encompass both
[15:21] <hazmat> jimbaker, az are low latency xc communication
[15:22] <hazmat> 3-10ms, avg 5ms afaicr
[15:22] <bcsaller> hazmat: deploy openstack to create more than one Avail Zone
[15:22] <bcsaller> then use ensemble to manage services in that zone
[15:22] <bcsaller> we have new expectations at many levels of that deployment story
[15:22] <hazmat> bcsaller, cool, looking forward to the write up
[15:24] <bcsaller> hazmat: I am writing up a review of the AMI branch now, keep having to service interrupts though 
[15:27] <hazmat> bcsaller, no worries, i'm dealing with the same (low throughput, kaleb hanging on my arm literally atm)
[15:27] <bcsaller> hi kaleb
[15:38] <_mup_> ensemble/status-w-peers r264 committed by kapil.thangavelu@canonical.com
[15:38] <_mup_> sattus works with peer relations
[15:40] <bcsaller> I'll review that today as well ^^
[15:41] <hazmat> bcsaller, cool, thanks, i had some problems on some of the render tests, but they seem to predate the branch.. not sure, maybe missing i'm missing graphiz locally the error expressed oddly as unable to obtain zk handle
[15:42] <bcsaller> hazmat: if you have an old stack trace or can reproduce point me at it and I can take a look
[15:42] <bcsaller> I might not see it otherwise :-/
[15:43] <hazmat> bcsaller, https://pastebin.canonical.com/49276/
[15:43] <bcsaller> hazmat: and thats intermittent or not?
[15:44] <hazmat> bcsaller, constant
[15:44] <bcsaller> ok, looks like its not getting a handle to ZK in the dummy provider, that might indicate that something in setUp is failing 
[15:44] <hazmat> bcsaller, i get it on trunk as well
[15:44] <bcsaller> I can take a look
[15:44] <bcsaller> doh!
[15:45] <hazmat> when just running the status tests
[15:45] <hazmat> bcsaller, hmm.. might be my local setup.. something seems strange
[15:45] <hazmat> lots of tests fail for me on trunk.. randomly.. i might have a bg zk
[15:47] <bcsaller> yeah, I don't get that, just ran the branch w/o issue 
[15:47] <bcsaller> hazmat: thats what it sounds like to me
[15:52] <hazmat> bcsaller, cool
[15:55] <kirkland> hazmat: yo!
[15:55] <kirkland> hazmat: how goes :-)
[15:55] <hazmat> kirkland, wasup? any good irish explorations ?
[15:55] <hazmat> kirkland, pretty good.. hanging at the beach w/ my son, watching wimbledon, fixing bugs, but wishing i was in dublin
[15:55] <kirkland> hazmat: heading to scotland next week for some hiking and whisky tasting
[15:55] <kirkland> hazmat: nice
[15:55] <hazmat> kirkland, awesome
[15:56] <kirkland> hazmat: bcsaller says you've got the bug about ensemble requiring its own images
[15:56] <kirkland> hazmat: we're blocking a bit on that right now
[15:56] <hazmat> kirkland, yeah.. i fixed it earlier this week, its in review, ensemble will be able to use any cloud-init enabled ubuntu image
[15:56] <kirkland> hazmat: as we're developing formulas for cloudfoundry that require a) oneiric and b) much larger instances
[15:56] <kirkland> hazmat: rock! 
[15:57] <kirkland> hazmat: can you point me to that bug/merge?
[15:57] <hazmat> kirkland, alternatively you can use the ./bin/ensemble-make-image script to make a 64 bit image for use with the environments.yaml config 'default-instance-type'
[15:57] <hazmat> and setting the image id
[15:57] <hazmat> kirkland, https://bugs.launchpad.net/ensemble/+bug/791501
[15:57] <_mup_> Bug #791501: Ensemble should use standard amis and the ppa <Ensemble:In Progress by hazmat> < https://launchpad.net/bugs/791501 >
[15:58] <kirkland> hazmat: sweet, thanks
[15:58] <hazmat> kirkland, cutting out the use of bzr with the ppa, makes the initial install time about the same as a prebaked image (minus the bootstrap node which needs to install java)
[15:58] <hazmat> twisted takes a long time to install as well oddly.
[15:59] <kirkland> hazmat: nice
[16:04] <hazmat> kirkland, any other blocker bugs for you guys?
[16:04] <hazmat> kirkland, i'm working on auto adding relations for peers that troubled negronjl yesterday
[16:04] <kirkland> hazmat: hmm, well, we've filed a handful this week
[16:04] <hazmat> kirkland, well most of them seemed like wish list items
[16:05] <kirkland> hazmat: lynxman and negronjl  have been filing them;  i've been marking them "confirmed" and adding additional notes
[16:05] <kirkland> hazmat: i think our highest is the generic images/bigger instances
[16:05] <kirkland> hazmat: let me check the others ....
[16:05] <hazmat> the templating one is a wish list, the parallel ssh exec one is going to need some more thought
[16:06] <hazmat> security stuff i'm going to be making a big push on for most of july
[16:06] <kirkland> hazmat: good, security stuff will be essential before we can actually start pushing this to real customers
[16:06] <hazmat> kirkland, i updated that bug with a link to the security spec that forms the initial design
[16:07] <kirkland> hazmat: the templating one, yeah, wishlist;  something we really need/want, so that we can define tunables and just put our selections in the config
[16:08] <kirkland> hazmat: absolutely necessary for formula reuse
[16:08] <hazmat> kirkland, absolutely.. the last branch for config/tunables is in review
[16:08] <hazmat> its had a rough time through review, but it should land hopefully next week
[16:11] <kirkland> hazmat: cool, that's more important to me than the templating itself
[16:11] <kirkland> hazmat: which we can handle as formula writers
[16:11] <kirkland> hazmat: but we need help from ensemble on the config side
[16:11] <hazmat> kirkland, sounds good
[16:11] <hazmat> kirkland, how so?
[16:12] <hazmat> beyond offering the configuration options on a service
[16:17] <kirkland> hazmat: heh, just that -- offering the config options and getting them through to the install hook
[16:18] <hazmat> kirkland, cool
[16:19] <hazmat> off to grab some lunch, bbiab
[16:24]  * hazmat  tries to tap the packaging experience in the room
[16:29] <_mup_> ensemble/expose-provision-machines r273 committed by jim.baker@canonical.com
[16:29] <_mup_> Test to verify that machine applies firewall policy after unit has been assigned