#ubuntu-ensemble 2011-06-27
<_mup_> ensemble/trunk r261 committed by jim.baker@canonical.com
<_mup_> [trivial] By default tests redirect ZK log to /dev/null [r=hazmat][f=719520]
<_mup_> ensemble/expose-provision-service-hierarchy r302 committed by jim.baker@canonical.com
<_mup_> Test naming and docstrings
<_mup_> ensemble/expose-provision-service-hierarchy r303 committed by jim.baker@canonical.com
<_mup_> Merged trunk
<pindonga> morning
<hallyn> morning
<hazmat> off to mongodb conference for the day. bbiab
<kim0> jimbaker: I'm not really sure what the decision was for bug 766317
<_mup_> Bug #766317: debug-log should show relation settings changes <Ensemble:In Progress by jimbaker> < https://launchpad.net/bugs/766317 >
<_mup_> ensemble/expose-provision-service-hierarchy r304 committed by jim.baker@canonical.com
<_mup_> Test comments
<_mup_> ensemble/expose-provision-service-hierarchy r305 committed by jim.baker@canonical.com
<_mup_> Cleanup - PEP8, PyFlakes, removed debugging
<kim0> So that's the list of happenings around Ensemble I could collect this week â http://paste.ubuntu.com/633726/
<kim0> let me know anything to add or remove 
<_mup_> ensemble/trunk r262 committed by jim.baker@canonical.com
<_mup_> merge expose-provision-service-hierarch [r=niemeyer,hazmat][f=787069]
<_mup_> Implements the management of watches on services, service exposing,
<_mup_> and service unit ports in the provisioning agent such that any changes
<_mup_> being watched for will trigger the firewall mgmt for the opening
<_mup_> and/or closing of ports.
<_mup_> ensemble/trunk r262 committed by jim.baker@canonical.com
<_mup_> merge expose-provision-service-hierarchy [r=niemeyer,hazmat][f=787069]
<_mup_> Implements the management of watches on services, service exposing,
<_mup_> and service unit ports in the provisioning agent such that any changes
<_mup_> being watched for will trigger the firewall mgmt for the opening
<_mup_> and/or closing of ports.
<kim0> niemeyer: has config-set landed yet ? I'm writing the report
<_mup_> ensemble/trunk r263 committed by jim.baker@canonical.com
<_mup_> merge expose-provision-service-hierarchy [r=niemeyer,hazmat][f=787069]
<_mup_> Implements the management of watches on services, service exposing,
<_mup_> and service unit ports in the provisioning agent such that any changes
<_mup_> being watched for will trigger the firewall mgmt for the opening
<_mup_> and/or closing of ports.
<jimbaker> needed to retry twice times... first to get the commit message correct, second because bcsaller just pushed up another trunk merge
<jimbaker> (that would be just twice OR two times :)... typos, typos)
<_mup_> Bug #802678 was filed: Update watch_ports <Ensemble:In Progress by jimbaker> < https://launchpad.net/bugs/802678 >
<kapilt> jimbaker: How's Dublin?
#ubuntu-ensemble 2011-06-28
<bcsaller> niemeyer: https://bugs.launchpad.net/ensemble/+bug/771348
<_mup_> Bug #771348: status barfs when checking newly deployed service units (no state) <Ensemble:New> < https://launchpad.net/bugs/771348 >
<niemeyer> bcsaller: Cheers
<jimbaker> hazmat, dublin is fun + productive
<kim0> A new screencast springs into existence â http://www.youtube.com/user/ubuntucloud#p/a/u/0/AMHcy63wRL0
<kim0> "Ensemble deploy and scale cloud apps with ease"
<jimbaker> kim0, sounds great! need to get my headphones out
<kim0> if you spot a horrible mistake .. let me know .. can still fix it
<_mup_> ensemble/expose-status r263 committed by jim.baker@canonical.com
<_mup_> Tests to verify that open-ports is not set in status unless service is exposed
<_mup_> ensemble/expose-status r264 committed by jim.baker@canonical.com
<_mup_> Do not set open-ports unless service is exposed
<_mup_> Bug #802931 was filed: Modify provisioning agent so that the machine opens/closes ports <Ensemble:New> < https://launchpad.net/bugs/802931 >
<kim0> Does anyone agree "ensemble shutdown" should have a -y option ?
<ivoks> can ensemble formula be interactive?
<ivoks> meaning, can it ask for some user input before for information required when packages are installed
<kim0> ivoks: afaik not really .. but we'll get that soon with config-set (passing variables to formulas)
<kim0> devs correct me if I'm wrong
<ivoks> ok, that's good enough
<ivoks> how does expose/unexpose work?
<ivoks> even if i don't expose, all the services from my formula are available from outside
<kim0> ivoks: I believe expose is for opening up ports using ec2 security groups (current provider)
<kim0> but today .. I believe all ports are open by default
<ivoks> ok
<ivoks> i'm working on mail-stack formula
<ivoks> i got it working fine, with exception of not being able to define relayhost :)
<kim0> Ben is working hard to add passing params
<kim0> should hopefully land soon
<ivoks> what i'd like to do more is to separate postfix to one instance and dovecot to the other
<kim0> ivoks: btw https://ensemble.ubuntu.com/docs/ has more info on exposing :)
<ivoks> yeah, i know
<ivoks> :)
<hazmat> ivoks, atm things are open, but in future, when the ports stuff land it will be closed to public by default but open to internal traffic (till we adjust the ports work to take into account relations)
<ivoks> ok
<hazmat> ivoks, re how ports work.. the formula hooks can designate which hooks they want to open and close via an ensemble cli api (open-port, close-port).. the ensemble user/admin will need to expose the service for these hook defined settings to take effect
<ivoks> ok
<hazmat> underneath the hood their using ec2 security groups as kim0 pointed out
<hazmat> kim0, are you in dublin this week?
<kim0> Yes 
<kim0> you are not right
<hazmat> kim0, nope.. just wondering where the dev team is hanging out at
<hazmat> kim0, i did a mongodb conference yesterday.. and did a quick demo of the mongodb formula i've been working on
<hazmat> but it also needs config-set for the sharding configuration
<kim0> woohoo
<kim0> hazmat: share that formula man :)
 * hazmat looks up the how to share with principia docs
<hazmat> niemeyer, hallo!
<ivoks> destroy-service doesn't really do anything yet?
<hazmat> ivoks, it does.. it removes relations, destroys units, it should wipe the service from the environment
<ivoks> hm
<ivoks> everything is still running on my instance :)
<hazmat> hmm.. 
<hazmat> ivoks, have you implemented the stop hook?
<ivoks> there are no relations, but dovecot, postfix and amavis are still running
<ivoks> yes; it might be wrong
<ivoks> service dovecot stop
<ivoks> same for postfix and amavis
 * hazmat checks the implementation
<hazmat> SpamapS, how's the reception of lxc-provider at the rally?
<hazmat> ivoks, so it doesn't appear that destroying a service will actually inform the unit it should suicide, it just tells the machine agent to kill the unit agent
<hazmat> ivoks, so in the destroy-service case with the service processes still running, do you see any unit agent processes in the process listing?
<hazmat> so the relation-broken hooks are invoked, but not afaics the stop hook
<ivoks> i see unit agent
<ivoks> /usr/bin/python -m ensemble.agents.unit -n --pidfile /var/lib/ensem....
<hazmat> hmm
<ivoks> it's not listed in ensemble status
<ivoks> ah, don't worry about it
<ivoks> i have other things to solve first
<niemeyer> hazmat: Hey man!
<niemeyer> hazmat: Missing you here
<hazmat> ivoks, i'll keep digging into it
<ivoks> like 'restart service after 'install''
<hazmat> niemeyer, how's it going so far
<niemeyer> hazmat: Fantastic
<hazmat> niemeyer, awesome.. any highlights so far? or just a good productive vibe going around?
<ivoks> niemeyer: i bet it's raining all the time :)
<niemeyer> ivoks: :-)
<niemeyer> hazmat: Quite a few.. still have to digest
<niemeyer> hazmat: What are you working on right now?
<hazmat> i think its rained every day at wimbledon but like two.. out of 10
<niemeyer> hazmat: and how much time would it take for you to get rid of our pre-baked images?
<hazmat> niemeyer, atm digging into destroy-service
<hazmat> niemeyer, maybe 2.5 hrs
<niemeyer> hazmat: Awesome!
<niemeyer> hazmat: Can you push that?
<hazmat> niemeyer, sounds good, i'll put an issue to have a look at destroy-service later
<niemeyer> hazmat: Yeah, destroy-service actually looks important, but I wonder if it should be postponed and solve LXC first
<niemeyer> hazmat: destroy-service will likely have a different story in that light
<bcsaller> hazmat: also there is some hinting at more review on my branch which would be helpful
<hazmat> niemeyer, not sure that its significantly different, in light of lxc.. perhaps.. the machine agent does a destroy on the unit deployment, and the relations are broken first.. i guess it depends on how we structure the agents for the container, if a machine agent lives in each container in the provider case, or if we just have a singleton machine agent on the physical machine creating containers (which would translate well to pushing the work to ec2)
<hazmat> bcsaller, indeed i do need to follow up on that
<niemeyer> hazmat: Yeah, maybe it's not that different indeed
<niemeyer> hazmat: What about the security stuff?
<niemeyer> hazmat: Would be good to get some action happening on that front sooner rather than later
<_mup_> Bug #802995 was filed: Destroy service should invoke unit's stop hook, verify/investigate this is true <Ensemble:New> < https://launchpad.net/bugs/802995 >
<hazmat> niemeyer, i've got some comments on the review, i'll try to kick that back out today..  so reviews and image work on my plate for today
<niemeyer> hazmat: Cool.. I'm just concerned we're not getting any action for a while
<niemeyer> hazmat: We can continue evolving the spec and the overall design, but would be good to define what are the first few steps and get these going
<ivoks> hazmat: note that this was reproduced on my custom formula (maybe the formula is broken)
<hazmat> niemeyer, the mongodb conference yesterday was fun.. their roadmap has some cool features.. the aggregation stuff (map/reduce replacement) is really nice
<niemeyer> hazmat: It's awesome indeed.. really looking forward to it
<hazmat> off to dog-walk, back in a few
<niemeyer> hazmat: Have fun
<SpamapS> hazmat: re the lxc provider, its a bit rough getting going, but m_3 was able to get it going
<hazmat>  SpamapS nice, you doubled the user population ;-)
<_mup_> ensemble/expose-provision-machines r264 committed by jim.baker@canonical.com
<_mup_> Initial commit
<SpamapS> hazmat: I really want to get all of the agents running under upstart, that would be cool.
<hazmat> SpamapS, i had a look after we discussed last month, it looked pretty straightforward
<hazmat> i'll see if i can get in this week if no else gets to it
<_mup_> Bug #803042 was filed: The "bootstrap node" needs high availability <Ensemble:New> < https://launchpad.net/bugs/803042 >
<ivoks> i could help with that :)
<niemeyer> Yos
<niemeyer> ivoks: Why aren't you here? :)
<ivoks> don't know :)
<_mup_> ensemble/sans-ami r264 committed by kapil.thangavelu@canonical.com
<_mup_> use standard ubuntu amis
<_mup_> ensemble/sans-ami r265 committed by kapil.thangavelu@canonical.com
<_mup_> add missing natty release image data for tests
<_mup_> ensemble/expose-provision-machines r265 committed by jim.baker@canonical.com
<_mup_> Now loops
<_mup_> ensemble/expose-provision-machines r266 committed by jim.baker@canonical.com
<_mup_> Removed debugging sleeps from new test
<_mup_> ensemble/sans-ami r266 committed by kapil.thangavelu@canonical.com
<_mup_> allow image specifications to select an ami, these specs are not passed to user data.
<_mup_> ensemble/sans-ami r267 committed by kapil.thangavelu@canonical.com
<_mup_> use the ensemble ppa for software installation by default, but also support branch installations.
<_mup_> ensemble/sans-ami r268 committed by kapil.thangavelu@canonical.com
<_mup_> tweak install commands to include state and log dir creation
<_mup_> ensemble/sans-ami r269 committed by kapil.thangavelu@canonical.com
<_mup_> correct name for txzookeeper, and normalize checkout dirname
 * niemeyer breaths and heads to some reviewing..
<niemeyer> A bit sleepy.. may not last long. :)
#ubuntu-ensemble 2011-06-29
<niemeyer> Okay, some great code reviewed, and now it's really time for bed.
<niemeyer> Night all
<ivoks> idea: does it make sense to use micro for example?
<niemeyer> ivoks: You mean the micro instance type?
<ivoks> yes
<ivoks> for bootstrap
<niemeyer> ivoks: It's too small
<niemeyer> ivoks: Well.. maybe for bootstrap it's not
<niemeyer> ivoks: But in general it's too slugish
<niemeyer> sluggish
<ivoks> i know
<niemeyer> ivoks: We actually used it across the board until very recently
<ivoks> but for learning and playing with ensemble...
<niemeyer> ivoks: But we had complaints about it being extremely unpredictable in terms of performance
<ivoks> amazone gives two micro instances for free for 1 year
<niemeyer> ivoks: So we gave up on the idea to avoid burning the project image just because the instance is unreliably slow
<ivoks> for someone that wants to try and play with ensemble (no production), those would be enough
<ivoks> ok
<niemeyer> ivoks: Yeah, it's a bit of a pity, but there's a reason why they give it for free :-)
<ivoks> i know :)
<_mup_> ensemble/expose-provision-machines r267 committed by jim.baker@canonical.com
<_mup_> Add test re assigning a machine to a service unit with opened ports, opens ports on that machine
 * hazmat waits for the coffee to kick in
<niemeyer> hazmat: ping
<hazmat> niemeyer, pong
<niemeyer> hazmat: Yo
<niemeyer> hazmat: All good there?
<hazmat> just chatting with bcsaller about the config-set branch and orchestra
<hazmat> niemeyer, all good.. wimbledon on in the background
<hazmat> niemeyer, working on security doc, and upstart today
<niemeyer> hazmat: Ah, awesome
<hazmat> niemeyer, ready for the meeting anytime 
<niemeyer> hazmat: How's the image stuff?
<hazmat> niemeyer, in review
<niemeyer> hazmat++
<bcsaller> hazmat: I expect to have a draft doc of the co-located services proposal out soon too, so you can hear our thinking on that 
<hazmat> niemeyer, looking good, start time for the bootstrap node is a little longer, but the rest seems quite workable
<niemeyer> hazmat: Can I trow something else on your table without you falling down?
<niemeyer> throw even
<hazmat> niemeyer, shoot, we'll see if it sticks ;-)
<niemeyer> hazmat: Juan had difficulties with peer relations
<niemeyer> hazmat: It's my understanding that they were supposed to work already, is that right?
<hazmat> niemeyer, hmm.. how so.. i've been using them a little for the mongodb stuff, but we don't have many formulas that use them
<bcsaller> hazmat: the peer events didn't seem to fire, we haven't looked into it yet
<hazmat> actually any
<hazmat> hmm
<niemeyer> hazmat: Hah, sweet
<hazmat> niemeyer, i'll dig into that
<niemeyer> hazmat: So it's likely something minor
<niemeyer> hazmat: Let me send you his formula
<hazmat> niemeyer, which formula .. thanks
<hazmat> i can prioritize that, its probably a minor one
<niemeyer> hazmat: Should be in your inbox
<hazmat> niemeyer, not seeing it yet
<hazmat> niemeyer, got it
<niemeyer> hazmat: Sweet
<hazmat> niemeyer, no attachment though
<niemeyer> hazmat: Meeting in 1:40.. will try to dial you in a little earlier
<niemeyer> hazmat: The branch is there
<hazmat> niemeyer, ah right.. regular phone or skype?
<niemeyer> hazmat: What phone number should I dial?
<hazmat> niemeyer, in priv msg
<niemeyer> hazmat: Thanks
<hazmat> negronjl, so the problem with the tomcat formula is that its using relative addressing to files
<hazmat> 2011-06-29 11:38:28,105 unit:tomcat6/1: hook.output ERROR: sed: can't read /usr/lib/ensemble/txzookeeper/../templates/tomcat-server-xml-top.tmpl: No such file or directory
<hazmat> hth
<hazmat> that output is from the debug-log which is very useful for debugging problems..  as is the debug-hooks cmd
<hazmat> negronjl, there's a branch in review to ensure that formula hooks are always executed from the formula dir
<hazmat> but it hasn't landed, atm the hook PWD isn't something that can be relied on by hooks
<hazmat> and addressing paths from there should be absolute
<hazmat> negronjl, i guess in this case your trying to get to some formula included config?
<hazmat> negronjl, which will become more easier once bug 761015 is fixed
<_mup_> Bug #761015: FORMULA_DIR should be available to hooks <Ensemble:In Progress by hazmat> < https://launchpad.net/bugs/761015 >
 * hazmat grabs some food
<niemeyer> hazmat: ping?
<hazmat> niemeyer, pong
<niemeyer> hazmat: Ring.. ring.. ring..
<niemeyer> hazmat: Ring.. ring.. ring..
<hazmat> niemeyer, no ring here
<hazmat> niemeyer, phone or skype?
<niemeyer> hazmat: We just talked to your answering machine
<hazmat> niemeyer, hmm.. my phone is low let me plug-in and see if works better
<niemeyer> hazmat: Again
<hazmat> niemeyer, bcsaller, jimbaker, anyone.. i dropped.. pls callback
<niemeyer> hazmat: Calling
<hazmat> niemeyer, thanks
<hazmat> niemeyer, bcsaller, jimbaker, anyone.. i dropped again.. pls callback
<niemeyer>  hazmat Calling
<hazmat> niemeyer, bcsaller, jimbaker, anyone.. argh! i dropped again.. pls callback
 * hazmat kicks his phone company where the light don't shine
<hazmat> niemeyer, bcsaller, jimbaker, anyone.. argh! i dropped again.. it might be time to give up.. 
<hazmat> niemeyer, i've been curious while looking at zeromq go bindings, how the general problem of thread local resources is handled in go, since there isn't any easy way of distinguishing thread from a goroutine (ie their mapped m to  n to threads)
<hazmat> seems like its also relevant in the context of gui apps where typically only one thread is managing interacting with the gui run loop
<hazmat> the only thing variant i've seen for this seems a bit like overkill (runtime.LockOSThread etc)
<niemeyer> hazmat: We have exactly the same problem with zk
<niemeyer> hazmat: It's handled by giving it its own thread
<niemeyer> hazmat: But the zeromq Go author wasn't very clever at that
<niemeyer> hazmat: Which means it will blow up on people's face
<hazmat> indeed, it looks experimental at best
<hazmat> interestingly the erlang for (zeromq) version does much the same (separate thread allocation)
<hazmat> niemeyer, thanks, i'll take a look at the gozk impl for ideas
<niemeyer> hazmat: Haven't seen that, but it's the only option that makes sense.. goroutines with free threading is a feature which we shouldn't undermine
#ubuntu-ensemble 2011-06-30
<kiranm> ensemble status results in 'connection timeout' or 'broken pipe'. Running in verbose mode shows the the messages - http://paste.ubuntu.com/635647/
<_mup_> Bug #803852 was filed: 64-bit instances and large instances <Ensemble:New> < https://launchpad.net/bugs/803852 >
<_mup_> Bug #803855 was filed: peer hooks are not invoked <Ensemble:New> < https://launchpad.net/bugs/803855 >
<hazmat> kiranm, hmm.. if you retry do you get the same error, ie is it a persistent problem, also
<kiranm> hazmat, it turned out to be my corporate firewall blocking outbound traffic. when i tried on a different setup it is working fine. bootsrap, deploy mysql wordpress works fine. but add-relation fails with ambigious endpoints.... i'm using formulas from principia-tools and not from the examples directory
<kiranm> hazmat, let me try the whole sequence again - deploy mysql, deploy wordpress, check status, add relation
<hazmat> kiranm, there might be some new relations types in the mysql formula
<hazmat> which would requires specifying the relation names
<kiranm> hazmat, alright let me check with the example formulas first
<_mup_> Bug #803866 was filed: On ambigious relation endpoint error from add-relation should show the possible relations  <Ensemble:New> < https://launchpad.net/bugs/803866 >
<kiranm> hazmat, the exact error for the ambiguous endpoints....
<kiranm> mbiguous endpoints: ([(RelationEndpoint(service_name='wordpress', relation_type=u'mysql', relation_name=u'db', relation_role='client'), RelationEndpoint(service_name='mysql', relation_type=u'mysql', relation_name=u'db', relation_role='server')), (RelationEndpoint(service_name='wordpress', relation_type=u'mysql', relation_name=u'db', relation_role='client'), RelationEndpoint(service_name='mysql', relation_type=u'mysql', relation_name=u'db-admin', r
<kiranm> elation_role='server'))],)
<kiranm> 2011-06-30 17:39:35,243 ERROR Ambiguous endpoints: ([(RelationEndpoint(service_name='wordpress', relation_type=u'mysql', relation_name=u'db', relation_role='client'), RelationEndpoint(service_name='mysql', relation_type=u'mysql', relation_name=u'db', relation_role='server')), (RelationEndpoint(service_name='wordpress', relation_type=u'mysql', relation_name=u'db', relation_role='client'), RelationEndpoint(service_name='mysql', relation_type=u'mysql
<kiranm> ', relation_name=u'db-admin', relation_role='server'))],)
<hazmat> kiranm, so you need to add the relation via .. ensemble add-relation wordpress mysql:db
<hazmat> kiranm, a second named relation was added to the principia version to support administrative program access to mysql, so now the usage needs to be qualified
<kiranm> hazmat, ok understood. 
<kiranm> hazmat, are these kind of changes documented some where, so that people who test the formulas regularly can be enlightened
<kiranm> hazmat, especially the formulas from principia-tools would be the focus of testing IMO
<hazmat> kiranm, at the moment principia is pretty early in its development, the only change log is the revision log.. we will eventually get to releases/frozen version sets of principia formulas with change logs between them
<kiranm> hazmat, thx again
<hazmat> kiranm, np
<_mup_> Bug #803892 was filed: Ensemble should have a config templating system <Ensemble:New> < https://launchpad.net/bugs/803892 >
<_mup_> Bug #803895 was filed: Ensemble should have a parallel execution mechanism <Ensemble:New> < https://launchpad.net/bugs/803895 >
<_mup_> Bug #803902 was filed: provide a way to remove relationship entries <Ensemble:New> < https://launchpad.net/bugs/803902 >
<lynxman> hazmat: ping
<hazmat> lynxman, pong
<lynxman> hazmat: hey just added myself to bug #616900 since I'm trying to do a macports version of the ensemble client
<_mup_> Bug #616900: Make a release <txzookeeper:New for hazmat> < https://launchpad.net/bugs/616900 >
<lynxman> hazmat: any chance this can be done in the near future?
<lynxman> (near future being a couple of weeks or so)
<hazmat> lynxman, sure, there's one branch in review thats needed for a new release, it is already available off pypi
<lynxman> hazmat: oh cool, do you have the URL for the tarball?
<hazmat> lynxman, its basically a pure python package, the nesc compilation bit is getting zookeeper-python bindings installed
<hazmat> lynxman, http://pypi.python.org/pypi/txzookeeper
<lynxman> hazmat: lovely, thanks
<hazmat> lynxman, that version might be a little old for ensemble..
<hazmat> trunk that is
<lynxman> hazmat: well if you push a new tarball that'll be cool :) meanwhile I'll do my port work based on that one
<hazmat> lynxman, we've in the past run the full unit tests on osx with just one or two issues due to different fs handling of tempfiles
<hazmat> but for the most part it should just work
<lynxman> hazmat: that's good news :)
<lynxman> hazmat: just making the portfiles to submit them to macports for now
<hazmat> lynxman, you also want to make sure you have the most recent zk release version in macports
<lynxman> hazmat: as soon as it's working I'm sure there'll be bugs to fill
<hazmat> lynxman, awesome
<hazmat> lynxman, sounds good
<lynxman> hazmat: yeah slowly going through the dependencies
 * hazmat likes macports
<hazmat> very easy to contrib to
<lynxman> :)
<lynxman> hazmat: indeed, I'm also contributing byobu
<koolhead11> SpamapS, hello
<hazmat> although now a days i here everyone talking about homebrew.. only reason i can think of is the install into /usr/local makes it easy for lookups 
<koolhead11> hi all
<hazmat> when compiling additional extensions at the price of getting a potential mismash of libraries
<hazmat> koolhead11, hi
<hazmat> or perhaps its just tcl v ruby
<koolhead11> hi hazmat 
<koolhead11> i am working on the phpmyadmin but having some trouble with preseed configuration
<koolhead11> i see some bugs regarding the same on debian mailing list
<hazmat> koolhead11, i'm a bit out of my element regarding php.. but the deb mailing list or the lp bug tracker for phpadmin is probably the right place to follow up those issues
<hazmat> koolhead11, do you have a link?
<koolhead11> yes
<koolhead11> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=578960  
<koolhead11> when i am doing dpkg-reconfigure phpmyadmin i have to enter everything manually and then it writes to the DB :
<hazmat> koolhead11, hmm.. are you purging and reinstalling or just preseeding debconf selections
<koolhead11> i was doing purge earlier
<koolhead11> now i have installed new VM with snapshot
<koolhead11> am doing it on LTS
<hazmat> nice. http://morepypy.blogspot.com/2011/06/global-interpreter-lock-or-how-to-kill.html
 * koolhead11 heads back home
<_mup_> ensemble/expose-provision-machines r268 committed by jim.baker@canonical.com
<_mup_> Fixes to handle topology changes wrt watches
<_mup_> txzookeeper/session-event-handling r52 committed by kapil.foss@gmail.com
<_mup_> merge trunk
<_mup_> txzookeeper/session-event-handling r53 committed by kapil.foss@gmail.com
<_mup_> add a disconnected client test for failures, dead connections can exhibit either notconnected or connection exceptions adjust tests to reflect.
<_mup_> txzookeeper/session-event-handling r54 committed by kapil.foss@gmail.com
<_mup_> address jim's review comments, grammar fixes, additional param valiadation tests
<hazmat> bcsaller, do you mind if i post your email to the merge proposal along with my reply?
<bcsaller> hazmat: I don't remember specifically what you're talking about but I think its fine
<bcsaller> hazmat: the scheduler/workflow stuff?
<hazmat> bcsaller, yup
<bcsaller> hazmat: thanks for looking into that :)
#ubuntu-ensemble 2011-07-01
<_mup_> Bug #804120 was filed: add-relation between add-unit instances gives instances out of range <Ensemble:New> < https://launchpad.net/bugs/804120 >
<_mup_> Bug #803855 was filed: peer hooks are not invoked <Ensemble:Confirmed> < https://launchpad.net/bugs/803855 >
<hazmat> SpamapS, negronjl my first comment on the bug shows the output of the peer hook..
<negronjl> hazmat:  checking the bug now...
<hazmat> negronjl,  relations aren't added between units but between services
<hazmat> a peer relation is self-referential.. i gave the example in the bug of adding it for the tomcat service
<negronjl> hazmat:  my last post on the bug shows the error that ensemble returns when I try to add-relation between services
<hazmat> negronjl, it shows adding a relation between units which isn't valid
<hazmat> oh.. nm
<hazmat> negronjl, i see now
<SpamapS> hazmat: it looks like theres a relation with no services listed in zk
<hazmat> negronjl, but that command does add the relation and the hooks do fire
<hazmat> negronjl, i've succesfully run debug-hooks against the peer relation hooks in your formula, and watched them execute after i got gustavo's first email re
<hazmat> negronjl, that error in status is a bug though
<negronjl> hazmat:  I really don't understand your position on this.  I just showed you the command + output.  What am I doing wrong here?
<negronjl> hazmat:  Are you running the latest version from PPA or something newer ?
<hazmat> negronjl, i'm saying the relation is added by the ensemble add-relation tomcat6 tomcat6.. and that the hooks do fire
<hazmat> negronjl, i agree the status command appears to have issues displaying peer relations
<hazmat> negronjl, i also agree that peer relations should probably be auto added by deploy
<hazmat> but my position is that peer hooks do work now
<SpamapS> [zk: localhost:2181(CONNECTED) 7] get /relations/relation-0000000000/peer    
<SpamapS> {name: cluster, role: peer}
<hazmat> SpamapS, what do you expect to see there?
<hazmat> SpamapS, ls on that same node
<negronjl> hazmat:  I'll try again with the debug-hooks running but, the output of ensemble status is still a bug even if I see hooks running...I'll get back to you in a minute so I can try again
<hazmat> negronjl, agreed re status
<SpamapS> hazmat: two units
<hazmat> SpamapS, as expected
<SpamapS> hazmat: this code sees no  "rel_services" in status...
<hazmat> SpamapS, i think we've established there's a bug in status
<SpamapS>             rel_service = rel_services[0]
<hazmat> ill fix the status bug now
<SpamapS>     # filter out 'self'
<SpamapS>             rel_services = [rsn for rsn in rel_services if rsn.service_name !=
<SpamapS>                             service.service_name]
<hazmat> yeah.. its unfortunate that the status doesn't have any peer tests
<hazmat> SpamapS, that code is from status i assume
<SpamapS> yes, line 220
<SpamapS> removing it fixes it
<SpamapS> But probably will cause bad output on non peer relations
<hazmat> negronjl, did that work?
<negronjl> hazmat:  still spinning up.  one more minute
<hazmat> sounds like its been an exciting week, wish i could have been sprinting with you guys
 * hazmat looks at status
<hazmat> SpamapS, thanks for the pointer
<SpamapS> hazmat: its been a little nuts yeah
<negronjl> hazmat:  just checked it and it works but, status is busted
<hazmat> negronjl, cool, i'm working on status right now, it will be in the review queue in an hr or two, but it probably won't get enough reviews to go in b4 next week
<hazmat> the PWD for hooks to be the formula directory has been in review for a week as well..
<hazmat> negronjl, you could probably bug bcsaller and jimbaker  to do a review and we can get it pushed tomorrow if you need to demo it
<_mup_> Bug #804135 was filed: peer relations should be automatic <Ensemble:New> < https://launchpad.net/bugs/804135 >
<hazmat> i'll see if i can get to that one as well
<hazmat> tonight
<negronjl> hazmat:  I will.  Thanks for the help hazmat
<hazmat> negronjl, np... sorry if i seemed a little short... its just that i saw it working, i didn't understand what the issue was
<negronjl> hazmat:  no worries
<hazmat> negronjl, SpamapS btw if you guys are doing a demo, its a little bit of a cheat, but in the bzr history, there's an auto-deploy rev, that will auto deploy dependent formulas and add required relations automatically when deploying
<hazmat> very magical ;-)
<SpamapS> hazmat: heh.. demo time is in about 7 hours .. I'm just going to demo my LXC branch. ;)
<SpamapS> if I wake up in time
<hazmat> SpamapS, nice
<hazmat> SpamapS, get some sleep.. its late there i assume
<SpamapS> hazmat: hacking on interfacing w/ cobbler. ;)
<SpamapS> and yeah its about time to hit it
<hazmat> SpamapS, via xml-rpc of cobbler?
<SpamapS> hazmat: yep
<SpamapS> lp:~clint-fewbar/ensemble/orchestra-provider
<SpamapS> VERY VERY raw
<hazmat> SpamapS, nice
<_mup_> Bug #804199 was filed: Ensemble needs a way to pin a formula to a specific package version <Ensemble:New> < https://launchpad.net/bugs/804199 >
<_mup_> Bug #804202 was filed: Ensemble needs to enforce configuration sanity <Ensemble:New> < https://launchpad.net/bugs/804202 >
<_mup_> Bug #804203 was filed: Ensemble needs to communicate securely with Zookeeper <Ensemble:New> < https://launchpad.net/bugs/804203 >
<m_3> http://paste.ubuntu.com/636296/
<m_3> negronjl: http://paste.ubuntu.com/636296/
<m_3> negronjl: http://paste.ubuntu.com/636296/
<m_3> sorry... irssi retard
<_mup_> Bug #804226 was filed: Relation Get should show all elements <Ensemble:New> < https://launchpad.net/bugs/804226 >
<_mup_> ensemble/expose-provision-machines r269 committed by jim.baker@canonical.com
<_mup_> Watch service unit assignment on machines
<_mup_> ensemble/expose-provision-machines r270 committed by jim.baker@canonical.com
<_mup_> More guards
<_mup_> ensemble/expose-provision-machines r271 committed by jim.baker@canonical.com
<_mup_> Refactored tests
<hazmat> zookeeper multi-node functionality lands
<_mup_> Bug #804284 was filed: REST API for managing ensemble environments, aka expose cli as ensemble daemon <Ensemble:New> < https://launchpad.net/bugs/804284 >
<_mup_> Bug #804327 was filed: Implement formula include <Ensemble:New> < https://launchpad.net/bugs/804327 >
<_mup_> ensemble/expose-provision-machines r272 committed by jim.baker@canonical.com
<_mup_> Robust testing without sleep on working with machines
<hazmat> argh.. includes
<jimbaker> hazmat, yeah, we need stacks :)
<bcsaller> hazmat:  bare metal deployments changes the timeline on the multi availability zone stuff. I am writing up the conversation I just had, we need to chat about this soon
<hazmat> jimbaker, indeed, noted the same on the bug comment
<jimbaker> bcsaller, so they want this to work in a high-latency env?
<hazmat> bcsaller, not sure i see the relation between the two.. multi-availability zone is basically rack level awarness in physical terms.. a --location param seems to encompass both
<hazmat> jimbaker, az are low latency xc communication
<hazmat> 3-10ms, avg 5ms afaicr
<bcsaller> hazmat: deploy openstack to create more than one Avail Zone
<bcsaller> then use ensemble to manage services in that zone
<bcsaller> we have new expectations at many levels of that deployment story
<hazmat> bcsaller, cool, looking forward to the write up
<bcsaller> hazmat: I am writing up a review of the AMI branch now, keep having to service interrupts though 
<hazmat> bcsaller, no worries, i'm dealing with the same (low throughput, kaleb hanging on my arm literally atm)
<bcsaller> hi kaleb
<_mup_> ensemble/status-w-peers r264 committed by kapil.thangavelu@canonical.com
<_mup_> sattus works with peer relations
<bcsaller> I'll review that today as well ^^
<hazmat> bcsaller, cool, thanks, i had some problems on some of the render tests, but they seem to predate the branch.. not sure, maybe missing i'm missing graphiz locally the error expressed oddly as unable to obtain zk handle
<bcsaller> hazmat: if you have an old stack trace or can reproduce point me at it and I can take a look
<bcsaller> I might not see it otherwise :-/
<hazmat> bcsaller, https://pastebin.canonical.com/49276/
<bcsaller> hazmat: and thats intermittent or not?
<hazmat> bcsaller, constant
<bcsaller> ok, looks like its not getting a handle to ZK in the dummy provider, that might indicate that something in setUp is failing 
<hazmat> bcsaller, i get it on trunk as well
<bcsaller> I can take a look
<bcsaller> doh!
<hazmat> when just running the status tests
<hazmat> bcsaller, hmm.. might be my local setup.. something seems strange
<hazmat> lots of tests fail for me on trunk.. randomly.. i might have a bg zk
<bcsaller> yeah, I don't get that, just ran the branch w/o issue 
<bcsaller> hazmat: thats what it sounds like to me
<hazmat> bcsaller, cool
<kirkland> hazmat: yo!
<kirkland> hazmat: how goes :-)
<hazmat> kirkland, wasup? any good irish explorations ?
<hazmat> kirkland, pretty good.. hanging at the beach w/ my son, watching wimbledon, fixing bugs, but wishing i was in dublin
<kirkland> hazmat: heading to scotland next week for some hiking and whisky tasting
<kirkland> hazmat: nice
<hazmat> kirkland, awesome
<kirkland> hazmat: bcsaller says you've got the bug about ensemble requiring its own images
<kirkland> hazmat: we're blocking a bit on that right now
<hazmat> kirkland, yeah.. i fixed it earlier this week, its in review, ensemble will be able to use any cloud-init enabled ubuntu image
<kirkland> hazmat: as we're developing formulas for cloudfoundry that require a) oneiric and b) much larger instances
<kirkland> hazmat: rock! 
<kirkland> hazmat: can you point me to that bug/merge?
<hazmat> kirkland, alternatively you can use the ./bin/ensemble-make-image script to make a 64 bit image for use with the environments.yaml config 'default-instance-type'
<hazmat> and setting the image id
<hazmat> kirkland, https://bugs.launchpad.net/ensemble/+bug/791501
<_mup_> Bug #791501: Ensemble should use standard amis and the ppa <Ensemble:In Progress by hazmat> < https://launchpad.net/bugs/791501 >
<kirkland> hazmat: sweet, thanks
<hazmat> kirkland, cutting out the use of bzr with the ppa, makes the initial install time about the same as a prebaked image (minus the bootstrap node which needs to install java)
<hazmat> twisted takes a long time to install as well oddly.
<kirkland> hazmat: nice
<hazmat> kirkland, any other blocker bugs for you guys?
<hazmat> kirkland, i'm working on auto adding relations for peers that troubled negronjl yesterday
<kirkland> hazmat: hmm, well, we've filed a handful this week
<hazmat> kirkland, well most of them seemed like wish list items
<kirkland> hazmat: lynxman and negronjl  have been filing them;  i've been marking them "confirmed" and adding additional notes
<kirkland> hazmat: i think our highest is the generic images/bigger instances
<kirkland> hazmat: let me check the others ....
<hazmat> the templating one is a wish list, the parallel ssh exec one is going to need some more thought
<hazmat> security stuff i'm going to be making a big push on for most of july
<kirkland> hazmat: good, security stuff will be essential before we can actually start pushing this to real customers
<hazmat> kirkland, i updated that bug with a link to the security spec that forms the initial design
<kirkland> hazmat: the templating one, yeah, wishlist;  something we really need/want, so that we can define tunables and just put our selections in the config
<kirkland> hazmat: absolutely necessary for formula reuse
<hazmat> kirkland, absolutely.. the last branch for config/tunables is in review
<hazmat> its had a rough time through review, but it should land hopefully next week
<kirkland> hazmat: cool, that's more important to me than the templating itself
<kirkland> hazmat: which we can handle as formula writers
<kirkland> hazmat: but we need help from ensemble on the config side
<hazmat> kirkland, sounds good
<hazmat> kirkland, how so?
<hazmat> beyond offering the configuration options on a service
<kirkland> hazmat: heh, just that -- offering the config options and getting them through to the install hook
<hazmat> kirkland, cool
<hazmat> off to grab some lunch, bbiab
 * hazmat  tries to tap the packaging experience in the room
<_mup_> ensemble/expose-provision-machines r273 committed by jim.baker@canonical.com
<_mup_> Test to verify that machine applies firewall policy after unit has been assigned
#ubuntu-ensemble 2011-07-02
<_mup_> Bug #804892 was filed: Traceback when accessing relation data in relation-broken <Ensemble:New> < https://launchpad.net/bugs/804892 >
#ubuntu-ensemble 2011-07-03
<m_3> 1
