#ubuntu-ensemble 2011-07-11
<kim0> Morning everyone
<TeTeT> hi kim0 
<kim0> TeTeT: Hey o/ Morning 
<koolhead11> hi all
<kim0> hey :)
<koolhead11> hi kim0 TeTeT 
<TeTeT> hi koolhead11 
<hazmat> fwereade, how's the spring location?
<niemeyer> Hey there
<niemeyer> Part of the team is sprinting in Ensemble this week
<niemeyer> So apologies if conversations feel a bit confusing, in advance
<niemeyer> Feel free to join in, though :)
<robbiew> :)
<smoser> RoAkSoAx, so... the preseed file that we were starting with (which i believe is incomplete at this point) is orchestra/./provisioning-server/etc/orchestra/ubuntu-orchestra-client.seed
<RoAkSoAx> smoser: ok. That's not in the archives yet I presume
<RoAkSoAx> so we'll have to build that from source
<RoAkSoAx> smoser: would it be possible for you to do a small howto of the things you were doing so we can work that from here?
<smoser> well, its not exactly what we want anyway
<smoser> right. thats what i'll work on.
<smoser> getting you a running system and doc on how its working
<RoAkSoAx> smoser: cool, I'm setting up my own cobbler /orchestra server here
<smoser> RoAkSoAx, http://paste.ubuntu.com/641956/ is an example of a late_command
<_mup_> ensemble/expose-provision-machines r281 committed by jim.baker@canonical.com
<_mup_> Stubs for directly testing open_close_ports_on_machine
<smoser> that is generated with ./update-latecmd oneiric-amd64-preseed.cfg update-latecmd from lp:~smoser/+junk/cobbler-devenv/
<_mup_> ensemble/expose-provision-machines r282 committed by jim.baker@canonical.com
<_mup_> Merged trunk
<RoAkSoAx> smoser: ok cool
<jcastro> niemeyer: do you have any slides from any ensemble presentations you might have done? I'm making a collection.
<smoser> RoAkSoAx, http://paste.ubuntu.com/641966/ would be how i think preseed might look for us.
<RoAkSoAx> smoser: right, all those variables are provided by cobbler, orchestra, or ensemble?
<smoser> i'm not sure about EXTRA_PACKAGES
<smoser> but the idea is that we will start hte installer from ensemble and pass it ks_args to populate ENSEMBLE_LATE_COMMAND
<RoAkSoAx> smoser: ahh I see, so those are ks_args then
<negronjl> Hi guys:  Regarding ensemble and standard AMIs ( Bug: 791501 ).  I have two questions:  Is it released in the ppa already and, how do I specify the AMI/size ?
<_mup_> Bug #808854 was filed: need a skeleton for a cobbler provider <Ensemble:New> < https://launchpad.net/bugs/808854 >
<jimbaker`> negronjl, as i understand it, the ppa should be built against trunk; bug 791501 is marked released and the code is in there
<_mup_> Bug #791501: Ensemble should use standard amis and the ppa <Ensemble:Fix Released by hazmat> <Principia Ensemble:Triaged> < https://launchpad.net/bugs/791501 >
<negronjl> jimbaker':  thx.  How do I specify AMI/size for ensemble to use ?
<jimbaker`> re the way to specify it: default-instance-type and default-ami
<jimbaker`> (i was on vacation last week, so i haven't had a chance to use it)
<fwereade> hazmat, sorry I missed you, I kinda forgot IRC existed this morning :)
<hazmat> indeed, that's it, ensemble is configured entirely through cloud-init with the new branch, it no longer uses pre-built images
<negronjl> jimbaker':  thx
<hazmat> so any standard ubuntu ami (natty or latter due to cloud-init  fixes)
<hazmat> er. 10.10 should work
<negronjl> hazmat:  Is there a way to dynamically specify the AMI/size when deploying a formula with ensemble?
<hazmat> fwereade, no worries, just looking over the orchestra skeleton branch
<negronjl> hazmat:  ie:  ensemble deploy <formula> <AMI> <size> 
<hazmat> negronjl, not atm
<jimbaker`> hazmat, negronjl - that would be something that the stack concept should be handle
<negronjl> hazmat:  then I would use the default-instance-type and default-ami in the ~/.ensemble/environment.yaml file ?
<jimbaker`> able to handle
<jimbaker`> negronjl, correct
<hazmat> negronjl, yeah.. at the moment's that the only expose way of configuring them
<fwereade> hazmat it's smal, but I bet it's got some sort of problem :p
<negronjl> thx hazmat and jimbaker'
<hazmat> negronjl, tests ;-)
<negronjl> hazmat:  I'll be testing today.  I'll report back
<m_3> hazmat: do we have to add a default-ami line to environment.yaml?
<m_3> hazmat: I've been having problems with the new AMIs in r268
<hazmat> m_3, hmm are you on oneric or natty?
<hazmat> m_3, it should default to the natty image
<hazmat> m_3, what sort of issues are you seeing?
<m_3> hazmat: the ami (ami-06ad526f) is coming up, but any ensemble command stays in "...will wait"
<m_3> hazmat: let me try on natty locally first... then I'll ping you
<hazmat> m_3, i just verified it, i do see the wait problem, the agents on the bootstrap machine are running though.. i'm investigating
<Daviey> o/
<kim0> Daviey: o/
<m_3> hazmat: same behavior for natty and lucid (don't know about oneiric yet)... both default to same ami(^^)
<hazmat> m_3, i'm testing a patch right now
<m_3> hazmat: cool... I'll grab coffee... lemme know if I can help
<_mup_> ensemble/test-using-system-zookeeper r261 committed by gustavo@niemeyer.net
<_mup_> Minor styling changes from review.
<niemeyer> fwereade: Can you please merge this and see if it works for you: lp:~niemeyer/ensemble/test-using-system-zookeeper
<Daviey> hey kim0 
<niemeyer> fwereade: That's just SpamapS' branch with stylistic fixes
<niemeyer> fwereade: If it does I'll commit to trunk
<hazmat> niemeyer, jimbaker could you guys look at this trunk trivial diff, it looks like i missed a shelved commit when merging the sans-ami branch.. https://pastebin.canonical.com/49564/
<jimbaker> hazmat, looking
<hazmat> m_3, that diff is the fix for the issue your seeing
<fwereade> niemeyer, works fine for me
<niemeyer> hazmat: +1
<m_3> hazmat: I'll apply it locally and give it a spin
<_mup_> ensemble/trunk r269 committed by gustavo@niemeyer.net
<_mup_> Merged branch test-using-system-zookeeper from Clint [a=spamaps] [r=hazmat,niemeyer]
<_mup_> This enables tests to be run using the system installed ZooKeeper.
<_mup_> Note: I've applied irrelevant style changes to the original branch.
<jimbaker> hazmat, looks good to me too
<RoAkSoAx> smoser: any luck?
<m_3> hazmat: that worked... thanks!
<niemeyer> jcastro: I don't have any useful slides myself.. the talk from last week was in pt_BR, and the ones before are out-of-date and in a crazy format
<smoser> no. working on it right now.
<jcastro> ok, I'm organizing a bunch now
<smoser> will focus on gettin gthat to you.
<jcastro> PSA: If anyone has ensemble slides send them my way
<niemeyer> jcastro: bcsaller, Nick Barcet, and Davie Duffey had good ones, though
<hazmat> m_3, yeah.. i missed that one, it should have been part of the sans-ami merge. glad its working for you.
<jcastro> I've got Nick's already, I'll check with the other 2, thanks!
<_mup_> ensemble/trunk r270 committed by kapil.thangavelu@canonical.com
<_mup_> [trivial] Fix bootstrap initialization with ensemble ppa installation [r=jimbaker,niemeyer]
<_mup_> This was a missing commit from the sans-ami merge. It allows for any ensemble-admin cli within
<_mup_> the path to be used instead of hardcoding it to /usr/local.
<niemeyer> jcastro: David's was not specific to Ensemble, though
<niemeyer> jcastro: Not sure if it'd be useful
<niemeyer> jcastro: It had an awesome video cut out from a screencast
<RoAkSoAx> smoser: thanks!
<niemeyer> jcastro: Which I found handy to reuse
<jcastro> niemeyer: even raw material will be useful
<kim0> does anyone know if that hadoop formula can be used today, as is
<kim0> thinking about coming up with some cool demo for it
<m_3> kim0: let's coordinate... I'm working on similar demos
<kim0> m_3: sure thing
<kim0> well basically the simple plan for me was for every working formula, I'd blog an article about it and perhaps a screencast. And I'll try to push to the interested communities
<kim0> m_3: ofc there's enough cool demos for everyone :) if you think you'll be doing lots of those, then perhaps let's coordinate with bugs?
<niemeyer> Folks, we'll attempt to run through the normal procedure here in the sprint
<m_3> kim0: we should certainly cover http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/
<kim0> m_3: btw, where do you blog, I'd like to add you to cloud.ubuntu.com
<niemeyer> including branches, merge proposals, reviews, etc
<m_3> kim0: but the single/multi in one tutorial... it'd be great if we could get him to repoint to an ensemble-based demo as "the" getting-started
<niemeyer> as one exception, we'd like to run with one review for the bits which are specific to Orchestra support
<niemeyer> (rather than changing existing logic)
<kim0> m_3: I hope the ensemble formula is gonna make it way much simpler than this :D
<niemeyer> So that we can run a tighter loop here
<niemeyer> E.g. for stuff like this: https://code.launchpad.net/~fwereade/ensemble/cobbler-integration/+merge/67568
<kim0> m_3: probably once we have a dead simple article demo'ing that, we can point him, he'll surely add that
<niemeyer> hazmat, bcsaller, jimbaker: Is that alright?
<m_3> kim0: blog.markmims.com, but it's still under construction atm
<jimbaker> niemeyer, +1
<kim0> m_3: once you have a cloud tag, give me the rss I should add
<m_3> kim0: dude, it'd totally be simpler for that getting started guide
<niemeyer> hazmat, bcsaller, jimbaker: Even then, you should feel free to come over and review the logic, before or after merging
<kim0> m_3: so the formula is ready? do we have like a readme of what needs to be done to get to crunching map/red jobs ?
<niemeyer> We'll be happy to address any points
<m_3> kim0: I want some more interesting ones too though
<kim0> m_3: is it a good idea to create a spreadsheet now to track cool demos we can think of ?
<kim0> I was thinking more like, as each formula is done, I'd figure out a cool demo for it
<kim0> but if you have ideas, let's write'em down
<m_3> kim0: nope... those forms not ready yet... I'll devote some time today to test juan's formulas out and add a readme
<m_3> kim0: spreadsheet yes, definitely
<kim0> m_3: that would be awesome :)
<kim0> m_3: cool then .. creating it
<niemeyer> fwereade: Re. XML-RPC on Twisted: http://twistedmatrix.com/documents/current/web/howto/xmlrpc.html
<hazmat> niemeyer, that sounds good
<niemeyer> Cool, thanks
<Daviey> Is twisted really needed for this?
<kim0> Here's a spreadsheet to track cool demo ideas for formulas that are being written http://j.mp/peTAKG
<kim0> m_3: jcastro ^
<niemeyer> Daviey: It's not needed for anything in theory, and I hope we manage to simplify things quite a bit in the near future
<niemeyer> Daviey: But we do have to use Twisted right now for implementing that
<niemeyer> Daviey: Not using Twisted for XML-RPC blocks every other activity in the system
<Daviey> niemeyer: it's like 4 xmlrpc calls?
<Daviey> blocking for <0.05s is a big deal?
<niemeyer> Daviey: It's a remote call, and it stops every other activity in the system.. if it takes longer for whatever reason, nothing else works.
<niemeyer> Daviey: This is the price for using Twisted.. I very much want to get rid of it, but we can't be in the middle
<niemeyer> We're stepping out for lunch
<Daviey> have fun
<_mup_> ensemble/security-principal-def r269 committed by kapil.thangavelu@canonical.com
<_mup_> security principals for ensemble
<_mup_> ensemble/security-principal-def r270 committed by kapil.thangavelu@canonical.com
<_mup_> principal identity token database.
<_mup_> Bug #808935 was filed: ensemble needs principals and way to store/retrieve principal identity tokens <Ensemble:In Progress by hazmat> < https://launchpad.net/bugs/808935 >
<negronjl> guys:  running ensemble r270 from ppa.  I have modified my environment.yaml by adding default-instance-type: m1.large and default-ami: ami-4eab5127 (oneiric).  I then run ensemble bootstrap and I get: http://pastebin.ubuntu.com/642099/
<negronjl> any help would be appreciated.
<negronjl> a simple search for ami-06ad526f finds it at:  /usr/share/pyshared/ensemble/tests/test_utils.py  in case that makes a difference.
<niemeyer> negronjl: hazmat would be the right person to help you
<negronjl> niemeyer:  thx.
<negronjl> hazmat: ping
<hazmat> negronjl, looking
<niemeyer> hazmat: Danke
<negronjl> hazmat:  thx
<hazmat> negronjl, that paste is to the environments.yaml? the instance launched is not the ami specified?
<negronjl> I'm not following.
<hazmat> negronjl, i'm trying to understand what the problem is
<_mup_> ensemble/expose-provision-machines r283 committed by jim.baker@canonical.com
<_mup_> Explicit direct testing of corner cases in open_close_ports_on_machine
<negronjl> hazmat:  Here is what I did.
<negronjl> hazmat: I am running natty btw.
<negronjl> hazmat:  I modified my environment.yaml file by adding the default-instance-type and default-ami values
<negronjl> hazmat:  after I saved the file, I ran "ensemble bootstrap"
<negronjl> hazmat: ...and got the error I pasted.
<negronjl> hazmat:  that's all I have done so far.
<hazmat> negronjl, ah.. i see i missed that at the bottom of the pastebin, i thought it was just the environment.yaml content
<negronjl> hazmat:  no worries.  I appreciate the help :)
<niemeyer> fwereade, RoAkSoAx: For reference, this is the cobbler/remote.py file which contains the server-side XML-RPC API: http://j.mp/cobbler-remote
<hazmat> negronjl, the setting is 'default-image-id'
<negronjl> hazmat:  ahh.. that may be it....let me change/try again.
<negronjl> hazmat:  heh....look at that...it seems to be working so far.
<negronjl> hazmat:  thx for the help
<hazmat> negronjl, cool
<hazmat> np
<hazmat> negronjl, it looks like we led you astray on irc re the param name, but  fwiw it is documented correctly here https://ensemble.ubuntu.com/docs/provider-configuration-ec2.html
<negronjl> hazmat:  cool.. no worries.  I'm just glad it's working now.  Thanks for the help
<_mup_> Bug #808969 was filed: determine the instance type at deploy <Ensemble:New> < https://launchpad.net/bugs/808969 >
<_mup_> ensemble/expose-provision-machines r284 committed by jim.baker@canonical.com
<_mup_> Comments/docstrings
<_mup_> ensemble/expose-provision-machines r285 committed by jim.baker@canonical.com
<_mup_> Test some corner cases
<_mup_> ensemble/expose-provision-machines r286 committed by jim.baker@canonical.com
<_mup_> Cleanup
<_mup_> ensemble/expose-provision-machines r287 committed by jim.baker@canonical.com
<_mup_> Merged trunk
<niemeyer> hazmat: As a heads up, we're attempting some refactoring work on the EC2 provider
<hazmat> niemeyer, great
<hazmat> niemeyer, based on what clint already did?
<_mup_> ensemble/expose-provision-machines r288 committed by jim.baker@canonical.com
<_mup_> PEP8
<niemeyer> hazmat: The first few branches may not be entirely obvious, because it's just untangling things
<hazmat> SpamapS, not sure if those branches are pushed re refactoring
<hazmat> niemeyer, sounds good
<niemeyer> hazmat: We hope to move some bits to a common place til the end of the day
<niemeyer> hazmat: No, it's not based on Clint's work.. we're hacking the EC2 provider itself to extract logic out of it
<niemeyer> hazmat: Hopefully the bootstrap logic inside it will get very thin
<hazmat> niemeyer, sounds good
<niemeyer> hazmat: SpamapS work will be most useful to us once get past that initial phase, because it has very good hints about how to structure the work with relation to the cobbler interaction
<_mup_> ensemble/expose-provision-machines r289 committed by jim.baker@canonical.com
<_mup_> Properly guard against topology state changes
<jimbaker> niemeyer, sounds good about ec2 refactoring - will be useful for the ec2 firewall mgmt piece i have queued up
<niemeyer> jimbaker: Likely
<_mup_> ensemble/expose-provision-machines r290 committed by jim.baker@canonical.com
<_mup_> Removed debugging statements
<_mup_> Bug #809005 was filed: inconvenient EC2Operation inheritance <Ensemble:New> < https://launchpad.net/bugs/809005 >
<_mup_> Bug #809007 was filed: Modify provisioning agent so that port changes are made against a machine <Ensemble:In Progress> < https://launchpad.net/bugs/809007 >
<niemeyer> hazmat: ping
<_mup_> ensemble/expose-status r265 committed by jim.baker@canonical.com
<_mup_> Merged trunk & resolved conflict
<_mup_> ensemble/expose-status r266 committed by jim.baker@canonical.com
<_mup_> Fix open-ports output in ensemble status to report [] if service is exposed but the service unit has no open ports
<hazmat> niemeyer, pong
<niemeyer> hazmat: Hye
<niemeyer> hazmat: Curious about a test problem here
<hazmat> niemeyer, shoot
<niemeyer> hazmat: I suspect it may be caused by a left-over/unfinished branch, but not sure
<hazmat> niemeyer, what's the failure?
<niemeyer> hazmat: test_watch_user_callback_invocation_delays_node_watch
<niemeyer> hazmat: ensemble/state/tests/test_relation.py
<hazmat> hmm
<niemeyer> hazmat: See the assertions at the end of the test?
<hazmat> niemeyer, seems to work fine on trunk
 * hazmat looks over the test
<niemeyer> hazmat: That "state: connected" string doesn't seem to be anywhere
<niemeyer> hazmat: I suspect you may have a local uncommitted change in your txzookeeper branch
<hazmat> niemeyer, update your txzk
<niemeyer> hazmat: Already did
<hazmat> hmm
<niemeyer> hazmat: let me confirm I don't have paths mixed up
<niemeyer> hazmat: fwereade sees the same issue, FWIW
<hazmat> niemeyer, hmm my python path is pointing to a txzk branch..
<niemeyer> hazmat: Uh oh
<niemeyer> :)
<hazmat> yeah.. i was trying to make sure ensemble trunk ran with the branch before i merged.. investigating
<_mup_> ensemble/trunk r272 committed by jim.baker@canonical.com
<_mup_> merged expose-status [r=niemeyer,hazmat][f=767414]
<_mup_> Modifies "ensemble status" to report whether a service is exposed, and
<_mup_> if so, the open ports for each of its service units, if any.
<niemeyer> hazmat: Well, that's exactly what is happening!  ensemble trunk runs with the branch before you merged. 8-)
<hazmat> niemeyer, what revno do you have for txzk
<hazmat> niemeyer, it works fine for me with rev 42 of trunk
<hazmat> on txzk
<hazmat> hmmm
<niemeyer> hazmat: Well, something funky just happened
<niemeyer> hazmat: I checked it moments ago, on revision 42, and ClientEvent wasn't changed
<jimbaker> hazmat, niemeyer - i saw this problem this morning, it went away with the update to txzkookeeper trunk
<niemeyer> hazmat: I've updated back to 40 to test, and now updated to 42, and it is there
<niemeyer> hazmat: Super weird
<hazmat> niemeyer, trial doesn't clear out stale pyc files, not generally an issue..
<niemeyer> hazmat: Let me check with fwereade's branch before he touches it
<hazmat> k
<niemeyer> hazmat: No game.. he's using the package, so the issue is expected
<niemeyer> hazmat: Just so you don't think I'm crazy, this is from my history:
<niemeyer> ..eeper-bzr/trunk]% bzr update
<niemeyer> Tree is up to date at revision 42 of branch bzr+ssh://bazaar.launchpad.net/~ensemble/txzookeeper/trunk 
<niemeyer> hazmat: I could reproduce the bug after that..
<hazmat> niemeyer, i believe you.. although they have some good rum in miami i hear ;-)
<hazmat> perhaps the universal answer confused the issue
 * hazmat looks for a towel
<jimbaker> i just cleaned up pyc files in trunk txzk and ensemble, our test suite still works fine for me
<niemeyer> hazmat: :-)
<niemeyer> hazmat: So, we need to fix that package.. will look at that
<niemeyer> jimbaker: Sweet, thanks for confirming
<_mup_> Bug #809030 was filed: [ReOpen] Ensemble should use standard amis and the ppa <Ensemble:New> < https://launchpad.net/bugs/809030 >
<niemeyer> hazmat: Did you enter the date by hand?
<hazmat> niemeyer, ? what date
<niemeyer> hazmat: The changelog one in txzookeeper
<hazmat> niemeyer, i did
<_mup_> ensemble/trunk r43 committed by gustavo@niemeyer.net
<_mup_> Add seconds to the changelog date. (/me looks at hazmat)
<hazmat> niemeyer, it looked like the other lines, but it still doesn't like it
<niemeyer> hazmat: "date -R" is what you want
<kirkland`> hazmat: hi, could we get https://bugs.launchpad.net/ensemble/+bug/791501 reopened?  the fix doesn't appear to be working for us ...
<jimbaker> niemeyer, when you have a chance, my branch debug-log-relation-settings-changes has been waiting for a review for 2.5 weeks (i made all changes that hazmat requested)
<_mup_> Bug #791501: Ensemble should use standard amis and the ppa <Ensemble:Fix Released by hazmat> <Principia Ensemble:Triaged> < https://launchpad.net/bugs/791501 >
<kirkland`> hazmat: it seems that LP has recently disabled the ability to reopen bugs in projects :-)
<kirkland`> hazmat: in the mean time, we've filed https://bugs.launchpad.net/ensemble/+bug/809030 as a placeholder
<_mup_> Bug #809030: [ReOpen] Ensemble should use standard amis and the ppa <Ensemble:New> < https://launchpad.net/bugs/809030 >
<niemeyer> jimbaker: Ok.. it will wait for at least another one, sorry about that
<hazmat> kirkland`, k, i'll try it out now
<jimbaker> niemeyer, np. this should be a pretty useful branch for formula developers, so just wanted to get merged in :)
<niemeyer> jimbaker: Yeah, I know.. I still want to check it out for sure, but it's also a low priority feature.. this week we really have to get orchestra cranking
<jimbaker> niemeyer, ok
<hazmat> niemeyer, fwereade re the ec2op refactoring, i feel like i'm missing the point about what value this brings
<hazmat> could one of you elaborate on where you going
<niemeyer> hazmat: Hehe :-)
<niemeyer> hazmat: I anticipated that
<hazmat> niemeyer, all i see is
<hazmat> 56	-            d = self.ec2.describe_instances(instance_id)
<hazmat> 57	+            d = self._provider.ec2.describe_instances(instance_id)
<hazmat> which is a little unclear as far refactoring towards orchestra
<niemeyer> hazmat: We need higher bandwidth
<hazmat> or others
<niemeyer> hazmat: Yeah, I know
<niemeyer> hazmat: Which is why I've briefed you
<hazmat> niemeyer, skype | google+ ?
<niemeyer> hazmat: Google+
<niemeyer> Anyone else want to join?
<kirkland> hazmat: try "what" out now?  changing the state of the bug?
<kirkland> hazmat: or launching ami's?
<hazmat> kirkland, launching amis
<hazmat> kirkland, going to do this meeting, and i'll check back in on it
<kirkland> negronjl: what version of ensemble do you test the ami launching?
<kirkland> hazmat: let me check with negronjl to see which version he tested
<negronjl> kirkland, hazmat:  latest ensemble ppa ( r270 )
<kirkland> hazmat: which rXXX do you want us to test, specifically?
<negronjl> kirkland, hazmat: 0.5+270-0ensemble1~natty1
<kirkland> hazmat: also, could you check that we're configuring it correctly?  it's possible that we have a bad configuration
<hazmat> kirkland, un momento, doing a chat with niemeyer
<kirkland> hazmat: k
<negronjl> kirkland, hazmat:  the config that I am using:  http://pastebin.ubuntu.com/642192/
<niemeyer> hazmat: Oh, btw, before you leave, would you be able to have a quick look at the branch?
<niemeyer> hazmat: Should be pretty much a trivial now that you have the context
<niemeyer> hazmat: Btw, txzookeeper package updated..
<niemeyer> hazmat: It was the missing seconds indeed
<hazmat> niemeyer, sure
<hazmat> re quick look at the branch
<hazmat> niemeyer, done
<hazmat> kirkland, negronjl it's working for me.. using the same config.. all m1.large oneiric instances.. http://ec2-174-129-90-179.compute-1.amazonaws.com/wp-admin/install.php
<hazmat> well same param names and same ami and machine size
<hazmat> kirkland, negronjl what problems are you seeing? your on the latest rev, so that should all be good
<hazmat> niemeyer, thanks re seconds
 * hazmat wanders off to dinner
<negronjl> hazmat:  i am unable to deploy any formulas at all.
<niemeyer> hazmat: no worries
<niemeyer> negronjl: What problem are you seeing there?
<negronjl> niemeyer:  I don't see anything at all.. nothing happens.
<negronjl> ensemble deploy <formula>
<negronjl> then ensemble status
<negronjl> and status is always null
<adam_g> hey guys, stumbling in here late but it i can confirm 0.5+270-0ensemble1~natty1 doesn't deploy
<adam_g> it bootstraps the first node fine
<negronjl> ensemble ssh <machine> hangs as well
<adam_g> but deploying nodes does not use the new userdata
<niemeyer> adam_g: new userdata?
<adam_g> one sec
<adam_g> niemeyer: the userdata that cloud-init uses to install and bootstrap ensemble on the stock ubuntu AMIs
<adam_g> from ensemble boostrap: http://paste.ubuntu.com/642216/
<adam_g> from ensemble deploy: http://paste.ubuntu.com/642217/
<niemeyer> adam_g: Hmm
<hazmat> it shouldn't be doing the bzr stuff there
<adam_g> i believe hazmat committed something last week to produce userdata from the first pastebinit, to allow ensemble to bootstrap stock AMIs
<adam_g> but 'deploy' is still producing it in the way pre-commit
<niemeyer> Ok, understood
<hazmat> adam_g, are you using an existing environment perhaps?
<hazmat> hmm
<niemeyer> hazmat: MACHINE_ID=1.. probably not
<adam_g> hazmat: perhaps i am? http://paste.ubuntu.com/642225/
<hazmat> the "bzr up" line isn't in trunk (rev 270) anymore is why i ask
<adam_g> hazmat: im using the enviornmnet ensemble originally generated, with a new default-image-id 
<adam_g> ill wipe my environment and try again.. strange, though, 'ensemble bootstrap' brought up the first node as expected
<hazmat> adam_g, but you've shutdown and bootstrap since then?
<adam_g> hazmat: yup. bootstrap'd for the first time this week after upgrade
<hazmat> bootstrap on existing environment is a no-op.. the environment has to be shutdown and then bootstrap again to get a fresh install
<hazmat> negronjl, perhaps the same applies for you? i was able to shutdown/bootstrap/deploy formulas/add-relations with trunk (r270)
<hazmat> using the ppas for the ensemble install on the environment nodes
<negronjl> hazmat:  I did:
<negronjl> hazmat:  ensemble shutdown
<negronjl> hazmat:  updated environment.yaml
<negronjl> hazmat:  ensemble bootstrap
<negronjl> hazmat:  ensemble deploy <whatever> 
<negronjl> hazmat: waited a while
<negronjl> hazmat:  ensemble ssh < machine >
<negronjl> hazmant:  nothing
<hazmat> negronjl, if you ec2-describe-instances can you login into the machine directly?
<hazmat> i'm curious if the machine and unit agent are running 
<adam_g> negronjl: can you pastebinit /var/lib/cloud/instance/scripts/runcmd?
<negronjl> hazmat:  bootstrapping now
<adam_g> hazmat: i see the same as juan. i can ssh directly, there is nothing installed or running
<negronjl> hazmat:  waiting for bootstrap to complete
 * hazmat checks the ppa
<negronjl> hazmat:  do you want me to ssh into the bootstrap instance or try to deploy and then check that ?
<hazmat> negronjl, the service unit instance (post deploy)
<negronjl> hazmat: ok.  waiting
<niemeyer> hazmat: Please note txzookeeper was _just_ updated.. if they were using the PPA, it was out of sync with Ensemble trunk
<hazmat> looks like the current ppa rev is 272 (updating)
<negronjl> hazmat:  upgrading ensemble to latest ppa 
<negronjl> hazmat:  the bootstrap process on the previous attempt timed out.
<hazmat> i'm running through the same steps with the latest
<hazmat> negronjl, the bootstrap is waiting on the remote end, so i'm not clear what its timing out on
<hazmat> negronjl, you mean the command just hangs?
<hazmat> that would imply a problem connecting to ec2
<negronjl> hazmat:  ensemble returned a time out error ( took longer than 30 seconds )
<negronjl> hazmat:  bootstrapping now
<negronjl> hazmat:  bootstrapped.  should I deploy something now ?
<hazmat> negronjl, yes
<negronjl> hazmat:  trying to deploy tomcat6.   waiting
<hazmat> negronjl, i've been using the example formulas as a known working base set
<negronjl> hazmat:  that's ok.  If this one doesn't provide definitive results, I can try one of the included ones.
<hazmat> "2011-07-11 18:56:36,260 ERROR SSH forwarding error: @@@@@@@"
<hazmat> strange
<negronjl> hazmat: I now have an instance.  What should I do now ?
<hazmat> negronjl, does it show up in ensemble status?
<negronjl> hazmat:  it does
<negronjl> hazmat:  after the upgrade.  it seems to be working now.
<negronjl> hazmat:  let me do some testing .... hold on
<hazmat> negronjl, k
<hazmat> negronjl, i'm going to grab dinner but i'll be back shortly
<negronjl> hazmat: ok
<negronjl> brb
<negronjl> back
<hazmat> negronjl, how'd it go?
<negronjl> hazmat:  so far, it seems good. but, I am having issues with aws/ec2.  two of my instances are still pending.
<adam_g> after upgrade, cloud-init stuff looks consistent across boostrap + deploy'd nodes.
<adam_g> i too ran into a timed out instance, but it may be ec2 related. it also took an unusually long time to provision the instance
<negronjl> ditto that.  ec2 is having issues
<negronjl> hazmat:  it's all working well for me now.
<hazmat> negronjl, great
<negronjl> hazmat:  thx for the patience and help :)
<hazmat> negronjl, np
<hazmat> adam_g, afaics your bug and patch are against an older version of the codebase
<hazmat> adam_g, is your local ensemble from the ppa?
<hazmat> ensemble doesn't do bzr up anymore.. it either does the bzr checkout, or defaults to installing from ppa on the environment machines. it looks like your local installation might be out of date
<adam_g> hazmat: yeah, that was last week before you committed the sans-AMI stuff
<hazmat> adam_g, ah.. gotcha
<hazmat> adam_g, are you still having issues running with oneric images?
<adam_g> hazmat: nope, that last upgrade ~30mins ago seems to fix the issues i was seeing
<hazmat> adam_g, great
<_mup_> Bug #809070 was filed: ensemble should allow execution of formula specific/defined hooks <Ensemble:New> < https://launchpad.net/bugs/809070 >
#ubuntu-ensemble 2011-07-12
<_mup_> ensemble/expose-ec2-provider r271 committed by jim.baker@canonical.com
<_mup_> Merged trunk
<SpamapS> hazmat: I have the ec2->common stuff ready, will push soon now
<kirkland> hazmat: awesome, thanks for working with negronjl 
<kirkland> hazmat: glad to see we're cranking ;-)
<ixxvil> hey sladen , kim0 
<etneg> sladen: ping
<etneg> kim0: ping
<kim0> hey
<kim0> wonder what time is it for sladen 
<sladen> 13:37
<kim0> :)
<kim0> cool then
<kim0> etneg: how's it going 
<sladen> etneg: are you the same as ixxvil?
<sladen> etneg: (one disappeared, as the other appeared)
<etneg> not too bad
<etneg> oh ye
<etneg> :D
<etneg> did you guys see some rocky guy's logo?
<etneg> that looked like an atom
<etneg> nobody commented on it, just thought i'd give you a headsup
<etneg> http://ubuntuone.com/p/13yI/
<etneg> i haent done any new stuff, kinda got tied up but ill do a few today and upload
<koolhead17> etneg, capital E would be much better :P
<etneg> koolhead17: for which one?
<etneg> the one i just posted?
<etneg> that's not my design, it's a robbie williamson's concept
<koolhead17> etneg, hmm. okey
<etneg> robbie@ubuntu.com
<etneg> but ye leave comments
<etneg> :D
<etneg> more options on the table
<koolhead17> he comes here too i suppose
<etneg> ah ok
<etneg> well either way we have more people working on it,good sign
<koolhead17> etneg, indeed
 * koolhead17 looks at SpamapS 
<etneg> i sitll need someone to start coloring mine:/
<_mup_> Bug #809319 was filed: zookeeper finding should be the machine provider's responsibility <Ensemble:New> < https://launchpad.net/bugs/809319 >
<hazmat> g'morning
<niemeyer> hazmat: Yo!
<niemeyer> hazmat: How did it go yesterday?
<hazmat> niemeyer, pretty good, got some good momentum on the security policy stuff
<hazmat> niemeyer, negronjl and i were able to work through some of the deploy issues, afaics it was just an update issue
<niemeyer> hazmat: Hmm.. how did the issue go away?  It felt like it was reproduced after a clean bootstrap.. was that not the case?
<hazmat> niemeyer, not after updating and doing a clean bootstrap
<hazmat> the issue went away
<niemeyer> hazmat: Ah, sweet
<hazmat> niemeyer, re robbiew's email on weekly releases.. i'm a little wary atm, but i'm also unclear on how often packages get pushed to the distro repo for ubuntu-next (oneiric)
<niemeyer> hazmat: Probably not too often
<niemeyer> hazmat: Weekly integration releases sounds sensible
<hazmat> to me the question would be if we're hitting backwards compatibility issues
<niemeyer> hazmat: We have some time to allow for wild backwards incompatible changes in terms of deployments
<hazmat> niemeyer, what's involved in making that happen, can the daily build recipe be adjusted to a weekly?
<niemeyer> hazmat: We'll soon have to put that in control, though
<niemeyer> hazmat: I suggest we address this on the Austin sprint
<hazmat> niemeyer, sounds good
<niemeyer> hazmat: From then on, it has to handle the situation cleanly
<niemeyer> hazmat: Even if that means saying to admin "You can't use that version" in some tricky situations
<niemeyer> hazmat: We'll need a hand from real Ubuntu Developers there :-)
<koolhead17> hi robbiew 
<robbiew> hi
<niemeyer> fwereade: Can you please have a look at this branch when you have a sec: https://code.launchpad.net/~hazmat/ensemble/security-principal-def/+merge/67618
<fwereade> niemeyer, ofc
<robbiew> hazmat: so with this treadmill desk thing you got setup...will we need to move our sprint from the hotel conference room to the gym? :P
<hazmat> robbiew, i'm game. while the view might be nicer, the logistics are a bit harder to get a good roundtable setup or whiteboard usage. we might as well just sprinting from the pool treading water ;-)
<robbiew> hazmat: hmm...they make waterproof laptops?
<SpamapS> they do
<hazmat> robbiew, toughbooks probably have a model for that
<robbiew> nice
<robbiew> hazmat: on the serious side...about how many calories do you burn a day now...or miles walked
<hazmat> robbiew, about 6 miles a day walked, i take desk breaks where i work/sit at a desk, i've been trying to start the day with a mile run to kick things off.
<robbiew> hazmat: nice
<niemeyer> hazmat: Would you mind to have a look at fwereade's branch which is a follow up on the previous conversation: https://code.launchpad.net/~fwereade/ensemble/zkaw-in-provider/+merge/67703
<hazmat> niemeyer, sure
<fwereade> btw, what's the protocol for merging to trunk?
<fwereade> request reviews from hazmat and niemeyer, wait for both, go?
<fwereade> well, wait for both to *approve*, I guess ;)
<SpamapS> fwereade: btw, welcome. :)
<fwereade> SpamapS, thanks :)
<fwereade> SpamapS, very glad to be here (and especially *here*)
<hazmat> fwereade, basically, the second reviewer is also supposed to set the merge proposal to approved, for work done at the sprint, we're going with one review, to try and keep things a bit more fluid. i typically do a bzr-pipeline when i've got multiple related things in review, so i can keep forward progress on development
<hazmat> for related things
 * SpamapS is going to have to look into pipelines
 * fwereade is looking into the right now :)
<fwereade> hazmat, anyway, I'm good to merge now then?
<hazmat> fwereade, can you add a separate test for the new provider method
<kim0> SpamapS: Howdy Clint. So did you check out Ryan's email from mediawiki. I see this as a good opportunity, and I imagine either you or m_3 should get involved. Any thoughts ?
<hazmat> else it lgtm, +1
<fwereade> hazmat, ofc
<SpamapS> kim0: I nearly spit my turkish coffee out when I read that.. *very* excited about it.
<kim0> SpamapS: wow very good :)
<kim0> so the ball is in our playground 
<kim0> I think we'll need someone to help convert that infrastructure
<SpamapS> kim0: I know some ex-wikia people and have met Ryan personally. I'll reach out to them soon and maybe go up there and play with it a bit with them.
<kim0> SpamapS: Awesome :) so we're in good hands
<SpamapS> Ryan and I think very much alike.
<kim0> good stuff indeed
<fwereade> hazmat, niemeyer: re tests: a lot of the stuff I want to test is currently in the tests for EC2Connect, and (almost certainly) EC2LaunchMachine and EC2Bootstrap
<jcastro> Hi everyone, here's the weekly report so far, anyone have anything to add? http://pad.ubuntu.com/ensemble-report
<fwereade> what's the local consensus on where such things should be?
<jcastro> kim0: I don't know how to describe william's ec2zookeeper aware thing.
<fwereade> jcastro: it's just some preliminary refactoring we think we need before we can do a cobbler provider
<fwereade> jcastro: there have been no actual changes in functionality
<jcastro> ok
<fwereade> hazmat, niemeyer: on one hand, duplication bad: if we end up changing the precise operations performed we need to change tests in several places
<kim0> fwereade: Glad you explained it :) indeed I had not idea what it was doing
<fwereade> kim0: heh, sorry :)
<fwereade> on other hand, redundant verification good: if we didn't have this style of test it would have been much harder to refactor in this specific instance
<fwereade> ...but I'm still reluctant to introduce *more* duplication until someone tells me it's a good idea :p
<kim0> SpamapS: I can't wait for the storage formulas nfs/ceph/...etc This is gonna rock so hard ;)
<hazmat> jcastro, i'm doing some of the initial security implementation work
<hazmat> jcastro, else most of what i've been landing has been following stuff up from last week
<kim0> hazmat: is the config-set work still in review?
<hazmat> kim0, yes.. it hasn't landed yet.
<jimbaker> kim0, SpamapS - any thoughts on what's the planned mapping of ceph to the cloud infrastructure? that sounds quite interesting
<SpamapS> jimbaker: ceph is a ring.. it should work a lot like Cassandra actually.
<SpamapS> jimbaker: the holy grail is to be able to just say 'requires: storage\ninterface: shared-mount' in any program that needs a shared mount point.
<koolhead17> hey all
<SpamapS> jimbaker: so ceph and nfs and gluster are interchanagable
<SpamapS> changeable even
 * SpamapS wishes irssi had spell check.. maybe time for a foray into X-chat again.
<kim0> koolhead17: hey 
<koolhead17> apt-cache showpkg moodle  shows mysql-client as deps but does not install it. Using dbconfig-common for installation throws error saying mysql-client needs to be installed before this process to finish. do i need to file a bug for this?
 * kim0 wishes unity had bell support so he can hear irssi's bells
<koolhead17> i asked the same on u-server
<m_3> SpamapS: do we have an interface list yet?
<kim0> r u falling in love with dbconfig-common
<m_3> SpamapS: I chose "filestore" and "mount"... we should standardize
<SpamapS> m_3: no we *definitely* should standardize though
<m_3> storage and mount/shared-mount are great
<m_3> does mount need 'master' or is that something for 'shared-mount'?
<SpamapS> m_3: the relation name should always be a reference to what function inside the formula it performs..
<m_3> SpamapS: right, but filestore and storage would both work
<SpamapS> Yeah
<SpamapS> m_3: I forget, did I hand off the NFS formula entirely to you or did we agree I should complete it?
<m_3> SpamapS: I do like storage better
 * SpamapS feels his head wanting to turn hard right, starting the spin...
<koolhead17> kim0, well indeed i am
<m_3> SpamapS: ha!  I've been working on it, but was expecting to merge... I can finish if you want
<SpamapS> m_3: Just push up what you've got into a branch.. I wanna see. :)
<m_3> yup, will do... I've been having problems with legal branch names in lp... just using +junk now
<m_3> is 
<m_3> 'master' there for a reason? or is that just thinking about a 'shared-mount' interface?
<koolhead17> SpamapS, so i forget dbconfig-common while writing ensemble formulas :D
<SpamapS> koolhead17: well you can use it!
<SpamapS> koolhead17: since it knows how to configure the service for database access, its not totally useless. Its just *confusing* is all.
<koolhead17> SpamapS, am stuck at phpmyadmin because a dbconfig-common bug which was reported in debian too
<SpamapS> that reminds me
<SpamapS> adam_g: did you hack the mysql formula to work w/ shared db's yet?
<adam_g> SpamapS: yes!
<koolhead17> hey obino 
<adam_g> SpamapS: i was gonna clean it up just a bit and send a proposal.
<adam_g> SpamapS: https://code.launchpad.net/~gandelman-a/ensemble/mysql   ive been using it a bunch. works well
<SpamapS> adam_g: *sweet* .. I think it will be useful for phpmyadmin as well
<adam_g> SpamapS: cool. ill send a merge proposal before lunch hopefully
<SpamapS> adam_g: no rush.. just wanted to see how it was going
<adam_g> one thing ive noticed, tho
<adam_g> maybe this is the only formula that has multiple peers relating to it? but, the list of related peers for mysql unit in 'ensemble status' only lists the last related peer, not a list of all peers related.. if that makes sense
<SpamapS> adam_g: I noticed it too
<SpamapS> adam_g: I think the status output is just masking the actual list of relations
<adam_g> obviously not a huge deal, but it would be convinient to have them listed there as well
<SpamapS> adam_g: I think I saw a bug on that, did you file that?
<adam_g> SpamapS: no, i haven't
<SpamapS> adam_g: status' output is critical actually.. for a number of reasons.
<m_3> adam_g: dude, the shared-db-relations look great
<adam_g> cool thans
<adam_g> ive also got a rabbitmq formula that needs a bug worked out and ready to submit, and a buncha nova stuff
<adam_g> (nova-compute, nova-cloud-controller, glance)
<SpamapS> adam_g: swift?
<adam_g> no swift yet, was gonna try to tackle that this week
<SpamapS> seems swift should be pretty straight forward as a formula.. the consumers just have to list and connect to the nodes right?
<jimbaker> SpamapS, i take it that ceph would simply use the instance storage?
<adam_g> SpamapS: well, yeah. but constructing the swift proxy and all of the storage nodes is a bit less trivial. its almost like building a raid array
<SpamapS> jimbaker: yeah I think thats the simplest way
<SpamapS> adam_g: sounds like I need to re-read the swift architecture page again
<jimbaker> SpamapS, yeah, that does keep it simple. i wonder how that works out costwise in comparison to EBS
<SpamapS> jimbaker: Actually I think I'd recommend we use EBS root instances.
<SpamapS> jimbaker: at least until we have a better storage management story.
<jimbaker> SpamapS, yeah, that's why i was curious :)
<SpamapS> RoAkSoAx: re the orchestra-provider branch, I'm playing with it now.. can you try running './test ensemble.ftests.test_orchestracontrol' ? does that work with the environments.yaml I pastebinned earlier?
<RoAkSoAx> SpamapS: let me check
<RoAkSoAx> SpamapS: ZOOKEEPER_PATH what should I set in the environment variable
<SpamapS> RoAkSoAx: hm shouldn't need that.. 
<SpamapS> RoAkSoAx: do you have zookeeper installed?
<RoAkSoAx> SpamapS: yeah working now
<kim0> SpamapS: m_3 You are invited to add Ensemble sessions to https://wiki.ubuntu.com/UbuntuCloudDays/Timetable
<jcastro> so are we keeping the principia name or not? I'd like to just rename the wiki page to /Formulas
<koolhead17|afk> robbiew, saw the ensemble logo idea of yours. Capital E would be better :P
<robbiew> koolhead17|afk: heh...that's pretty easy to switch...but I thought the lower case looked better
<kim0> reminds of IE though :)
<SpamapS> somebody should maybe grab this https://launchpad.net/formulas
<SpamapS> robbiew: Oh! yeah, the logo is *sweet*. As much as I like the oxygen reference with the number of electrons.. I think it might be too busy ;)
<jcastro> SpamapS: I'll snag that on launchpad
<SpamapS> RoAkSoAx: http://pad.ubuntu.com/orchestra-setup-for-ensemble .. working on it now
<jcastro> lp.net/ensemble-formulas perhaps?
<robbiew> SpamapS: yeah...had that thought too
<robbiew> that's easily reduced though
<robbiew> maybe just three
<robbiew> to hint at the heads in the ubuntu logo
<RoAkSoAx> SpamapS: awesome, thank you!
<robbiew> that's actually how I originally had it
<kim0> m_3: thanks for the session \o/
<smoser> RoAkSoAx, so is the pad doc there working for you?
<RoAkSoAx> smoser: yes
<smoser> cool. i will reproduce here too.
<m_3> kim0: np... lemme know if you'd like another topic (i.e., if we need one for data instead)
<m_3> kim0: I figured maybe the etherpad-lite node app would be cool to see
<jcastro> what's the preferred license of a formula? Ideally whatever the upstream project's license is I would assume?
<m_3> jcastro: maybe more open than that... formulas won't nec be written by the people maintaining the upstream project
<jcastro> m_3: right, so in general we don't really care as long as it's open? Like we don't have an overarching guideline?
<smoser> RoAkSoAx, i'm guessing its just user error
<smoser> but /etc/apache2/conf.d/dav.conf doesnt work for me
<smoser> Syntax error on line 5 of /etc/apache2/conf.d/dav.conf:
<smoser> Invalid command 'Dav', perhaps misspelled or defined by a module not included in the server configuration
<SpamapS> sudo a2enmod dav
<SpamapS> adding
<smoser> ok. so that was fixed.
<smoser> now Unknown DAV provider: filesystem
<SpamapS> hrm
<m_3> a2enmod dav_fs?
<SpamapS> maybe
<m_3> jcastro: we want to promote formula re-use... don't know what else
<SpamapS> jcastro: the formula has to be redistributable under a free license so that we are in the clear and users know they can use it w/o any problems.
<SpamapS> smoser: thats printing when you restart apache?
<smoser> works now.
<smoser> needed dav_fs
<smoser> updated etherpad
<SpamapS> Note that I just updated the pre-seed template to point at the right 'repo' instead of 'pkgs'
<_mup_> ensemble/security-node-policy-def r271 committed by kapil.thangavelu@canonical.com
<_mup_> implement security policy, returns acls for new nodes by path.
<niemeyer> I love robbiew's logo: http://ubuntuone.com/p/148H/
<m_3> niemeyer: +1
<m_3> goes right along with the physics theme... "canonical ensemble"
<m_3> we really do need some options that allow for "grand canonical ensemble" or "micro canonical ensemble"
<SpamapS> Ensemble: Now w/ less electrons. ;)
<SpamapS> niemeyer: +1 from me too.. I think we should adopt it.
 * SpamapS begins work on creating an "Ensemble Man" out of The Simpsons' Radioactive Man
<m_3> like little ladder operators... from the vacuum state to an excited state in two shakes
<m_3> SpamapS: rofl
<m_3> SpamapS: set to the tune of 'triangle man' from theymightbegiants
<SpamapS> haha ensemble man beats cranky-ops man. ensemble wins
<SpamapS> cranky ops man hates ensemble man. they have a fight. ensemble wins. ensemble man.
<m_3> nice
<m_3> ok, we'll have a recording session in Austin
<SpamapS> you bring the accordion
<m_3> tub-bass is more my talent
<_mup_> Bug #809498 was filed: intended providers.ec2 submodule visibility is unclear <Ensemble:New> < https://launchpad.net/bugs/809498 >
<robbiew> apparently my only key contribution to ensemble will be the logo
<robbiew> lol
<SpamapS> robbiew: don't forget you also gave us a theme song
<SpamapS> Or, I guess, its a mission statement more than a song. ;)
<SpamapS> RoAkSoAx: any progress?
<m_3> negronjl: the masters/slaves aren't set by the package, but it evidently doesn't matter... working fine so far
<zul> ensemble queries the uec-images.ubuntu.com by default right? how do we get to use our own images
<_mup_> ensemble/expose-ec2-provider r274 committed by jim.baker@canonical.com
<_mup_> Merged trunk
<SpamapS> zul: right now ensemble only uses ec2
<SpamapS> zul: oh sorry I misunderstood your question
<SpamapS> zul: there's a config parameter in environments.yaml
<SpamapS> zul: where you set the ami
<zul> yeah i tried that i didnt get any further 
<zul> SpamapS: http://pastebin.ubuntu.com/642836/
<RoAkSoAx> SpamapS: just got back from lunch
<RoAkSoAx> working on it now
<zul> SpamapS: and my environment http://pastebin.ubuntu.com/642837/
<SpamapS> RoAkSoAx: I updated the kickstart template again
<SpamapS> zul: that looks more like a problem connecting to the ec2-uri than the ami
<SpamapS> if not, then thats a crappy error message
<SpamapS> zul: give it '-v' to get a full backtrace
<zul> SpamapS: yeah i figured out what is wrong with that error message different ip now trying on the bucket stuff
<SpamapS> as in ensemble -v bootstrap
<zul> now it cant find the s3 bucket
<RoAkSoAx> SpamapS: k
<jcastro> SpamapS: when you have a few minutes like when you take a break or something can we skype or google hangout? I have some dumb questions for ya
<m_3> kim0: that 'getting started with hadoop on ubuntu' blog post simplifies a bunch
<SpamapS> jcastro: still trying to get out the door to eat.. how about in about 1 hour?
<jcastro> SpamapS: sure, whenevs
<serue> SpamapS: go eat!
<serue> forget that, give in and call dominos
<_mup_> ensemble/expose-ec2-provider r275 committed by jim.baker@canonical.com
<_mup_> Refactored _ensure_groups
<RoAkSoAx> SpamapS: zookeeperd should also be installed and running right?
<kim0> m_3: simplifies? it sounded like it needed simplification :)
<_mup_> ensemble/expose-provider-ec2 r276 committed by jim.baker@canonical.com
<_mup_> Fix callback refactoring
<m_3> kim0: yeah, dude... rocks
<kim0> m_3: did you have time to write that readme file you mentioned
<smoser> SpamapS, http://pad.ubuntu.com/orchestra-setup-for-ensemble . 'ensemble-branch' should probably match the orchestra branch, rigth?
<m_3> kim0: just ran through the demo and am writing it up now
<m_3> kim0: still a little ugly... references +junk repos, and lots of steps that won't be there after release
<SpamapS> smoser: yes but thats actually not important as that functionality is currently disabled. :-P
<SpamapS> smoser: the way to achieve the same thing is to put that branch's packages in the repo
<SpamapS> which also has me thinking that there's work to be done in making ensemble deploy itself more cleanly.
 * niemeyer mumbles something about cloud-init
<SpamapS> Right, I think cloud-init may be the right way to do what I'm thinking. I'm more concerned with making sure ensemble can install without access to ppa.launchpad.net or bazaar.launchpad.net.
<SpamapS> I was just thinking that the provisioning agent should be able to encapsulate the desired ensemble version and deliver it to the machines.
<SpamapS> Probably by setting a repo address and serving said repo itself... and then it can just build the .deb or .egg or whatever and serve it up.
<niemeyer> SpamapS: I believe hazmat has already fixed part of that porblem in trunk
<niemeyer> SpamapS: In revision 266, more specifically
<niemeyer> "porblem" is an awesome typo, btw
<SpamapS> Sounds delicious
<SpamapS> niemeyer: right, so the private cloud story is to also setup a bzr server available to your machines. I'm thinking that it should be part of the provisioning agent to just do that automatically.
<niemeyer> SpamapS: I don't think I understand what you mean thre
<niemeyer> SpamapS: The private cloud story should work the same way as the public cloud story
<niemeyer> SpamapS: As far as protocols and procedures go
<niemeyer> SpamapS: One talks to public services, the other doesn't
<SpamapS> niemeyer: it has no access to lp:ensemble or ppa:ensemble/ppa .. just a local mirror of ubuntu.
<niemeyer> SpamapS: Yes, and we won't need that.. trunk is able to deploy Ensemble without bzr branches available
<niemeyer> SpamapS: It needs packages, of course, and we'll need to ensure there is a repository available
<niemeyer> SpamapS: But that's somewhat of a need anyway.  Or at least that's my understanding
<SpamapS> niemeyer: yeah, the story is handled.. I'm just not entirely happy with it at the moment. Still too dependent on ppa/lp ..
<niemeyer> SpamapS: We don't depend on it.. we depend on an archive
<SpamapS> are you guys going to EOD at 17:00 or 18:00 ? I still haven't eaten.. so much to handle after 2 weeks away from home. :p
<niemeyer> SpamapS: It's trivial to implement support for looking at a different archive
<niemeyer> SpamapS: Oh, 18+, please go for it
<SpamapS> ok.. bbiab
<niemeyer> SpamapS: Enjoy
<jcastro> kim0: alright! our first community question, http://askubuntu.com/questions/52840/differentiator-between-ensemble-and-front-runners-puppet-and-chef
<_mup_> ensemble/expose-provider-ec2 r277 committed by jim.baker@canonical.com
<_mup_> Remove default inbound fw rules
<SpamapS> jcastro: wanna chat now?
<m_3> jcastro: first question cuts right to the chase
<SpamapS> jcastro: btw I'm going to answer that question with, hopefully, a short answer
<jcastro> SpamapS: heh
<jcastro> SpamapS: yeah I can chat now
<SpamapS> jcastro: I have yet to try google's video chat.. and with Skype being under the scourge of new ownership... I'm anxious to give it a whirl
<m_3> kim0: here's my first pass at a hadoop getting started post http://blog.markmims.com/ensemble/2011/07/12/hadoop-on-ubuntu
<m_3> feedback from all pls... kim0 lemme know if you want something done differently
<jimbaker>  m_3, nice
<m_3> jimbaker: thanks... trying to walk the balance between short/sweet and too many 'ensemble status' snippets
<jimbaker> m_3, you may also want to show ensemble status using a filter
<jimbaker> m_3, we probably should take an arg for add-unit so you don't have to do that list. i think this has been discussed before
<jimbaker> m_3, lastly - you can do add-relation as soon as the services on each side of the relation have been deployed. no need to wait
<SpamapS> m_3: have you tried the ensemble status svg output?
<m_3> SpamapS: is that the dot one?
<m_3> jimbaker: awesome thanks... that saves a step of waiting/checking
<SpamapS> jcastro: btw, here is the draft docs for how service configs should work https://ensemble.ubuntu.com/docs/drafts/service-config.html
<_mup_> Bug #809599 was filed: ensemble add-unit should support adding more than one unit at a time <Ensemble:New> < https://launchpad.net/bugs/809599 >
 * SpamapS also wants a "provision-machines" command so he can just tell the provision agent to start 10 ahead of time.
<m_3> I also _so_ want a --stack arg to deploy
<m_3> enseble deploy --stack svc1 svc2 svc3
<jimbaker> m_3, the other thing that comes to mind is that ideally the formula for hadoop should leave it in a good state such that it can always have jobs run against it w/o observing ensemble status by a user
<m_3> but I'm so lazy I've got two character aliases for all of this
<m_3> jimbaker: I don't follow
<jimbaker> m_3, i have my own deploy-stack alias ;) - basically you should be able to do everything in that listing all at once, as long as the commands are run sequentially
<jimbaker> m_3, that is, alias deploy-stack='ensemble status && ensemble deploy --repository=examples mysql && ensemble deploy --repository=examples wordpress && ensemble add-relation mysql wordpress && ensemble status'
<m_3> the 'ensemble status' observations are to wait until the service is related before trying to ssh in and use the thing
<jimbaker> m_3, yeah, i do wonder about using ensemble ssh in this way. it's really meant for debugging a system
<m_3> jimbaker: there are lots of times that status reports the service started and the relation up, but there's still another several minutes (literally) of stuff to go before the relation is really up
<jimbaker> m_3, maybe the right way to do this is have some sort of job submission service... not certain
<m_3> dunno
<SpamapS> isn't there a web interface for hadoop?
<m_3> yup, but that's fer sissies
<m_3> yeah we can show that too
<jimbaker> m_3, precisely. but if you have a service that depends on hadoop being in a good state, there's no need to poll ensemble status. that might be the web interface SpamapS mentions, i don't know
<m_3> I was just trying to reproduce the seminal posts on the topic... only changing ensemble
<jimbaker> m_3, definitely it would be nice to have the web interface demonstrated
<_mup_> Bug #809607 was filed: EC2-specific code is mixed in with generic machine-launching code <Ensemble:New for fwereade> < https://launchpad.net/bugs/809607 >
<SpamapS> If ssh is the normal way to use it.. then I think ssh is fine.
<jimbaker> SpamapS, certainly a reasonable point
<SpamapS> But maybe ssh is the normal way its used because it has always been a PITA to get more structured access methods setup
<m_3> yeah, I'd imagine the best way to use it would be from your laptop... letting other tools do the ssh-ing for you
<m_3> BTW, these use cases are crying out for osx ensemble tools... lots of people use ubuntu only in the cloud
<jimbaker> m_3, as far as i know, ensemble works fine on os x
<SpamapS> Yeah I'd be surprised if it didn't.
<jimbaker> m_3, although i haven't tried for a while
<SpamapS> python setup.py
<SpamapS> tho maybe there are 2.7'isms now
<m_3> was just thinking of the friends I would send the post out to
<SpamapS> I think Lion may have 2.7
<m_3> good deal of them develop on mac but use ubuntu for cluster deployments
<jimbaker> definitely depends on 2.6
<SpamapS> jimbaker: Snow Leopard has 2.6
<jimbaker> SpamapS, cool. as you mentioned, it might also be at 2.7. not sure. i should get out my MBP and try out the test suite again
<SpamapS> We actually should setup jenkins tests and have a mac slave running somewhere
<jimbaker> SpamapS, that would be very nice
<SpamapS> Hmm.. I wonder if the test suite will run in a buildd's isolation..
<_mup_> Bug #809634 was filed: Ensemble should allow additional user-defined userdata/cloud-config <Ensemble:New> < https://launchpad.net/bugs/809634 >
#ubuntu-ensemble 2011-07-13
<_mup_> ensemble/expose-provider-ec2 r278 committed by jim.baker@canonical.com
<_mup_> Update mock expectations for launch
<_mup_> ensemble/expose-provider-ec2 r279 committed by jim.baker@canonical.com
<_mup_> Fix EC2MachineLaunchTest mocks
<_mup_> ensemble/expose-provider-ec2 r280 committed by jim.baker@canonical.com
<_mup_> Fix EC2BootstrapTest mocks
<_mup_> ensemble/expose-provider-ec2 r281 committed by jim.baker@canonical.com
<_mup_> Reuse security group listing
<jimbaker> fwereade, glad in reading over it just now that your branch proposal is orthogonal to what i'm working on :)
<jimbaker> looks like a nice change
<fwereade> jimbaker, heh: it's actually a bit less horrifying thaan it looks
<jimbaker> (i'm also touching the launch code, to manage security groups)
<fwereade> ...but then I would say that, I've been breathing it all day ;)
<fwereade> the security group stuff is totally untouched I think
<jimbaker> fwereade, are you in miami?
<fwereade> yeah
<fwereade> it's nice :)
<jimbaker> fwereade, cool. from one beachfront to another, i guess
<fwereade> jimbaker, I actually came from the UK this time
<jimbaker> fwereade, btw, i understand you worked on the ironpython c ext api work
<fwereade> yeah, that was a lot of fun
<fwereade> but babies are the great enemy of open source, as they say ;)
<jimbaker> fwereade, cool. i work on jython. we plan to add that functionality sometime. probably next summer
<fwereade> ah, nice
<fwereade> definitely look me up
<jimbaker> fwereade, oh sure. that's why i'm scheduling it out so far in advance ;) - i have 2 kids myself
<fwereade> the last thing I did was a pile of abstraction intended to make that easier
<fwereade> jimbaker, cool :)
<fwereade> if I try really hard I might be able to remember how some of it works ;)
<jimbaker> maciej gave me a nice rundown on his work last year to help on the pypy impl
<fwereade> I seem to recall they took quite a different approach
<jimbaker> but in general, it seems like cextapi is the right approach
<fwereade> and frankly a better one... binary compatibility is kinda cool but it was never going to be a 100% solution
<jimbaker> fwereade, i think the details will vary. we have a codegen system in place that we can leverage for jython
<fwereade> jimbaker, excellent, we can put our respective codegen systems in a machine together and watch them fight it out
<fwereade> take bets on the outcome :p
<jimbaker> the more interesting question will be how to make cextapi lowcost
<fwereade> I seem to recall that one thing that was especially painful in ironclad was chatty APIs with callbacks that kept having to set up managed contexts again and again
<jimbaker> fwereade, there's a student working on invokedynamic support, it's looking quite promising for overall jython performance. for cextapi, it's about leveraging zerocopy with NIO bytebuffers and figuring out how to get the GIL cost down
<fwereade> awesome
<fwereade> I tried really hard to kill the GIL... no dice ;)
<jimbaker> fwereade, yeah, i can see that as particularly bad. basically chattiness kills inlining
<jimbaker> for the obvious reasons
<fwereade> yep :)
<fwereade> anyway, this has been a great day in which I keep losing track of time... but that then means I really must sleep ;)
<fwereade> nn
<jimbaker> fwereade, sounds good, take care!
<kim0> Morning all
<SpamapS> kim0: howdy
 * SpamapS hasn't quite switched back to PDT yet :-P
<kim0> SpamapS: hi there 
<kim0> up early I guess
<kim0> hehe
<kim0> SpamapS: before the channel gets busy, is there any interesting topic you'd like to present in Cloud Days ?
<SpamapS> kim0: ensemble seems like a good topic. ;)
<kim0> hehe
<kim0> I was thinking of doing a basic getting started with Ensemble, and Mark is doing node+mongo on ensemble
<kim0> You can have the intro session if you want, or you can do something more exciting :)
<kim0> SpamapS: You can add yourself to https://wiki.ubuntu.com/UbuntuCloudDays/Timetable
<SpamapS> Yeah I'm just not sure what non-general topic should be covered.
<kim0> SpamapS: do you know if orchestra is in a state that can be demo'ed somehow ?
<SpamapS> kim0: no
<SpamapS> and I'm hesitant to demo the LXC stuff I did because it may not ever see trunk in its current form.
<kim0> well .. the people wouldn't see the code :)
<kim0> it's just a demo
<kim0> SpamapS: can I sign you up for LXC session, and you can change it later if you think of something else ?
<SpamapS> no like, it won't even work the way I did it
<kim0> ok np, I'm sure you'll find another cool topic :)
<SpamapS> Dunno... seems like bad timing as everything is heavily in flux.
<SpamapS> There are *tons* of formulas now tho.. that is quite interesting.
<kim0> Yeah .. formulas are cool to demo :)
<kirkland> SpamapS: yeah, i'm jetlaggin too
<kirkland> kim0: actually, check with RoAkSoAx;  parts of orchestra are demo-able (even if the ensemble bits are not)
<kim0> kirkland: thanks .. I sure will
<kim0> RoAkSoAx: please ping me when you're up :)
<kim0> wonder if Orchestra can be demo'ed on ec2 !
<kirkland> kim0: certainly not the cobbler/pxe/install bits
<kirkland> kim0: on ec2
<kim0> I know someone who runs virtualbox on ec2
 * kim0 scratches head
<kim0> wasn't there some ajaxterm formula .. I think it'd be cool for such irc events .. cant find it in https://code.launchpad.net/principia
<SpamapS> kirkland: ^^ you were working on byobu classroom weren't you?
<kirkland> SpamapS: yeah, i was;  i'll finish it today, actually
<kirkland> SpamapS: i'm giving a presentation in irc tomorrow
<kirkland> SpamapS: and i'm planning on using that
<kirkland> SpamapS: presentation is on dotdee
<kirkland> SpamapS: but i'm going to point people to the URL/ajaxterm to watch
<kim0> kirkland: I thought your dotdee session is today ? https://wiki.ubuntu.com/UbuntuDeveloperWeek
<kirkland> kim0: so it is
<kirkland> kim0: okay, i'll get right onto that formula :-)
<kim0> hehe :)
 * SpamapS looks at clock.. 4:40am .. to sleep, or not to sleep.. that is the question
<kim0> life is too exciting to sleep!
<koolhead17> SpamapS, don`t sleep :D
<koolhead17> hello kim0 niemeyer and all :)
<niemeyer> koolhead17: Yo!
<niemeyer> koolhead17: How's it going?
<RoAkSoAx> kim0: I'm here
<kim0> koolhead17: hey o/
<kim0> RoAkSoAx: Hey o/ 
<koolhead17> niemeyer, am great. in love with buggy dbconfig-common
<koolhead17> RoAkSoAx, hi :D
<kim0> So we're having Ubuntu cloud days on the 25th-26th
<RoAkSoAx> koolhead17: hi!
<kim0> RoAkSoAx: Dustin was just telling me perhaps some parts of Orchestra might be ready for a demo
<RoAkSoAx> kim0: \o/
<kim0> can you dazzle the world with a session :)
<RoAkSoAx> kim0: heh when?
<koolhead17> RoAkSoAx, you had time to play with koan recently?
<kim0> 25th or 26th :)
<kim0> RoAkSoAx: here's the time tablet https://wiki.ubuntu.com/UbuntuCloudDays/Timetable
<RoAkSoAx> koolhead17: yes
<kim0> Would be awesome to grab a session
<kim0> RoAkSoAx: I wonder if we can demo the pxe and deployment part somehow ?!
<kim0> it's gonna be hard for sure
<kim0> if that doesn't work .. feel free to demo any other part :)
<koolhead17> RoAkSoAx, cool
<kim0> RoAkSoAx: what say you 
<RoAkSoAx> kim0: yeah I guess I could do that
<kim0> Awesome! yipeee
<RoAkSoAx> koolhead17: play with it every day
<kim0> \o/
<kim0> RoAkSoAx: so which part to demo .. or session title ?
<koolhead17> RoAkSoAx, :D i need to learn a bit kvm/xen to do koan 
<RoAkSoAx> kim0: I guess for that matter it would be cobbler rather than exactly orchestra
<kim0> RoAkSoAx: still dazzling :)
<kim0> RoAkSoAx: so please pick a session title
<RoAkSoAx> kim0: ok, will do by the EOF as i'm gonna do a call in a bit
<kim0> sure thing 
<kim0> RoAkSoAx: thanks man .. cheers
<jcastro> kirkland: ccze is that log prettyizer I showed you
<kirkland> jcastro: awesome, thanks
<niemeyer> Aram: Welcome!
<Aram> hi!
<robbiew> welcome Aram
<SpamapS> RoAkSoAx: so did you guys give up on trying to use my branch? ;-)
<fwereade> SpamapS, I fully expect to pull in all the meat from your branch, it's just the structure I've been going at with an axe ;)
<fwereade> SpamapS, the pre-existing structure
<fwereade> bzr diff
<koolhead17> kim0, 
<SpamapS> fwereade: awesome. :)
<kim0> koolhead17: hey
<SpamapS> fwereade: I'm glad you've got the fire in your belly necessary to blow apart that EC2 branch. It was a mess. :-P
<fwereade> SpamapS, I love doing this sort of thing :)
<fwereade> SpamapS, the tests seem really solid though, makes life *much* easier
<fwereade> (and, ofc, we'll find out what assumptions I didn't even know I was making pretty soon ;))
<koolhead17> kim0, do we have members from europe belin/athens/madris/rome
<koolhead17> that way i could meet some of us
<kim0> koolhead17: we? the global ubuntu community ?
<SpamapS> fwereade: indeed.. ensemble has made me a believer in TDD ;)
<kim0> I'm sure hell yeah :)
<SpamapS> koolhead17: where are you based?
<koolhead17> SpamapS, am from India. i am travelling to berlin for desktop summit and after that travelling to these places i mentioned earlier
<koolhead17> i am thinking of having lug meetup in all the cities when i am there so that i can meet local linux folks
<koolhead17> :P
<SpamapS> koolhead17: By chance, most of the core ensemble devs are in the Americas somewhere. But there are tons of European Ubuntu folks. :)
<koolhead17> SpamapS, yeah i know that. i am thinking of mailing the local lug for some meetup. let see if am lucky
<RoAkSoAx> SpamapS: I did, though adding some bootstrap process at the moment
<koolhead17> kim0, will we have ensemble logo in place soon. I would like to get it printed on my t-shirt :D
<kim0> koolhead17: lol, robbie's one seems to be winning :)
<RoAkSoAx> SpamapS: as discussed here with niemeyer , the idea is that ensemble bootstrap starts a new system that will run zookeeper as how it does it in ec2
<koolhead17> i don`t like small e
<koolhead17> it makes me feel internet explorer
<koolhead17> :(
<kim0> IE'ish 
<kim0> yeah
<koolhead17> so i asked robbie for capital E
<koolhead17> :D
<kim0> maybe it should be engraved like enlightenment or something
 * kim0 jumps back to writing some stuff
<koolhead17> well we need to do sumthing to the circle around that e
<koolhead17> :P
<SpamapS> RoAkSoAx: yeah thats fine. We had a debate in Dublin and decided it really didn't matter either way.
<SpamapS> RoAkSoAx: or at least, the end of the discussion was "lets move forward from this because it doesn't matter for a proof of concept"
<RoAkSoAx> SpamapS: yeah smoser updated me on the matter
<SpamapS> RoAkSoAx: by system, I hope you mean a vm using koan. ;)
<RoAkSoAx> SpamapS: yes
<RoAkSoAx> SpamapS: for the PoC i'm working atm i'm using koan
<SpamapS> RoAkSoAx: alright, so .. did you actually get something to boot?
<RoAkSoAx> SpamapS: yeah more or less 
<RoAkSoAx> will send you a difff later for you to see
<RoAkSoAx> SpamapS: but yesterday we were concentrating on the best approach to pass coud-init stuff usoing the preseed
<SpamapS> Yeah I'm wondering how that can be done too
<SpamapS> At this point a late_command that acceps a massive base64 encoded string is probably the best bet unless you found something better?
<RoAkSoAx> SpamapS: we have that already, hold on ;)
<RoAkSoAx> SpamapS: http://paste.ubuntu.com/643355/
<RoAkSoAx> SpamapS: http://paste.ubuntu.com/643356/
<SpamapS> RoAkSoAx: I've had difficulty getting the system <-> actual mac bits working right
<etneg> kim0: uploading a few rough concepts now
<etneg> sladentoo ^^
<RoAkSoAx> SpamapS: what do you mean exactly?
<SpamapS> RoAkSoAx: I think I just figured it out actually
<kim0> etneg: awesome :)
<kim0> sladen: ^
<SpamapS> RoAkSoAx: yeah, I didn't assign the MAC right before. Its working fine.
<RoAkSoAx> SpamapS: ok ;) yeah you have to assign and interface first, and then the MAC address it is a tricky thing that's not being validated
<etneg> launchpad takes awhile to send an email
<etneg> i forgot my pass and wanted to reset, still havent received a confirmation code to reset..
<etneg> k since its taking awhile to reset the password. cant post it in the bug now but here it goes
<etneg> http://i54.tinypic.com/2yn00b6.jpg
<etneg> 1 concept for it
<etneg> http://i55.tinypic.com/sq34fs.jpg
<etneg> ignore the colors pls, that was just to give an idea
<etneg> and the last one http://i51.tinypic.com/28jguvt.jpg
<etneg> ignore the coloring:D
<etneg> comments, critique! if it sucks it sucks, but let me know!
<kim0> etneg: thanks for the great work man :)
<kim0> many options to choose from is always a good thing
<etneg> kim0: sure:P
<etneg> yeah
<etneg> keep options open
<kim0> sladen: ^
<sladen> etneg: do you want me to link those into the bug report, or can you do it yourself directly?
<etneg> but till someone colors it up not sure how it's going to look
<etneg> sladen: i could but i forgot my password to login
<kim0> The oxygen atom concept is still on the top of my list though :)
<etneg> and was trying to reset, launchpad hasnt sent me the confirmation code yet 
<kim0> etneg: check ur spam folder ?
<etneg> empty
<sladen> etneg: btw, you should submit something to the wallpaper contest
<etneg> unless osomeone can color it out, i dont know if i could
<etneg> i could do the concepts but not sure thats what they'd want
<etneg> kim0: the oxygen atom idea is quite digitized though
<etneg> mine are still in pencil
<etneg> :P
<kim0> yeah :)
<etneg> from this http://i52.tinypic.com/t62vzl.jpg
<etneg> to
<sladen> etneg: details of the Ubuntu 11.10 contest at  http://design.canonical.com/2011/07/get-excited-and-make-things-wallpaper-edition/
<etneg> to http://i55.tinypic.com/2bckrn.png
<sladen> etneg: what's your username, I can try asking one of the Launchpad people to look at the mail and see how far it got
<etneg> make s a huge difference in the coloring
<etneg> mellgoth29@gmail.com
<sladen> etneg: Launchpad doesn't seem to think there's anyone with that email address (I've tried subscribing it)
<etneg> Forgotten your password?
<etneg> Weâve just emailed mellgoth29@gmail.com (from noreply@launchpad.net) with instructions on resetting your password.
<etneg> i dont have an account on launchpad, i just subscribed to the ensemble list, that was it and now to the bugs list so i can post it there
<etneg> but i forgot my password so i can post to the lists
<sladen> etneg: ahhh, no Launchpad account, that'll be why I can't subscribe you to the bug report :)
<etneg> i'll  need a launchpad account to post to the bugs report?
<sladen> etneg: yes, cunning spam avoidance mechanism
<sladen> etneg: there are a couple of wish-list bugs about eg. enabling posting by Debian Developers, or GPG-signed emails without having an account first
<sladen> etneg: or for allowing OpenID (my preference for a number of years)
<etneg> ah ok
<etneg> ah ok launchpad created
<adam_g> need suggestions here on a formula im writing. ive got a serivce unit that has many clients relating to it. i need a hook that will fire (for each relation) after a minimum number of peer relations is reached, and again for every new relation added. any ideas? SpamapS m_3 ?
<etneg> if i click show bugs by heat
<etneg> i get an (Error ID: OOPS-2020AT233) 
<sladen> etneg: I've used the example and added another comment to the requests for OpenID:  https://bugs.launchpad.net/launchpad/+bug/210943
<_mup_> Bug #210943: Launchpad only permits user authentication via the Ubuntu single-signon OpenID server <feature> <lp-foundations> <openid> <openidrp> <Launchpad itself:Triaged> < https://launchpad.net/bugs/210943 >
<sladen> etneg: and subscribed you to:  https://bugs.launchpad.net/ubuntu-branding/+bug/807100
<_mup_> Bug #807100: Develop Ensemble logo (ensemble.ubuntu.com) <ubuntu-branding:In Progress> < https://launchpad.net/bugs/807100 >
<etneg> sweet thanks sladen 
<sladen> etneg: would you like to introduce + link to the latest sketches in your own words?
<etneg> na na i just want post it and get some feedback i guess
<etneg> possibly get someone to color it
<etneg> :D
<sladen> etneg: okay, perhaps just "here are three more colour sketches I did, I'd love feedback and anyone who can help with colouring them"
<etneg> ok
<sladen> etneg: well, maybe not colour sketches, if you want help colouring them :)
<sladen> etneg: but this one with the Nemo fishes, I would definately put that foward for the wallpaper contest http://i55.tinypic.com/2bckrn.png
<etneg> ohhh no way
<etneg> someone else is using it
<etneg> i just wanted to give a rough idea on how coloring makes a difference
<etneg> i just did the pencil part and someone else did the coloring part
<sladen> etneg: out of interest, what's using it?
<etneg> noticed you
<etneg> basically i got my fingers in every pie :P
<etneg> but we were done with the wallpaper stuff, all finalised, so figured i'll do some logo stuff and bumped into ensemble:D
<etneg> sladen: thanks for the headsup in the bugs list
<robbiew> kim0 so what's the process with the logo?
<robbiew> are we gathering submissions and then doing a vote?
<robbiew> I don't think we want this dragging on for much longer
<sladen> robbiew: the only basic requirement is that we shop it past marcushaslam (Ubuntu Brand Lead) to ensure that whatever is proposed fits with the brand (and we'll probably get ideas on how to improve it if not)
<robbiew> sladen: uh...I don't think so
<sladen> robbiew: *blink*
<robbiew> this is more project oriented...nothing to do with wallpapers
<etneg> ok done
<sladen> robbiew: correct, this has nothing to do with wallpapers
<sladen> robbiew: and the wallpapers has nothing to do with Ubuntu Branding
<robbiew> neither does Ensemble, really
<sladen> robbiew: but this here Ensemble logo probably has to tie in with the Ubuntu/Canonical brand eco-sphere to some degree (it would be good if you could clarify to what extent---there's a question about just that on the bug report)
<robbiew> <sigh>....this seems overly complicated
<sladen> robbiew: I think the Design Team would love to be useful, but need some metrics about how to be useful :)
<etneg> oh we'll need canonical as part of the logo as well?
<robbiew> they are useful
<etneg> cause all of mine are geared towards ubuntu+ensemble
<sladen> robbiew: right, so the Design Team's long-suffering techie sought to enquire 'A big question I'd like to know is how closely is this going to be tied to the Ubuntu or Canonical brands, or is it intended to be stand-alone? Eg. will it always be "Ubuntu Ensemble", or "Canonical Ensemble", or just "Ensemble".'  ( https://bugs.launchpad.net/ubuntu-branding/+bug/807100/comments/5 )
<_mup_> Bug #807100: Develop Ensemble logo (ensemble.ubuntu.com) <ubuntu-branding:In Progress> < https://launchpad.net/bugs/807100 >
<robbiew> it will be Ensemble
<robbiew> just like Upstart is Upstart
<etneg> if nobody can color mine i guess there's nothing much left to decide or vote
<etneg> its the oxygen atom concept
<robbiew> one point, and I believe niemeyer has made this before, is that ensemble is not just for clouds
<robbiew> it's service orchestration
<robbiew> which can occur on bare-metal machines as well
<sladen> robbiew: https://bugs.launchpad.net/ubuntu-branding/+bug/807100/+addcomment?field.text=For+the+branding+it+will+just+be+Ensemble,+just+like+Upstart+is+upstart
<_mup_> Bug #807100: Develop Ensemble logo (ensemble.ubuntu.com) <ubuntu-branding:In Progress> < https://launchpad.net/bugs/807100 >
<etneg> but cloud is still part of it
<robbiew> indeed...but doesn't define it
<etneg> so where is cloud being defined?
<etneg> or orchestration in any of the logos though
<sladen> robbiew: https://bugs.launchpad.net/ubuntu-branding/+bug/807100/+addcomment?field.comment=For+the+branding+it+will+just+be+Ensemble,+just+like+Upstart+is+upstart  lets try that again
<_mup_> Bug #807100: Develop Ensemble logo (ensemble.ubuntu.com) <ubuntu-branding:In Progress> < https://launchpad.net/bugs/807100 >
<etneg> here's something to think about from an artistic point o view and not from a technical aspect
<etneg> people give two shits about a 2hr lecture on how the logo was made, maybe 10% would care, 
<etneg> rest is if the logo looks good, you're done
<etneg> everyone's happy, and it'll stay in people's head
<etneg> waht you want is people to remember it
<etneg> not about the long explanation on bare metal machines or cloud or orchestration, sure the people designing it could think about all those aspects but the audience really just wants to see a nice good looking logo
<etneg> atleasts in my experience:D
<etneg> atleast
<etneg> we could just have a plain E and then color it just enough to look good, but if it acually is well done, nothing else will matter
<etneg> all the orchestration and cloud and any other theme we thought of would just slowly fit into it simply because people like it
<etneg> like it= like the logo
<robbiew> i guess...I'm staying out this
<robbiew> good luck all!
<etneg> ?
<etneg> ohhh
<etneg> you're the guy who designed the oxygen atom concept
<etneg> heh
<etneg> either way i think your idea is the winner, we got nobody to color mine
<sladen> etneg: yup, meet superman^W^W robbiew, Server Team Lead 
<etneg> oh ok
<etneg> he isn't very happy at the moment lol
<etneg> hi
<etneg> ok i guess i can stop with the concepts then, instead of dragging this like robbiew  said, go with the oxygen idea even kim0 likes it
<robbiew> not unhappy...just busy with some other stuff
<robbiew> seriously...I'm fine with anything chosen
<robbiew> not pushing my idea...I just don't want this to drag on for months
 * etneg nods
<kim0> Glad everyone is happy, and indeed let's finish this off
<SpamapS> RoAkSoAx: how goes the battle? ;)
<etneg> well anyohw nice working with you guys, if you need anything else let me know
<etneg> later sladen , kim0 
<smoser> RoAkSoAx, bug 810044 for the cloud-init race on mutiple interfaces
<_mup_> Bug #810044: cloud-init will have race conditions for cloud-config with multiple network adapters <amd64> <apport-bug> <oneiric> <cloud-init:New> <cloud-init (Ubuntu):New> < https://launchpad.net/bugs/810044 >
<smoser> SpamapS, i'd like your suggestions for upstart magic that would handle "all itnerfaces up" (i think there is something in upstart now for that)
<SpamapS> No I'm supposed to add that.
<SpamapS> the closest thing is 'started networking' because it comes after 'ifup -a'
<SpamapS> err
<SpamapS> stopped networking actually
 * SpamapS curses the task
<SpamapS> smoser: I'm supposed to be adding one for oneiric that happens when all statically configured interfaces are up.
<smoser> you ahve a bug opne for that ?
<smoser> that i cna reference?
<serue_> yeah stopped networking is what i'm using in libvirt
<SpamapS> Its in a blueprint
<SpamapS> smoser: https://blueprints.launchpad.net/ubuntu/+spec/server-o-boot-experience
<RoAkSoAx> SpamapS: just came back from lunch, should have something more solid later today
<RoAkSoAx> smoser: let's see
<SpamapS> RoAkSoAx: cool. I've been playing and I keep running into the "how do we know when to turn off netboot for a system" problem..
<SpamapS> RoAkSoAx: seems like we need to have something that turns that off in the latecmd.
<RoAkSoAx> smoser: so, as you mention, checking if we can access a public ubuntu archive give us the impression that there';s outside world connectivity?
<smoser> well, it would indeed give that impression
<smoser> but that could be useless information
<smoser> especially in the case where it has an internal archive that it was told to use by the preseed.
<RoAkSoAx> SpamapS: in the cobbler "system" you mean?
<RoAkSoAx> smoser: right, but I mean, accessing an always available Ubuntu archive on the internet
<SpamapS> RoAkSoAx: yeah, basically my box gets installed correctly and then reboots and because it is defined as a "system" it starts installing itself again rather than defaulting to local boot
<RoAkSoAx> SpamapS: is that in a VM or in real HW?
<SpamapS> real
<smoser> SpamapS, i agree, you need something there.
<smoser> RoAkSoAx, why would it matter real or virt?
<smoser> the issue is that the system is set to netboot, and cobbler is told to netboot it.
<smoser> cobbler needs to be told to stop netbooting it
<smoser> how is this general problem addressed
<smoser> it is not specific to ensemble , or even ubuntu
<SpamapS> I think there's some snippets for kickstarts that do it
<smoser> how?
<SpamapS>     ## PXE JUST ONCE
<SpamapS>     #if $pxe_just_once in [ "1", "true", "yes", "y" ]
<SpamapS>         #if $breed == 'redhat'
<SpamapS>             #set nopxe = "\nwget \"http://%s/cblr/svc/op/nopxe/system/%s\" -O /dev/null" % (srv, system_name)
<SpamapS> thats from kickstart_done
<smoser> that seems easy enough
<smoser> no?
<RoAkSoAx> SpamapS: yeah that's it with $pxe_just_once
<SpamapS> but that is only in kickstart snippets
<RoAkSoAx> but seems disabled by default
<smoser> well we're providing our own preseed file.
<SpamapS> the variable seems meaningless with pre-seeds w/o another snippet
<smoser> and i can't imagine any case where you would want to install again and again
<RoAkSoAx> I actually haven't run into the problem
<SpamapS> if you have hardware that only PXE's when you ask it to, not by default, this isn't an issue
<smoser> just put into our preseed 
<smoser>   wget \"http://%s/cblr/svc/op/nopxe/system/%s\" -O /dev/null" % (srv, system_name)
<SpamapS> smoser: that needs to be run as part of the latecmd
<smoser> right
<smoser> so let it be
<smoser> by default
<smoser> always
<RoAkSoAx> SpamapS: that's easy to achieve
<smoser> do not put this in ensemble though
<smoser> just put it in the late-command by default
<SpamapS> Right
<smoser> (or early command)
<SpamapS> late
<smoser> meh. 6 in one half a dozen in the other.
<SpamapS> I see some advantages to either
<smoser> yeah
<smoser> so i dont care
<smoser> it does seem strange to me that the option to turn off pxe is not acl'd
<SpamapS> smoser: couldn't you make cloud-init's nocloud seed data "pre-seedable" ..
<smoser> ie, anyone can do that.
<SpamapS> smoser: that way you wouldn't have to use latecmd.. you could just say  cloud-init cloud-init/nonet-base64 string asdflkja1383185a0easdfasdfffa09
<smoser> SpamapS, there is no issue really with utilizing late-command multiple times.
<SpamapS> smoser: its additive?!
<smoser> ie, we could definitely do that in cloud-init though
<smoser> the way we've got it now , it is easily additive
<SpamapS> ah well then i'll stop bike shedding
<smoser> hold on. let me dig it up
<smoser> SpamapS,  see line 105 and 106 at
<smoser> http://bazaar.launchpad.net/~smoser/+junk/cobbler-devenv/view/head:/cobbler-server/late_command.sh
<smoser> just insert a :
<smoser> # wget "http://%s/cblr/svc/op/nopxe/system/%s\" -O /dev/null % (srv, system_name) 
<smoser> before the ensembel line
<SpamapS> Not sure I understand how it is 'string true' .. but I'll take your word for it. :)
<smoser> i dont know how you get 'srv' set in that snippit.
<smoser> but i just tested rendering of:
<smoser> http://paste.ubuntu.com/643453/
<smoser> SpamapS, the 'true' is just part of the string
<smoser> only there so that it is [apparently not very] obvious where you could insert additional commands
<SpamapS> smoser: anything in 'sudo cobbler system dumpvars --name=x' is available
<smoser> yeah, and i dont see any 'server' there. or anything that indicates my cobbler server
<SpamapS> smoser: oh, srv is special
<smoser> do you?
<SpamapS> #set srv = $getVar('http_server','')
<smoser> nice.
<smoser> my cobbler thinks that
<smoser> http_server : 127.0.3.2
<smoser> dont know where it gets that
<SpamapS> haha
<SpamapS> I think thats actually in settings
<SpamapS> server: cobbler
<SpamapS> /etc/cobbler/settings
<_mup_> ensemble/expose-provider-ec2 r282 committed by jim.baker@canonical.com
<_mup_> EC2 port ops
<RoAkSoAx> SpamapS: by any chance do you have a user-data sample file of an ensemble zookeeper machine?
<RoAkSoAx> s/machine/instance
<SpamapS> RoAkSoAx: it just installs zookeeperd
<SpamapS> RoAkSoAx: the rest is some run commands to initialize the instance of zookeeper
<RoAkSoAx> SpamapS: ok
<smoser> http://paste.ubuntu.com/643466/
<smoser> so that works.
<SpamapS> smoser: *nice*
<SpamapS> btw that is a truly glorious latecmd ;)
<smoser> well, the really nice thing is that its so easy to tell whats going on
<smoser> :)
<m_3> RoAkSoAx: user-data from my bootstrap instance is http://paste.ubuntu.com/643463/
<smoser> RoAkSoAx, we should add that "pxe-once" snippet to any preseeds we ship
<RoAkSoAx> m_3: thanks!!
<RoAkSoAx> smoser: yeah cool
<adam_g> hey python people, any idea why this may be failing to install from install hook, but installs fine manually and cleansup fine with 'apt-get -f install' after failure?
<adam_g> http://paste.ubuntu.com/643455/
<smoser> i still think its strange that anyone can just turn off pxe boot for anyone else
<SpamapS> adam_g: there are some transitions going on w/ python.. might be just borken in oneiric
<adam_g> SpamapS: in what way? using the same AMI, manually installing the same packages (even executing the hook manually) works fine
<SpamapS> adam_g: try it with 'apt-get -y install foo | cat'
<SpamapS> adam_g: its possible that there's something broken in their config scripts that depends on the terminal.. though that sounds unlikely
<smoser> DEBIAN_FRONTEND=noninteractive
<SpamapS> yeah thats worth a go as well
<adam_g> ive already set to noninteractive
<adam_g> ill give natty a shot in a min
<RoAkSoAx> SpamapS: http://paste.ubuntu.com/643476/ --> so this is just a test bootstrap procedure
<RoAkSoAx> niemeyer: http://paste.ubuntu.com/643476/
<SpamapS> +        #log.info("Run ensemble-admin --instance_id orchestra-server --admin_id %s" % self._admin_id)
<SpamapS> RoAkSoAx: IIRC, ensemble-admin initialize is run as part of the scripts from common.. I think
<RoAkSoAx> SpamapS: I have yet to fully discover ensemble :) but just wanted to provide a quick demo using what you worked on and the approach discussed here
<RoAkSoAx> SpamapS: of course this needs to be integrated correctly with the refactoring that fwereade is working on
<RoAkSoAx> smoser: http://paste.ubuntu.com/643476/ that's over SpamapS' branch
<RoAkSoAx> smoser: so the only thing missing would be to install by default cloud-init in the target system to run what we are passing through the preseed right?
<smoser> RoAkSoAx, yeah, so i have two thoughts on that.
<smoser> in my preseeds i've had EXTRA_PACKAGES as a ks value
<smoser> we *could* use that, similarly '$ENSEMBLE_PACKAGES'
<smoser> but i think we can also just assume that cloud-init is in the packages list.
<smoser> and i lean towards assuming it is.
<SpamapS> I think for completeness of solution, ensemble bootstrap should actually install this preseed file if cobbler will allow it.
<RoAkSoAx> SpamapS: unless we ship it with orchestra as an ensemble preseed
<SpamapS> That seems like a good fallback. "Orchestra - Now with Ensemble Support" ;)
<smoser> SpamapS, cobbler will not allow it
<smoser> well, you can only change existing kickstarts.
<smoser> i thik...
<niemeyer> This code, in Jim's branch, looks awesome
<smoser> maybe i'm wrong, but there was some limitation there.
<niemeyer> http://paste.ubuntu.com/643484/
<smoser> oh...
<niemeyer> Let's have more of it
<smoser> SpamapS, i think it is not desireable for ensemble to insert the kickstart itself
<RoAkSoAx> i think we should have the kickstart provided
<SpamapS> Ok. That is a simpler solution.
<smoser> this is because in any real world situation, the admin will have a taylored preseed for their systems.
<SpamapS> Eh, but thats just the kind of thing we don't want.
<RoAkSoAx> and when we have system's in foo-available, we set the kickstart to the one for ensemble
<smoser> it is the kind of thing we *do* want
<smoser> and admins want
<SpamapS> Err.. ok.
<smoser> you buy a honking large server with a 3200RPM disk on / and a high end raid attached
<SpamapS> (I'd err on the side of pure machines so we can continue to make assumptions in formulas)
<smoser> you want to tell your preseed to use the high end disk
<smoser> not jsut accept the defaults of the installer
<SpamapS> smoser: I think we can do that w/ snippets
<smoser> do what with snippets?
<SpamapS> Well, s/that/what we need to/
<smoser> well, we already have that basically
<SpamapS> We can include an ensemble snippet, and a default pre-seed that does not disk config..
<smoser> ensemble needs one hook in
<smoser> you put that text in your preseed and you're good
<smoser> we provide a documented example in orchestra that shows what it needs
<SpamapS> Then let people make their own pre-seed that just includes the ensemble snippet.. rather than copy/paste
<smoser> meh
<RoAkSoAx> we can have a bsic preseed and also chainload another preseed or use snippets, it will work either way
<SpamapS> Yeah, I'd keep the magic in the snippets
<smoser> copy and paste a snippet or copy and paste a bit of code
<RoAkSoAx> I think adding a snippet would be easier
<SpamapS> a call to a snippet is going to evolve with releases and updates..
<RoAkSoAx> indeed
<SpamapS> copy/pasted code will not
<smoser> thats possibly true, but a preseed file will *also* evolve with releases
<SpamapS> tho one thing that sucks.. the snippets live in /var/lib don't they?
<SpamapS> yeah.. suck.. that means if we update them in the package they don't get updated in your installation
<smoser> and a snippet doesn't really solve all your problem.
<SpamapS> a unicorn would
<smoser> i think as much as possible we shoudl rely on only late_command
<smoser> and so a snippet for late_command, that allows the user to add their own commands is not terribly prettier
<SpamapS> Thats fair, but I'm concerned with people doing things in a weird way on these servers that will make formulas fail.. until we have container support in the machine agent, thats going to be a real danger.
<smoser> they're going to do things.
<smoser> and there are very good reasons for wanting to modify a preseed.
<SpamapS> yeah, I want the default to allow them to do those things w/o copy/pasting something we need to change later.
<smoser> but i dont think you can.
<smoser> i'm saying keep it to only touching late_command
<smoser> we can do just about anything inside of a late command
<smoser> and we can inject anything we want in there.
<smoser> and a snippet wouldn't help you there, really.
<smoser> since its basically a patch collision on likely modified code
<SpamapS> You can define a python function, with default arguments.. and the default kickstart would be something like
<SpamapS> $SNIPPET('ensemble')
<SpamapS> ## put your custom pre-seed here
<SpamapS> $ensemble_latecmd() # Add an argument if you need to add late commands as well.
<SpamapS> Now the magic is hidden in ensemble_latecmd()
<_mup_> ensemble/expose-provider-ec2 r283 committed by jim.baker@canonical.com
<_mup_> Adjust provider interface for port mgmt to take machine_id
<smoser> SpamapS,  i don't folow
<SpamapS> you can fix the magic, and even put it in a python lib which the snippet imports
<smoser> oh. sure.
<niemeyer> jimbaker: expose-provision-machines has a review
<_mup_> ensemble/expose-provider-ec2 r284 committed by jim.baker@canonical.com
<_mup_> Merged expose-provision-machines
<jimbaker> niemeyer, thanks
<niemeyer> hazmat: That's a good candidate for a review break: https://code.launchpad.net/~jimbaker/ensemble/expose-provision-machines/+merge/67623
<smoser> so, sure. i'm fine saying a snipet.
<smoser> and then on the late_command line using $ensemble_latecmd
<smoser> but you still have cut and paste
<smoser> so really you'd want
<smoser> late_command string $SNIPPET('ensemble_latecommand')
<SpamapS> $SNIPPET() is like an include
<SpamapS> but yeah details details.. I do like having the explicit latecmd line there so its clear that this is where late_command is set.
<smoser> and where the user would want to add their own.
<SpamapS> yeah, that makes it easier not to screw up
<smoser> so all you did is change what they will copy and paste. ;-)
<jimbaker> niemeyer, looks good. re try-finally for observation, this is to ensure the observation happens even if there's an error (eg  ProviderInteractionError)
<niemeyer> jimbaker: If you put at the top there's no possible error that may happen besides from itself
<niemeyer> jimbaker: Is there a reason why it can't be at the top?
<jimbaker> niemeyer, sure, but i also want it to happen last, otherwise the timing gets off
<jimbaker> i should note, this only occurs with repeated looping
<jimbaker> otherwise the ordering is unnecessary
<niemeyer> jimbaker: This feels a bit vague
<niemeyer> jimbaker: How does the timing get off?
<niemeyer> jimbaker: What is depending on that order?
<jimbaker> niemeyer, sorry i'm just recalling the torture :)
<jimbaker> as you may know, i fully support the move to go based on this work ;)
<SpamapS> smoser: yes I want to change what they will copy and paste so that it has no logic in it
<smoser> $ENSEMBLE_COMMAND has no logic
<smoser> only the spelling of the word
<smoser> but this is not worth arguing
<jimbaker> niemeyer, the observation is there to ensure that a specific sequence of activities associated with a particular event (open_close_ports) has completed. otherwise, it is possible for a callback to still be in process, which defeats the observation rationale
<niemeyer> jimbaker: Cool, please just paste that as the answer to that review point then.  Thanks for the explanation.
<jimbaker> niemeyer, sounds good!
<niemeyer> jimbaker: In terms of the migration, it will certainly help us getting rid of those issues, but it's not a free pass to be lax in terms of understanding why/how things work right now.
 * jimbaker doesn't want to write such complex code in twisted again
<jimbaker> niemeyer, indeed
<jimbaker> niemeyer, fortunately i think we have a very good pattern in place with this code. it is robust, if complex. it just needs some doc in place to explain the why behind the necessary complexity
<jimbaker> so i will address that :)
<niemeyer> jimbaker: Kind of.. there are still tricks in there which clearly are made out of explosions noted while testing
<niemeyer> jimbaker: if not self._running, etc
<niemeyer> jimbaker: StateChangeds..
<jimbaker> niemeyer, sure
<niemeyer> jimbaker: Hopefully the final pattern we'll get to will be more resilient ("if this call failed, then 
<niemeyer> ..")
<jimbaker> niemeyer, none of which seem to be necessary when the tests are just run once, only when looped. but justified after the fact
<niemeyer> jimbaker: That's exactly my worry
<jimbaker> also seen in supporting state code, so needing to add guards against topology changes
<niemeyer> jimbaker: What you're really saying is "None of that fails when thing happen on that very precise order"
<niemeyer> jimbaker: Which is artificial
<niemeyer> jimbaker: I know.. your guards are sane
<niemeyer> jimbaker: The problem I see isn't that you've put them in place.. but rather that I'm sure you've figured they were needed because things failed while running
<niemeyer> jimbaker: This means it's still harder to come up with correct logic than it ought to be
<niemeyer> jimbaker: We'll get there, though.. don't worry :)
<jimbaker> niemeyer, yeah, i'm certainly looking forward to golang making it easier to express things deterministically
<_mup_> ensemble/expose-provider-ec2 r285 committed by jim.baker@canonical.com
<_mup_> Now pass in the machine_id to port ops in provisioning agent
<SpamapS> niemeyer: I'm curious why only twisted is used for non-blocking code. I have zero experience w/ python threads.. but.. are they that useless?
<niemeyer> SpamapS: It's a bit like the opposite
<niemeyer> SpamapS: Twisted runs in a main thread
<niemeyer> SpamapS: and it has a dispatching loop as most event based systems do
<niemeyer> SpamapS: Like most Python code, though, Twisted is not thread safe
<SpamapS> That much seems straight forward.. the couple of twisted based things I've written were just a protocol handler hanging off the reactor.
<niemeyer> SpamapS: So it has its own thread, and it orchestrates calls between pending events
<niemeyer> SpamapS: The trickiness comes when you want to execute a blocking call
<SpamapS> Ah.. I recall it was a huge breakthrough when libevent became thread safe.
<SpamapS> memcached got an order of magnitude faster w/o adding much code complexity.
<niemeyer> SpamapS: The issue is that _everything_ happening through Twisted logic (deferreds), happens in that one thread
<niemeyer> SpamapS: So if you call something that blocks within that one thread, it wedges the whole thing
<niemeyer> SpamapS: So the right way to do it is to defer to a thread
<SpamapS> Yeah.. as is the usual case with non-blocking event based systems.
<niemeyer> SpamapS: (which is why I said it was the opposite)
<SpamapS> I read a good white paper on why event based systems always start simpler than threaded systems, but usually end up more complex over time.
<niemeyer> SpamapS: Even more when it's an after-thought
<niemeyer> SpamapS: .. as in, Python itself as a language wasn't designed like that
<SpamapS> http://www.usenix.org/events/hotos03/tech/full_papers/vonbehren/vonbehren_html/
<niemeyer> SpamapS: Oh, that looks interesting, thanks!
<niemeyer> SpamapS: I'm sending to Ubuntu One for the reading queue right now :)
<niemeyer> "is being uploaded to your personal cloud." is sooooo cheesy, btw :-)
<SpamapS> I don't let U1 tell me anything.. I ask it occasionally if it did something. ;)
<adam_g> are there any published formulas that serve as a good example of peer relations vs require/provides?
<SpamapS> adam_g: I think negronjl has some
<SpamapS> adam_g: tomcat6 maybe
<adam_g> cool
<m_3> maybe cassandra?
<SpamapS> Yeah I don't know if the cassandra formula is published just yet
<niemeyer> adam_g: I'm not sure I'm really answering your question there, to be honest
<niemeyer> adam_g: Are you trying to do a ring, or 1-N relationship?
<niemeyer> adam_g: Both are doable.. I just want to make sure we're on the same page
<adam_g> niemeyer: 1-N. the term "ring" is specific to swift in this case i think
<adam_g> what you've described on the list sounds like a good solution. 
<niemeyer> adam_g: Hmm, ok
<niemeyer> adam_g: I wasn't sure because peer relations work within the same service
<niemeyer> adam_g: So when you need two _different_ formulas/services, it's a 1-N relationship
<adam_g> i basically need relation-changed to be fired on all nodes participating in the relationship
<niemeyer> adam_g: Which just works.. you don't have to do anything about it
<adam_g> ok wait.. 
<adam_g> i'll have 1 swift-proxy that relates to many swift-storage-nodes
<adam_g> when a new storage-node joins, i need relation-changed to be fired on all other storage-nodes to get the new ring configuration
<niemeyer> adam_g: Ok.. are all the swift-storage-nodes deployed with the same service, or with different services?
<adam_g> niemeyer: not sure i follow the question. they'll all be using the same swift-storage-node formula, if that means anything
<niemeyer> adam_g: Or, in more simple words, how do you add a new swift-storage-node?
<niemeyer> adam_g: Ok.. so you can have a peer relation
<niemeyer> adam_g: and the list conversation makes sense
<niemeyer> adam_g: Well, one more question:
<niemeyer> adam_g: Do you want a change in _swift-proxy_ to affect all nodes?
<niemeyer> adam_g: Or the nodes themselves to be self-aware?
<niemeyer> (aware of the ring they're in)
<_mup_> ensemble/expose-provider-ec2 r286 committed by jim.baker@canonical.com
<_mup_> Mock test for EC2OpenedPorts
<adam_g> the workflow is basically this. swift-proxy is alone with no ring config. storage-node joins. proxy configures ring with 1 node, sends config to storage-node1. storage-node2 joins. rign is reconfigured with 2 nodes, config is sent to storage-node1 + storage-node2
<niemeyer> adam_g: Ok, that's a NOOP, I think :-)
<niemeyer> adam_g: I mean, it just works
<adam_g> in the context of peer relations?
<niemeyer> adam_g: If you have one formula for swift-proxy, and one formula for swift-storage-node
<adam_g> or in general?
<niemeyer> adam_g: in general
<niemeyer> adam_g: Then you do
<niemeyer> adam_g: add-relation proxy storage
<niemeyer> adam_g: You get a relation established between these
<niemeyer> adam_g: Then, you can do
<niemeyer> adam_g: add-unit storage
<niemeyer> adam_g: add-unit storage
<niemeyer> adam_g: and get a couple of new units (3 in total now)
<niemeyer> adam_g: These are units of the same service (storage) and all of them are in a relation with the proxy
<niemeyer> adam_g: When the proxy does "relation-set whatever=foo"
<niemeyer> adam_g: all the units get the notification
<niemeyer> adam_g: through relation-chagned
<niemeyer> adam_g: Is that what you wanted?
<adam_g> ok. that is exactly what i needed. 
<adam_g> ive not used 'add-unit' yet
<niemeyer> adam_g: Ok.. that's awesome.. it should just work
<adam_g> but i think now would be a good time to start
<niemeyer> adam_g: Oh, you should try it!  I personally find that one of the most awesome things about the model we have
<niemeyer> adam_g: all of the units of a single service share the same relations and the same configuration
<niemeyer> adam_g: Makes it trivial for the user to scale things up at will
<niemeyer> ensemble add-unit <service-name>, profit
<adam_g> ah
 * SpamapS hearts add-unit too
<adam_g> for another formula that has a provides/requires metadata layout, does add-unit 'just work' or it needs to be configured as peer relation?
<m_3> adam_g: lp:principia/haproxy under the reverse-proxy-relation-xxx uses "relation-list" too
<niemeyer> adam_g: add-unit just work for all kinds of relations
<adam_g> great
<adam_g> a second issue
<niemeyer> adam_g: Of course, the service has to be aware of it needs to support multiple units in the same relation
<niemeyer> adam_g: In the near future we'll define max-units
<niemeyer> or similar
<adam_g> instead of using relation-set to pass a KEY=VALUE to the peer, i need to send a gzip archive. :)
<niemeyer> adam_g: Uh oh :)
<niemeyer> adam_g: That's generally not great to be sending that way, but for now it should work to base64 encode it
<niemeyer> adam_g: The detail to be aware of is that this is going to memory
<adam_g> right
<niemeyer> adam_g: I really want us to have a storage mechanism tied to Ensemble itself
<adam_g> i was thinking of dumping it to s3 and passing the url via relation-set, but that'd require credentials 
<niemeyer> adam_g: So that we can do integrated file storage operations like this at will
<niemeyer> adam_g: yeah
<niemeyer> adam_g: Is the file big?
<adam_g> no, ~200k
<niemeyer> adam_g: I'd say base64-encode it, to get going
<adam_g> actually, s3 is a stupid idea. swift provides s3 storage. s3 is only an option when testing in ec2
<niemeyer> adam_g: It's a nice problem to have :)
<SpamapS> it would be useful if ensemble abstracted a shared file storage mechanism away, since it is guaranteed to have one available for formulas.
<niemeyer> adam_g: 200k in memory won't be a big deal right now.. once we smooth out other corners, we can fix that in a more elegant fashion
<niemeyer> SpamapS: Agreed
<SpamapS> adam_g: What about just popping up a webserver on an alternate port and serving it up that way? I did that for mysql replication snapshots.
<SpamapS> when slaves join I feed them a url to fetch the latest snapshot
<adam_g> thats an option as a work around
<SpamapS> niemeyer: https://launchpad.net/~gophers/+archive/go is that the official go pa ?
<niemeyer> SpamapS: This is dying, pretty much
<niemeyer> SpamapS: But it is
<SpamapS> adam_g: It works pretty well really.. I lock it down so only the slaves can grab the snapshot
<SpamapS> niemeyer: its not even working. :(
<niemeyer> SpamapS: We'll be using packages from Debian
<SpamapS> niemeyer: well I want it on natty
<niemeyer> SpamapS: Why not?
<SpamapS> W: Failed to fetch http://ppa.launchpad.net/golang/ppa/ubuntu/dists/natty/main/source/Sources  404  Not Found
<niemeyer> Huh
<SpamapS> oh wait
<SpamapS> thats the other one
<SpamapS> n/m
<niemeyer> I have no idea about what'd be going on there
<adam_g> SpamapS: seems strange to be worrying about a utility webserver from within a formula, tho. maybe it would make sense to run something like that on the bootstrap node, and formulas can publish to / grab from as needed
<niemeyer> Ok
<SpamapS> adam_g: in the case of the mysql server, that would be 100GB files.. :-P
<_mup_> ensemble/expose-provider-ec2 r287 committed by jim.baker@canonical.com
<_mup_> More mocks for EC2 port ops
<SpamapS> adam_g: eventually I want the url to stream the results of mysqldump if its big.
<niemeyer> adam_g: Yeah
<m_3> adam_g: because of your proxy<->storage relation, you might not need real "peer" relations between storage nodes
<_mup_> ensemble/expose-provider-ec2 r288 committed by jim.baker@canonical.com
<_mup_> Change example formula
<m_3> SpamapS: I've got an interface naming problem...
<m_3> SpamapS: mysql provides relation 'db' with interface 'mysql'
<m_3> SpamapS: postgresql provides relation 'db' with interface 'postgresql'
<m_3> but now when something wants to consume these...
<m_3> it needs to do two optional interfaces?
<SpamapS> uh, no?
<SpamapS> the relation name is meaningless
<SpamapS> call it db-mysql and db-postgresql if they can do either/or
<m_3> I'm getting no endpoints on add-relation unless I do something like
<m_3> http://pastebin.com/9jaRyy3g
<m_3> the formula doesn't care which db it is
<m_3> oh, wait, I see what you mean
<m_3> thanks
<SpamapS> I suggest that you *always* explicitly say which relation you mean
<SpamapS> the implicit stuff is cool, but leads to confusion
<m_3> they still both have to be optional though
<m_3> so it's the same difference if it's the same hooks
<m_3> but I'll use the more specific relation names
<SpamapS> optional meaning not "required" ?
<SpamapS> I think the word Requires: is a bit misleading in the metadata.
<SpamapS> many of these are just things that the service *can* use
<m_3> optional: true
<SpamapS> not that it must use.
<SpamapS> is that a valid used metadata flag?
<m_3> seems to be, but I guess it's redundant based on what you just said
<m_3> it's not like it waits for relations before setting the service state to 'started'
<SpamapS> m_3: right but when we start doing a lot of auto-resolution.. it may matter
<m_3> SpamapS: yeah, looks like one of those planned lockdowns according to the docs
<m_3> SpamapS: how long have you been up now?  you were in the logs waaaaay early
#ubuntu-ensemble 2011-07-14
<etneg> kim0 sladen you guys wont be needing anything more on the logos right? oxygen atom idea is confirmed?
<etneg> let me know if it's not confirmed and if you need further more.
<etneg> later
<_mup_> ensemble/expose-provider-ec2 r289 committed by jim.baker@canonical.com
<_mup_> More logging
<_mup_> ensemble/expose-provider-ec2 r290 committed by jim.baker@canonical.com
<_mup_> Some more logging in the provisioning agent
<_mup_> ensemble/expose-provider-ec2 r291 committed by jim.baker@canonical.com
<_mup_> Guard against EC2 errors from txaws
<_mup_> ensemble/expose-provider-ec2 r292 committed by jim.baker@canonical.com
<_mup_> Fixed incorrect CIDR format for filtering IP permissions in security group
<kim0> Morning all
<koolhead17> moning kim0 
<koolhead17> *morning
<kim0> koolhead17: Morning o/
 * SpamapS shakes off sleep and braces for the morning deluge of email
<hazmat> g'morning
<niemeyer> Yo all!
<koolhead17> morning and email
<SpamapS> howdy
 * SpamapS wanders downstairs to make a massive omlette
<kirkland> howdy
<kirkland> having some trouble here with a formula
<kirkland> http://paste.ubuntu.com/644085/
<kirkland> that's the log
<kirkland> the formula install hook is:
<kirkland> http://paste.ubuntu.com/644087/
<kirkland> any ideas?
<kirkland> looks like maybe pipes in the install hook are throwing things off
<SpamapS> 2011-07-14 13:00:32,673: hook.output@DEBUG: hook install ended, exit code 0.
<SpamapS> Kind of looks like it worked fine.
<SpamapS> what is the fail?
 * SpamapS will re-join the discussion in 20 min after his omlette has been devoured
<kirkland> SpamapS: um, there's hook.output@ERROR all over the place
<kirkland> SpamapS: starting with 2011-07-14 13:00:11,320: hook.output@ERROR: dpkg-preconfigure: unable to re-open stdin: 
<koolhead17> SpamapS, massive omlette :)
<koolhead17> he kirkland 
<koolhead17> *hey
<kirkland> koolhead17: howdy
<koolhead17> kim0, find byoubu very heavy :)
<hazmat> hmm
<jamespage> hello
<jamespage> in order to use specific AMI images for an environment am I going to need to do anything other than set the default-ami parameter in the environment configuration?
<hazmat> kirkland, does it work though despite the error output (which looks non fatal to me)
<jamespage> can't seem to make ensemble use anything other than the default natty one ATM
<kirkland> hazmat: no, sorry
<kirkland> hazmat: those error commands didn't actually take effect
<hazmat> jamespage, default-image-id is the magic parameter to be set in environments.yaml per the docs here https://ensemble.ubuntu.com/docs/provider-configuration-ec2.html
<kirkland> hazmat: for instance, we're trying to write to the file /usr/share/byobu/profiles/classroom
<kirkland> hazmat: and that just doesn't happen (that file doesn't exist)
<jamespage> hazmat: ta - thanks for the pointer
<hazmat> kirkland, is there an escaping issue around that escape sequence?, i see the hook output has the file test and the echo to create the file, and then tries to start the session right after that
<kirkland> hazmat: hmm, is that script just executed by bash?
<kirkland> hazmat: or is it processed in some more obscure way?
<kirkland> hazmat: because if you just run it with bash, it works fine
<hazmat> kirkland, its executed by whatever shell is specified with the #! at the top
<hazmat> kirkland, in this case #!/bin/sh
<kirkland> hazmat: it's just when run in ensemble mode that it doesn't work right
<kirkland> hazmat: right, sh, i mean (dash)
<hazmat> kirkland, perhaps changing the install hook  to using #!/bin/bash?
<kirkland> hazmat: hmm, okay, i can try that i suppose
<jamespage> FYI I  requested a sync of zookeeper for oneiric from Debian this morning - this will push the version up to 3.3.3 and will superceded what's currently in the ensemble PPA for oneiric only
<niemeyer> jamespage: Uhmm
<niemeyer> jamespage: I'm not sure the Debian one has the needed patches
<niemeyer> jamespage: Or the needed packaging changes, to be more precise
<niemeyer> (our patches ahve been integrated upstream, IIRC)
<niemeyer> jamespage: Is this going to kill any local changes we have?
<jamespage> niemeyer: hmm - lemme take a look
<niemeyer> jamespage: Regarding the AMI, please note we have as a goal using stock AMIs, and using formulas/etc to do the tweaking
<niemeyer> jamespage: We still lack some features in that area, like formulas being able to specify which release they're targeting, but that's the goal
<niemeyer> kirkland: You can use debug-hooks as well, to debug hooks in general
<jamespage> niemeyer: how are you managing your additional patches in the PPA version of zookeeper - I can't see anything in debian/patches that I don't have in the debian version
<niemeyer> jamespage: Does it have the same subpackages et al?
<niemeyer> jamespage: Maybe it's all good and it makes no difference
<niemeyer> jamespage: I recall we've had to do changes in the package originally
<niemeyer> jamespage: But I suppose these were synced upstream by someone more careful than we are :)
<kirkland> niemeyer: i see the debug hooks are still using tmux :-/
<niemeyer> kirkland: Hmm.. yeah, it will continue using it until someones change?
<niemeyer> someone hanges
<niemeyer> changes!
<niemeyer> kirkland: For the reasons we discussed
<niemeyer> kirkland: As you know, I don't personally care about whether we use screen or tmux
<jamespage> niemeyer: if Â lp:~ensemble/ensemble/zookeeper-package is a good place to look the package structure is the same
<niemeyer> kirkland: But I care about introducing further hacks to use screen
<niemeyer> jamespage: Sounds fantastic.. I bet doko did the syncing work
<jamespage> so I picked up the upgrade to 3.3.3 and fixed a load of issues that mean't it was going to get dropped in Debian
<niemeyer> jamespage: Oh, you mean you uploaded these fixes to Debian, and then synced back?
<jamespage> niemeyer: broadley yes - https://bugs.launchpad.net/ubuntu/+source/zookeeper/+bug/810320 contains the changelog since the last release in Ubuntu
<jamespage> niemeyer: yours truly has now adopted the package in Debian (although SpamapS will be helping me - won't you :-))
<niemeyer> jamespage: Man.. what can I say.. that's why you're called James Page I guess
<kirkland> niemeyer: I gave you a one-liner to use screen, that wraps it with the 'run-one' command, which is in Natty, Oneric and beyond, and generally solves the problem of ensuring one, single, unique flocked copy of a given process running on the system at a time
<jamespage> niemeyer: for some reason (can think why) I thought it might be important for Ubuntu :-)
<niemeyer> kirkland: Yeah, we went over that already.. I'm not introducing further locking in that already messy logic to fix an internal bug of screen
<niemeyer> kirkland: What we have now works well for the purposes we have, and once screen fixes that we can easily switch back
<niemeyer> kirkland: Last we talked you were going to fix screen.. any progress there?
<kirkland> niemeyer: i have not looked at fixing screen, other than reproducing and confirming the limitation with the upstream maintainer in #screen
<kirkland> niemeyer: I suspect that even if I fix that bug in screen, you're going to give me another hoop to jump through;  then another;  then another ;-)
<niemeyer> Jun 17 14:35:49 <kirkland>      niemeyer: would you prefer that I patched screen?
<niemeyer> Jun 17 14:37:17 <niemeyer>      kirkland: Yes, I'd certainly wish that screen implemented this logic by itself in a reliable way
<niemeyer> kirkland: You're being silly..
<kirkland> niemeyer: heh, maybe a little, but I've seen your branch reviews :-P
<niemeyer> kirkland: We've been discussing this for a long time, and now you bring back as if I had said I'd switch back to screen, and that somehow I'm trying to prevent you from using screen.
<niemeyer> kirkland: I'm using tmux because screen has a bug, and tmux works.
<kirkland> niemeyer: imagine if I said "niemeyer: I'm using puppet/chef/cfengine because ensemble has a bug, and puppet/chef/cfengine works."
<niemeyer> kirkland: I'd have no issues with that.. I'd politely disagree, and continue pushing Ensemble forward because I believe in what we are doing.
<kirkland> niemeyer: i believe in what you're doing too; and i'm a strong proponent of ensemble
<niemeyer> kirkland: Doesn't look like so.. you're creating a major issue out of something I specifically attempted to help you with.
<kirkland> niemeyer: it's hardly a major issue
<kirkland> niemeyer: and I specifically gave you a simple workaround, that you rejected
<kirkland> niemeyer: a temporary workaround
<niemeyer> kirkland: You suggested an approach for me to implement, that involves further locking in an already messy area that was full of bugs as you noticed in the review I pointed you to.
<hazmat> fwereade, just to clarify a point on your merge proposal "  Made a start on rearranging tests as well, but IMO that will be an ongoing process (especially over the next day or two) and so shouldn't hold up a merge.
<hazmat> ".. The existing tests are all passing?
<fwereade> hazmat: yes, everything is passing and I'm pretty certain that everything that was originally covered is still covered
<hazmat> fwereade, cool
<fwereade> hazmat: however, there are a number of EC2 tests that are really only hitting the common stuff
<fwereade> hazmat: and I think it would be sensible to gradually transition them
<hazmat> sounds good
<fwereade> hazmat: but I think the appropriate boundaries will become clearer as we get a cobbler bootstrap integrated
<fwereade> hazmat: ofc, don't trust me, run them yourself too :p
<hazmat> always.. but i wanted context for the comments, thanks
<fwereade> hazmat: a pleasure :)
<kirkland> for anyone who was watching the niemeyer/kirkland paintball match, we took it private and hugged it out ;-)
<kirkland> I have a tmux profile for byobu that will land in the archive very soon ;-)
 * kim0 bugs both kirkland and niemeyer :)
<kim0> hugs* even :)
<kim0> \o/ woohoo for byobu on tmux
<kim0> kirkland: is your formula for setting up a shared cloud byobu ready for public consumption .. I'd love to use it for a session today
<kirkland> kim0: damn close, has a bug somewhere in the formula
<kirkland> kim0: the formula is at lp:~kirkland/+junk/byobu-classroom if you want to try it
<kirkland> kim0: something's wrong with the shell script processing
<kim0> kirkland: is there some manual workaround ?
<kirkland> kim0: yeah;  launch and ec2 instance;  and run the install hook as root by hand
<kirkland> kim0: works just fine
<kirkland> kim0: does not work when ensemble calls it
<kim0> got it
<kim0> ok I can do that
<kim0> kirkland: thanks a lot
<kirkland> kim0: i'm trying a small change from /bin/sh to /bin/bash now
 * kim0 nods
<niemeyer> hazmat: Not sure if you're already on it, but I think fwereade's branch is now ready for other reviews: 
<kirkland> hmm, i just updated to the latest ensemble, and i'm not able to get status on my newly bootstrapped environment:
<kirkland> http://paste.ubuntu.com/644173/
<niemeyer> hazmat: lp:~fwereade/ensemble/tweak-launch-machine
<niemeyer> SpamapS: This might be interesting for you as well, and you may have some feedback to William there
<niemeyer> SpamapS: It'll be more clear why we didn't just merge the logic
<fwereade> be brutal, it's the only way I'll learn ;)
<SpamapS> niemeyer: I've been following the branch and I think William's direction, however different from my hackery, is far more maintainable long term.
<SpamapS> I was hacking in the things I needed to get going.. not paying any real debt back. ;)
<niemeyer> SpamapS: He just pushed some changes today, not sure if you've seen those
<niemeyer> SpamapS: Yeah, as we discussed before, your hacks are greatly appreciated for real
<niemeyer> SpamapS: andreas has been hacking on top of them to enable the cloud-init support pretty much the whole time
<SpamapS> w00t
<niemeyer> SpamapS: and we'll be basing the XML-RPC work on them
<niemeyer> SpamapS: Sorry, that was Andres
<niemeyer> or RoAkSoAx for the friends
<SpamapS> While I was doing the XML-RPC stuff I kept wondering if there was already a twisted xmlrpc plugin.
<SpamapS> shallow googling produced nothing
<niemeyer> SpamapS: There is XML-RPC support in twisted, not specifically to cobbler though
<SpamapS> Ah, so I was looking externally where I should have probably just RTFM'd
<SpamapS> if you look at OrchestraControl .. it just uses the python xmlrpclib .. there's nothing specific to cobbler itself.
<SpamapS> actually one thing that is broken in the OrchestraControl is that it only logs in at __init__ time ... needs to re-login whenever the token times out.
<SpamapS> Important since the provisioning agent keeps the provider object around forever.
<niemeyer> SpamapS: Hmm, interesting
<niemeyer> SpamapS: That's a good hint, thanks
<niemeyer> Good mornings
<m_3> niemeyer: o/
<SpamapS> m_3: its funny that you asked yesterday how long I had been up, because it was about 4:00pm and I took a little nap... had been up since 03:30
<niemeyer> m_3: Hey Mark
<kim0> m_3: hey o/
<niemeyer> SpamapS: :)
 * SpamapS is almost back on PDT .. slept till 05:30 today
<m_3> SpamapS: yeah, I saw traffic from horribly early hours
<fwereade> hazmat, SpamapS: any further thoughts on my branch?
<SpamapS> fwereade: I haven't had time to take another look, but I did late yesterday and I was happy to see things shaping up in a more sane way. :)
<fwereade> SpamapS: thanks :)
<fwereade> SpamapS: the really awful bit is gone now :)
<kim0> Getting ambiguous endpoint error â http://paste.ubuntu.com/644210/
<kim0> can someone please help ?
<SpamapS> I for one was happy to find some awful code in Ensemble.. no successful project is ever made up of 100% pure cane sugar. There's always some bitterness buried in the pressure to be timely. ;)
<jimbaker>  kim0, you're going to get that if there is more than one possible way to relate the services
<kim0> jimbaker: I don't think there is
<jimbaker> kim0, use the relation name to further specify
<kim0> jimbaker: this is the very simple mysql from examples with one interface only
<fwereade> SpamapS: what I've seen looks pretty solid to me :)
<kim0> with my drupal formula .. also one interface
<kim0> jimbaker: and I'm demo'ing this within 45mins :)
<jimbaker> kim0, sorry about that
<jimbaker> kim0, one thing that never made it in the codebase was more guidance in the error message
<kim0> jimbaker: I'm mostly sure there's only one relation on both sides
<kim0> I tried specifying it as well
<kim0> still not happy
<SpamapS> fwereade: right, but some of the stuff in the ec2 provider was clearly thrown together in haste. :)
<jimbaker> kim0, on mydb, there are two relation names (db-admin, db) for relation role of server
<fwereade> SpamapS: sure... but I've seen far, far worse ;)
<SpamapS> I have been working on an outline for a book I plan to write .. "Your Code Must Suck - How to succeed in software by writing crappy code."
<fwereade> haha
<jimbaker> kim0, just specify which one you are adding. if you want to add both, you need to do so explicitly
<kim0> jimbaker: hm, I specified 'db' and it worked .. but looking at http://paste.ubuntu.com/644222/ I cannot see this other db-admin relation ?!
<jimbaker> kim0, you need to break it up
<hazmat> fwereade, looking it over, its a lot of churn to digest
<jimbaker> kim0, the error message is definitely unfriendly - we need to fix that
<fwereade> hazmat: yep, sorry about that
<fwereade> hazmat: it's *mostly* moving stuff around
<kim0> jimbaker: can you please point me where to see this db-admin relation ?
<fwereade> hazmat: the significant interface changes are where I fixed the machine/instance confusion
<fwereade> hazmat: ...and that extended its own independent set of tentacles through the codebase :(
<jimbaker> kim0, http://pastebin.ubuntu.com/644225/
<jimbaker> kim0, i just reformatted the specific text from one of the error lines
<kim0> jimbaker: I understand the error says it exists .. however looking at metadata.yaml .. it's not there right ?
<kim0> jimbaker: as in http://paste.ubuntu.com/644222/
<jimbaker> kim0, there's no relation named db-admin as you mentioned in this specific metadata.yaml
<kim0> jimbaker: so this is indeed confusing ?
<jimbaker> kim0, indeed. i think you got a version out of whack here
<kim0> ok .. glad it works now though
<kim0> once done .. I'll see if I can reproduce it
<kim0> jimbaker: thanks a ton :)
<jimbaker> kim0, sounds good about reproducing *later* and good luck with your demo *soon* ;)
<jimbaker> kim0, i will file a bug report on this ambiguous endpoint stuff
<jimbaker> kim0, also in looking at the docs, some of the details on add-relation seemed to have disappeared with respect to ambiguous endpoints and using relation names
<jimbaker> kim0, that was in an earlier version of the docs, we should presumably restore some aspects of it
<jimbaker> kim0, this is the original spec for add-relation/remove-relation, https://code.launchpad.net/~jimbaker/ensemble/rel-mgmt-cli - a quick reading indicates that it's still a correct description of the behavior of these commands
<kim0> I'll read that soonish
<SpamapS> fwereade: BTW, that is fantastic (fixing the machine/instance confusion). I spent probably a whole day screwing that up when playing with lxc.
<fwereade> SpamapS: thanks :D
<_mup_> Bug #810600 was filed: Ambiguous endpoints error in ensemble add-relation or remove-relation is confusing <Ensemble:New> < https://launchpad.net/bugs/810600 >
<jimbaker> SpamapS, fwereade - i should look at that, it's definitely was a pain point in adding support of ec2 firewall mgmt (which almost works, just need to fix a bug in the provisioning agent itself for re-exposing a service)
<fwereade> jimbaker: cool
<fwereade> jimbaker: a lot of it is just renaming variables to make it clear when we have a machine or an instance
<jimbaker> fwereade, i think my favorite example in that chunk of code is MachineProvider vs ProviderMachine...
<jimbaker> fwereade, i suppose that means something. but symmetry like that is generally not good ;)
<SpamapS> negronjl: hey, there's already a memcached formula in principia!
<negronjl> SpamapS:  duh!!!  sorry about that :)
 * SpamapS reads it anyway .. maybe its better. ;)
 * negronjl thinks it isn't :)
<SpamapS> err.. nope. :)
<SpamapS> negronjl: its part of the whole mediawiki demo that I do.
<negronjl> SpamapS:  cool...I'll be using soon :)
<SpamapS> https://code.launchpad.net/~ensemble-composers/principia/oneiric/memcached/trunk
<kirkland> arses
<kirkland> okay, my bad
<kirkland> i forgot to bump the version of the formula
<kirkland> so i've been deploying the same code for 4+ hours now
<kirkland> :-P
<kirkland> \o/  \o/  \o/  \o/  \o/  \o/  \o/  \o/  \o/  \o/  \o/  \o/  \o/  \o/  \o/  \o/  \o/  \o/  \o/  \o/  \o/  
<kirkland> that formula version bumping got me again
<SpamapS> kirkland: its high time we opened that bug...
 * SpamapS does it
<kirkland> jamespage: working byobu-classroom formula at bzr+ssh://bazaar.launchpad.net/~kirkland/%2Bjunk/byobu-classroom/
<kirkland> jamespage: m_3 is about to remove the 443 https dependency
<kirkland> jamespage: rather, i think he's going to put ajaxterm on both 80 and 443
<kirkland> m_3: ^ that's what I'd like to see anyway
<kirkland> m_3: so that the user can choose if they want ssh or telnet :-P
<m_3> sounds good
<kim0> kirkland: thanks for byobu-classroom :)
<kim0> worked great for me
<_mup_> Bug #810648 was filed: The revision number in formula metadata is not very useful to users <Ensemble:New> < https://launchpad.net/bugs/810648 >
<m_3> kim0: yeah, caught first part of the class... awesome man
 * ahs3 ^5 kim0 -- well done
<_mup_> Bug #810649 was filed: Revision number should be optional in metadata <Ensemble:New> < https://launchpad.net/bugs/810649 >
<kim0> ahs3: o/ Thanks :)
<SpamapS> Ok, two bugs filed for that.
<SpamapS> kirkland: might I suggest you review those, comment on your experiences, and +1/Confirm them? Thx. :)
<kim0> Got a good question as I demo'ed ensemble
<kim0> QUESTION: I didn't see a formula for apache in the list. Is that implicit? Can one swap out a different http server?
<kim0> like can I deploy wordpress, on nginx not apache
<SpamapS> kim0: *programs* are handled by packaging. Formulas are for configuring programs.
<SpamapS> kim0: yes, that might be called the 'wordpress-nginx' formula though
<kirkland> SpamapS: sure
<kim0> a separate copy .. cant it be a config option ?!
<SpamapS> kim0: it could be yes
<kim0> cool that's better :)
<SpamapS> but I haven't seen config settings actually available yet... which is a pity. ;)
<kim0> I really can't wait for options to land
<kim0> yeah :)
<SpamapS> distractions.. :-P
<kirkland> SpamapS: commented
<SpamapS> kirkland: danke
<kirkland> SpamapS: marked confirmed
<adam_g> SpamapS: do i send this mysql merge proposal to lp:ensemble or some other?
<SpamapS> adam_g: if you branched from lp:principia/mysql then you  just push to launchpad and do 'bzr lp-propose' .. it will propose the merge against the original source branch.
<adam_g> ok, ill rebranch and do it that way
<SpamapS> adam_g: how did you do it?
<SpamapS> adam_g: you shouldn't have to re-branch, you can target a merge proposal at anything
<adam_g> i have no idea, i branched from lp:ensemble a while back it looks like
<SpamapS> adam_g: but it typically is easier/smoother if you're proposing to merge with the branch you started from.
<SpamapS> wait
<SpamapS> you did this on the examples formula?
<SpamapS> :(
<SpamapS> principia has a much richer mysql formula that includes one-way replication.
<adam_g> ill just rebranch and merge, its not a big deal. the share-db stuff doesn't touch anything else
<SpamapS> Did you branch the principia mysql formula or the examples one though?
<SpamapS> they're totally different
<adam_g> principia, according to branch.conf
<SpamapS> ok good
<SpamapS> bzr info you mean?
<adam_g> yeah, that too
<SpamapS> Ok you don't need to re-branch then
<SpamapS> just push it to somewhere on launchpad and propose it for merging into lp:principia/mysql
<adam_g> 10-4
<kim0> SpamapS: is it possible to use principia's mysql formula, without replication or munin ..etc. i.e. just like the simple examples/ mysql formula ?
<SpamapS> kim0: they're interchangable in the examples yes. The only reason its different is that the example one is trying to be the simple clear example.. whereas the principia one is trying to be highly useful.
<kim0> cool!
<kim0> so when it says require: munin, it's lying :)
<SpamapS> require is a lie yes
<SpamapS> mediawiki works fine w/o memcached
<hazmat> SpamapS, probably should have optional: true re mysq/munin
<SpamapS> hazmat: we talked about this.. required things that are then made optional is.. silly ;)
<SpamapS> There should be a new section, Consumes
<m_3> the docs make it sound like required will change behavior at some point
<niemeyer> SpamapS: That article was a good reading, btw 
<SpamapS> niemeyer: yeah really makes a strong case for exactly what coroutines give you. :)
<SpamapS> light weight simplified thread-like behavior.
<niemeyer> SpamapS: Yeah
<niemeyer> SpamapS: It was also nice to see such arguments about callback-based logic written down
<hazmat> niemeyer, what article was that?
<niemeyer> hazmat: http://www.usenix.org/events/hotos03/tech/full_papers/vonbehren/vonbehren_html/
<niemeyer> bcsaller: ping
<bcsaller> niemeyer: pong
<bcsaller> client locked up there 
<niemeyer> bcsaller: Hey
<bcsaller> hey
<niemeyer> bcsaller: Would you mind to review this branch from Kapil: https://code.launchpad.net/~hazmat/ensemble/security-node-policy-def/+merge/67972
<bcsaller> will do
<niemeyer> bcsaller: Thanks
<robbiew> SpamapS: around?
<niemeyer>    # object has changed due to parent rename, get a new handle
<niemeyer>    sid = server.get_system_handle("system1", token)
<niemeyer> fwereade, RoAkSoAx: ^^^
<niemeyer> Persistent object ids are such a novel idea :-(
<fwereade> niemeyer: gaaaaah
<SpamapS> robbiew: yeah, sup?
<hazmat> kirkland, was that install hook problem just a shell issue (dash v. bash)? and did your environment status work?
<negronjl> SpamapS:  ping
<SpamapS> negronjl: pong, gotta run in 2 minutes, sup?
<negronjl> SpamapS: issues deploying mysql formula: http://pastebin.ubuntu.com/644396/
<negronjl> SpamapS:  whenever you get a chance 
<SpamapS> negronjl: thats quite strange... maybe a regression of some kind?
<SpamapS> I haven't deployed it in a while
<SpamapS> anyway, I'm late
<negronjl> SpamapS:  later 
<_mup_> Bug #810765 was filed: cannot deploy mysql <Ensemble:New> < https://launchpad.net/bugs/810765 >
<hazmat> negronjl, that looks like an invalid metadata.yaml file that doesn't conform to yaml spec
<negronjl> hazmat:  Did anything change on the yaml modules within ensemble?  I don't think the mysql formula has been changed in a while.  I'm currently playing with it nonetheless.
<hazmat> negronjl, no.. i kinda of doubt its an ensemble issue given the traceback
<negronjl> hazmat:  I'm checking right now.  I'll keep you posted on my findings.
<negronjl> hazmat:  this is happening on all formulas.  It does appear to be an Ensemble issue
<hazmat> ugh..
<negronjl> hazmat:  I just updated to the latest and greatest ensemble too....
 * hazmat updates as well
<kirkland> hazmat: well, the spurious @ERRORS was because I had set -x in my script
<kirkland> hazmat: (regardless of dash/bash)
<kirkland> hazmat: set -x prints each command that gets executed on stderr
<kirkland> hazmat: ensemble was slurping those up and labeling them "@ERRORs"
<kirkland> m_3 helped me figure that out
<hazmat> kirkland, but what was the issue with the script not putting the echo'd config file on disk?
<negronjl> hazmat:  Where you able to reproduce the issue?
<hazmat> negronjl, not using the examples.. i'm trying with principia atm
<hazmat> negronjl, also works with principia
<hazmat> negronjl, i've got python-yaml_3.09-5ubuntu2 installed
<negronjl> hazmat:  me too
<negronjl> hazmat:  a bit of background.  I was working on a formulat and it was working just fine ( i was doing some last minute checks on it ) when I noticed that ensemble had an upgrade.  I shutdown the environment, upgraded it and bootstrapped it.  Now I cannot deploy any of the formulas that were working 10 minutes befoe the update 
<hazmat> negronjl, the issue is strange, from the traceback you posted, it looks like python-yaml is saying the metadata.yaml file is invalid... 
<hazmat> negronjl, can you download this paste http://pastebin.ubuntu.com/644418/ and run python downloaded.py abs_path_to_mysql_metadata.yaml
<negronjl> hazmat ( or anyone else interested ).  Can you reproduce ?
<hazmat> negronjl, i'm not able to reproduce
<hazmat> if parses it should just print it stdout, if it fails, the metadata.yaml file is indeed invalid
<negronjl> hazmat:  http://pastebin.ubuntu.com/644420/
<hazmat> hmm
<hazmat> negronjl, your using a ppa ensemble installation?
<negronjl> hazmat:  i am
<_mup_> ensemble/expose-provision-machines r291 committed by jim.baker@canonical.com
<_mup_> Properly supporting re-exposing a service
<negronjl> hazmat:  I opened a bug (https://bugs.launchpad.net/ensemble/+bug/810765)
<_mup_> Bug #810765: cannot deploy any formulas <Ensemble:New> < https://launchpad.net/bugs/810765 >
<negronjl> hazmat:  I'll be back in a minute.
<adam_g> ive got an install hook that succeeds and exits 0, but ensemble.executor never logs it as complete and does not then fire the start hook.
<adam_g> if i destroy the service and re-deploy it to the same machine, it re-installs and fires 'start'
<m_3> negronjl: I ran into a problem with one formula in the repository directory screwing up others... you might check metadata of all or clean up your repo
<adam_g> if i destroy the service, remove the packages from the system, and redeploy, ensemble gets stuck in 'install' again.. any ideas?
<hazmat> adam_g, when you say its stuck in install.. what does ensemble status show?
<hazmat> m_3, good point.. any broken formula metadata in the repo can hose the deploy
 * hazmat updates the bug
<adam_g> hazmat: null
<m_3> I had an accidentally named config.yaml file in one recipe... it took a _while_ to figure that out
<m_3> adam_g: crickets all the way man
<m_3> s/recipe/formula/g
<adam_g> hazmat: "hook.executor@DEBUG: Hook complete: ...." is never logged the first time around
<adam_g> (For install)
<hazmat> adam_g, if you log into the machine the first time a round, is it still running?
<hazmat> ie. are we sure the hook has exited?
<adam_g> yes, positive
<adam_g> and sure that has exitted successfully (logging in the script to make sure)
<adam_g> lp:~gandelman-a/+junk/rabbitmq if you'd like to test, its 100% reproducable
<hazmat> adam_g, can you file a bug.. i'm about to head out for dinner, but i'll have a look latter
<adam_g> definitely
<niemeyer> m_3, negronjl: In the near future "ensemble publish" will catch those issues
<m_3> niemeyer: haven't seen that one...
<niemeyer> m_3: It doesn't exist yet.. it's part of the repo work
<niemeyer> m_3: We'll validate formulas before publishing them
<m_3> awesome
<jimbaker> niemeyer, i found a bug in expose-provision-machines with respect to re-exposing a service. the solution is simple: do the firewall checks per service unit with open_close_ports regardless of whether the service is exposed
<jimbaker> niemeyer, however to fix this, i needed to do the guard against the connection being closed in _watch_topology, much like other watches
<jimbaker> niemeyer, should i put that 2-line change in a separate branch and test it? my only concern is that the one example i have seen that tests connections is in ensemble.state.tests.test_agent.AgentDomainTest.test_connect_agent, which is not reliable when looped
<niemeyer> jimbaker: Hmm
<jimbaker> niemeyer, with these changes, and a downstream branch, we should have port exposing working as desired on ec2. the only bit left is 1. change the example formula (that's also 2 lines, including comment); 2. support exposed/unexposed hooks
<niemeyer> jimbaker: Not sure I understand
<niemeyer> jimbaker: You found a bug in expose-provision-machines, and that also needs a guard
<jimbaker> #2 is a bit more work, but it also seems more of a specialized usage (eg, maybe a formula would respond to exposing a service by starting a web-based admin)
<niemeyer> jimbaker: Sounds like we need a branch which addresses these two issues?
<jimbaker> niemeyer, sounds like a good plan
<jimbaker> niemeyer, i will create a branch to address
<niemeyer> jimbaker: Cool, thanks!
<niemeyer> Alright, I'm stepping out to the room.. see you all later
#ubuntu-ensemble 2011-07-15
<_mup_> Bug #810808 was filed: Formula install hook never completes <Ensemble:New> < https://launchpad.net/bugs/810808 >
<kim0> Morning all
<DigitalFlux> Morning kim0 :)
<kim0> DigitalFlux: Morning o/
<DigitalFlux> what is the status of deploying Ensemble's formulas into LXC ?
<kim0> an experimental branch exists .. however it's certainly not 'ready'
<kim0> you can play freely on ec2 however .. launch micro instances :)
<DigitalFlux> i'm guessing this one https://code.launchpad.net/~clint-fewbar/ensemble/lxc-container ?
<kim0> yeah should be
<DigitalFlux> yeah, that's my plan for today
<kim0> what are you going to be working on
 * DigitalFlux need to find where to configure ensemble to use micro instances ..
<DigitalFlux> The Trac formula
<kim0> oh I see
<DigitalFlux> https://bugs.launchpad.net/principia/+bug/795480
<_mup_> Bug #795480: Formula needed: Trac <bitesize> <Principia Ensemble:New for ahmedelgamil> < https://launchpad.net/bugs/795480 >
<kim0> yaay \o/
<kim0> let me help with configuring the t1.micro thing
<DigitalFlux> kim0: please, go ahead
<kim0> DigitalFlux: check out http://fewbar.com/2011/06/so-what-is-ensemble-anyway/
<kim0> search for m1.small
<kim0> You need to add     default-instance-type: m1.small
<DigitalFlux> Ahh, that was a long article :)
 * DigitalFlux checking ..
<kim0> to ~/.ensemble/environments.yaml
<kim0> in your case .. it would be t1.micro of course
<DigitalFlux> hehe, looks like i already did that before :)
<kim0> cool :)
<DigitalFlux>     default-instance-type: t1.micro
 * kim0 nods
<DigitalFlux> ok neat ..
 * DigitalFlux coding his first Ensemble formula :)
<kim0> \o/
<kim0> DigitalFlux: feel free to grab me any time once you have a question
<DigitalFlux> kim0: i will indeed :), Thanks kim0
<kim0> RoAkSoAx: morning o/
<DigitalFlux> kim0: hmm, here is some point that i want to discuss
<kim0> RoAkSoAx: I see you haven't added your cool session yet to https://wiki.ubuntu.com/UbuntuCloudDays/Timetable .. please add one today :) Thanks
<kim0> DigitalFlux: shoot
<DigitalFlux> kim0: so i looked over the current principia formulas
<DigitalFlux> like in case of checking the wordpress formula
<DigitalFlux> i see that not all the underlying components are obvious
<kim0> like ?
<DigitalFlux> where the web server layer is not that shiny here
<DigitalFlux> you just install the wordpress package
<DigitalFlux> i understand that this may get apache with the default config
<DigitalFlux> but is this the perfect way to go .. from the prespective of code reuse ?
<DigitalFlux> or will it better to create a separate apache formula and relate it to the wordpress formula ?
<kim0> ah .. we just had this discussion yesterday :)
<kim0> good question indeed
<kim0> so basically .. Ensemble should be getting settable parameters soon
<DigitalFlux> I think we're talking about CM vs. packaging here ..
<kim0> which would let you choose which web server for example you'd like to use
<DigitalFlux> setting the standards in this for Ensemble from the early beginning will solve a lot of formula 'styling' in the future BTW !
<kim0> the need is recognized .. the code is already sitting in review queue
<kim0> good catch however
<DigitalFlux> aha, awesome
<DigitalFlux> wow, lots of hooks in mysql :)
<DigitalFlux> my expectation is that the mysql formula will have the largest amount of hooks around :) 
<kim0> DigitalFlux: yes it is growing :)
<kim0> DigitalFlux: you only need to worry about db-relation only for starters
<DigitalFlux> kim0: well, i originally check it in order to know how it 'automatically' creates and hands random passwords for WP
 * kim0 nods
<DigitalFlux> kim0: I guess i should do that for Trac/SVN integration ..
<kim0> although the beautiful part .. is you don't have to understand it
 * DigitalFlux nods
<DigitalFlux> Okie
<DigitalFlux> How can i set some service config 'parameters' for some formula that should be deployed
<DigitalFlux> like where i'm creating a Trac project here, how can i set it's name ?
<hazmat> good morning
<kim0> hazmat: Morning o/
<kim0> DigitalFlux: that's the configuration stuff I talked about earlier, the one that is not ready yet .. for now hardcode some names
<DigitalFlux> kim0: i see, ok, good to know, Thanks
 * kim0 can't wait for the config-set stuff to land either ;)
<hazmat> bug 810765
<_mup_> Bug #810765: if a formula in a repo has invalid metadata, it can stop any deploy from the repo. <Ensemble:New> < https://launchpad.net/bugs/810765 >
<_mup_> ensemble/deploy-tolerates-broken-formulas-bug-810765 r276 committed by kapil.thangavelu@canonical.com
<_mup_> repository.find ignores invalid formula metadata.
<DigitalFlux> I guess there is no Ubuntu EC2 images that can boot on instance-store micro-size, right /
<DigitalFlux> ?
<hazmat> DigitalFlux, there are
<DigitalFlux> only EBS ..
<hazmat> oh.. instance-store
<hazmat> DigitalFlux, ensemble tries to use ebs for all ec2 instances 
<hazmat> it should work with instance-store instances though
<DigitalFlux> hazmat: aha, i was just booting an instance manually for testing some stuff ..
<hazmat> you have to specify the image in default-image-id in environments.yaml
<DigitalFlux> lets use the ebs then ..
<DigitalFlux> hazmat: yeah, did that, Thanks :)
<hazmat> morning niemeyer, fwereade 
<niemeyer> hazmat: Yo!
<fwereade> hazmat: heyhey
<fwereade> how's it going?
<niemeyer> Man, there's a beep-beep-beep going on here that is driving me crazy
<fwereade> hazmat: thanks for the review
<fwereade> hazmat: did you get a chance to think over my response?
<hazmat> fwereade, np, parts of it still digesting
<hazmat> as far i can see even though bootstrap code is reusable, it still needs modification on a per provider basis 
<hazmat> at least to set the attribute
<hazmat> i'm still recovering from a bit of a cold the last few days.. i'll be glad when its gone
<fwereade> agreed, but that attribute does actually differ across providers
<RoAkSoAx> kim0: howdy! Yeah I'll do this weekend as I'm still unsure on what to do
<fwereade> the actual logic -- start a machine with specific variables if there isn't one running already -- is completely generic
<hazmat> fwereade, niemeyer last day of the sprint, any conclusions on cobbler integration?
<fwereade> and I'm concerned that if we subclass FooLaunchMachine every time we will end up with noticeable duplication
<fwereade> hazmat: we have something that runs for andres
<hazmat> fwereade, it doesn't need a subclass of launch machine to avoid the pre callback
<fwereade> I need to make sure I can duplicate it on my machine before I go
<fwereade> and then there'll be a fair amount of work before it's production-ready
<fwereade> but it feels like a good first sprint to me :)
<fwereade> hazmat: please explain
<hazmat> fwereade, cool, so there's another branch that does the cobbler integration?
<hazmat> fwereade, the bootstrap has the launch class, it can instantiate, get the variables, modify, and invoke start machine
<fwereade> hazmat: at the moment we have lp:~ensemble/ensemble/bootstrap-cobbler
<fwereade> hazmat: strongly disapprove of that approach, it makes it hard to change LaunchMachine without changing Bootstrap
<hazmat> fwereade, their already intimately connected
<hazmat> try changing the scripts output of launchmachine and you have to change them both
<fwereade> hazmat: fair point
<fwereade> hazmat: I have a half-formed idea that says that would be best addressed by making machine_data/variables more structured
<hazmat> its already invoking a public method of launchmachine, this just swithes it to another, cleanups up the data passing, and avoids pushing callback logic manipulating variables into multiple places
<fwereade> hazmat: so I *think* it's a data-format connection more than a code-flow connection
<fwereade> hazmat: well: it makes it call 2 methods, instead of passing one callback into one method
<fwereade> (good point about the unnecessary post_ callback, ty for that)
<hazmat> passing callbacks arounds obscures flow, its better to just invoke the method, consume the value, and invoke another method
<fwereade> hazmat: my issue is that that means duplicating the existing flow in LaunchMachine.run
<fwereade> if we change that, we also have to change bootstrap
<fwereade> however, if bootstrap is just using run on its own, which is tested separately, we can be confident that changes to LM that don't affect the interface to run won't affect bootstrap
<hazmat> fwereade, it means simplifying the flow in launch_machine.run. imo.. it removes the callback parameter insertion into the deferred chain
<hazmat> fwereade, start_machine is also a public method on LM, and tested
<fwereade> hazmat: ok, so the choice is between 2 simple flows, in different places, that have to stay the same; or one slightly-more-complex flow in just one place
<hazmat> fwereade, re the same, as i said, their already fairly intimately connected, simplifying the flow doesn't change that
<fwereade> hazmat: would your perception of the connections be different if variables was a custom object instead of a class?
<fwereade> hazmat: because I think the intimacy of that connection is a result of the unstructured nature of that data
<fwereade> hazmat: (besides, I thought you were the one arguing for a simpler flow? I'm advocating a slightly more complex flow that eschews duplication)
<kim0> RoAkSoAx: okie thanks :)
<hazmat> fwereade, no re custom object, i think i've already stated why.. its the callback param to mutate data in a particular order that i take issue with
<hazmat> fwereade, we're talking about a single line of duplication
<hazmat> and we're removing several lines of complexity and the callback parameters
<fwereade> hazmat: well, 2 lines -- I agree it's not really a big issue
<fwereade> hazmat: but I've already removed one callback
<fwereade> so you seem to be arguing that:
<fwereade>         if pre_start_machine is not None:
<fwereade>             launch_chain.addCallback(pre_start_machine)
<fwereade> is excessively complex, and is better addressed by duplicating a simple flow
<fwereade> hazmat: is that an accurate statement of your position?
<niemeyer> hazmat: What's that callback param that mutates data in particular order?
<hazmat> fwereade, i'm saying that variables = lm.get_machine_variables(); modify_variables(variables); do_something_with_variables();... is cleaner than.. do_something_else(modify_variables_callback)
<fwereade> hazmat: in isolation, yes
<niemeyer> hazmat, fwereade: Where's that logic?
<niemeyer> Just want to follow it
<hazmat> niemeyer, common.launch.LaunchMachine.run
<fwereade> ensemble.providers.common.launch
 * niemeyer looks
<niemeyer> I agree with hazmat, I think
<niemeyer> Whoever runs run() can do anything before and after it, if needed
<niemeyer> hazmat: Is that your point?
<hazmat> niemeyer, well i this case the before part is manipulating data internal to that method, the after part callback was already removed
<niemeyer> hazmat: So what's your suggestion to address that data?
<hazmat> i'm saying the bootstrap can just get the data using the public api of the launchmachine, manipulate it, and call the start_machine api of it
<fwereade> niemeyer: but it's not soing stuff before run
<fwereade> niemeyer: it's before start_machine
<fwereade> hazmat: I think I may be starting to be convinced
<niemeyer> fwereade: Yeah, got it
<fwereade> hazmat: but I'm still concerned that a change to LaunchMachine.run would require a non-obvious change to Bootstrap
<niemeyer> Well..
<niemeyer> Hmm
<hazmat> fwereade, run shouldn't have any subclass responsibility, they need to implement start_machine.. and changes to the data would incur a need to change.. we need to trust the api we construct, not create holes in them so that the provider can hide from the consumer
<niemeyer> That whole machine_data blob of unstructured data is pretty sad, to be honest
<hazmat> agreed
<niemeyer> Who owns the data?
<hazmat> both cloud-init and the provider
<niemeyer> If you invert it, get_machine_variables within LaunchMachine will have to take into consideration that someone else is setting packages
<hazmat> its constructed from api params, ensemble config, cloud-init defaults, and provider details
<niemeyer> hazmat: Not really.. no one owns it.. we just pass it around, and everyone sticks the data in as if it was fine
<fwereade> IMO changes to the data are a red herring here, and we can all agree that a more structured solution would make life easier
<niemeyer> This should really be a proper object, rather than a dictionary
<fwereade> (ok, the whole purpose of the code is to change the data, but it's the control flow we're debating)
<niemeyer> fwereade: It is relevant.. if you just invert the order, it will break the logic
<fwereade> niemeyer: yes... I don't think I said otherwise
<niemeyer> fwereade: You said it was a red herring.. I'm just saying it's relevant
<niemeyer> So, yeah.. I suggest we don't address this change right now..
<niemeyer> We'll have to change the way in which these methods alter the data
<niemeyer> and if we do this, we should really be using a structured object which has add_packages() methods and the such
<niemeyer> so that we don't have to worry about who's doing what in which order
<hazmat> niemeyer, it doesn't really effect the discussion at hand.. because the underlying data here is ordered
<niemeyer> I also agree with fwereade that it's a good thing we have a single place for the run() logic that starts a machine.
<hazmat> and its a discussion of the implementation used to effect that ordering, either hidden behind a callback from the consumer, or explicitly ordered by the consumer
<niemeyer> Actually
<niemeyer> Hmm
<fwereade> niemeyer: precisely, I don't know how run() is going to change in the future but I don't want to have to keep Bootstrap's reimplementation i sync
<hazmat> the actual api contract for the provider is start_machine, which is the same in either case.. the only purpose of run is to provide data before start_machine
<hazmat> bootstrap just does the same
<hazmat> imo
<hazmat> instead of obscuring flow with a callback only used by bootstrap, just make bootstrap provide the params
<fwereade> hazmat: that hadn't been my perspective but perhaps you're right
<niemeyer> hazmat: No, the contract is really run
<fwereade> hazmat: I'd seen the get_machine_variables and start_machines methods as public just because we expect them to be overridden
<niemeyer> hazmat: Hopefully the whole LaunchMachine thing can go away, and bootstrap should actually use provider.start_machine
<niemeyer> hazmat, fwereade: All these variables that are being set in the pre_start_hook should really be injected upfront
<niemeyer> hazmat: Passed to the method
<daker> hello
<niemeyer> That would make it a single interface, without any ordering issues
<daker> i need some help! http://paste.ubuntu.com/644790/
<niemeyer> daker: Hey!
<fwereade> niemeyer: but for that we do need a more structured machine_data
<fwereade> basically just for the ordering of scripts
<niemeyer> fwereade: Not necessarily.. as long as we change get_machine_variables within LaunchMachine to take into consideration that there's pre-exisitng data that may be meaningful, instead of blindly killing everything
<fwereade> which I freely stipluate is a mess, and has giant flashing lights saying "this will bite us in the future"
<niemeyer> Which was my original point
<fwereade> niemeyer: ah, I see
<fwereade> niemeyer: but without structured machine_data I think we're still just shuffling the points at which we assume too much around the code
<hazmat> niemeyer, its not just pre-existing data, its data that needs to merged in particular order.. the underlying data structure here is a list, the bootstrap needs to have some of its elements added prior to the end of any machine .. to ensure the zk tree is initialized prior to the machine agent firing
<RoAkSoAx> daker: try: $PYTHONPATH=`pwd` ./bin/ensemble shutdow
<niemeyer> hazmat: Yeah.. which is a terrible terrible interface
<RoAkSoAx> daker: try: $PYTHONPATH=`pwd` ./bin/ensemble shutdown
<hazmat> fwereade, niemeyer alternatively we could just have this as a flag to get machine variables 
<niemeyer> hazmat: and why I'm suggesting we don't touch this before we actually want to fix it properly
<hazmat> and pass that bootstrap flag to launchmachine.run or provider.start_machine
<niemeyer> hazmat: This, specifically, should _really_ go away:
<niemeyer>         scripts = vars_out["scripts"]
<niemeyer>         machine_agent_script = scripts.pop()
<niemeyer>         scripts.extend([
<hazmat> niemeyer, agreed
<hazmat> niemeyer, the best way to do that atm is to unify the data construction
<hazmat> and parameterize it with a bootstrap flag
<niemeyer> hazmat: That sounds pretty good, actually
<hazmat> it exists the way it does due to its origin as a subclass method
<daker> RoAkSoAx, doesnt' work :/ No such file or directory
<niemeyer> fwereade: ?
<daker> RoAkSoAx, http://paste.ubuntu.com/644795/
<fwereade> hazmat, niemeyer: I think so... I was opposed to making B an LM subclass, because it isn't and LM... but I think it's fine to have LM know how to construct a zk instance specifically
<fwereade> yep, I think I like that a lot
<hazmat> cool
<fwereade> make that change and propose again?
<fwereade> we maintain machine_data ickiness but at least it's all in one inheritance tree ;)
<hazmat> fwereade, make that change, and just add a comment to the mp to that effect, i'll have  a look and approve
<hazmat> fwereade, and it also gets rid of the callback param
<fwereade> cool, thanks very much
<fwereade> indeed :)
<fwereade> I was never proposing the callback as a perfect solution... just, to my mind, the least worst of the ones we'd considered :)
<fwereade> much appreciated
<hazmat> fwereade, np, thanks for discussing
 * daker <= SOS
<hazmat> daker, your installing ensemble from bzr?
<hazmat> daker, we generally recommend people not developing ensemble code to just the ppa which is updated daily
<hazmat> daker, in this particular case its a PYTHONPATH issue
<daker> there was one from a ppa an i removed it
<RoAkSoAx> daker: remove the $ from $PYTHON....
<daker> RoAkSoAx, ah
<hazmat> daker, RoAkSoAx suggestion looks good to me
<hazmat> with the $ removed
<daker> works thanks hazmat RoAkSoAx 
<fwereade> hazmat, niemeyer: it crosses my mind that then the get_instance_id_command method then moves to LaunchMachine
<hazmat> fwereade, sounds good.. perhaps renamed get_bootstrap_instance_id()
<hazmat> one less provider subclass responsibility on bootstrap.. bootstrap becomes a true generic.. you could even move the launch class to the provider itself and make bootstrap truly generic
<hazmat> i guess bootstrap goes away entirely then
<hazmat> as a separate command cls
<fwereade> hazmat: precisely, I was abut to write exactly that before I got distracted :)
<fwereade> hazmat: and you may be right about it going away entriely
<fwereade> hazmat: cool :)
<niemeyer> Woohay things going away entirely
<fwereade> there's nothing quite so nice as deleting code :)
<fwereade> anyway all that will have to wait until I have a working cobbler environment
<fwereade> which I need before I go away from andres
<fwereade> but I think we have a plan :)
<fwereade> hazmat, niemeyer: although that actually implies a common MachineProvider base class with a .bootstrap() method
<fwereade> ...which then implies that actually a *lot* of methods can be defined on the base class
<niemeyer> fwereade: Hm?
<niemeyer> fwereade: I don't see how one thing implies the other
<fwereade> niemeyer: hmm, perhaps not, just a sec
<fwereade> niemeyer: agreed, not implied; we'll see how it looks when we have 2 implementations
<jcastro> kirkland: I agree (wrt. figuring out who is working on what in principia)
<kirkland> jcastro: well, that, and an easy way to see what's currently in principia
<kirkland> jcastro: an equivalent of apt-cache search
<jcastro> kirkland: any idea when the whole repository will get hooked up?
<jcastro> (heh, that's exactly what I was going to ask, apt-cache search)
<niemeyer> jcastro: I've heard Launchpad is going to land the needed changes this week
<niemeyer> jcastro: Work starts immediately
<jcastro> oh, perfect, that's great timing.
<niemeyer> jcastro: and yeah, we'll have ensemble search :)
<niemeyer> Oh, and it's Friday.. maybe it's even there arleady
 * robbiew notes we need to sort out "principia" as a name
<kirkland> jcastro: dunno, SpamapS was on that
<jcastro> ok I'll ping him when he's around
<jcastro> robbiew: I thought we're just going with "ensemble formulas"?
<jcastro> and "ensemble core"?
<robbiew> jcastro: well the thread didn't really conclude...so maybe I'll do that now
<robbiew> ;)
<daker> kim0, check now https://code.launchpad.net/~daker/ensemble/small-fix/+merge/68038
<robbiew> jcastro: not so sure we need an "Ensemble core"..."Ensemble" is fine, imo
 * jcastro nods
 * kim0 disk seems to be dying
<lynxman> hey guys *waves*
<lynxman> is there anywhere that we'll keep versions of Ensemble in a tar archive?
<lynxman> finishing my macports implementation of the Ensemble client :)
<hazmat> lynxman, you mean like a release of ensemble. or a release of formulas?
<lynxman> hazmat: release of ensemble, the port need somewhere to download a tar from
<m_3> lynxman: awesome... tons of devs on mac using ubuntu in the cloud
<lynxman> m_3: that's the idea :) hit first and hit nicely
<hazmat> lynxman, any chance it could construct one on the fly with a bzr export?
<lynxman> hazmat: hmm you'll be making my life miserable doing that :)
<lynxman> hazmat: that would mean installing bzr then pulling and so forth, completely foreign to how ports works
<hazmat> lynxman, atm we don't have a release.. is my only concern.. i think we where planning on having one in the next few months to coincide with the release of oneiric
<hazmat> lynxman, right.. but there are ports that pull from vcs i thought?
<lynxman> hazmat: yes but they do nasty stuff
<hazmat> niemeyer, release time? ^
<lynxman> hazmat: just saying it out loud :)
<hazmat> niemeyer, this actually dove tails nicely with what robbiew was mentioning about setting some sort of more stable weekly release
<niemeyer> hazmat: robbiew was talking about the sync into Oneiric, IIRC
<lynxman> hazmat: oh, that's true
<niemeyer> hazmat: We can do a weekly release, though
<robbiew> right...and sync them into Oneiric ;)
<lynxman> niemeyer: would be okay to get a tar.gz in the ensemble project for the last milestone?
<lynxman> niemeyer: that would be easy for me to parse
<niemeyer> lynxman: We can do tarballs for sure.. what do you mean by last milestone?
<niemeyer> robbiew, hazmat: Yeah, good point
<hazmat> lynxman, trunk is stable so we can cut directly from it
<niemeyer> robbiew, hazmat: My point was just that we don't need release tarballs to sync
<robbiew> ack
<niemeyer> robbiew, hazmat: But might be interesting to be more careful about what is going into Oneiric either way
<SpamapS> does setup.py produce a good dist output?
<lynxman> niemeyer: I mean looking at the ensemble project page, right now last milestone is capetown
<lynxman> niemeyer: something like that
 * SpamapS just emailed the list with his weekly release plans
<niemeyer> SpamapS: bzr export --format=.tar.gz
<SpamapS> niemeyer: *OH* right.. thats much simpler. :)
<niemeyer> lynxman: CUrrent one is Dublin
<niemeyer> lynxman: WE should update it
<lynxman> niemeyer: so.. dublin :)
<hazmat> SpamapS, it should
<niemeyer> lynxman: It's pretty old
<SpamapS> niemeyer: at this point, with Oneiric 3 months away, would it make sense to make the next milestone "orlando-2" ?
<niemeyer> lynxman: I mean, the last one
<niemeyer> lynxman: I'd recommend bzr export --format=.tar.gz
<SpamapS> or austin? :)
<niemeyer> lynxman: To get going
<hazmat> next release name.. startswith 'E', dublin release ends at the end of july afaicr
<niemeyer> SpamapS: We'll need something with an E :-)
<SpamapS> oh you guys have *got* to stop using the cities we're in
<SpamapS> that is *CONFUSING*
<SpamapS> I remember now you explained that already
<hazmat> dublin was coincidence
<niemeyer> SpamapS: It's actually the opposite
<lynxman> niemeyer: yeah, the thing is that I can either curl a tar from somewhere (what I was asking at first) or go around doing a whole bzr export, was asking which is better for you guys
<niemeyer> SpamapS: Canonical should stop putting sprints in our milestone names!
 * SpamapS votes for Entendre as the next release name.. 'ensemble entendre' has a nice french comedic ring to it
<niemeyer> SpamapS: ;-)
<lynxman> SpamapS: maybe Ecublens? It's a nice city in the French part of Switzerland ;)
<lynxman> http://maps.google.com/maps?hl=en&q=ecublens%2C%20ch
<lynxman> it's where the EPFL is based
<SpamapS> Ooo I like lynxman's
 * SpamapS dreams of swiss beer and pretzls
<lynxman> niemeyer: so you prefer a bzr branch then... just from trunk?
<SpamapS> niemeyer: to lnxman's point, it would be prudent to start branching trunk when a new milestone is started...
<niemeyer> lynxman: Just while we don't have actual releases, that would be preferable
<lynxman> niemeyer: okay, will try to make it that way, thanks
<SpamapS> Once there are releases.. it will be critical... but yeah, w/o a release, they're just points in time.
<niemeyer> SpamapS: Not necessarily.. milestones are more organizational than anything else
<niemeyer> SpamapS: Weekly releases with proper tagging should be enough
<SpamapS> should we just tag w/ the date? 0.5.2011.07.15 ?
<niemeyer> SpamapS: Probably just a sequence number on the version itself might be fine
<niemeyer> SpamapS: We always have the date out of the revision information
<SpamapS> I actually like the way it works now.. 0.5+bzrXXX
<SpamapS> because its very clear how to go get that release
<niemeyer> SpamapS: Well, we might also do this
<niemeyer> SpamapS: Using the revision number
<niemeyer> SpamapS: 0.5.XXX
<SpamapS> thats what XXX is
<SpamapS> oh
<SpamapS> the only reason I don't like that, is that its pretending to be a release
<niemeyer> LOL
<niemeyer> SpamapS: It's not just pretending.. ;)
<SpamapS> or at least, pretending to be more of a release than the 0.5 already is pretending
<niemeyer> SpamapS: It _is_ a release
 * SpamapS thinks a massage is a nice release too.. :)
<SpamapS> yeah.. its really not that important...
<SpamapS> Oh I was thinking on one other thing..
<SpamapS> how about dropping the debian dir from the trunk and nesting the Ubuntu packaging for the daily builds?
<SpamapS> Would be good to have a single place for packaging maintenance.
<hazmat> SpamapS, sounds good to me.. the ec2-build script should probably get yanked then
<SpamapS> hazmat: is that used anymore?
<hazmat> SpamapS, nope
<kim0|backup> Alright folks .. my disk drive decided to die now, seems like I'll begin a badblocks party .. I'm off next week, so may not be online all the time. see you soon
<negronjl> SpamapS, niemeyer, hazmat, m_3, anyone:  why is this happening.  I need help on this.  https://pastebin.canonical.com/49825/
<SpamapS> negronjl: IIRC, there's a bug where if any of the formulas in your repository have bad metadata, none of them are deployable.
<negronjl> SpamapS:  I fixed that already.  this is different.
<SpamapS> oh
 * SpamapS waits for the second timeout-refresh of pastebin
<jimbaker> negronjl, we really need the debug log to see what's going on
<SpamapS> maybe the start hook failed?
<SpamapS> or is still running?
<jimbaker> negronjl, also, i think it would be best if we used paste.ubuntu.com
<negronjl> jimbaker.   ahh...let me re-do it all with debug-hooks, debug-log and pastebin.ubuntu.com ... I'll be back
<jimbaker> negronjl, i don't think you need debug-hooks yet :)
<SpamapS> negronjl: you can just ssh to the box and look at the formula.log
<negronjl> SpamapS:  re-doing it.  I'll get all that info in a minute.
<SpamapS> negronjl: /var/lib/ensemble/units/cloudfoundry-server-0/formula.log
<jimbaker> negronjl, SpamapS is right... that's all debug-log is doing is aggregating the logs together
<jimbaker> in fact one problem with debug-log is that it misses some log info... but as far as i know, just very early in the agent startup process now
<niemeyer> negronjl: Best way to find out is log in the machine and read the log
<negronjl> niemeyer:  doing that now :)
<negronjl> I'll paste it back in a minute
<niemeyer> negronjl: We have to create a neat way to just find out the recent logs
<niemeyer> negronjl: Not really hard.. just need to put it in place
<niemeyer> negronjl: Would be tremendously helpful in these cases indeed
<SpamapS> some helpers would do that
<SpamapS> sudo 'ensemble-admin tail-logs' would work
<jimbaker> SpamapS, sounds like an after the fact debug-log
<jimbaker> SpamapS, in general i always run debug-log, might be nice if we made it easier to use, eg, ensemble bootstrap --debug to start debugging immediately
<m_3> can debug-log take an arg for unit?
<jimbaker> however, the debug log was never designed to be more than a simple solution for formula dev. some sort of log collection tool would be nice to have integrated at some point (to bring back some previous discussion we've had)
<SpamapS> m_3: yeah
<SpamapS> jimbaker: if it were changed to a ring buffer and always done.. then debug-log would just tap into the ring buffer and keep tailing it.
<jimbaker> m_3, it's quite flexible: ensemble debug-log --help http://paste.ubuntu.com/644868/
<SpamapS> jimbaker: think 'dmesg'
<m_3> ok, didn't register that include/exclude took unit-names
<m_3> think maybe the solution to _that_ problem is more coffee
<negronjl> Here is the pastebin for the formula.log:  http://pastebin.ubuntu.com/644870/
<negronjl> jimbaker, SpamapS, niemeyer, hazmat, m_3:  http://pastebin.ubuntu.com/644870/
<niemeyer> negronjl: Can't see any issue there
<niemeyer> negronjl: 2011-07-15 16:24:39,240: statemachine@DEBUG: unitworkflowstate: transition state (installed -> started)
<niemeyer> negronjl: It actually says it went into started
<negronjl> niemeyer:  hence the question and confusion.  here is what I get when I run ensemble status:  http://pastebin.ubuntu.com/644872/
<m_3> looks like it started the transition from installed->started, but never completed the transition
<m_3> your start hook is not returning?
<m_3> what's it look like?
<negronjl> m_3:  it returns just fine when I do it manually
<niemeyer> negronjl: Hmm
<m_3> niemeyer: is that a correct interp of the log?
<niemeyer> negronjl: It actually looks like the log is partial
<niemeyer> negronjl: 2011-07-15 16:24:39,241: hook.executor@DEBUG: Running hook: /var/lib/ensemble/units/cloudfoundry-server-0/formula/hooks/start
<niemeyer> negronjl: There's no "Hook complete" there
<niemeyer> negronjl: Can you paste your start hook please?
<negronjl> niemeyer:  sure....give me a sec
<negronjl> http://pastebin.ubuntu.com/644886/
<negronjl> niemeyer:  ^^
<niemeyer> negronjl: Can you please ps auxw on the server and see if "vcap start" is still running?
<negronjl> niemeyer:  checking
<m_3> background that puppy
<m_3> or really, prob better off putting into a real upstart script to daemonize properly
<negronjl> m_3:  thinking of just adding that all powerful & at the end of bin/vcap start :D
<niemeyer> negronjl: Was it running?
<negronjl> niemeyer:  still checking as I had shutdown so now, I'm re-deploying
<niemeyer> negronjl: It at least looks like so from the output in the log
<m_3> negronjl: quick enough to try just a background... sometimes it's not enough to kick your controlling term though
<niemeyer> negronjl: It ends without any other feedback about the hook
<niemeyer> negronjl: or transitions
<niemeyer> negronjl: Just "echo" something after the command is run, just to be sure
<negronjl> niemeyer:  just what i did.  added echo something at the end.  deploying now...waiting..
<SpamapS> W00t!
<SpamapS> http://www.oscon.com/oscon2011/public/schedule/detail/18367
<m_3> looks like fun
<negronjl> hey SpamapS, you're famous now :D
<niemeyer> SpamapS: Wow, sweet
<niemeyer> Heading to lunch, biab
<robbiew> SpamapS:  *\o/*
<niemeyer> robbiew: Man, that looks like a cheer leader
<m_3> pom-poms ftw
<negronjl> SpamapS: ping
<robbiew> niemeyer: it was ;)
<niemeyer> *\o/*
<niemeyer>   |
<niemeyer> ____
<niemeyer> Arg.. Almost
<niemeyer> *\o/*
<niemeyer>     |
<niemeyer> LOL
<niemeyer> I can't type
<niemeyer> *\o/*
<niemeyer> __|__
<niemeyer> Sorry.. back to work
<_mup_> Bug #811226 was filed: Ensemble needs to enable backing storage <Ensemble:New> < https://launchpad.net/bugs/811226 >
<kirkland> jcastro: would you mind confirming https://bugs.launchpad.net/ensemble/+bug/811226 ?
<robbiew> *\o/*
<robbiew>    /^\
<_mup_> Bug #811226: Ensemble needs to enable backing storage <Ensemble:New> < https://launchpad.net/bugs/811226 >
<kirkland> robbiew: um, you're a cheerleader who was severed at the waist?
<robbiew> that was the skirt
<robbiew> lol
<kirkland> robbiew: :-)
<SpamapS> negronjl: pong, sup?
<negronjl> SpamapS: https://bugs.launchpad.net/principia/+bug/811228
<_mup_> Bug #811228: mysql formula user names sometimes exceed 16 characters <Ensemble Formulas:New> < https://launchpad.net/bugs/811228 >
<negronjl> SpamapS:  mysql usernames sometime exceed the max number of characters (16)
<SpamapS> negronjl: ahh, adam_g was working on that
<negronjl> SpamapS:  Is there a bug number?
<SpamapS> negronjl: or rather, his recently submitted shared db functionality solves it .. and I think we can solve it for all cases.
<SpamapS> negronjl: no it was just something he found while working on the openstack stuff.
<SpamapS> negronjl: an easy fix is just to generate the username semi-randomly.
<SpamapS> like, use the first 8 chars of the service name, and then 8 random chars.
<negronjl> SpamapS:  ahh...a bug number on these ( and probably any issues ) would have prevented me from debugging and duplicating the work that adam may already be working on
<negronjl> SpamapS:  I submitted a patch tht uses uuid and uses the last 12 digits
<negronjl> SpamapS:  that way is more random than just chopping the service name or something silly like that.
<SpamapS> negronjl: I'd err on the side of using pwgen rather than uuid .. the only reason being I don't know what the randomness guarantees of the first 12 chars of the uuid generator are... but I do know them for pwgen.
<SpamapS> negronjl: But anyway, if you have a patch, you can push it yourself, as you're a composer.
<negronjl> SpamapS:  I know, I was just being polite by bringing it to your attention and take the oportunity to complain about the lack of a bug from adam_g about the same issue that I was working on :)
<negronjl> SpamapS:  I'll look into pwgen
<SpamapS> negronjl: Ah.. ok.. just making sure you know.. I don't want people to think I'm special in this regard. :)
<SpamapS> I think at this point you've written more formulas than anybody else.
<jsalisbury> negronjl, I'm starting to play with ensemble.  Does the current version only run against EC2?  Or can I have my own OpenStack cloud, say on a VM, that I can use for testing/playing with ensemble?
<kirkland> negronjl: SpamapS: what's the context of the pwgen comment?
<niemeyer> jsalisbury: It runs against EC2 ATM, but if you set up OpenStack and use the EC2 API it "should" work
<niemeyer> jsalisbury: We haven't put time on testing it yet, but any reasons for it not to work are bugs we want to fix in the short term
<niemeyer> jsalisbury: So if you're interested in getting that in place, just poke us with issues you find and we'll give a hnad
<jsalisbury> niemeyer, great, thanks.  Then I can possible get two birds with one stone ;-)
<niemeyer> jsalisbury: Definitely :)(
<niemeyer> :)
<jsalisbury> niemeyer, thanks!
<kirkland> jsalisbury: that would be muy awesome :-)
<jsalisbury> kirkland, :-)
<jsalisbury> niemeyer, Would running OpenStack on a VM or single box be a valid test?  It's not very 'Real world', but the interaction with ensemble should be the same, correct?
<niemeyer> jsalisbury: Indeed
<jsalisbury> niemeyer, great
<adam_g> jsalisbury: this exists: https://bugs.launchpad.net/ensemble/+bug/805554  but i haven't tested myself, and figured it would "just work"  if you find the same, pleas confirm
<_mup_> Bug #805554: Ensemble from ppa does not work with nova. <Ensemble:Incomplete> < https://launchpad.net/bugs/805554 >
<jsalisbury> _mup_, boo.  I guess I can try to confirm that one.
<niemeyer> Alright, I'm taking off for now.. see you all later
<robbiew> SpamapS: http://paper.li/OnXcloud/1291324328#!art-entertainment
<robbiew> heh...apparently my tweet of your OSCON talk got picked up by some paper.li publisher
<robbiew> Onxcloud...created by OnX Enterprise Solutions...www.onx.com...never heard of them
<robbiew> canadian...maybe zul has :P
<hazmat> SpamapS, awesome re oscon
<negronjl> SpamapS: you around?
<SpamapS> negronjl: here now, sup?
<negronjl> SpamapS:  nm.  I figured it out.
<SpamapS> negronjl: silence is golden. ;)
#ubuntu-ensemble 2011-07-16
<adam_g> win 14
<_mup_> ensemble/expose-provision-machines r291 committed by jim.baker@canonical.com
<_mup_> Addressed review comments
#ubuntu-ensemble 2011-07-17
<m_3> adam_g: ping
