#ubuntu-ensemble 2011-06-06
<kiranmurari> Executing 'bin/ensemble', gives the error "ImportError: No module named txzookeeper.utils"
<hazmat> kiranmurari, how did you install ensemble (ppa.. bzr.. etc) ?  looks like your missing a dependency.
<kiranmurari> hazmat, downloaded the code from lp:ensemble and executed setup.py install
<kiranmurari> hazmat, followed the getting started document https://ensemble.ubuntu.com/docs/getting-started.html
<hazmat> kiranmurari, so that won't install any of the c extension dependencies, and it will install older versions of pure python deps from pypi
 * hazmat looks
<hazmat> kiranmurari, are you using a virtualenv or the system python (/usr/bin/python)?
<kiranmurari> hazmat, i'm using a virtual environment
<hazmat> kiranmurari, awesome... that's a little easier to fix then
<kiranmurari> hazmat, this has been reported as bug 788950 on lp
<_mup_> Bug #788950: Getting Started Documention is missing dependancies <Ensemble:New> < https://launchpad.net/bugs/788950 >
<kiranmurari> _mup_, i was referring to the same one
<hazmat> kiranmurari, so we have a daily ppa now.. which is probably easiest for getting started... but your most of the way through a source install.. to finish the source install you'll need to cd .. & bzr branch lp:txzookeeper && cd txzookeeper && python setup.py develop
<hazmat> kiranmurari, with the virtualenv python
<hazmat> kiranmurari, fwiw mup is a bot.. bugs  info and commit messages.. along with logs at irclogs.ubuntu.com
<hazmat> kiranmurari, thanks for filing the bug
<kiranmurari> hazmat, the bug was already in place ;)
<hazmat> ugh... time to fix it then
<niemeyer> Good mornings!
<hazmat> niemeyer, g'morning
<kiranmurari> hazmat, thx for the pointer. let me try your suggestion
<kiranmurari> Is there a Ubuntu docs page for txzookeeper like the ensemble docs page
<hazmat> kiranmurari, there's an older version of txzookeeper on pypi that got install into the virtualenv ... which is what caused the error
<hazmat> kiranmurari, not at the moment re separate txzookeeper docs, its more of a library atm
<kiranmurari> hazmat, the error is resolved. Time to move and ahead and start testing ensemble...
<hazmat> kiranmurari, awesome
<_mup_> ensemble/update-install-docs r240 committed by kapil.thangavelu@canonical.com
<_mup_> update installation docs
<_mup_> ensemble/update-install-docs r241 committed by kapil.thangavelu@canonical.com
<_mup_> add python-yaml to source install package deps
<niemeyer> I'll get some lunch.. biab
<jimbaker> hazmat, just thinking about the review in https://code.launchpad.net/~jimbaker/ensemble/expose-watch-exposed-flag/+merge/63066. niemeyer refers to line 52 of the diff, in which exists_d is unpacked, but not used
<jimbaker> hazmat, so the question i have is, is there a meaningful use of exists_d there, or should we simply unpack just the watch_d?
<jimbaker> hazmat, this watcher nested function follows exactly the same pattern as in watch_resolved
<hazmat> jimbaker, waiting on exists_d would work although its extraneous.. or discarding the value
<hazmat> the discard does entail lost bg activity.. but its not clear how the data associated is useful in this context
<jimbaker> hazmat, yeah, i think the standard pattern of _, watch_d = self._client.exists_and_watch(...) would be appropriate in this case, as you mention it is not clear how it can be usefully worked with
<jimbaker> the point of the watcher is to be called upon a watch event after all
<hazmat> jimbaker, indeed.. but all watches are associated to retrieving data
<hazmat> it does make sense to go ahead and wait till that data is retrieved b4 proceeding with the watch
<hazmat> even if its not used
<jimbaker> hazmat, so just use it as an additional sync point, eg yield exists_d ?
<jimbaker> hazmat, as expected, that would simply just work (and verified by tests - incidentally i started splitting test_service into separate test suite classes with this branch)
<jimbaker> hazmat, so maybe something like this would be better - it reorders the watch setup with the callback such that any StopWatcher can be trapped before attempting to reestablish the watch - http://paste.ubuntu.com/620074/
<jimbaker> and adds in the additional sync point of yield exists_d
<jimbaker> hazmat, forget about that ordering change - it will occasionally fail on the slow callback tests when looped
 * hazmat catches up
<hazmat> was on the phone with verizon support.. my dsl has been going down like 8 times a day
<hazmat> jimbaker, yes re additional sync point.. that's a little surprising re slow callback failure.. what's the failure out of the test with the ordering change?
<jimbaker> http://paste.ubuntu.com/620083/ - i changed both watch_exposed_flag, watch_resolved for this experiment
<jimbaker> hazmat, ^^^
<hazmat> jimbaker, so the problem with the ordering change is that it introduces a gap while the callback is executing that changes may occur un-noticed.. 
<hazmat> so the change isn't viable
<niemeyer> hazmat: I believe it never makes sense to set up a watch without taking the data in consideration
<niemeyer> Sorry, that was
<niemeyer> jimbaker, hazmat: I believe it never makes sense to set up a watch without taking the current state in consideration
<niemeyer> What's the point of waiting for a change if you don't know what the current value is?
<jimbaker> hazmat, yeah, not too surprising re callback ordering, just doesn't look obvious from the code
<hazmat> niemeyer, in these cases, its the callback responsibility to fetch the current state, as the current state is unknown when the watch fires
<hazmat> niemeyer, potentially we could/should restructure these,so the callback takes the current state
<jimbaker> hazmat, that's certainly my understanding of how we've been using watches
<niemeyer> hazmat: It still doesn't make sense..
<niemeyer> hazmat: You're asking zookeeper to notify you when something changes from an arbitrary point in time..
<niemeyer> hazmat: when you don't know what the value was before that
<niemeyer> hazmat: Why is it important to know the state on moment X if you've lost the changes on X-1?
<niemeyer> hazmat: The callback receives the current state..
<hazmat> niemeyer, yes, effectively..its the callback responsibility to fetch current state and effect any behavior in response to the change
<niemeyer> hazmat: It doesn't matter what the callback does
<niemeyer> hazmat: It's bogus to ask zookeeper to tell you about a change on something you didn't know the previous value
<hazmat> niemeyer, in these cases the callback recieves a change event, although the first time it recieves a node state 
<hazmat> niemeyer, i think that's fair.. esp. towards constructing a more state based api.. instead of a notification api.. and removes state retrieval from the callback responsibility
<hazmat> i can incorporate that into the state protocol work
<niemeyer> hazmat: That's not the point.. the state may still be retrieved by the callback
<niemeyer> hazmat: The point is that if the logic is _detecting changes_ with a txzookeeper watch, and the current state is discarded, there's a bug.
<hazmat> niemeyer, than what's the problem? if we know the callback operates against current state, and is responsible for fetching it, what's the functional problem with the current setup?
<hazmat> hmm
<niemeyer> hazmat: The problem is what I described above.
<niemeyer> hazmat: Every single time you set up a watch and you discard the current state there's a bug.
<hazmat> the bug being?
<niemeyer> hazmat: Because you don't know what you're watching
<niemeyer> hazmat: Are you waiting for the node to be changed?  What about the changes you've discarded by ignoring the current state?
<hazmat> we know exactly what we're watching.. that discard change handling.. is encompassed by invoking the callback which retrieves against current state.. ie. its not discarded
<niemeyer> hazmat: Are you waiting for the node to be removed?  It already was!
<niemeyer> hazmat: No, necessarily, you can't know what you're watching if you have _ignored_ the _current_ state. :-)
<hazmat> niemeyer, its not ignored
<hazmat> niemeyer, the callback is invoked for the change
<niemeyer> hazmat: It is.. the exists_d parameter is not used.
<niemeyer> hazmat: and it reflects the current state.
<niemeyer> hazmat: You don't know if you're watching for it to be removed or added.
<hazmat> niemeyer, yes the watch api get result is ignored, but the callback is invoked after the watch is set.. and it retrieves the current state
<niemeyer> hazmat: Does that make sense?>
<niemeyer> hazmat: It doesn't matter..
<niemeyer> hazmat: It's a pretty basic issue.
<niemeyer> hazmat: Imagine..
<niemeyer> callback() => get data, the file exists, cool
<niemeyer> file gets removed
<niemeyer> set watch in case it changes
<niemeyer> callback() never called again
<hazmat> that's not the case
<hazmat> the callback is invoked after the removal
<hazmat> we set watch, invoke callback, attach callback to new watch
<niemeyer> hazmat: How can you tell?
<niemeyer> hazmat: You are _ignoring the current state_
<niemeyer> hazmat: You are asking zookeeper to tell you when something changes, but 10000 revisions may have gone by
<hazmat> the watch setter is ignoring the callback, but all of the watch apis explictly state that its the responsibility of the callback to fetch current state
<niemeyer> hazmat: Dude..
<niemeyer> hazmat: Why do you need a watch?
 * hazmat sighs
<niemeyer> hazmat: Yeah, I know..
<niemeyer> hazmat: Why?
<hazmat> niemeyer, to get notified of change
<niemeyer> hazmat: Bingo..
<niemeyer> hazmat: So, what happens if one clock cycle before the watch gets set in zookeeper, the node got removed?
<niemeyer> hazmat: Does it matter what the callback looked at?
<hazmat> niemeyer, the watch gets set before the callback is invoked.. so if the node was removed, the callback will see the removal
<niemeyer> hazmat: The callback is invoked with the change event
<niemeyer> hazmat: 
<niemeyer>          callback_d = maybeDeferred(callback, bool(exists))
<hazmat> niemeyer, the callback must fetch the current state, per its signature
<hazmat> the doc strings for all the watch apis state this explicitly
<niemeyer> hazmat: What is that line above doing?
<hazmat> niemeyer, that's the first invocation of the callback, executing with a boolean of existence
<niemeyer> hazmat: What is that documentation saying:
<niemeyer> 15	                change event. The watcher always recieve an initial
<niemeyer> 16	                boolean value invocation denoting the existence of the
<niemeyer> 26	+               exposed flag. Subsequent invocations will be with
<niemeyer> 27	+               change events.
<niemeyer> +                yield callback(change_event)
<niemeyer> hazmat: I'll step out and get some coffee.. we can chat on Skype later if you still don't think there's a problem.
<jimbaker> hazmat, niemeyer - definitely would like to be part of the skype call when it happens
<hazmat> niemeyer, jimbaker lets
<jimbaker> especially since i have two branches pending on this :)
<jimbaker> but i'm glad that the expose-watch-exposed-flag branch is motivating what's a very important discussion
<hazmat> niemeyer, if we're quoting.. we might as well finish the quote.. "Its important that clients do not rely on the event as reflective                                                                                                                                                                     
<hazmat>         of the current state. It is only a reflection of some change                                                                                                                                                                          
<hazmat>         happening, the callback should fetch the current value via                                                                                                                                                                            
<hazmat>         the API, if needed."
<niemeyer> hazmat: Yeah, sure.. "Look.. here is the change event, and the current state, but.. DON'T TRUST IT!  It's just to make sure you've read that paragraph!"
<hazmat> niemeyer, the current state is never passed to the callback
<niemeyer> hazmat: +                yield callback(change_event)
<hazmat> niemeyer, a change event != current state
<niemeyer> hazmat: The change event is useful for..?
<hazmat> niemeyer,  to have notice that the current state needs processing
<niemeyer> hazmat: That's what the callback is for. The change event tells what the change is, but it can't be trusted because the next change event won't reflect the period while the callback was running.
<niemeyer> hazmat: This API is broken.  Let's fix it please.
<hazmat> niemeyer, the next change event will reflect the period b4 the callback was invoked
<jimbaker> it does seem to me that zk watches in general provide better guarantees with respect to ordering (and this is where niemeyer's point is especially relevant), and the current approach loses that
<hazmat> niemeyer, as i said at the beginning a more state based api might reflect better
<jimbaker> just knowing that a change has happened is so much weaker
<niemeyer> hazmat: No, because it is _ignoring the current state_!
<hazmat> niemeyer, the watch is set before the callback is invoked.
<niemeyer> We're going in loops..
<hazmat> so the current state is accounted for the next change
<niemeyer> Let's hang on mumble please
<hazmat> the only issue i  see is the callback may recieve a more recent state (for which a watch notification/change event is pending)
<hazmat> and will be reinvoked subsequently
<hazmat> which would be resolved with a more state based callback api to the watch api
<niemeyer> https://code.launchpad.net/~jimbaker/ensemble/expose-watch-exposed-flag/+merge/63066
<hazmat> bcsaller, mumble standup?
<bcsaller> mumble pretty much died out for me
<niemeyer> hazmat: Btw, I see your point and agree with you that if we guarantee that the exists_and_watch hits the server before the next exists, the watch from the first will enable the callback to be activated at least once.
<niemeyer> hazmat: More clearly then, as a summary from our conversation, the main issues are parameters being not-trustable, and events being dispatched multiple times for the same zk state version.
<niemeyer> hazmat: If the change_event is trusted, then state may effectively be lost.
<niemeyer> hazmat: Is that a fair summary we can agree on?
<hazmat> niemeyer, yes re non trustable parameters.. events being dispatched for the same state zk version.. is a little ambigious.. better to say that multiple callbacks invocations see the same zk state version
<hazmat> and yes that
<hazmat> that's a good summary
<niemeyer> hazmat: Cool, good to be on the same page
<niemeyer> hazmat: One thing I pondered about during that discussion is whether we guarantee that two sequential calls to the txzk API will necessarily happen in the same order within zk if we don't yield from the first one
<niemeyer> hazmat: It looks like so, but I couldn't tell for sure without looking at the internals of zk
<niemeyer> s/couldn't/can't
<hazmat> niemeyer, the calls are submitted to the zk client, which has ack'd, so my understanding is it should be sequentially ordered
<hazmat> ah
<hazmat> niemeyer, actually i take that back
<hazmat> the work is submitted in order, if the responses are returned in the same order is not guaranteed afaik.. say you did a sync, and a get
<hazmat> if you don't wait on the sync, its not clear to me that the sync will always happen b4 the get
<niemeyer> hazmat: We'll have to investigate that.. without this guarantee, we have more serious issues in the watch-before-callback pattern
<niemeyer> hazmat: It feels like it should be guaranteed, though
<niemeyer> hazmat: The issue goes like this:
<hazmat> niemeyer, it does.. i'd have to ask the zk devs to verify though
<niemeyer> hazmat: If we have two exists_and_watch(path) in sequence, if the former doesn't guarantee the watch being set before the following operation, we're in trouble
<hazmat> niemeyer, its confirmed responses are returned in order
<niemeyer> hazmat: Cool
<niemeyer> hazmat: So we're safe
<_mup_> ensemble/expose-watch-exposed-flag r243 committed by jim.baker@canonical.com
<_mup_> watch_exposed_flag provides the callback the current state, simplifying its API
#ubuntu-ensemble 2011-06-07
<niemeyer> Today was more difficult than usual.. it's funny how things aggregate around a similar time frame every once in a while
<_mup_> ensemble/expose-provision-service-hierarchy r265 committed by jim.baker@canonical.com
<_mup_> Merged upstream expose-watch-exposed-flag
<kim0> I'm pretty sure ensemble-log and set -eux are not logging for me (by default?) in the install hook
<TREllis> 20
<TREllis> bah
<kim0> a ha .. I think I stumbled over a serious bug
<kim0> Here are the steps to reproduce:
<kim0> - deploy a formula with an error in its install hook
<kim0> - state becomes, install_error
<kim0> - launch debug-hooks 
<kim0> - launch resolved -r <SU>
<kim0> - Broken hook is debugged under tmux
<kim0> - Now "status" mentions state is "installed" not "started"!
<kim0> Now I get the following traceback http://paste.ubuntu.com/620867/
<kim0> and issuing add-relation .. doesn't really help move it from state installed
<kim0> niemeyer: hazmat ^
<kim0> It seems the state-machine is stuck
<niemeyer> kim0: Yo
<kim0> hey :)
<niemeyer> kim0: Intresting
<kim0> indeed
<kim0> any manual way to move it into started ? 
<kim0> so I can continue writing the tutorial :)
<niemeyer> kim0: Yeah, it looks like a real bug indeed
<kim0> cool .. guess I'll just recreate a fresh env
<kim0> One note 
<niemeyer> kim0: Let's wait for hazmat.. he's coded that recently, and will likely be able to spot/fix what's up in no time
<kim0> awesome
<kim0> I also think the current functionality of fixing a broken hook is seriously lacking
<kim0> - fix remotely .. retry
<kim0> - copy to local machine
<kim0> - upgrade
<kim0> I wish debug-hooks would just be connected to local hooks dir some how
<niemeyer> kim0: Hmm.. that's a bit strange.. why?
<niemeyer> kim0: Oh, for copying?
<niemeyer> kim0: Remote => local?
<kim0> and then upgrading 
<niemeyer> Ok, interesting
<niemeyer> ahasenack, sidnei, zaid_h, wrtp: Hi all!
<sidnei> heya niemeyer
<zaid_h> o/
<wrtp> hi niemeyer
<rockstar> Is ensemble going to be put up on pypi?
<niemeyer> rockstar: Hmm
<niemeyer> rockstar: I hadn't thought of that
<niemeyer> rockstar: I guess it's doable, but I'm not sure about what's the real benefit when compared to having the PPA up
<niemeyer> rockstar: What's your thinking?
<rockstar> niemeyer, well, you want people other than Ubuntu folk using Ensemble, right?
<rockstar> It's kind of odd to say "you can use whatever language you want to write Ensemble formulas as long as you use our OS"  :)
<niemeyer> rockstar: That sounds good for sure, but our current focus is on Ubuntu obviously
<rockstar> niemeyer, yeah, for deployment, it's a no-brainer.
<niemeyer> rockstar: Ah, I see what you mean
<rockstar> But for the ensemble controller, it'd be nice to use what you want.
<niemeyer> rockstar: Yeah, that sounds nice indeed
<rockstar> The reason I ask is that I have people at the local LUG asking if I could give a demo at next month's meeting, so I wanted to know about usage.
<niemeyer> rockstar: Cool
<rockstar> niemeyer, I'd be happy to submit patches for any bugs I find on other OSes.
<niemeyer> rockstar: It shouldn't be hard, actually
<niemeyer> rockstar: Ensemble is really just a Python library at this point, with a tiny executable wrapper
<rockstar> niemeyer, it seems relatively ready for pypi, that's why I asked.
<niemeyer> rockstar: The dependencies are likely the boring part, though
<rockstar> It didn't seem like a technical barrier, so I wondered if there were principle issues.
<niemeyer> rockstar: I mean, I'm not sure python-zookeeper exists there, for instance
<rockstar> niemeyer, there's txzookeeper
<rockstar> I don't know if that's the same thing.
<niemeyer> rockstar: Yeah, but that's an easier one
<niemeyer> rockstar: That's a Python-only wrapper for Twisted
<niemeyer> rockstar: It still needs python-zookeeper on the backend
<rockstar> Ah.
<rockstar> I know txAWS is.
<niemeyer> rockstar: That's helpful already, and it's "the real thing"
<jimbaker`> rockstar, fwiw, i used to run the entire test suite on mac os - there was only one minor bug, around a small diff in how temp files are named
<jimbaker`> i didn't really look into it, but i  believe it was purely a problem in the test
<kim0> hazmat: Hi .. are you there 
 * niemeyer heads to lunch
<kim0> hazmat: just pinging you to check the bug I had reported earlier today, ping me if you have any questions
<hazmat> kim0, yeah
 * hazmat pokes
<kim0> cool 
<_mup_> ensemble/resolved-install-should-start r240 committed by kapil.thangavelu@canonical.com
<_mup_> on retry install (with or without hooks) use a success transition id to automatically transition to started.
<hazmat> kim0, is there a bug reported for the issue you mentioned earlier?
<_mup_> Bug #794129 was filed: install hook error, does not transition to start after resolved <Ensemble:New> < https://launchpad.net/bugs/794129 >
<hazmat> there is now ;-)
<kim0> hazmat: cool :) yeah I hadnt opened one
<hazmat> kim0, cool, the fix is in the review queue now
<kim0> hazmat: oh that fast! awesome :)
<hazmat> kim0, TDD ftw
<kim0> niemeyer: Just linked a rewrite of the formula writer's tutorial to bug 788255
<_mup_> Bug #788255: Ensemble needs a tutorial for writing a formula <Ensemble:In Progress by kim0> < https://launchpad.net/bugs/788255 >
<kim0> would be great if you could review it soonish, since I think I'll need to refer to it soon in a blog post
<kim0> thanks a lot
 * hazmat admires steve job's reality distortion field
<hazmat> the truth is in the cloud.. the cloud, it just works..
 * niemeyer waves
<niemeyer> hazmat: Yeah :-)
<niemeyer> Cloud just got 30% more hyped
<SpamapS> So.. plugins.. agents..
<hazmat> SpamapS, plugins.. cli or agent?
<SpamapS> I've been thinking about how mcollective works.
<SpamapS> And how easy that would be to add to ensemble.
<hazmat> SpamapS, agent side plugins then
<SpamapS> yes
<SpamapS> ensemble agent-plugin --service=wki-db --plugin=mysql-cmd "FLUSH LOGS"
<hazmat> SpamapS, that would be nice, but we'd have to some dynamic dispatch to allow ready overriding of behavior for fully pluggable.. else we can manage them as some sort of collection of objects with lifecycle, which work if their mostly independent from existing functionality and using a watch based protocol
<SpamapS> I mean, we have an agent, with an event driven comm channel to it. We should be able to use it to run stuff. :)
<SpamapS> hazmat: again, eep op glorp glok.. I am sorry that I don't understand the internals of ensemble better by now. How you do it is very interesting to me, but even more is, how feasible is it?
<hazmat> SpamapS, its quite feasible.. we have most of the building blocks available, we could isolate the plugins to separate processes if we want.
<SpamapS> We could just say nah we're not going to do that, but then we need to help people setup mcollective or parallel-ssh or something to do these things.
<niemeyer> SpamapS: We have to look at a few different use cases to see what would be the right thing to do
<niemeyer> SpamapS: Flushing logs, for instance, sounds like something we'd want in the agent itself default
<niemeyer> s/default/by default/
<hazmat> niemeyer, then it would need to be formula driven.. the agent is agnostic
<niemeyer> hazmat: Log management sounds like a pretty general problem
<hazmat> oh.. i guess a generic log flush is feasible
<SpamapS> niemeyer: flushing the mysql logs?
<hazmat> niemeyer, indeed, definitely also puts us pretty close into policy land in the agents
<niemeyer> SpamapS: Flushing the X logs
<niemeyer> SpamapS: Where X is whatever is running in the machine
<hazmat> where do the logs go, what log values do we relay. do we have an alert/event engine feeding on the logs.. etc
<SpamapS> We can't possibly know everything that every formula will want to expose to users on an arbitrary basis. In cassandra, there's DB compaction, log flushing, offlining for upgrades..
<SpamapS> I do like the idea of being able to provide arbitrary hooks in formulas
<niemeyer> SpamapS: DB compaction is formula business.. log flushing is a generic problem we can tackle on the agent.. offlining for upgrade is formula business to
<niemeyer> s/to/too
<SpamapS> niemeyer: DB compaction happens when the admin wants to do it for Cassandra... 
<SpamapS> how would formulas do that now?
<niemeyer> SpamapS: Sounds like a setting
<hazmat> SpamapS, cron job from setting
<SpamapS> no no no no cron!
<niemeyer> SpamapS: ensemble set run-compaction=true
<hazmat> :-)
<SpamapS> niemeyer: THAT I like
<hazmat> yeah.. cassandra has lots of knobs that it wants to expose the admin, that need to be turned in the right order
<SpamapS> hazmat: while standing on the right foot
<hazmat> niemeyer, with the flag being cleared post  compaction?
<SpamapS> Cassandra and Jenkins are both going to need a lot of the same knobs for the JVM .. I've been wondering about shared code.
<niemeyer> hazmat: Yeah, possibly
<SpamapS> Ultimately I'd want feedback from that setting.
<SpamapS> not necessarily from the 'set' command, but somehow I need to be able to know that it ran and failed/succeeded
<hazmat> SpamapS, i think we need some notion of hook inheritance.. it more tightly couples the formula namespace and inter formula deps.. but i dislike pushing down every hook into all formulas
<niemeyer> hazmat: It sounds like we can introduce a specific type of setting for that kind of flag 
<niemeyer> SpamapS: Agreed
<niemeyer> SpamapS: What kind of feedback?
<hazmat> niemeyer, perhaps.. alternatively we have some sort of service agent (not unit) that can operate on a wholistic service view
<niemeyer> hazmat:  Agreed, except s/sinheritance/composition/
<hazmat> ie. do round robin upgrades, rolling upgrades, clear the compact flag after its done on all units
<niemeyer> hazmat: Hooks are scripts, it's trivial to bundle a package with things for well known activities
<hazmat> niemeyer, indeed, it is composition
<SpamapS> Yeah I was thinking that a simple ensemble-formula-helpers package would do nicely
<SpamapS> think debhelper. :)
<niemeyer> hazmat: Rather than clearing the flag, it feels like it should be a timestamp based setting
<hazmat> its convience and reuse, one hook copy to edit, vs. 30 if your upgrading/changing a  behavior
<niemeyer> hazmat: Something resembling this: ensemble set ... compact=now
<hazmat> niemeyer, sounds good re set
<niemeyer> hazmat: So each unit can observe the setting, and tell if it should compact or not
<niemeyer> hazmat: No races, etc
<niemeyer> no need to reset too
<SpamapS> like just have the formula's setting hook call /usr/share/ensemble/helpers/hooks/jvm_settings .. 
<hazmat> niemeyer, if we don't reset to reflect current state, then we have a temporal disconnect for a unit coming back online or new unit.. that wouldn't nesc. need compaction
<niemeyer> SpamapS: Yeah!  Man, it's neat that this would just work..
<niemeyer> SpamapS: I mean, in the sense that the helper will have the environment
<SpamapS> yep
<niemeyer> hazmat: sorry, I actually meant compact=`date`
<SpamapS> keeping it simple and clear means more people will make use of them
<niemeyer> hazmat: Not with that syntax, of course, but that's the idea
<hazmat> niemeyer, cool, but even then you need coordination by something external to the units
<niemeyer> hazmat: Hm?
<hazmat> you don't want to compact on your cassandra cluster in parallel
<hazmat> you do it piecemeal because your putting an additional load on your data store
<niemeyer> hazmat: That's up to the formula to coordinate internally.. that's the kind of thing we have peer relations for
<hazmat> they can either coordinate or be coordinate by an external actor
<SpamapS> as long as hooks can change the settings they've been given, that should be fine
<hazmat> niemeyer, true
<niemeyer> hazmat: We can help, of course, but as far as the "how do I tell things should happen", the setting system should be a good start
<_mup_> ensemble/expose-provision-service-hierarchy r266 committed by jim.baker@canonical.com
<_mup_> Need more robust ending assertions that don't eventually fail with pending deferreds, but non edge case code now works properly with new stateful semantics for watch_exposed_flag
<niemeyer> SpamapS: It doesn't have to change the _settings_, I think we shouldn't allow it to, but there are ways it can store and relay information (peer relations, etc)
<SpamapS> One thing that mcollective has that this wouldn't give me, that sysadmins *love* is that the output of the agents can be structured and piped so admins can do their grep/sed/awk magic with the data returned.
<niemeyer> SpamapS: The idea so far is to reserve the settings framework for human interaction
<SpamapS> niemeyer: the run-compaction=1 thing would then run compaction on adding unit though, would it not?
<niemeyer> SpamapS: Yeah, we should add an easier way as discussed with hazmat above.. we could have something equivalent to run-compaction=`date` 
<niemeyer> SpamapS: So units can compare the timestamp
<niemeyer> SpamapS: and tell if they're supposed to compact or not
<SpamapS> sounds like we need a "message to the unit" interface not a "set a setting" interface
<niemeyer> SpamapS: This also solves a number of other issues.. e.g. which units have already compacted?  What if I ask to compact again?  etc
<niemeyer> SpamapS: "set a setting" is effectively a "message to the unit" interface
<niemeyer> SpamapS: With the advantage that messages are not lost
<niemeyer> SpamapS: Nor duplicated, etc
<SpamapS> that can make it more complex too since sometimes you mean "send it to the service" and sometimes you mean "send it to the units that exist now"
<niemeyer> SpamapS: "the service" in this sense is an abstract concept
<niemeyer> SpamapS: Communication will always take place with the underlying units
<niemeyer> SpamapS: I get your point, though, I think.. one has to understand how it works
<niemeyer> SpamapS: That said, this would likely be true either way..
<niemeyer> SpamapS: E.g. how to tell all units they have to compact, but they should do so one at a time, etc
<niemeyer> SpamapS: Whatever system we invent for facilitating this will still need some understanding of the issue at hand by the formula author
<SpamapS> definitely.. I'm sure it will remain as simple as the other bits of the formula too. :)
<niemeyer> SpamapS: Yeah, hopefully we'll evolve to that :)
<niemeyer> We have a _gigantic_ review queue on Ensemble..
<niemeyer> That's _awesome_.. :-)
<niemeyer> Just finishing some Go packaging work to upgrade to the latest weekly, and will do a full round.
<_mup_> ensemble/trunk-merge r223 committed by kapil.thangavelu@canonical.com
<_mup_> merge trunk
<_mup_> ensemble/auto-dependency-spec r224 committed by kapil.thangavelu@canonical.com
<_mup_> merge auto dep resolution spec
<_mup_> ensemble/trunk r241 committed by kapil.thangavelu@canonical.com
<_mup_> merge auto-dependency-resolution prototype [r=niemeyer][f=777816]
<_mup_> An auto resolution algorithm implementation for deployment of
<_mup_> dependent formulas and relations with constraints satisified from both
<_mup_> repository and environment.
<_mup_> ensemble/trunk r242 committed by kapil.thangavelu@canonical.com
<_mup_> merge ensemble-deploy-auto-resolve prototype [r=niemeyer][f=788735]
<_mup_> Integration of auto dependency resolution of formulas and relations 
<_mup_> into the deploy cli.
<_mup_> ensemble/trunk r243 committed by kapil.thangavelu@canonical.com
<_mup_> remove auto dependency resolution functionality, pending repository work, left in history for future ref.
<_mup_> ensemble/expose-provision-service-hierarchy r267 committed by jim.baker@canonical.com
<_mup_> Ensure test_watche_services does not tear down until exposed service watches have been observed to raise StopWatcher
<hazmat> niemeyer, bcsaller, jimbaker` standup?
<niemeyer> Do we need one today?
<jimbaker`> hazmat, sure
<niemeyer> I'm a bit behind myself, and feel slow in the last couple of days
<hazmat> niemeyer, i'm good without
<jimbaker`> no need on my part - i'm just reworking the timing issues introduced by the new stateful api for watch_exposed_flag
<hazmat> jimbaker`, bcsaller any blockers or other stuff you guys want to talk about today?
<niemeyer> If there are no important topics to sync upon/figure out, I'd prefer to keep hacking/reviewing/etc
<jimbaker`> was slow going this morning, but now i'm making good progress
<jimbaker`> in fact, excellent progress, so would like to focus on it
<hazmat> yeah.. me too
<bcsaller> yeah, I just need to finish what I'm working on
<hazmat> all right.. tomorrow then
<niemeyer> Super
<_mup_> ensemble/expose-provision-service-hierarchy r268 committed by jim.baker@canonical.com
<_mup_> Looping on all tests except test_watch_service_changes_removed_services, which needs to take in account new semantics
<_mup_> ensemble/expose-provision-service-hierarchy r269 committed by jim.baker@canonical.com
<_mup_> Fixed last test for new semantics
<_mup_> ensemble/security-specification r240 committed by kapil.thangavelu@canonical.com
<_mup_> security overview and todo spec draft
<niemeyer> Okay.. packages out of the way.. will step out for a moment and come back to that review queue
<niemeyer> hazmat: What was the outcome of our debate about the db-relation-joined issue on mysql again?
<niemeyer> hazmat: The one re. the fact it wasn't updating on re-joins
<hazmat> niemeyer, that the formula hook should check if credentials are already set on its relation, and if not it should create them, even if the db exists
<niemeyer> hazmat: Ah, with the relation id thing, ok
<niemeyer> hazmat: Thanks
<hazmat> niemeyer, np
<_mup_> ensemble/trunk r244 committed by gustavo@niemeyer.net
<_mup_> Merged fix-mysql-example-formula-publish-details-precreated-databases branch
<_mup_> [a=kim0][r=niemeyer]
<_mup_> This enables reusing the same example mysql formula multiple times
<_mup_> when deployed.
<_mup_> The proper way to fix this is actually to use a database named after the
<_mup_> relation id, and then create a new database on every new relation id we
<_mup_> get.  The problem is that we currently don't offer such a relation id for
<_mup_> the hooks, so this will do for now.
<SpamapS> re the formulas re-creating credentials.. the principia mysql formula touches a flag file when the relation is broken, and revokes the credentials at that time. When the flag is present, or database is missing, it creates new credentials.
<SpamapS> It has an additional check if its a slave since the db will be present but the credentials will be wrong in the db and need updating.
<_mup_> ensemble/trunk r245 committed by gustavo@niemeyer.net
<_mup_> Merged micro-is-sluggish branch [r=hazmat]
<_mup_> This changes the default instance type on EC2 to m1.small.
<_mup_> ensemble/expose-provision-service-hierarchy r271 committed by jim.baker@canonical.com
<_mup_> Additional cleanup re removing unnecessary observers
<_mup_> ensemble/trunk r246 committed by gustavo@niemeyer.net
<_mup_> Merge write-formula-tutorial branch [a=kim0] [r=niemeyer]
<_mup_> Yes, a formula writing tutorial!
<kim0> _mup_: are you being sarcastic :D
<niemeyer> kim0: Not at all.. :-)
<niemeyer> kim0: Something important we were lacking
<kim0> hehe  cool then :)
<_mup_> ensemble/config-set r240 committed by bcsaller@gmail.com
<_mup_> merge trunk
#ubuntu-ensemble 2011-06-08
<kim0> Morning everyone
 * hazmat gives up on unity
<hazmat> are there any union or overlay filesystems for which you can capture a useable delta against the base fs
<niemeyer> Hey Ensemblers!
<kim0> hazmat: aufs ?
<_mup_> ensemble/update-install-docs r242 committed by kapil.thangavelu@canonical.com
<_mup_> factor out source installation into a developer-install doc
<hazmat>  kim0 i don't think any of them real expose deltas is a particular useful way.. aufs has some  internal files it uses for inode pointers, which  it looks like which make it conceivable but potentially brittle
<hazmat> s/real/really
<kim0> zfs snapshots with 'zfs send' ? :)
<hazmat> kim0, hmm.. not sure if thats appropriate, there are some nice time slider interfaces for zfs auto snapshot integrated into nautilus but snapshot browsing is a pretty easy.. what i was wondering about was some sort of preview mode against a hook, that could take a snapshot of the fs, and the running processes, before the hook, compared to an after hook environment, that could be presented to an end user as the actions of a hook
<hazmat> along with the delta of changed relation data
<kim0> system wide diff
<kim0> pff
<kim0> hazmat: and it'd have to work on regular filesystems too right? (ext3/4?)
<hazmat> kim0, indeed, and its not clear to me if that's either realistic or useful by itself, the process changes is rather difficult to capture meaningfully, if there's an internal change in a process state..
<hazmat> just brainstorming on what dry run might look like
<kim0> Yeah
<kim0> perhaps it might be simpler to try hooking runs of service/cp/mv ?
<kim0> but that list is never complete
<hazmat> kim0, indeed.. some hooks might not use shell as well, but an arbitrary binary (formula in go? ;-) we'd have to hook syscalls and presenting that an intelligible diff to an admin might be a challenge
<kim0> hazmat: or perhaps another way to look at it, might be to request prepending commands with a sudo like prefix, just like (svn mv, svn rm) ...etc Instead of allowing running arbitrary executables .. pros and cons of course
<hazmat> kim0, its also unclear since its a dynamic system, if such a preview is truly reliable.. a remote state change could change the local behavior from the preview at any time.
<kim0> yeah, it's a hard problem :)
<_mup_> ensemble/trunk r247 committed by kapil.thangavelu@canonical.com
<_mup_> merge resolved-install-should-start [r=niemeyer][f=794129]
<_mup_> Resolving an install_error should automatically attempt a transition to start
<_mup_> if the install error is resolved successfully.
<niemeyer> Nice review queue again
<_mup_> ensemble/expose-provision-service-hierarchy r272 committed by jim.baker@canonical.com
<_mup_> More corner cases dealing with concurrently removed services
<jimbaker`> niemeyer, could you take a look at expose-hook-commands in review - it's independent of the two other branches i'm working on
<niemeyer> jimbaker`: Yeah, will clear it up today
<jimbaker`> i also just adjusted expose-provision-service-hierarchy to indicate it's WIP w/ the changes in expose-watch-exposed-flag, should have done that earlier
<jimbaker`> the good thing about that is that the move to the stateful api did in fact unexpose some subtle timing bugs
<niemeyer> jimbaker`: Interesting
<niemeyer> I'll get some lunch
<_mup_> ensemble/expose-provision-service-hierarchy r273 committed by jim.baker@canonical.com
<_mup_> test_service_unit_watching now loops properly
<kim0> niemeyer: hey, let's do the sync'up meeting in 90mins
<kim0> Another thing .. I'll need some help from the team to identify bitesized bug
<kim0> Basically, I'll start a campaign to go out and get people interested in contributing to Ensemble. For that, we'd probably need a list of bitesized bugs that we can point people to
<kim0> This has worked well for Unity, so hopefully for us as well
<kim0> so we'd need a tag a few bugs as "bitesized". Or maybe file new bugs if appropriate. This can be easy to fix bugs, or nice to haves that are not too uegent for the team and are low barrier for a newcomer
<kim0> if we can have 10-15 bugs tagged/created, that'd be awesome
<kim0> let me know what you guys think
<kim0> I'll probably file some new bugs for important formulas I'd like to see written, and I'll pimp them to the respective communities. But where I need the help most, is code bugs (improve test coverage, fix a small bug ..etc)
<_mup_> ensemble/expose-provision-service-hierarchy r274 committed by jim.baker@canonical.com
<_mup_> Remaining tests now also loop
<jimbaker`> kim0, meeting on #ubuntu-cloud in 30 minutes, right?
<kim0> jimbaker`: yeah
<jimbaker`> kim0, sounds good
<_mup_> ensemble/set-transitions r230 committed by bcsaller@gmail.com
<_mup_> merge forward
<kim0> uno, dos, tres
<_mup_> ensemble/expose-provision-service-hierarchy r275 committed by jim.baker@canonical.com
<_mup_> Assert expected log lines
<kim0> niemeyer: hazmat bcsaller jimbaker`: anyone up for the meeting 
<kim0> #Ubuntu-Cloud please
<niemeyer> kim0: Here, and there
<kim0> cool
<jimbaker`> kim0, over on #ubuntu-cloud, waiting for the meeting to start :)
<niemeyer> jimbaker`: The meeting ended..
<jimbaker`> niemeyer, yes
<niemeyer> jimbaker`: I though you were going to say something?
<kim0> koolhead17: hey o/
<koolhead17> kim0: hey man.
<jimbaker`> niemeyer, didn't really have anything to add to what was said about the progress on expose
<kim0> koolhead17: how's it going buddy
<koolhead17> just happy happy :)
<kim0> koolhead17: haha :)
<jimbaker`> there are some interesting details about ensemble internals, but not germane for that particular meeting i think
<koolhead17> kim0: and yes puppet CTO follows all the tweets wit hashtag puppet :D
<kim0> koolhead17: that would make sense :)
<kim0> koolhead17: working on some cool stuff lately ?
<hazmat> jimbaker`, perhaps for the standup then..
<koolhead17> kim0: yes. KVM and CFengine
<koolhead17> :D
<jimbaker`> hazmat, of course
<hazmat> bcsaller, niemeyer, jimbaker` standup time?
<niemeyer> Yep, let's go
<koolhead17> hazmat: niemeyer jimbaker` hello all :P
<jimbaker`> niemeyer, hazmat sounds good
 * koolhead17 wants to meet someone from ensemble team in berlin :)
<bcsaller> in mumble
<kim0> koolhead17: so what exactly are you doing with those I'm interested :)
<koolhead17> kim0: in KVM learning the networking modes and CFengine just started and finding very confusing. 
<koolhead17> you were right about CFengine
<koolhead17> :P
<kim0> koolhead17: haha :) Yeah that I'm sure about .. the oldest and cruftiest afaik
<kim0> koolhead17: man you should try ensemble :P We have a formula authors tutorial now Yaay https://ensemble.ubuntu.com/docs/write-formula.html
<koolhead17> i might say adious to it tomorrow  after spending some more hours. there is no ubuntu community documentation 4 CFengine and Puppet :(
 * koolhead17 clicks
 * kim0 joins real life .. back in an hour or two
<niemeyer> hazmat: Would you mind to lower down the mic volume a little bit?
<niemeyer> bcsaller: Re. [16], on this point:
<niemeyer> """
<niemeyer> While I made these changes its worth noting that they are not exactly
<niemeyer> the same as a missing config.yaml means the service takes no options.
<niemeyer> So, for example, file not found isn't considered an error.
<niemeyer> """
<niemeyer> bcsaller: This distinction is in usage of the interface, not on the interface itself
<niemeyer> bcsaller: I totally agree with your understanding of what the user should see when the file is missing, though
<bcsaller> gustavo: ok
<niemeyer> bcsaller: Our disagreement is just that the API of Metadata and Config should be equivalent, despite those details
<niemeyer> bcsaller: re [19], in many cases the parenthesis were simply NOOP
<niemeyer> bcsaller: When it's used for wrapping, there are no issues
<niemeyer> bcsaller: E.g.
<niemeyer> > +                    "%s is not a valid configuration option." % (option))
<niemeyer> bcsaller: This is a bit like saying (1) + (2), etc
<bcsaller> yeah, some got moved around and then not removed after not being needed for formatting
<niemeyer> bcsaller: On [22], the point raised was mostly about the ordering of arguments
<bcsaller> I was explaining the history, I got your point :)
<niemeyer> bcsaller: Not how regex_validator takes exactly the same arguments of match
<niemeyer> bcsaller: Ah, ok, I'm the one who misunderstood then, sorry
<niemeyer> bcsaller: I don't see how the change would keep the validation API, given that every other function takes a single argument and went through a different path entirely
<niemeyer> bcsaller: But now that things have changed, this is a moot point anyway
<bcsaller> yeah, that wasn't the plan orig :)
<niemeyer> bcsaller: The API is still not matching.. not sure if I have the latest work?
<bcsaller> I'll check it again, there should be one code path 
<niemeyer> bcsaller: It still has "optional", no parse_serialization_data, parse is taking different kinds of data, etc
<niemeyer> bcsaller: http://pastebin.ubuntu.com/622012/
<niemeyer> bcsaller: This is the relevant part of MetaData
<niemeyer> bcsaller: ConfigOptions is doing exactly the same thing in all of those cases.  It feels like a good idea to have the same API on both.
<bcsaller> I didn't know you needed that level of parity, I can work on changing it
<niemeyer> bcsaller: There are two kinds of file.  Both of them are within the formula.  They both require the same kind of API from the application.  Why would these two APIs be different?
<_mup_> ensemble/security-specification r241 committed by kapil.thangavelu@canonical.com
<_mup_> additional escalation scenarios, outline next steps broadly
<bcsaller> you're right, I was focused on the wrong things
<niemeyer> bcsaller: The watch_config has exactly the same issues re. exists_d than what we're discussing with jim
<niemeyer> bcsaller: Let's just move it on, though
<niemeyer> hazmat: One more for us to look at ^
<niemeyer> bcsaller: It's looking good.. [16] is the only mildly boring issue to look at
<niemeyer> bcsaller: I've posted a few other comments in the merge proposal, mostly stylistic, though
<bcsaller> ok, great
<niemeyer> bcsaller: It's approved too, thanks for pushing it through
<bcsaller> I'll give it another pass and then push it 
<niemeyer> bcsaller: Just sort that API issue first, please, and it's good to go.
<bcsaller> and move the next branch into review
<bcsaller> yes sir :)
<bcsaller> thanks for the review 
 * niemeyer quickly finds a dictator's hat to wear
<niemeyer> bcsaller: np, sorry for the trouble :)
<SpamapS> hmm
<SpamapS> starting to package txzookeeper
<SpamapS> mocker.py should be dropped and a dependency on mocker added..
<niemeyer> SpamapS: mocker was meant to be embedded in projects
<niemeyer> SpamapS: So this is fine
<niemeyer> SpamapS: It's not polluting the global space, and is only for testing
<SpamapS> If its not generated, it will likely get rejected by an overzealous archive admin. :-/
<niemeyer> SpamapS: The mocker upstream explicitly designed mocker for it to be embedded in projects.
<SpamapS> Yeah, thats a grey area. Since its only a build time thing, I'll cross my fingers.
<niemeyer> SpamapS: It's used for testing, and tests shouldn't ever break if mocker changes.
<niemeyer> SpamapS: It's a well considered thing in this case
<SpamapS> Debian policy doesn't really take into account upstream's intentions. ;)
<niemeyer> SpamapS: I was actually against even having a mocker package, but since the Ubuntu One guys (or Launchpad, maybe) wanted very much, I couldn't really see a reason to prevent it
<SpamapS> Ooo
<SpamapS> they changed the language
<SpamapS> I digress!
<SpamapS> It used to say must not
<SpamapS> "Debian packages should not make use of these convenience copies unless the included package is explicitly intended to be used in this way."
<niemeyer> "explicitly intended"!
<SpamapS> Yeah, that used to say must not use them, and it didn't take into account those intentions. I think.
<SpamapS> That or I was just being overly excited about something silly. (latter far more likely)
<niemeyer> SpamapS: I understand the perspective.. the issue in this case is that it'll be really bad if mocker changes and tests are broken because of that
<niemeyer> SpamapS: Tests are supposed to verify the application.. if the test framework breaks/changes, the whole purpose of the system is gone
<SpamapS> Yeah, and there's no sane reason to be using the test framework in two projects at the same time.
<SpamapS> That said, the mocker.py in txzookeeper is missing a license.
<SpamapS> Will consider it licensed under MIT since the setup.py licenses the whole thing as such.
<hazmat> hmm.. that should get updated to the released one with license info, i might have pulled that from landscape when i was first working on txzk
<hazmat> the mocker module that is
<niemeyer> SpamapS: trunk actually has one
<niemeyer> SpamapS: It's BSD, IIRC
<SpamapS> right
<SpamapS> I'm comfortable submitting it with a blanket MIT license because the BSD and MIT licenses allow re-licensing and the same basic rights.
<SpamapS> If you guys update it we can change the debian/copyright file then
<niemeyer> SpamapS: The license says the text should be preserved, but I won't fight over that. :-)
<niemeyer> SpamapS: I assure you the upstream will never go after you on that. ;-)
<SpamapS> Since the file as distributed had no such text.. we're safe. :)
<hazmat> its still technically a copyright violation if that was yanked
 * SpamapS decides he may as well re-read the license portion of the policy manual again to see if his memory is dead wrong on this too. :-P
<SpamapS> hazmat: I believe at the time you received the file, it had no license. ;)
<hazmat> actually that's still in there
<hazmat> SpamapS, i belive, your belief reflects a statement that in the scope of the multiverse, could be known as reality 
<hazmat> ;-)
<_mup_> ensemble/bootstrap-shutdown-environment r248 committed by kapil.thangavelu@canonical.com
<_mup_> bootstrap operates on only one environment, shutdown next.
<SpamapS> I'm reading the manual because this situation has come up so many times.. it has to have a better resolution than "bug the upstream to add a copyright and license header to everything"
<niemeyer> SpamapS: There isn't one, likely.. the problem is that something without a license defaults to being proprietary
<hazmat> SpamapS, that's an acceptable answer afaics, i can add it in
<niemeyer> Luckily, in this case, we have good access to the upstream, and nothing like that is necessary. :-)
<SpamapS> "Note that under international copyright law (this applies in the United States, too), no distribution or modification of a work is allowed without an explicit notice saying so. Therefore a program without a copyright notice is copyrighted and you may not do anything to it without risking being sued! Likewise if a program has a copyright notice but no statement saying what is permitted then nothing is permitted."
<SpamapS> http://www.debian.org/doc/debian-policy/ch-archive.html#s-pkgcopyright
<hazmat> there is an existing copyright notice
<hazmat> at least in the mocker.py of txzk
<SpamapS> explicit notice saying so.. I'd say Kapil's debian/copyright file is sufficient ;)
<_mup_> txzookeeper/mocker-license r39 committed by kapil.foss@gmail.com
<_mup_> update mocker to latest release, which includes license header
<SpamapS> hazmat: copyright yes. license, no.
<SpamapS> But
<SpamapS> I think the debian/copyright file you put in is technically a blanket license
<hazmat> niemeyer, can i get a +1 on this trivial http://pastebin.ubuntu.com/622062/
<niemeyer> hazmat: +1!
<hazmat> SpamapS, its not clear how that's possible on aggregate work.. but i guess deb frowns on those
<SpamapS> the question is, did you have a right to re-license Gustavo's work... the answer is of course yes, but I need to have evidence of that to put in the debian/copyright file.
<niemeyer> SpamapS: I think debian/copyright is something else than what we're debating about
 * SpamapS has had 2 uploads to Debian rejected for these reasons.
<niemeyer> SpamapS: If not, I'd be happy to learn
<SpamapS> well there are two copyright files in the discussion
<_mup_> txzookeeper/trunk r39 committed by kapil.foss@gmail.com
<_mup_> [trivial] merge mocker-license which updates mocker to 1.1 which includes a license header for packaging [r=niemeyer]
<hazmat> SpamapS, hopefully that helps
<niemeyer> SpamapS: Isn't debian/copyright the _package_ copyright (debian/*)?
<SpamapS> there's the one Kapil put in txzookeeper's trunk, which serves as a blanket MIT license for the rest of the code...
<SpamapS> and then there's the one I'm tidying up for upload to Debian
<SpamapS> The former is just a reference work at this point. The latter is used by the Debian and Ubuntu projects as an assurance to their users that they can in fact use the software.
<hazmat> off to dinner, back in a bit
<SpamapS> hazmat: thanks, that may be enough, I'm going to ask around a bit
<SpamapS> I think I've spent way too much time on this, since the new trunk has the licensed code.. the matter is pretty much resolved
<niemeyer> jimbaker`: expose-hook-commands looks very nice and tiny, thanks
<SpamapS> ./txzookeeper/tests/mocker.py: BSD (3 clause) 
<niemeyer> tidy
<SpamapS> yay
<jimbaker`> niemeyer, thanks
<SpamapS> Can we make a tarball release of txzookeeper?
<SpamapS> actually
<SpamapS> it seems that packaging from VCS is totally legit
<SpamapS> w00t
#ubuntu-ensemble 2011-06-09
<hazmat> SpamapS, i've got releases out on pypi for it (needs to be updated).. but else its legit
<_mup_> ensemble/config-set r241 committed by bcsaller@gmail.com
<_mup_> last review changes pending merge
<_mup_> ensemble/config-set r242 committed by bcsaller@gmail.com
<_mup_> merge trunk
<_mup_> ensemble/trunk r248 committed by bcsaller@gmail.com
<_mup_> Merge config-set [r=niemeyer] [f=778685]
<_mup_> Provide an `ensemble set` command. 
<_mup_> Allow formula to contain an options specification as a YAML file
<_mup_> Validate user input relative to options configuration
<_mup_> Extend formula state to include config options
<_mup_> ensemble/config-example-changes r232 committed by bcsaller@gmail.com
<_mup_> forward merge (trunk)
<kim0> Morning everyone
<kim0> Ensemble is written in python, uses twisted, txAWS and zookeeper
<kim0> did I miss any core technology ensemble uses ?
<hazmat> kim0, that's the run of it.. we also use yaml for config
<hazmat> kim0, txaws/ec2 are part of a pluggable machine provider architecture
<hazmat> though till we support additional providers.. its a debateable if thats claimable
<kim0> hazmat: cool thanks
 * niemeyer waves
<_mup_> ensemble/bootstrap-shutdown-environment r249 committed by kapil.thangavelu@canonical.com
<_mup_> shutdown operates and on only one environment, and prompts for confirmation.
<_mup_> ensemble/trunk r249 committed by kapil.thangavelu@canonical.com
<_mup_> merge update-install-docs [r=niemeyer][f=788950]
<_mup_> Updated install docs referencing the PPA as the preferred installation method.
<kim0> hazmat: shouldn't 792540 be fix released ? my branch was merged
<hazmat> kim0, oh.. wasn't sure.. the review looked it was approved, it wasn't clear that it was merged.. if its merge, feel free to update the status
<hazmat> ^like
<kim0> um
<kim0> I might be the one imagining things
<hazmat> kim0, hmm.. actually i might be.. odd 
<kim0> hazmat: Merged into lp:ensemble at revision 244
<kim0> that means it was really merged right ? :)
<hazmat> kim0, on the merge proposal it notes that its merged, on the bug report it just says its approved
<hazmat> odd indeed
<hazmat> oh.. that's just me.. being odd its correct
<kim0> okie dokie
<hazmat> kim0, updated to fixed released
<kim0> cool
<_mup_> ensemble/expose-provision-service-hierarchy r277 committed by jim.baker@canonical.com
<_mup_> Removed unnecessary workaround for agent restart scenario
<hazmat> lp seems rather slow today
<_mup_> ensemble/expose-provision-service-hierarchy r278 committed by jim.baker@canonical.com
<_mup_> PEP8
<_mup_> ensemble/expose-provision-service-hierarchy r279 committed by jim.baker@canonical.com
<_mup_> Observer cleanup
<hazmat> lunch bbiab
<_mup_> ensemble/expose-provision-service-hierarchy r280 committed by jim.baker@canonical.com
<_mup_> Relaxed assertion on teardown cleanup (need to make watches more deterministic...)
<niemeyer> hazmat: Initial review of sec-spec delivered.. nice stuff there
 * niemeyer => lunch too
<SpamapS> hazmat: sweet!
<SpamapS> hazmat: if you are going to be releating on pypi then I can add a watch file .. which would be nice.
<SpamapS> releasing even
 * SpamapS waits patiently for the caffeine to kick in
<_mup_> ensemble/expose-watch-exposed-flag r244 committed by jim.baker@canonical.com
<_mup_> Doc string for watch_exposed_flag, removed obsolete version of function
<_mup_> ensemble/expose-provision-service-hierarchy r281 committed by jim.baker@canonical.com
<_mup_> Merged upstream expose-watch-exposed-flag
<hazmat> SpamapS, there's an old release on pypi atm, after i finish up the session event handling work, i'll push an updated release there
<hazmat> SpamapS, there was some talk a few months back about sending the whole project upstream to zk contrib
<hazmat> i've got a little wary of that in the interim.. its just rather nice to be able to fix things as needed, than push to the upstream bug tracker and wait for a committer to respond on it
<SpamapS> hazmat: that would be cool, since thats where python-zookeeper lives too right?
<hazmat> SpamapS, it does indeed
<SpamapS> I think until its been stable and unchanged for some time.. you're right in keeping it
<SpamapS> yesterday's license fix is an example of the kind of thing that makes me want you to keep it forever. ;)
<SpamapS> thanks again btw. :)
<hazmat> :-) np
<_mup_> ensemble/expose-hook-commands r245 committed by jim.baker@canonical.com
<_mup_> Added missing assertion on whether port is an integer in hook commands
<koolhead17> hello all
<_mup_> ensemble/expose-hook-commands r246 committed by jim.baker@canonical.com
<_mup_> Merged trunk and resolved conflicts
<SpamapS> hazmat: FYI, I'm taking the man pages bug .. as its part of what I'm working on today (packaging ensemble for Ubuntu/Debian)
<jimbaker> so what's up with the test failures in trunk - are there any dependency updates i should be aware of, or is that what's covered in the debug-hook-fixes branch?
<jimbaker> i will hold my expose-hook-commands branch for the moment...
<hazmat> SpamapS, cool
<hazmat> jimbaker, what failures on trunk?
<jimbaker> hazmat, one moment for a paste
<hazmat> hmmm.. those are odd, they weren't there yesterday
<jimbaker> http://paste.ubuntu.com/622740/
<jimbaker> hazmat, i just wonder if this is related to what bcsaller was talking about in standup
<hazmat> bcsaller, afaics those errors where introduced by the config-set merge.. before that rev i can do full trunk test runs without error
<hazmat> now i get about 21 errors
<hazmat> jimbaker, yeah.. probably.. i hadn't realized the problem was so widespread though
<bcsaller> hazmat: I cannot, trunk did that for me before the merge 
<bcsaller> run in isolation different sets of those test will pass 
<hazmat> bcsaller, ugh
<bcsaller> thats what I was talking about 
<bcsaller> there is bleeding in some of the setup/teardown
<hazmat> bcsaller, i updated to before the config-set merge, and was able to run all the tests on trunk... this much breakage on full test runs is a problem
<hazmat> bcsaller, yeah... perhaps some environment / global stomping
<hazmat> bcsaller, it looks like a good number of the failures are because the logging config is changed, and expected outputs dont match anymore
<bcsaller> hazmat: hmm, I thought even some of those passed in isolation, have to check that again 
<jimbaker> at this point to prevent bleeding, i'm always running tests w/ -u
<jimbaker> not perfect, but catches some bad assumptions about teardown
<hazmat> so this needs some better resolution and practice than merging a branch which breaks a bunch of tests
 * hazmat pokes at the diff
<bcsaller> hazmat: again, I was seeing this in trunk as well 
<hazmat> bcsaller, ok, but we need to escalate it then.. i think i asked what sort of failures you where seeing on the call yesterday
<hazmat> bcsaller, the diff looks totally innocous 
<hazmat> jimbaker, if you update to before config-set ( bzr up -r 247) and run trunk tests, do you see failures?
<hazmat> ugh.. i really need to be doing reviews as well
<jimbaker> hazmat, i will check that too
<hazmat> okay found a few review items, and the problem
 * hazmat does a full test run to confirm
<hazmat> bcsaller, when you say you had the same problems on trunk... you mean you had the same failures on full test runs.. are you sure it wasn't that you tried to merge, did a test run, reverted the merge, and did another test run
<hazmat> bcsaller, trial has the unfortunate habbit of not ignoring/removing stale bytecode
<jimbaker> hazmat, r247 works just fine
<hazmat> jimbaker, thanks for confirming.. i've got the fixes for trunk
<jimbaker> i'm going to run it with -u now just to see how that works
<jimbaker> actually running it now like that
<jimbaker> hazmat, sure, once you have them i will certainly review
<hazmat> jimbaker, i get occasionally a failure about subprocess when looping tests on trunk in one test i forgot which
<hazmat> bcsaller, jimbaker, niemeyer standup?
<jimbaker> hazmat, on the second run of -u for r247, i got ensemble.agents.tests.test_unit.UnitAgentResolvedTest.test_resolved_stopped
<jimbaker> (failure in that)
<bcsaller> when I hit problem after a merge from trunk in my branch I  ran it from trunk proper. When I saw issues there I was confident it wasn't in my code
<niemeyer> hazmat: Yeah
<hazmat> bcsaller, well that's the thing.. you have to clean out all the bytecode on trunk after reverting the merge
<jimbaker> hazmat, sounds good about standup
<hazmat> bcsaller, else you'll get stale bytecode... we have multiple confirmations now that b4 the merge this didn't exist
<bcsaller> hazmat: I didn't revert a merge, I merged to branch, saw issues I wasn't having before and then ran it on trunk (w/o the merge)
<hazmat> bcsaller, ah
<hazmat> wierd
<bcsaller> yeah
<bcsaller> and its order dependent like I was saying, sometimes running the module alone doesn't produce an issue 
<jimbaker> eventually i do get an error on looping ensemble.hooks.tests.test_invoker, which looks like the log question we just talked about in standup
<jimbaker> so this is the relevant paste: http://paste.ubuntu.com/622776/
<SpamapS> oo.. learning about Go.. I like the defer statement
<niemeyer> SpamapS: +1
<jimbaker> however, i did confirm that the save/reset logging can work successfully w/ -u (in looking specifically at test_invoker.PortCommandsTest)
<jimbaker> doesn't change the need to make this fix, it's still tech debt
<jimbaker> SpamapS, definitely agreed about go
<niemeyer> jimbaker: Not just the fix, but it'd be good to have in the ML an explanation about why it worked like that, and a suggested way for everyone to implement equivalent logic the proper way
<jimbaker> niemeyer, sounds good
<hazmat> jimbaker, the question i had is why is it needed at all.. is there something internal to the app code that manipulates logging channels on the fly?
<jimbaker> hazmat, indeed, i will investigate that
<_mup_> Bug #795233 was filed: Investigate use of save_logging, reset_logging and remove them from tests <Ensemble:New> < https://launchpad.net/bugs/795233 >
<jimbaker> biab, also my home internet connection is now down after last night's thunderstorm, need to look into it
<hazmat> niemeyer, bcsaller can i get a +1 on this fix for the trunk tests... i leaked some pep8 cleanups and the fix to use a cli util function into it https://pastebin.canonical.com/48373/
<hazmat> still pretty small though
<niemeyer> hazmat: Looks good
<niemeyer> hazmat: +1
<hazmat> niemeyer, thanks
<_mup_> ensemble/trunk r250 committed by kapil.thangavelu@canonical.com
<_mup_> [trivial] fix for unit test failures due to spurious reset logging invocation [r=niemeyer]
<_mup_> ensemble/set-transitions r232 committed by bcsaller@gmail.com
<_mup_> adding service to missing hookcontext init and various pep8 cleanups
<niemeyer> bcsaller, hazmat, jimbaker: Btw, now that we have the 2 reviews rule, the workflow is a little unclear. It seems like the _second_ reviewer should be careful to move the branch to "Work in progress" or "Approved"
<niemeyer> I'm still a little wary of the additional delay the process will impose, but am keen on trying it out too
<_mup_> txzookeeper/session-event-handling r43 committed by kapil.foss@gmail.com
<_mup_> pull in the managed zk abstraction from ensemble
 * kirkland high fives niemeyer 
<niemeyer> kirkland!
<niemeyer> kirkland: Welcome!
<niemeyer> hazmat: I've reserved a review re. the watch conversation for you
<niemeyer> hazmat: On jim's branch, that is
<_mup_> ensemble/expose-hook-commands r247 committed by jim.baker@canonical.com
<_mup_> Merged trunk
<_mup_> ensemble/set-transitions r233 committed by bcsaller@gmail.com
<_mup_> merge trunk
<jimbaker> hazmat, just to make certain i got the new process - expose-hook-commands is ready for release with one review; this along with silence-deprecation-warnings are the last branches that only require one review, from niemeyer
<niemeyer> jimbaker: That sounds good, unless hazmat would like to have a look at it
<jimbaker> niemeyer, sounds good. i need to take off now, hopefully internet service is resumed at home
<SpamapS> hey guys I don't know if I agree that the debug-log should show all relation-sets ..
<SpamapS> re bug #766317
<_mup_> Bug #766317: debug-log should show relation settings changes <Ensemble:New for jimbaker> < https://launchpad.net/bugs/766317 >
<SpamapS> That should be entirely optional to the formula author.
<SpamapS> I'd concede showing which values were changed, but not the values themselves. At some point, you'll want to contain those information leaks to the members of the relation and zk/bootstrap.. debug-log is free for anybody.
<hazmat> hmm
<hazmat> SpamapS, its a question of debug utility vs. white noise and security?
<hazmat> SpamapS, from a security perspective .. its not  a free for all
<hazmat> SpamapS, any admin using the ensemble cli.. atm is considered a root.
<hazmat> hmm
<SpamapS> debug-hooks should remain unrestricted... but debug-log actually sticks it in a log file on disk on the sender..
<SpamapS> Yeah *that* may also be a bad idea.
<hazmat> niemeyer, i totally skipped multiple ensemble users and stack delegation in the security doc
<SpamapS> think about LOSA's ... they're not root but they can enact change.
<hazmat> SpamapS, yeah.
<SpamapS> hazmat: I think its totally fine to say that for now.. and add it to the "TODO" list. :)
<SpamapS> hazmat: but putting all that info in the log is my only concern
<hazmat> SpamapS, if your interested i put together a security overview.. and some of ensemble's next steps on security
 * hazmat digs up a link
<SpamapS> This may be a bit of sysadmin dogma seeping through, so I think a good logical analysis is in order, not an outright rejection of the idea.
<SpamapS> Yes very interested
<SpamapS> I imagine we'll get lots of questions at bofs and such
<hazmat> SpamapS, at the bottom of this https://code.launchpad.net/~hazmat/ensemble/security-specification/+merge/63921
<hazmat> its pretty high level
<SpamapS> bcsaller: I forget, are you talking at devops days or doing a velocity bof?
<SpamapS> high level is all we need for answering questions
<bcsaller> SpamapS: I am talking at Structure, I sent in a proposal at devop days but they didn't get back to me
<hazmat> its more vulnerability assessment that adaptation to multi-user or org workflows
<hazmat> s/that/than
<SpamapS> bcsaller: ah ok
<SpamapS> bcsaller: when is structure?
<hazmat> my plumbers talk bit the dust as well
 * hazmat wanders off into the night
<bcsaller> 22nd/23rd
<niemeyer> hazmat: That's fine.. we're not considering the multi-user scenario at this point
<niemeyer> hazmat: I mean.. there's no security separation between different users
<niemeyer> hazmat: So the implications for a single user and for multiple users are the same
<jimbaker> hazmat, are ok with my releasing expose-hook-commands, or do you want to review it too
#ubuntu-ensemble 2011-06-10
<niemeyer> Okay, I'm heading to Porto Alegre tonight..
<niemeyer> I'll see you guys tomorrow!
<hazmat>   jimbaker go for it
<jimbaker> hazmat, thanks
<_mup_> ensemble/trunk r251 committed by jim.baker@canonical.com
<_mup_> merge expose-hook-commands [r=niemeyer][f=767405]
<_mup_> Implements open-port and close-port hook commands.
<_mup_> ensemble/set-transitions r234 committed by bcsaller@gmail.com
<_mup_> add service to HookScheduler in tests
<_mup_> ensemble/set-transitions r235 committed by bcsaller@gmail.com
<_mup_> merge trunk
<niemeyer> Morning lads
<kim0> niemeyer: thanks for chmining in to the mediawiki thread :) I hope it does get them interested!
<niemeyer> kim0: Indeed!
<niemeyer> kim0: and np, good to see that kind of activity going on, thanks
<kim0> yeah
<niemeyer> Heading to lunch.. will be off to put the laptop in the safe.
<niemeyer> biab
<jimbaker> nice, a student working with me on jython had a talk accepted for the jvm language summit: http://openjdk.java.net/projects/mlvm/jvmlangsummit/agenda.html
<jimbaker> (that would be shashank at the univ of colorado)
<_mup_> ensemble/set-transitions r236 committed by bcsaller@gmail.com
<_mup_> change count on wait_for_hook in test_lifecycle
<_mup_> ensemble/standardize-log-testing r251 committed by jim.baker@canonical.com
<_mup_> Removed save_logging from test_status, using capture_logging instead
<_mup_> ensemble/standardize-log-testing r252 committed by jim.baker@canonical.com
<_mup_> Merged trunk
<_mup_> ensemble/standardize-log-testing r253 committed by jim.baker@canonical.com
<_mup_> Refactor to use our cooperative inheritance convention in test_status
<SpamapS> http://lists.debian.org/debian-devel/2011/06/msg00316.html
<SpamapS> "please be aware that I'm planning on orphaning ZooKeeper in the next days."
<SpamapS> His reasoning is a little.. weird.
<SpamapS> http://berlinbuzzwords.de/sites/berlinbuzzwords.de/files/thomas_koch_zookeeper.pdf
<SpamapS> PMD is "Project Mess Detector" .. pretty cool way to parse out code and find bad coding practices.. and PMD thinks Zookeeper sucks.
<niemeyer> SpamapS: Woah!
<niemeyer> SpamapS: Man.. :-)
<hazmat> SpamapS, some of the internal implementation on zk are a bit weak
<hazmat> as far as style
<hazmat> multi K conditional structures
<SpamapS> I'm responding that we will be happy to adopt ZK.
<niemeyer> Yeah, it definitely isn't a dream..
<niemeyer> Still, there are tens of thousands of servers relying on it
<niemeyer> Without anything better to rely on
<hazmat> yeah.. battle tested production beats most anything else
<niemeyer> Give us something better and we'll shift!
<SpamapS> Sometimes in order to achieve your short term goals you have to take on technical debt.
<niemeyer> :)
<niemeyer> Doozer is promising, but not there yet
<niemeyer> "Live is too short for crap." is the final sentence in the slide..
<niemeyer> Ok.. let's publish this and watch it burn.
<jimbaker> bcsaller, hazmat, niemeyer - standup?
<niemeyer> jimbaker: I'm up for skipping it today in favor of pushing things forward.
<niemeyer> I'll also have a short day today, and want to finish a review before I'm off
<jimbaker> niemeyer, that works for me.  i need something to eat badly right now :)
<_mup_> ensemble/set-transitions r237 committed by bcsaller@gmail.com
<_mup_> include config.yaml and default config-changed hook
<jimbaker> bcsaller, hazmat - is that ok for you to skip standup today?
 * jimbaker looks at the door, thinking "sandwich"
<bcsaller> fine for me
<niemeyer> jimbaker: Enjoy
<jimbaker> ok, sounds good :)
 * hazmat does a hop and skip
<hazmat> niemeyer, jimbaker, bcsaller you guys see about the new guy (ensemble devops) mark?
<niemeyer> hazmat: see about?
<hazmat> niemeyer, see the email about
<niemeyer> hazmat: Sorry, I don't understand what you mean.  I can see several emails about Mark.
<hazmat> niemeyer, i just saw the one..  i didn't realize that was an open position
<hazmat> thomas's presentation is pretty well informed
<niemeyer> hazmat: I still don't know what email you're talking about, but yeah, he's going to join the server team
<niemeyer> hazmat: Really?
<niemeyer> hazmat: To me it felt very silly
<hazmat> niemeyer, he notes its good points and its bad points... i agree the zk code has a bad smell
<hazmat> and attempts to improve that have been stymied for the quoted reason
<niemeyer> hazmat: Oh, sure, he makes some good points that there are things to be cleaned up in ZK's code base.
<niemeyer> hazmat: Most of his points are based on a static analysis tool.
<niemeyer> hazmat: and then mentions things like "feature bloat" => "multi-update command in the works for half a year"
<niemeyer> hazmat: Then, mentions "horrible concurrency", and points fingers saying they don't use the Java pattern for concurrency du-jour
<niemeyer> Much better! Actors!
<niemeyer> Seriously.. it's a terrible analysis.
<niemeyer> I don't understand how he can be running his stuff on the Linux kernel still.
<SpamapS> He's caught the java bug
<niemeyer> Yes, I agree, the code looks ugly
<SpamapS> the one that infects your brain with "yes, it can be perfect"
<niemeyer> Yes, I agree, I don't like coding/reading Java.
<hazmat> the linux kernel patches are reviewed for maintainability.
<hazmat> i don't mind, but the zk code reads like a c program in several places (request processor dispatch for example)
<niemeyer> hazmat: Heh.. have a look at how the Linux kernel looked like a few years ago
<SpamapS> Ultimately the linux kernel's maintainability and longevity can be attributed to the amount of shame laid upon those who submit bad patches. :)
<niemeyer> Anyway, enough ranting..
<niemeyer> The code isn't great.  There's nothing better.
<SpamapS> Right, I'm thinking we should take over maintenance, or request the debian java team to adopt it.
<niemeyer> Now go watch this and relax: http://www.youtube.com/watch?v=_AYEgwwCYWw
 * SpamapS <hearts> rock + classical instruments
<niemeyer> Re. the ZooKeeper rant: http://markmail.org/message/4dzk5osxssaibirt
<niemeyer> In the end the review took longer than expected..
<niemeyer> It's delivered though
<niemeyer> jimbaker: You've got some feedback there
<niemeyer> I'll step out for dinner.. have a good weekend folks
<_mup_> ensemble/set-transitions r238 committed by bcsaller@gmail.com
<_mup_> resolve divergent branch issue
#ubuntu-ensemble 2011-06-11
<_mup_> ensemble/standardize-log-testing r254 committed by jim.baker@canonical.com
<_mup_> Switched to using processEnded to ensure that output streams are ready for reading, removed unnecessary test timeout override, and removed some save_logging/reset_logging
<mhall119> okay, I'm checking out ensemble formulas, but they're not making much sense to me
<mhall119> is anyone around to help?
<_mup_> ensemble/standardize-log-testing r255 committed by jim.baker@canonical.com
<_mup_> Removed last parts of save_logging/reset_logging/assertInDefaultLog from codebase
<_mup_> Bug #795894 was filed: Add python-yaml to the dependencies <Ensemble:New> < https://launchpad.net/bugs/795894 >
