[00:13] <_mup_> ensemble/robust-hook-exit r290 committed by jim.baker@canonical.com
[00:13] <_mup_> Refactoring, doc strings, and better comments wrt review
[00:49] <_mup_> ensemble/debug-log-relation-settings-changes r274 committed by jim.baker@canonical.com
[00:49] <_mup_> Merged trunk
[07:30] <kim0> Morning all
[08:47] <jamespage> morning
[08:47] <jamespage> what do I need todo to get the jenkins and jenkins-slave formulas into principia? (see bug 793735)
[08:47] <_mup_> Bug #793735: import jenkins formula <new-formula> <Ensemble Formulas:In Progress> < https://launchpad.net/bugs/793735 >
[08:52] <jamespage> SpamapS: still around?
[08:59] <SpamapS> jamespage: insomnia seems to have caught me tonight. whats up?
[09:00] <jamespage> SpamapS: hey - I wanted to move forwards with the jenkins* formulas - wondered what needed to be done to get them accepted into principia?
[09:01] <SpamapS> Ok, I went ahead and approved you for ensemble-composers
[09:01] <SpamapS> bzr push lp:principia/jenkins will work.. but bzr push lp:principia/jenkins-slave will run into the dreaded "no package exists" ug..
[09:03] <jamespage> SpamapS: thanks - I fake something in a PPA to work around that :-)
[09:03] <jamespage> cheers
[09:03] <SpamapS> sweeeet
[09:32] <jamespage> SpamapS: both branches now pushed...
[09:32] <SpamapS> jamespage: fantastic
[09:33] <jamespage> I'll prob do some further work on them when jenkins lands in Oneiric
[09:33] <SpamapS> I saw that you are just blocked on NEW! :)
[09:34] <jamespage> yep
[09:34] <jamespage> 6  packages pending and then its in
[09:51] <adam_g> SpamapS: any specific reason why default_storage_engine is set to innodb on install of mysql via the formula?
[09:52] <SpamapS> adam_g: because MyISAM is a piece of S*** that should die an unholy death. :)
[09:52] <SpamapS> Err, I'll rephrase ina positive light
[09:53] <SpamapS> Because every time an alter table converts to InnoDB from MyISAM, an angel gets his wings.
[09:53] <adam_g> hehe
[09:54] <adam_g> wondering if it might be better to install defaults via install hooks, and let users tweak via their formula configs
[09:57] <SpamapS> Yeah its probably a good idea to just make it the default
[09:57] <SpamapS> I did it at first when I was doing the master/slave stuff to make the snapshot simpler.
[09:58] <adam_g> ah
[10:00] <SpamapS> wtf are you doing awake?
[10:00]  * SpamapS should be sleeping too.. :-P
[13:18] <niemeyer> Good morning all
[13:21] <wrtp> hiya
[13:21] <niemeyer> wrtp: Hey!
[13:24] <wrtp> niemeyer: i was looking at laptops to run ubuntu on. any good recommendations?
[13:24] <wrtp> (i saw the canonical recommended laptop list)
[13:25] <niemeyer> wrtp: I like the thinkpad series, and it's fairly common within Canonical
[13:25] <wrtp> i've still got an old thinkpad which i really liked
[13:25] <niemeyer> wrtp: Got the T410 at the moment
[13:25] <wrtp> (T21 possibly)
[13:25] <wrtp> do they still have three mouse buttons and a nipple?
[13:26] <niemeyer> wrtp: That's the second one.. my last one lasted 4+ years
[13:26] <wrtp> mine *should* be still working except the display went dodgy
[13:26] <niemeyer> wrtp: The model I use have both the nipple and the trackpad
[13:26] <niemeyer> wrtp: I tend to pop the nipple out
[13:26] <wrtp> which is a pity. i ran plan 9 on it for years.
[13:26] <niemeyer> (context is everything! ;-)
[13:27] <wrtp> i like the nipple much better than the trackpad. good for chording.
[13:27] <wrtp> :-)
[13:27] <wrtp> i liked the fact that the display was 1400x1200
[13:27] <wrtp> but i doubt i'd get one similar now
[13:28] <niemeyer> Yeah, they're pretty good laptops overall
[13:28] <wrtp> battery life?
[13:28] <niemeyer> wrtp: I'm pretty sure they still exist
[13:28] <niemeyer> wrtp: Long.. still get 5h+ nowadays
[13:28] <niemeyer> wrtp: Got an SSD as well
[13:28] <wrtp> not bad. and what about the X series? are they much smaller?
[13:29] <wrtp> SSD only? or SSD + hard drive?
[13:31] <niemeyer> wrtp: SSD only
[13:31] <niemeyer> wrtp: They are generally smaller, IIRC
[13:31] <wrtp> i bet that makes things fast.
[13:31] <niemeyer> wrtp: Like 13" or less
[13:31] <niemeyer> wrtp: It does.. boot times are unbelievable nowadays
[13:31] <wrtp> ok. i think T series it is
[14:00] <jcastro> wrtp: the T410's and 420's seems to be popular at Canonical
[14:04] <wrtp> jcastro: thanks. BTW what's the difference between T410 and T420? the levovo web site does not seem to want to talk about the T410
[14:07] <jcastro> wrtp: the 420 replaces the 410
[14:07] <wrtp> oh, that's easy then :-)
[14:10] <wrtp> i can't believe that displays have got *smaller* since i last bought a thinkpad in 2003!
[14:14] <kim0> SpamapS: hmm we're being compared to cloudformations .. http://cloud.ubuntu.com/2011/06/so-what-is-ensemble-anyway/#comment-1635 
[14:15] <kim0> I think cloudformations only launches a collection of machines and that's it .. it doesn't really manage them afterwards ?!
[14:26] <jcastro> wrtp: they make "s" series with higher res screens, they are hard to find and more expensive, search for 410s and 420s.
[14:26] <jcastro> kim0: wow, some of the comments in that article, heh
[14:39] <niemeyer> wrtp: Yeah, I'm not a big fan of the widescreen laptops either..
[14:40] <niemeyer> wrtp: But I guess everyone else has won. ;-)
[14:40] <hallyn> niemeyer: :(
[14:41] <wrtp> jcastro: s series are still only 900 vertical pixels. my old one was 1050...
[14:42] <wrtp> niemeyer: yeah. boo hiss. i like my vertical space. still i'll survive!
[14:44] <jcastro> yeah but it's like they all stopped making 4:3
[14:46] <hazmat> wrtp, the x220 is doing pretty nice by me, although i think there are some sandy bridge graphics updates that need onieric, i'm using classic no effects.. the x220 can take two ssd drives (msata, and regular), although the regular has to be a 7mm device. speed and battery life are both very nice
[14:46] <hazmat> the ips screen rocks as well
[14:47] <hazmat> they also sell an optional battery slice for the x220, which can extend battery life to the 20+ hr mark in addition to a 9-cell
[14:48] <hazmat> the 's' identifier on a series identifies switchable/discrete graphics afaik.. they do sell high quality ips screens on some models without the 's'
[14:48] <jcastro> hazmat: yeah I have an X220 and it's pretty awesome
[15:14] <hazmat> niemeyer, ping
[15:14] <niemeyer> hazmat: Yo
[15:15] <hazmat> i took a step back and lookedat what's needed to finish up security. there's a question, i'd like to get your feedback on.. do you have time for a voice chat?
[15:23] <niemeyer> hazmat: Not right now, I'm about to leave for lunch
[15:23] <niemeyer> hazmat: and some friends just arrived to have lunch with us (which is why it took a moment, sorry)
[15:23] <niemeyer> hazmat: But will be happy to talk after lunch
[15:24] <hazmat> niemeyer, nevermind, there really is only way to look at the problem.. i think i've got it worked out, thanks and enjoy
[15:24] <hazmat> i'll paste my internal dialogue later ;-)
[15:24] <niemeyer> hazmat: Cool.. I've started looking at your new security branch already, btw
[15:25] <niemeyer> hazmat: It worked this time
[15:25] <hazmat> niemeyer, re the question.. https://pastebin.canonical.com/50336/
[15:25] <hazmat> niemeyer, i'm going to add to the add_machine_state, add_unit_state to create otp principals and store the token directly on the relevant states
[15:27] <niemeyer> hazmat: Cool, I'll put some thinking on that after lunch.. now I really need to step out!
[15:27] <niemeyer> biab
[15:36] <_mup_> ensemble/expose-provider-ec2 r301 committed by jim.baker@canonical.com
[15:36] <_mup_> Assign the provider machine its machine id in the provisioning agent to simplify provider APIs
[15:36] <SpamapS> negronjl: ping?
[15:41] <robbiew> m_3: ping
[15:41] <_mup_> ensemble/expose-provision-machines r292 committed by jim.baker@canonical.com
[15:41] <_mup_> Merged trunk
[15:42] <_mup_> ensemble/expose-provision-machines-reexpose r301 committed by jim.baker@canonical.com
[15:42] <_mup_> Merged upstream branch expose-provision-machines
[15:47] <negronjl> SpamapS: pong
[15:53] <SpamapS> negronjl: wondering about your gem issue.
[15:54] <negronjl> SpamapS: testing a workaround now.  I'll let you know
[15:56] <negronjl> SpamapS:  successfully worked around the issue by packaging bundler into a .deb ( so no gem install anymore ).
[15:57] <negronjl> SpamapS:  however the issue with ensemble still persists.  we just found a fix around the issue.
[15:58] <_mup_> ensemble/expose-provider-ec2 r302 committed by jim.baker@canonical.com
[15:58] <_mup_> Remove direct passing of machine_id for EC2 provider implementation and fix incorrect usage of Instance instead of ProviderMachine
[15:59] <SpamapS> negronjl: heh.. it really is a bug in gem... it shouldn't expect a login environment.
[16:02] <robbiew> SpamapS: hey...do you have the link to that google doc we had in dublin, listing all the projects we wanted to work with, e.g. HotandHairy ;)
[16:02] <SpamapS> robbiew: I think jcastro and m_3 have it
[16:02] <robbiew> jcastro: ^^...can you share that with me?
[16:10] <jcastro> on it
[16:16] <wrtp> hazmat: thanks, that's useful
[16:16] <jcastro> robbiew: doc title is "Ensemble & Principia"
[16:16] <jcastro> I don't own the doc so it won't let me reshare it with you
[16:16] <robbiew> jcastro: thnx
[16:16] <jcastro> I can jet you a copy via mail though if you just want to read it
[16:17] <jcastro> also, all the hot and hairy ones I filed, and have a "hot" tag in lp
[16:17] <robbiew> jcastro: who owns the doc, sabdfl?
[16:18] <m_3> robbiew: hey
[16:19] <jcastro> robbiew: jono
[16:20] <jcastro> robbiew: he likely owns all the other messaging ones too, I'll send him a note to add you as an owner to all of them
[16:20] <robbiew> jcastro: ffs
[16:20] <robbiew> jcastro: send me the link
[16:20] <robbiew> I should be able to view it...not showing up in a search though
[16:22] <robbiew> m_3: nevermind...jcastro responded ;)
[16:22] <m_3> cool... yeah, I'm not the owner either
[17:11] <jamespage> jcastro: ep-lite rocks BTW
[17:12] <jamespage> sooo much lighter than etherpad - it even runs in a t1.micro
[17:16] <m_3> jamespage: are you using that standalone or as part of some other "conference" stack?
[17:17] <jamespage> m_3: so we setup etherpad for UDS - but it was hard and the package is pretty ugly
[17:17] <jamespage> I just tried out ep-lite - http://ec2-46-137-10-1.eu-west-1.compute.amazonaws.com:9001/p/pad-with-daviey
[17:17] <jamespage> really small footprint compared to etherpad
[17:17] <jamespage> really small
[17:17] <m_3> ep-lite should just require node and npm
[17:18] <jamespage> m_3: yep - that was pretty much it
[17:18] <m_3> awesome
[17:18] <jamespage> npm pulled the rest of the deps
[17:18] <jamespage> it can backend to sqlite or mysql
[17:19] <m_3> did you do a formula?  that's on my list for node apps
[17:20] <SpamapS> jamespage: what services does it use for data storage?
[17:20] <SpamapS> oh
[17:20] <SpamapS> should read the whole conversation before joining
[17:20] <jamespage> m_3: not yet
[17:21] <jamespage> m_3: would you mind if I had a go at one?
[17:21] <m_3> jamespage: sure man... go ahead
[17:21] <jamespage> m_3: ta
[17:22] <m_3> jamespage: I've got some node/npm boilerplate you're welcome to
[17:22] <jamespage> m_3: point me at it :-)
[17:22] <m_3> lemme get it into lp (in github at the moment)
[17:28] <jcastro> jamespage: oooh, very nice.
[17:28] <jcastro> jamespage: yeah the page made a bunch of performance claims, but I wasn't ready to believe it until someone tried it. Great to hear it's slimmed down though
[17:29] <jcastro> it'd be a heck of a nice formula to have handy for conferences, LUGs, etc.
[17:30] <jcastro> and would also be a nice demo one too, since you could just fire it up, give people  the url, and then people could play with it right there
[17:32] <m_3> jamespage: lp:~mark-mims/+junk/nodeapp
[17:33] <jamespage> m_3: ta
[17:34] <niemeyer> Yo!
[17:36] <m_3> niemeyer: hey
[17:36] <niemeyer> m_3: Hey Mark
[17:39] <niemeyer> hazmat: SO, in the paste you provided earlier, it's not clear to me what you mean with "OTP serialization directly to the relevant nodes"
[17:52] <hazmat> niemeyer, basically the otp principal.. creates a permanent named principal with credentials stored to the otp node, along with acl using the otp credentials, the otp principal can be serialized to give a path:otp_user:otp_password info that can be utilized to retrieve the permanent credentials
[17:53] <hazmat> niemeyer, in particular i'm saying that machine_state and unit states would have that otp serialization directly as part of their contents
[17:53] <niemeyer> hazmat: That's the bit I don't get
[17:53] <hazmat> this allows say the  provisioning agent to access it from the machine state and then launch the machine agent with it
[17:53] <niemeyer> hazmat: Wasn't the plan to have a location where these OTP details are managed?
[17:53] <niemeyer> hazmat: It sounds like you're saying that OTP will be spread around, but I'm not sure if that's the case.  Is it?
[17:54] <hazmat> niemeyer, right but the otp credentials themselves need to be passed between multiple processes, i thought it was just a one time transfer from the launching process to the launched process. but really its needed much earlier (at the cli time) so we can associate the proper acl to named principal on created nodes
[17:55] <niemeyer> hazmat: The OTP principle and the timing sounds fine.. I'm wondering about the organization of nodes
[17:55] <hazmat> niemeyer, the permanent principal that the otp stores is stored in only one place, the otp serialization (access to that permanent principal) is stored on the relevant domain object for an agent, so that the process launching the domain object can access it to pass along to the domain agent
[17:56] <niemeyer> hazmat: Hmm.. it sounded to me like having a location where these exchanges happen would be cleaner
[17:57] <niemeyer> hazmat: Otherwise we can't really tell at any given point in time which OTPs are unclaimed, for instance
[17:57] <niemeyer> hazmat: Without having to dig through node content
[17:57] <hazmat> niemeyer, the otp is gone after use, and the launching process should remove it from the state when launching the domain agent
[17:58] <niemeyer> hazmat: Yeah, the above comment take that into account
[17:58] <niemeyer> takes
[17:59] <hazmat> niemeyer, on phone with isp.. trying to resolve internet issues.. bbiam
[18:00]  * hazmat is over his isp.. time for a new one.. 
[18:01] <hazmat> Nicke, the unclaimed otps are those that still exist, the otp serialization on the domain state, is removed prior to the launch of the domain agent
[18:02] <hazmat> their is a defined location for the exchange the otp node itself, but that has an acl protection to only allow access to someone with the otp credentials
[18:03] <niemeyer> hazmat: Ok, so maybe I misunderstood what you mean there
[18:03] <niemeyer> hazmat: This makes it sound like otherwise, for instance: " the otp serialization (access to that permanent principal) is stored on the relevant domain object for an agent"
[18:03] <hazmat> the otp credentials are stored transiently on the domain state, so we can allow for the indirection nesc to reference the named principal during domain node creation, and so the process that will create the corresponding domain state agent to access it to pass along
[18:04] <hazmat> i originally was going to forgo this and just have the otp created by the launching process, but its needed much earlier by the cli to associate the acls
[18:04] <hazmat> to domain objects created by the cli
[18:04] <niemeyer> hazmat: Sorry.. I don't understand stil
[18:04] <niemeyer> l
[18:04] <niemeyer> hazmat: What I mean is this..
[18:05] <niemeyer> There's a machine node
[18:05] <niemeyer> /machines/machine-0
[18:05]  * hazmat nods
[18:05] <niemeyer> This node is protected by an ACL so that it can only be read by relevant parties
[18:05] <hazmat> and a corresponding /otp/otp-xyz node
[18:05] <niemeyer> Yes..
[18:06] <niemeyer> When that gets put in place, which protection is /machines/machine-0 taking, and what's inside /otp/otp-xyz?
[18:09] <niemeyer> hazmat: Lacking some interactivity.. maybe we should video?
[18:12] <hazmat> niemeyer, i disconnected again..  could you repeat last line?
 When that gets put in place, which protection is /machines/machine-0 taking, and what's inside /otp/otp-xyz?
 hazmat: Lacking some interactivity.. maybe we should video?
[18:13] <hazmat> niemeyer, these are my last comments not sure what got missed https://pastebin.canonical.com/50354/
[18:13] <hazmat> niemeyer, sure
[18:14] <niemeyer> hazmat: Ok, I think we're on the same page
[18:14] <niemeyer> hazmat: The point I was missing was really about the "the otp serialization (access to that permanent principal) is stored on the relevant domain object for an agent" comment
[18:15] <niemeyer> hazmat: It felt like the domain object itself (e.g. /machines/machine-0) was protected by the OTP, and the real password was within it
[18:15] <hazmat> niemeyer, okay.. so that get's removed effectively when the domain agent is launched.
[18:15] <niemeyer> hazmat: But the description in the paste clearly states otherwise, so it's all good
[18:15] <hazmat> niemeyer, cool
[18:15] <niemeyer> hazmat: What gets removed when the domain agent is launched?
[18:16] <hazmat> niemeyer, the otp serialization (otp_node_path:otp_user:otp_pass) on the domain state
[18:16] <niemeyer> hazmat: Ok, we're still out of sync apparently.. you mean the domain object (/machines/machine-0) is protected by the OTP?
[18:17] <hazmat> niemeyer, no.. but the domain object has the otp serialization in it, after the launch of the domain agent, that data is stale
[18:17] <niemeyer> hazmat: Why does it need it?
[18:17] <hazmat> niemeyer, the domain agent doesn't need the otp serialization in the state, but the process that launches the domain agent does so it can pass it to the domain agent
[18:18] <niemeyer> hazmat: Ahh, I think I get it, ok
[18:18] <niemeyer> hazmat: Nice, makes sense
[18:19] <niemeyer> hazmat: That was the confusion.. it sounded like the OTP was *protecting* the domain node
[18:19] <hazmat> niemeyer, yeah.. after i thought it about their really weren't any options to work around it.. we need the named principals referenceable from the cli at state creation time, by the time the system is getting around to launching the domain agent, its too late for most of the acl grants
[18:20] <hazmat> niemeyer, ah.. yeah. the otp is serialized in the domain node, and only protects the named principal in the otp node
[18:21] <hazmat> niemeyer, so i took a step back yesterday to try and figure out how many more branches and work are needed to get both identity and security policies activated.. being realistic its probably about 1.5 weeks.
[18:22] <niemeyer> hazmat: That's fine
[18:23] <niemeyer> hazmat: Things are looking good.. and I won't do anything else today before killing that review queue :)
[18:23] <hazmat> niemeyer, cool
[19:27] <niemeyer> hazmat: First review on https://code.launchpad.net/~hazmat/ensemble/security-connection-redux/+merge/69369 delivered
[19:27] <niemeyer> hazmat: We'll probably need a voice conversation on some of the topics there
[19:57] <_mup_> ensemble/expose-provider-ec2 r301 committed by jim.baker@canonical.com
[19:57] <_mup_> Recover from bzr error
[19:59] <_mup_> ensemble/expose-provider-ec2 r302 committed by jim.baker@canonical.com
[19:59] <_mup_> Merged trunk and resolved conflicts.
[20:02] <jimbaker> So much for that attempt... argh, the only thing worse than a complicated conflict is bzr breaking down
[20:02] <jimbaker> (in addition to the conflict!)
[20:05] <niemeyer> jimbaker: How's it breaking down?
[20:05] <jimbaker> niemeyer, lost some of its metadata
[20:05] <niemeyer> Huh
[20:06] <jimbaker> i can recover the diff, but this is the context of merging upstream and resolving all the conflicts that was generated with the recent provider refactoring
[20:07] <niemeyer> hazmat: ping
[20:07] <niemeyer> hazmat: Not sure if your connection is flaky still, or if you're off
[20:09] <hazmat> niemeyer, back, its been flaky still, should have a new isp on monday though
[20:10] <niemeyer> hazmat: Cool.. not sure if you got this:
 hazmat: First review on https://code.launchpad.net/~hazmat/ensemble/security-connection-redux/+merge/69369 delivered
 hazmat: We'll probably need a voice conversation on some of the topics there
[20:11] <hazmat> niemeyer, nope i didn't thanks for the replay
[20:11] <niemeyer> hazmat: Oh my..
[20:11] <jimbaker> i hate to do this, but i'm going to break the commit into two pieces this time (the merge AND the conflicts)
[20:11] <niemeyer> hazmat: I think I messed up something
[20:12] <niemeyer> hazmat: Feels like I already reviewed security-policy-rules
[20:12] <niemeyer> hazmat: Hmm.. the connection-redux branch probably had it embedded
[20:12] <hazmat> niemeyer, yeah.. there are a pre-requisites on all the merge proposals
[20:13] <niemeyer> hazmat: Ok.. I'll copy over the review..
[20:13] <niemeyer> hazmat: Feel free to address specific points on the respective branches.
[20:13] <niemeyer> hazmat: This way you can merge stuff faster
[20:14] <_mup_> ensemble/expose-provider-ec2 r301 committed by jim.baker@canonical.com
[20:14] <_mup_> Merged upstream expose-provision-machines-reexpose (conflicts later)
[20:16] <niemeyer> hazmat: Alright, I think I fixed the mess
[20:16] <hazmat> niemeyer, your point #8 on security-connection bears some thinking about
[20:16] <hazmat> that's definitely got some implications for the rest of the approach
[20:16] <niemeyer> hazmat: security-policy-rules is the one that requires conversations
[20:16] <hazmat> niemeyer, yup
[20:16] <niemeyer> hazmat: security-connection-redux is back in review with a +1
[20:16] <hazmat> niemeyer, cool
[20:17] <niemeyer> hazmat: Only minor points there
[20:18] <hazmat> niemeyer, re why #2 on connection, the reason for the mixin is to ease testing, ssh conn requires ability to ssh into local host non-interactively.
[20:18] <hazmat> to test based on usage
[20:24] <niemeyer> hazmat: It can be a base then, and live within the same package
[20:25] <niemeyer> hazmat: Multiple-inheritance, separate packages, mixins.. feels like a lot for overloading a method
[20:26] <hazmat> niemeyer, sounds good, i'll probably end up with a second mixin to have policies active b4 we have transport level security for intra-environment communications
[20:26] <hazmat> er. second connection class using the mixin
[20:28] <niemeyer> hazmat: You don't need to have a second class using the mixin.. Just use that same base
[20:28] <hazmat> niemeyer, ah.. base, right, yeah.. that makes more sense
[20:29] <niemeyer> hazmat: Yeah ZK => SZK => SSHSZK
[20:29] <hazmat> SZK name is a bit of misnomer.. since its not transport level.. if you have an idea on rename.. i'm all ears. i was thinking PolicyConnection.. but that sounds strange as well.
[20:45] <_mup_> ensemble/expose-provider-ec2 r302 committed by jim.baker@canonical.com
[20:45] <_mup_> Resolved text conflicts, but some merge problems remain
[20:46] <niemeyer> hazmat: That wasn't a naming suggestion :-)
[20:46] <niemeyer> hazmat: Just illustrating the inhertiance
[20:47] <niemeyer> hazmat: Hmm
[20:47] <hazmat> niemeyer, understood, just making a naming request additional to that
[20:47] <niemeyer> hazmat: PolicyConnection?
[20:48] <hazmat> niemeyer, yeah.. i guess that makes as much sense.. it just felt strange
[20:48] <niemeyer> hazmat: RuledConnection :-)
[20:49] <hazmat> TheOneRing with s/connect/wear, doc string.. "And in the darkness bind them"  ;-)
[20:49] <hazmat> yeah.. policyconn sounds good
[20:59] <_mup_> ensemble/states-with-principals r298 committed by kapil.thangavelu@canonical.com
[20:59] <_mup_> a new method to OTPPrincipal, that creates a principal, adds it to token db, and returns the otp serialized data in one shot, also a general reuse test function to enable otp cleanup from tests that may encounter them.
[21:05] <_mup_> ensemble/expose-provider-ec2 r303 committed by jim.baker@canonical.com
[21:05] <_mup_> Added missing machine_id settings to mocks in test_launch
[21:07] <hazmat> niemeyer, the only place we really need topoloy data for things that might not be in the topology is the unit, since we need to retrieve the service
[21:08] <hazmat> niemeyer, we could remedy by just putting the service in the initial data for the unit node
[21:08] <hazmat> although that would change the rule interface to make them content based..
[21:08] <_mup_> ensemble/expose-provider-ec2 r304 committed by jim.baker@canonical.com
[21:08] <_mup_> Fix mock tests around the merged ProviderMachine
[21:08] <hazmat> perhaps a slippery slope
[21:09] <niemeyer> hazmat: Not sure I get what you mean.. there are more cases of topology being used for things that might not be in the topology in that branch
[21:09] <hazmat> niemeyer, most/all of the other use is for relations
[21:11] <niemeyer> hazmat: Yeah.. but e..g.
[21:11] <niemeyer> hazmat: How can you assign permissions to /relations/relation-10 based on the users of this relation, given that the id was just created?
[21:11] <hazmat> niemeyer, yeah.. the relation top level node also needs some consideration
[21:12] <niemeyer> hazmat: The path doesn't even include the sequence number!
[21:12] <hazmat> niemeyer, it could also be addressed the same way, using content
[21:12] <hazmat> as part of the rule interface
[21:13] <niemeyer> hazmat: Using content in what sense?>
[21:14] <hazmat> yeah.. as i noted in the mp.. i regard that branch as the most incomplete of the bunch.. its definitely going to need more work.. i just wanted to get more discussion ont he approach
[21:14] <hazmat> niemeyer, node data 
[21:14] <hazmat> niemeyer, for a relation including the services and their roles.. for the unit node including its service
[21:15] <hazmat> the rule could then dispatch both on path and content, which as you correctly point out is needed for any sequence node acl to be contextually aware
[21:15] <niemeyer> hazmat: Do we have that data there today?
[21:16] <hazmat> niemeyer, not in the relation-id node.. we do have them in the role container node below
[21:16] <hazmat> most of the domain object nodes are empty nodes at create time
[21:17] <hazmat> machines, units, relations, etc.
[21:17] <niemeyer> hazmat: It feels awkward to be sending data to zookeeper to fix a deficiency in our API
[21:18] <hazmat> niemeyer, its effectively just identity storing identity information on the relevant domain nodes.
[21:18] <niemeyer> hazmat: It's storing data in zookeeper in a place that is not necessary, besides for fixing a deficiency in our API
[21:19] <niemeyer> hazmat: It's indeed a slippery slope..  I'd like to take a step back and ponder for a while about possible approaches that avoid that kind of cross-dependency entirely
[21:19] <hazmat> niemeyer, well.. we can choose not to store it ;-)
[21:19] <niemeyer> hazmat: LOL
[21:21] <_mup_> ensemble/expose-provider-ec2 r305 committed by jim.baker@canonical.com
[21:21] <_mup_> Fixed remaining mocks in ec2 provider tests
[21:21] <hazmat> one of the other nice things i like about the policy rule approach, its very easy to apply the acls to an entire tree, or diff a tree and see if there acls missing/etc, or do an upgrade on them en mass. its just a tree walk reusing the same rule interface
[21:22] <hazmat> niemeyer, in some sense the api we have right now is the strange, one we have domain objects with no identity information, because we store identity in a secondary index, but the object in isolation lacks any context.
[21:23] <hazmat> i'll think about alternate approaches and we can revisit, i should continue on with the identity work
[21:24] <niemeyer> hazmat: That's not entirely true
[21:24] <niemeyer> hazmat: The object owns its identity, and there is context about what is there in most contexts
[21:25] <niemeyer> hazmat: The "secondary index" is not an index, but a relationship description
[21:28] <niemeyer> hazmat: that's also interesting, because what is being looked for is not information about the domain object either, but about its relations
[21:29] <niemeyer> Well, for units that's not true
[21:43] <_mup_> ensemble/expose-provider-ec2 r306 committed by jim.baker@canonical.com
[21:43] <_mup_> Pass through machine data to DummyLaunchMachine.start_machine to fix common tests
[21:52] <niemeyer> hazmat: Btw, _very_ nice break up of branches
[21:52] <niemeyer> hazmat: Thanks a ton for that
[21:52] <niemeyer> It'd be crazy otherwise
[21:52] <hazmat> niemeyer, np... hoping to continue that with the rest of the work
[21:53] <niemeyer> hazmat: I'm getting to the end of the pile.. it feels like that issue we're both brainstorming on is the only critical..
[21:53] <niemeyer> hazmat: Let's think some more on that and tomorrow we can catch up for a decision
[21:53] <hazmat> niemeyer, yeah.. i'm going to think about it some more, we should reconnect about it tomorrow.
[21:53] <niemeyer> hazmat: +1 :)
[21:56] <hazmat> niemeyer, re identity its not true for units or relations nodes, they only have identity with the topology, and those are both the ones the topology gets consulted by the rules for.
[21:58] <niemeyer> hazmat: The _identity_ is the node name..
[21:58] <hazmat> niemeyer, unit-000000 ? relation-00000? without consulting the topology.. the unit doesn't know it services, nor the relation it services.
[21:59] <hazmat> its
[21:59] <niemeyer> hazmat: Exactly.. it doesn't know its relationships..
[22:00] <hazmat> niemeyer, its more than just relationships.. although arguable the relationships are lending identity here.. but for example the unit doesn't even know its name
[22:00] <niemeyer> hazmat: A bit like saying that the identity of /home/hazmat is in /etc/passwd..
[22:01] <niemeyer> hazmat: There's information in there about it, but it stands on its own
[22:01] <hazmat> niemeyer, but i know hazmat is the user name just from the path.. in the case of units.. the name itself is unknown
[22:01] <niemeyer> hazmat: hazmat is your id.. like unit-000000
[22:01] <niemeyer> hazmat: Your name is K.T.
[22:01] <niemeyer> hazmat: and is in /etc/passwd
[22:02] <niemeyer> hazmat: This is another thing I'm wondering about that.. it's not clear why we're using the service name rather than the id on the ACLs
[22:02] <hazmat> niemeyer, we probably should be using the id everywhere for acls
[22:03] <hazmat> niemeyer, i started doing that when i was working on machines more recently.. the machine_state.id == 0, 1, 2.. is not particularly useful in this context for identification
[22:03] <hazmat> er. internal_id everywhere
[22:04] <niemeyer> hazmat: Uh.. why not?  It's effectively the same thing?
[22:05] <hazmat> niemeyer, because the acl is a global namespace of principals, we should be able to identify a principal from its name, names like '0', '1', are rather ambigious compared to 'machine-xyz'
[22:05] <niemeyer> hazmat: Gotcha
[22:06] <hazmat> niemeyer, and in the case of service names, we don't prevent service name reuse
[22:06] <niemeyer> hazmat: Either way, "0" in this context is akin to "wordpress/2" for a unit name.. it's really oriented for users
[22:06] <hazmat> yup
[22:06] <niemeyer> hazmat: When we're handling it internally, internal_id is good
[22:12] <hazmat> hmm.. hard to store the otpprincipal data on the domain state with a known principal name against a sequence node, the name is ..
[22:13] <hazmat> perhaps an additional topology index matching domain objects to principal names
[22:16] <niemeyer> Yeah.. trying to lock the drawer with the key inside
[22:17] <niemeyer> hazmat: Actually, not really.. what's the actual issue?
[22:17] <niemeyer> hazmat: The otp just needs to be created beforehand
[22:17] <niemeyer> hazmat: so that the name may be stored within the domain node, if I get what you mean
[22:20] <niemeyer> Okay.. I'm heading to a dinner with a friend that is in town.  Will be happy to talk about these details tomorrow morning.
[22:24] <hazmat> niemeyer, the otp creates a named principal, typically that should correspond to the identity of the domain object, but for sequence nodes, identity is rather ambigious.
[22:24] <hazmat> i could just use random names based on type...
[22:25] <hazmat> and store in the topology, but then the lookup process for the identity token is complicated
[22:47] <hazmat> easier for sequence nodes to just be updated with acls post creation
[23:39] <jimbaker> negronjl_, i need to do some additional refactoring and docs on robust-hook-exit branch per the review. then i can merge it into trunk
[23:39] <negronjl_> jimbaker: thx I appreciate it
[23:41] <jimbaker> negronjl_, np. occasionally these branches prompt some appraisal that there's too much previous tech debt, so that's why it's taking some additional time
[23:53]  * SpamapS thinks the only tech debt that is troubling is all the hoops we jump through marked "Twisted"
[23:58] <jimbaker> SpamapS, i think the attention we are paying to move to deterministic tests is a good one, but it is hard to do for sure
[23:59] <jimbaker> it was so much easier to just write tests that would simply sleep 200 ms and sweep the problems under the rug