=== daker__ is now known as daker [00:16] <_mup_> ensemble/states-with-principals r308 committed by kapil.thangavelu@canonical.com [00:16] <_mup_> agent tear down uses relies on base class for zk tree cleanup, also increase status test timeout for running with security === otubo[AFK] is now known as otubo === otubo is now known as otubo[AFK] === daker_ is now known as daker [11:21] * hazmat yawns [12:05] <_mup_> Bug #820892 was filed: machine_data is a formless blob < https://launchpad.net/bugs/820892 > === otubo[AFK] is now known as otubo [13:25] Hi there [13:26] does anybody know if ensemble only works with AWS so far? Or is it possible to use it with other providers as well [13:28] doitdistributed: we're working on orchestra support, to use it on bare metal with cobbler [13:28] doitdistributed: it's not ready for primetime yet, but we have a sprint next week and we hope to make some good progress there [13:28] cool [13:29] doitdistributed: right now though, if you want to play with it, AWS is the way to go [13:29] yes I'm already done ;-) [13:29] I like it and as well the architectural concept [13:29] doitdistributed: cool, I hope it's working nicely for you :) [13:29] doitdistributed: awesome :D [13:30] I try to figure out if it could be part of my Ph.D. thesis as well, but therefore the multicloud approach must be supported ;-) [13:31] doitdistributed, the ec2 api also works against openstack and eucalyptus for private cloud providers [13:31] I think I will give an introduction about ensemble in the next AWS User Group meetup I'm hosting === otubo is now known as otubo[AFK] [13:32] doitdistributed, at the moment we don't support cross cloud communications [13:32] @hamat cool I will try it with EUCA [13:32] for a single environment [13:32] and openstack [13:32] doitdistributed, cool, let us know if you have any issues [13:32] hazmat: ha, yes, the API makes me think of them as basically AWS ;) [13:33] I will give you a feedback and as well what the AWS Users think. [13:35] doitdistributed: thanks very much! [13:35] no problem ;-) [13:43] Hey, are you interested in an article about Ubuntu Cloud or ensemble in our Cloud Computing magazine? [13:44] here are the links http://issuu.com/symposiajournal [13:45] http://symposiajournal.de/ [13:45] I think it would fit good [13:48] doitdistributed: I bet we would be :) [13:50] doitdistributed: unhappily, my german was not very good 10 years ago, and is now almost gone, so I'm making only very slight sense of those pages ;) [13:50] ;-) No problem [13:51] maybe we could refresh it sometimes with a beer or so [13:55] doitdistributed: I'd be delighted -- but I don't suppose you're in Malta? ;) [13:55] not daily ;-) [13:55] ok everyone, here's the ensemble report for the week, I think I've covered most everything this week: http://pad.ubuntu.com/ensemble-report [13:55] any feedback would be appreciated [13:55] doitdistributed: yeah, it's not exactly on the beaten track ;0 [13:56] But mayby there http://cloud-devcon.com [13:57] doitdistributed: hey, that looks cool [13:58] Yeah I'm one of the organizer [13:58] it will be a great event, with cool people, a big party and all inclusive [13:59] doitdistributed: awesome, I will try to figure out if I can make it, all I can remember is that my travel plans are confused for the next few months ;) [14:00] joining Canonical has been a bit of a shock to the system on that front ;) [14:00] Would be great!!! And we could have a beer or two ;-) [14:01] doitdistributed: absolutely :) [14:05] Do you think the price is ok? I mean we are new to the "business" and we want to make no business ;-) We only want cool events from the community to the community [14:19] Hello Ensemblers! [14:19] niemeyer: morning [14:19] kirkland: Yo === otubo[AFK] is now known as otubo [14:35] fwereade: howdy! [14:35] RoAkSoAx: heyhey! [14:35] fwereade: how's it going today? [14:35] niemeyer: and hey :) [14:35] fwereade, RoAkSoAx: Hey lads! [14:36] RoAkSoAx: good, I think, the structure for the shared shutdown stuff is on the tip of my tongue (fingers?) [14:36] RoAkSoAx: how about you? [14:37] fwereade: go for it if your finger tips are itching :P [14:38] RoAkSoAx: I'm back in thinking mode, but I've got some stuff that looks very similar, it's just a matter of drawing the boundaries in the right place, and hoping it doesn't end up too baroque [14:39] fwereade: well the shutdown stuff should be pretty simple [14:40] fwereade: btw.. this would be the connect.py http://paste.ubuntu.com/658669/ [14:40] RoAkSoAx: it is, the tricky bit is making that and the ec2 stuff work in the same way so we don't end up missing features on one side or the other [14:40] fwereade: i'll take a look at it later today if you want and maybe I can contribute some more ideias on how to do it [14:40] RoAkSoAx: thanks :) [14:41] fwereade: anywa,s I have one question about the connect stuff, _wait_for_initialization that client.exists_and_watch("/initialized") where's that at? that's done by the zookeeper? [14:41] RoAkSoAx: I've got the impression that your additions to cobbler.py are pretty close to the ec2 interface, which is really handy [14:42] RoAkSoAx: my understanding is that when we initialise the admin, the last thing it does is create /initialized [14:42] RoAkSoAx: and so when that's there we the system is ready [14:42] we know^^^ [14:43] fwereade: ok, so that does not need to have any changes thne [14:43] fwereade: as it should work exactly the same way as with ec2 [14:43] RoAkSoAx: yep, exactly [14:43] fwereade: now, the only thing is that we cannot use _check_machine_age as it seems that machine.launch_time comes from ec2, correct? [14:43] RoAkSoAx: yeah, that's right [14:44] fwereade: the only way we could provide that information would be to pass it the time on when the command was executed and *supposed* to launch a machine with orchestra [14:44] fwereade: which might be suboptimal [14:45] RoAkSoAx: yeah, it doesn't feel right [14:46] fwereade: alright, I guess i'll take note of this one too [14:47] fwereade: I think that in the future we'll have to make orchestra server (or cobbler) a bit smarter xD [14:47] RoAkSoAx: yeah definitely ;) [14:49] fwereade: ok, so iterate.py, accessor.py and connect.py should be done [14:49] fwereade: however, all of that needs to be tested with deploying machine [14:49] fwereade: which is something i'm gonna work on next [14:50] fwereade: after that I'll provide you with one branch for you to review, and start making smaller branches [14:50] does that sounds good? [14:50] RoAkSoAx: cool, shadow-trunk should be up to date (including the stuff you wanted yesterday) [14:50] RoAkSoAx: sounds like a plan [14:54] fwereade: yeah just pulled from there [15:02] niemeyer: how set in stone is the MachineProvider interface? I'd quite like to change shutdown and shutdown_machine [15:03] fwereade: Nothing is set on stone.. what would you like to achieve there? [15:03] niemeyer: I'm not totally sure yet... but it feels like shutdown_machines, taking a list of machines, is going to make everything else fit better [15:04] niemeyer: plain old shutdown would actually be unchanged, it's just get all machines and pass them to shutdown_machines [15:05] fwereade: Sounds quite reasonable.. how do we have to change MachineProvider for that? [15:06] niemeyer: those are part of MachineProvidr's interface [15:07] niemeyer: I'll experiment, hopefully I'll have something to show soon [15:07] fwereade: Wait.. duh, ok [15:07] fwereade: Yeah, happy with that [15:08] fwereade: When you said MachineProvider, in my mind I thought about the Machine interface itself [15:08] My bad [15:08] niemeyer: no worries :) [15:10] <_mup_> ensemble/states-with-principals r309 committed by kapil.thangavelu@canonical.com [15:10] <_mup_> add another security check around removing units, fix an old service test that running (missing inlinecallbacks) === daker is now known as daker_ [15:54] <_mup_> ensemble/states-with-principals r310 committed by kapil.thangavelu@canonical.com [15:54] <_mup_> subsume some of the module level helper functions into the security adapter class. [15:55] bcsaller, niemeyer, fwereade team meeting in a few minutes [15:55] yep :) [15:56] Ouch.. forgot to have lunch [16:01] is that google+ then? [16:03] Okay, is it time? [16:03] * niemeyer starts a hang out [16:04] fwereade: Just invited you [16:04] bcsaller: and you [16:04] hazmat: and you [16:55] later all [17:09] <_mup_> ensemble/states-with-principals r311 committed by kapil.thangavelu@canonical.com [17:09] <_mup_> move otp consumption onto the security adapter. [17:10] Lunch time.. biab [17:17] <_mup_> ensemble/states-with-principals r312 committed by kapil.thangavelu@canonical.com [17:17] <_mup_> remove an unused group api method, move some otp helper api implementation from its class to the security adapter. [17:22] <_mup_> ensemble/states-with-principals r313 committed by kapil.thangavelu@canonical.com [17:22] <_mup_> additional test for ACL grants multiple times against the same node version. [17:56] <_mup_> ensemble/states-with-principals r314 committed by kapil.thangavelu@canonical.com [17:56] <_mup_> add an environment variable to allow for testing the entire system with security enabled, security integration with relation state creation. [18:19] <_mup_> ensemble/trunk-merge r274 committed by kapil.thangavelu@canonical.com [18:19] <_mup_> merge trunk [18:25] Man.. open source is truly awesome.. [18:25] I can't emphasize that enough [18:26] feedbooks.net is using mgo to serve all of their book covers, in production, today. [18:26] feedbooks.com, sorry [18:27] I feel more confident our repository will work fine with it now.. ;-) [18:29] <_mup_> Bug #821074 was filed: security integration, sequence nodes or topology dependencies require explicit acl application, otp principal integration with state creation < https://launchpad.net/bugs/821074 > [18:56] hazmat: Ugh.. 2.5k lines of diff [18:56] hmm [18:57] niemeyer, ugh.. i forgot the prerequiste branch spec [18:58] Phew [18:59] niemeyer, fixed, diffstat shows just under 700 lines [18:59] hazmat: Phew, that's awesome, thanks [19:00] niemeyer, that should be the last/only large branch for the security work [19:12] niemeyer: are there any more orchestra-related branches yet to be merged to trunk? [19:12] RoAkSoAx: There are [19:12] <_mup_> ensemble/security-policy-rules r285 committed by kapil.thangavelu@canonical.com [19:12] <_mup_> evaporate branch, to be resurrected latter in the pipeline. [19:13] RoAkSoAx: This should always provide a snapshot of what is going on: http://ensemble.ubuntu.com/kanban/dublin.html [19:14] niemeyer: ok, cool I was hoping to see ensemble deploying from trunk by tomorrow :) [19:14] RoAkSoAx: If you see branches from fwereade in the center column on in In Progress, there's likely something to be merged [19:14] RoAkSoAx: Sweet! [19:14] RoAkSoAx: I'm doing some spec writing ATM, but I hope to clean up that review queue by today still, so that William has it ready by the time he wakes up [19:15] niemeyer: cool! I will submit my work to him for tomorrow when he wakes up to make sure is according to standards and then I guess those branches will be proposed for merging [19:15] RoAkSoAx: Woot [19:17] niemeyer: cause I have it deploying already from shadow-trunk [19:17] which is the branch I'm working on top of [19:17] RoAkSoAx: That's awesome news indeed! [19:17] RoAkSoAx: How's the Cobbler side of things? [19:18] niemeyer: its good for now. there's some things that I'd like to discuss at the sprint to get the interaction improved [19:18] RoAkSoAx: Cool [19:19] RoAkSoAx: Do we have existing hacks still to be merged, or is that trunk interaction already working with stock Cobbler from Ubuntu? [19:20] niemeyer: it is working with stock cobbler, but, there's some changes in preseeds and stuff like that yet to be merged into orchestra meta-package (probably orchestra-ensemble) [19:21] niemeyer: that will land next week [19:21] niemeyer: and autoconfiguration of a webdav server as well [19:21] RoAkSoAx: Aha, gotcha [19:21] RoAkSoAx: You rock dude [19:21] all of that will be done in orchestra [19:22] niemeyer: heh... thanks but SpamapS openned the door and fwereade did an awesome job witht he refactoring [19:23] RoAkSoAx: Yeah, other folks rock too :-) We wouldn't be where we are without all of these fitting together. [19:23] Awesome team work [19:24] smoser should be in that list too [19:25] SpamapS, bcsaller, hazmat, RoAkSoAx, m_3, jcastro, all: Quick bikeshed [19:25] We need a prefix for formulas in the repo.. [19:25] like what, a designation that it's .... ? [19:25] I don't want to use lp: to avoid confusion between the formula namespaces and the branches [19:25] niemeyer: indeed [19:25] We need something like foo:oneiric/formula [19:25] What's "foo"? [19:27] ensemble:oneiric/formula ? [19:27] ensemble deploy ensemble:... doesn't look nice [19:27] I was just thinking form, frm, or just formula [19:27] frm.. hmmm [19:27] being palindromic.. I like recursion [19:27] because that's what you're thinking in your head [19:27] frm is nice [19:27] ensemble deploy $formula from $here [19:27] this is the prefix that says "use the default repositories" ? [19:28] SpamapS: No, this is the prefix that says "this is about the repository" [19:28] ah [19:28] repo: might also apply [19:28] "frm".. "repo"... [19:28] What else? [19:28] source? [19:28] efr [19:28] so if its not there.. what do you get? [19:28] efr.. [19:28] efr =ensemble formula repository [19:28] niemeyer, ensemble deploy ubuntu:xyz ? [19:28] L) [19:29] SpamapS: We're sorting the naming convention only.. the spec with details is coming [19:29] hazmat: ubuntu.. [19:29] "ubuntu", "frm", "repo", "efr".. [19:29] niemeyer: i'd would agree with hazmat too [19:31] Anything else? [19:31] although perhaps that doesn't capture distro.. but it could be a nice symbolic for default distro latest and qualified with ubuntu:oneiric, ubuntu:natty etc [19:31] niemeyer: mmk. Given that limited context, repo seems generic and at the same time memorable enough. [19:31] "forma" is latin for form. [19:31] ubuntu: has special meaning in bzr now too.. its an alias for lp:ubuntu/ [19:31] "store" [19:31] store:oneiric/wordpress [19:31] "ubuntu", "frm", "repo", "efr", "store".. [19:32] jcastro: Feels too close to formula.. reads like a typo almost [19:32] "fr", along the lines of RoAkSoAx suggestion (Formula Repository) [19:32] repo or store seems easiest to remember, we use "ubuntu" everywhere, so that might be confusing, frm and efr are just acronyms and would be harder to remember. [19:32] have to step out for a bit, but repo: is, I think, my vote [19:32] "ubuntu", "frm", "repo", "efr", "fr", "store".. [19:33] SpamapS: Cool, thank you [19:33] I'll likely send this to the list [19:33] * jcastro agrees with SpamapS for "repo", straight forward, easy to remember [19:38] with a fallback to the environment? [19:38] re: the distro [19:39] bcsaller: Hm? [19:40] I was thinking about what hazmat was saying about ubuntu:natty or whatever, I don't know that you'd want that in the repo naming so much as in the environment you're deploying to [19:40] but using it the name of the package like in the email is fine [19:41] when is it not this magic token? [19:41] wouldn't that be the only time we'd need it? [19:41] _:formula where _ is only needed when its not the default [19:42] surprised no on listed principia: as well [19:42] bcsaller: This is just about the prefix really.. the details semantics will be posted to review [19:42] I thought that name was going away [19:43] jcastro: I think you're right but I'm not sure [19:43] nor me [19:43] Right, principia is being debated.. principia may be finita [19:43] None of us is sure, actually [19:44] niemeyer: I only mean we don't need a prefix for the default which is the place we are trying to name, its only when its not that default that you need a prefix, no? [19:44] It's being debated by Elfos.. [19:45] bcsaller: Maybe.. [19:45] bcsaller: That's an interesting point [19:45] unless I misunderstand what the prefix is, I thought it was the mapping between a namespace of packages and the place to get them from, where you don't name the default [19:45] bcsaller: ensemble deploy --repository=/tmp/oneiric deploy foo [19:45] bcsaller: Where does foo come from? [19:46] whats now principa (or the default repo for your environment) [19:46] but even in the case in ()'s its only named if you step away from the default [19:46] bcsaller: Ok.. another issue.. [19:47] bcsaller: ensemble deploy ~bcsaller/foo [19:47] bcsaller: What does that do? [19:47] throw an error, there is no package named that in the namespace [19:47] --repository is needed to specify a local repo path [19:48] bcsaller: Yeah, but I was actually talking about a remote formula [19:48] bcsaller: What does this do: ensemble deploy repo:~bcsaller/foo [19:48] ? [19:48] bcsaller: Quite obvious, right? [19:49] bcsaller: So you make an interesting point.. maybe we can get away with the prefix.. but there are a few problems which the prefix solves that I don't have a good answer for myself [19:49] no? I think it would be namespace:package where that a reference to a name mapping somewhere else like apt sources almost [19:50] bcsaller lp:~bcsaller/ensemble in a config file [19:50] bcsaller: No? You don't get the idea that repo:~bcsaller/wordpress is bcsaller's wordpress in the repo? [19:50] then deploy bcsaller:foo [19:50] to me that assumes too much lp, but maybe I'm wrong there [19:50] Maybe.. hmm [19:52] because other SCM systems will still be mapped into lp on publish it could work, but I don't know if the best path fwd [19:52] I actually like the idea of oneiric/wordpress etc [19:52] <_mup_> ensemble/states-with-principals r316 committed by kapil.thangavelu@canonical.com [19:52] <_mup_> merge conflict from removal of security-policy-rules [19:52] I don't like joe:oneiric/wordpress too much, though [19:53] It's conflicting.. [19:53] having indirection there like I suggested in the namespace would create its own set of issues though, mostly that following instructions would be hard if people had different prefix names [19:53] yes [19:53] I think what I just said kills the idea, its less repeatable [19:54] Yeah.. another issue is that everyone here knows what "bcsaller" is [19:54] What about [19:54] obsoleted:oneiric/wordpress [19:55] That's not nice [19:55] store:~obsoleted/oneiric/wordpress [19:55] That's more obvious [19:56] yeah, I think the issue that repo: implies 1 to me was the issue [19:56] whats the company local repo called that overlays the public one? [19:57] bcsaller: In which sense (implies 1)? [19:57] Hmm [19:57] repo always refers to lp, no? [19:57] bcsaller: No.. it refers to our repository system [19:57] bcsaller: It's not necessarily a 1-1 translation [19:58] bcsaller: Hmm [19:58] directly exposing lp names, but ok, I'll give you that [19:58] bcsaller: Maybe we can break the proposal in half [19:58] E.g. [19:58] <_mup_> ensemble/security-policy-with-topology r317 committed by kapil.thangavelu@canonical.com [19:58] oneiric/wordpress, but personal:bcsaller/oneiric/wordpress [19:58] <_mup_> policies with topologies [19:59] bcsaller: No, it's not directly exposing _lp_ names [19:59] bcsaller: It's not a 1-1 translation [19:59] bcsaller: Multiple repo names can refer to the same branch, for instance [19:59] bcsaller: The precise semantics will be in the spec, and will be up for debate [20:00] niemeyer: ok, I can see how that works, but what about the company local repo? It seems like there will be 'main', some companies collection and then maybe an developers private repo on top of that [20:00] <_mup_> Bug #821109 was filed: security policies need to have an accessor for topology and utilized with a modified topology < https://launchpad.net/bugs/821109 > [20:01] it could be that those are merged under the repo: prefix by configuring access to that service though. just thinking out loud [20:01] bcsaller: Yeah, the companies collection can be put on a custom namespace [20:01] bcsaller: Maybe even referred to by URL [20:01] niemeyer: I know you have a plan for this, I don't mean to make you explain it all here [20:01] bcsaller: Same way Bazaar enables lp: but full blown URLs too [20:02] bcsaller: Well, maybe I don't have a good plan yet.. [20:02] heh [20:03] bcsaller: What's your specific concern regarding company repos? [20:03] bcsaller: Seriously, I'm not being factitious [20:04] I just want it to be clear and convenient to specify that you want a more custom version from another repo than the one in main [20:04] bcsaller: Just saying I'd like to debate semantics in the context of the spec, since there are more explanations written down, but still interested on your idea of why company prefixes are problematic [20:04] bcsaller: Cool, sounds good [20:07] bcsaller: Ok, thanks a lot [20:08] bcsaller: I think there's a very good middle path there that we can follow based on your idea of avoiding a prefix when feasible [20:08] bcsaller: Will try to put it down in the spec [20:08] niemeyer: happy to talk about it next week if you want [20:09] bcsaller: Absolutely [20:18] <_mup_> ensemble/security-policy-rules-redux r317 committed by kapil.thangavelu@canonical.com [20:18] <_mup_> resurrect security-policy-rules [20:21] <_mup_> ensemble/security-policy-rules-redux r318 committed by kapil.thangavelu@canonical.com [20:21] <_mup_> bring back some additional parts of security-policy-rules [20:22] <_mup_> ensemble/security-policy-rules-redux r319 committed by kapil.thangavelu@canonical.com [20:22] <_mup_> yank accidental add of merge file [20:42] bcsaller: Posted an answer in the list about the intermediate plan.. please let me know if it's along the lines of what you had in mind [20:42] (and solves the issues you've foreseen) [20:42] checking [20:44] niemeyer: that looks good [20:46] I'm tempted to start s/repository/store/ too across the board.. such an easier word to speak about :) [20:57] <_mup_> ensemble/security-agents-with-identity r301 committed by kapil.thangavelu@canonical.com [20:57] <_mup_> provisioning agents launching machines with machine agent identities/principals [21:35] I feel like I'm writing a book.. [21:35] Maybe I should start sending individual sections for review [21:45] jcastro: ping [21:45] robbiew: pong [21:53] <_mup_> ensemble/security-policy-rules-redux r320 committed by kapil.thangavelu@canonical.com [21:53] <_mup_> resurrect policy w/ topology, got lost in the merge. === otubo is now known as otubo[AFK] [22:17] <_mup_> ensemble/security-policy-rules-redux r321 committed by kapil.thangavelu@canonical.com [22:17] <_mup_> update security rules to latest api. [22:18] Ok, a couple of pieces pushed [22:26] <_mup_> ensemble/security-policy-rules-redux r322 committed by kapil.thangavelu@canonical.com [22:26] <_mup_> merge and resolve conflict. [22:37] We need a review.. [22:37] Anyone is up for it: https://code.launchpad.net/~fwereade/ensemble/webdav-storage [22:38] https://code.launchpad.net/~fwereade/ensemble/webdav-storage/+merge/69453 [22:38] That's the actual URL, actually [23:01] <_mup_> ensemble/security-connection r289 committed by kapil.thangavelu@canonical.com [23:01] <_mup_> address review comments [1][2]