[00:41] <_mup_> ensemble/debug-log-relation-settings-changes r277 committed by jim.baker@canonical.com [00:41] <_mup_> Delay __str__ of relation setting changes [00:51] Time to bake a new mgo release [00:54] niemeyer: talk went very well [00:54] Finished a bit early but had tons of questions. [00:54] SpamapS: Oh, tell me about it! [00:55] SpamapS: Was it videoed? [00:55] sadly no [00:55] Would have been good to have it on tape actually [00:55] Awww [00:56] SpamapS: So, how was it received? [00:56] Very well [00:56] There was one.. semi-hostile question asking how we could possibly do hardware provisioning w/ cobbler... [00:56] But I presented that as "not done yet" so it had no real legs. [00:56] SpamapS: Huh [00:57] He was just surprised we'd even try that when Open Stack is doing something similar but basically rolling their own cobbler. [00:57] Heh [00:57] It was the only non-excited question [00:58] SpamapS: Sweet [00:58] People saw very well how it is not the same as puppet or chef, and how the relationship definitions make it easier to deploy complex setups. [00:58] *Most* people are interested in when stacks will be ready... even though I barely mentioned them.. people see the value in the idea of repeating the entire deployment. [00:59] SpamapS: Cool, yeah, that's important indeed [00:59] A couple people said they had done similar things w/ zookeeper and thought it was a good choice for this type of info. [00:59] SpamapS: True.. I do hear about people using zk in a more custom setup, but doing similar stuff for config deployment and discovery [01:00] The Nebula guys were interested in how Ensemble could be used to add services to the openstack dashboard. [01:00] SpamapS: Hah, neat [01:00] SpamapS: Any interesting follow up conversations from there? [01:00] I didn't get a lot of people saying they wanted to write formulas.. which I was hoping for.. but.. the interest level was as much in joining the development of ensemble itself as anything.. so thats a nice surprise. [01:02] The best follow up was w/ the Nebula guys. They said they were trying to think of a way to encompass services in a generic way and so they liked the idea of just using ensemble for that. [01:02] SpamapS: That's half expected.. we'll have a _lot_ more people using it than developing [01:02] (hopefully ;-) [01:02] <_mup_> ensemble/debug-log-relation-settings-changes r278 committed by jim.baker@canonical.com [01:02] <_mup_> Modified output format of change items [01:02] SpamapS: That's very promising [01:02] niemeyer: well the audience is open source developers.. so its not a surprise that they'd want to get deep into code [01:03] Anyway, I invited a lot of people to the bof which starts in 55 minutes.. will send an email with my report on both events tomorrow. [01:03] SpamapS: OSCON seems quite varied in terms of audience [01:03] niemeyer: indeed.. all over the place [01:04] OpenStack seems to be the belle of the ball tho. :) [01:04] SpamapS: Fantastic [01:04] SpamapS: That's good.. I hope people make it nice :) [01:04] SpamapS: glad to hear it went well [01:04] jcastro: as predicted, lolcats were lol-flat [01:04] SpamapS: when you get back let's do a call, I want all the nitty gritty reactions, etc. [01:04] I told you dude, last year's joke! [01:05] jcastro: only joke that hit was me acting excited that we were ushering in the singularity .. one step away from Skynet. ;) [01:05] jcastro: I wish I had time to change it. :-P no worries, the content was quite compelling. [01:05] heh [01:06] glad to hear people didn't confuse it with puppet/chef [01:06] I've been getting alot of that [01:06] jcastro: A couple of people did make the point that puppet/chef can do some of the same things. But once they get that the formula is a self contained unit they let go of that. [01:06] * jcastro nods [01:06] SpamapS: hey did jono attend your talk? [01:07] no haven't seen him [01:07] big conference center.. and our tracks do not collide. :-P [01:07] adam_g was there.. and can maybe offer his own impressions. [01:15] <_mup_> ensemble/debug-log-relation-settings-changes r279 committed by jim.baker@canonical.com [01:15] <_mup_> Test new formatting for long strings [01:23] <_mup_> ensemble/debug-log-relation-settings-changes r280 committed by jim.baker@canonical.com [01:23] <_mup_> Changed use of nonlocal in apply_changes [01:47] <_mup_> ensemble/debug-log-relation-settings-changes r281 committed by jim.baker@canonical.com [01:47] <_mup_> Fix test_invoker with respect to new format [01:49] <_mup_> ensemble/debug-log-relation-settings-changes r282 committed by jim.baker@canonical.com [01:49] <_mup_> Capitalization [01:54] <_mup_> ensemble/debug-log-relation-settings-changes r283 committed by jim.baker@canonical.com [01:54] <_mup_> Log testing of hooks must wait for the ended deferred [02:06] <_mup_> ensemble/debug-log-relation-settings-changes r284 committed by jim.baker@canonical.com [02:06] <_mup_> PyFlakes === kooolhead11|afk is now known as kooolhead11 [07:28] Morning everyone [07:40] hi kim0 [08:15] raphink: hey [09:08] hi all, when I do the ensemble add-relation mydb:db myblog, what is the ":db" means? [09:08] are there any other options that I can use? [09:09] it's not just a relationship name, it has to be db, otherwise the system won't take it [09:19] shang: Hi there [09:19] shang: seems like you were at cloud days ? [09:19] kim0: hi, thanks for the great session the other day [09:20] cool [09:20] kim0: I was actually watching ur youtube video and start plaing with ensemble :-) [09:20] kim0: the zero to ensemble in 5 minutes was awesome!! [09:20] ah great .. looking into the formulas to answer your question [09:21] shang: ok so basically, looking at metadata.yaml .. you will find that the mysql formula provides many possible relations [09:21] like, db, db-admin, shared-db, master .. [09:21] ah! [09:21] so that :db that you add there, is needed to know exactly which relation to use [09:22] right, initially, I thought the name could be anything [09:22] it could be anything for the formula writer :) [09:22] just a name for the relation [09:22] right, :D [09:22] shang: so how are you enjoying your ensembling [09:23] yeah, very fun.... [09:23] awesome :) [09:23] kim0: still looking into how the ensemble and orchestra work together tho [09:23] You'll love it even more when you start writing a formula [09:23] kim0: have to read through the logs again [09:23] a ha .. that's still work in progress [09:24] yeah, I will definitely give that a try [09:24] playing with ensemble on ec2 might be the path of least resistance for now [09:24] but sure .. have fun :) [09:24] shang: let me know once you feel confident enough to write your first formula [09:24] yeah, I test a couple times on ec2 already [09:24] kim0: haha, I will be sure to let u know! :D [09:25] cool :) [09:25] kim0: so the idea is to be able to use ensemble on openstack (like ec2 ) and deploy openstack (orchestra)? [09:26] I think that's correct yes, bu I might be missing some details [09:26] ensemble will definitely use openstack like it does ec2 [09:27] and ensemble will integrate with orchestra but I'm not entirely clear on the details there [09:27] ok [09:27] i will catch up with the logs and see what's going on :) [09:28] cool [09:28] shang: and you can wait for the us to wake up and ask again, you'll get more details :) [09:29] kim0: yeah, I will do more reading first :) [09:29] * kim0 nods .. have fun [09:29] kim0: thanks again for all the hard works! [09:29] * kim0 works on a new and cool ensemble blog post [09:30] * shang wrote a few ensemble 101 post, but will write more fun stuff in the future! [09:30] shang: throw them over and I can tweet them from ubuntucloud :) [09:31] kim0: was very basic, but this is like a ensemble journey to me ;) http://voices.canonical.com/shang.wu/category/ensemble/ [09:32] cool :) === daker_ is now known as daker [09:49] shang: nice writeup, will follow it eventually and do some more testing. Still need to write my very first own formula [09:50] TeTeT: yeah, I will try the same! [13:04] Greetings! [13:20] Hey, happy sysadmin day! [13:53] <_mup_> ensemble/expose-provision-machines r293 committed by jim.baker@canonical.com [13:53] <_mup_> Merged trunk [13:53] <_mup_> ensemble/expose-provision-machines-reexpose r302 committed by jim.baker@canonical.com [13:53] <_mup_> Merged upstream expose-provision-machines [13:56] <_mup_> ensemble/expose-provider-ec2 r307 committed by jim.baker@canonical.com [13:56] <_mup_> Merged upstreadm expose-provsion-machines-reexpose [14:06] Got to restart.. kernel upgrade pending for a while [14:21] <_mup_> ensemble/expose-provider-ec2 r308 committed by jim.baker@canonical.com [14:21] <_mup_> Removed EC2 operation classes in favor of functions for security group functions [14:42] <_mup_> ensemble/expose-provider-ec2 r309 committed by jim.baker@canonical.com [14:42] <_mup_> PEP8 & PyFlakes [14:49] <_mup_> ensemble/expose-provider-ec2 r310 committed by jim.baker@canonical.com [14:49] <_mup_> Reverted a side PEP8 fix, apparently it's possible to have 'd.has_key(key)' and 'key in d' to be different [14:51] too bad, i've been annoyed about that pep8 violation for a while in test_base.py, but it's definitely not in scope for this branch to figure out why that would be the case [15:15] Ensemble crunching UFO data with hadoop :) http://cloud.ubuntu.com/2011/07/ubuntu-takes-ufos-to-the-cloud/ [15:18] fwereade: Wow.. Lots of new branches in review.. were all of these just pending bugs? [15:18] kim0: nice! [15:18] kim0, i think you meant corpus, not corpse ;) [15:18] lol .. fixing [15:18] but when it's about ufos, who knows? [15:18] ROTFL [15:18] ah, corpse fits in with the theme though [15:18] niemeyer: they were all features tidied and moved from andres' branch, which I added bugs to yesterday to ensure they showed up in the queue ;) [15:19] fwereade: and I thought I was free from reviews.. tsc tsc [15:19] niemeyer: if yu weren't expecting them, I may have misunderstood you yesterday? [15:19] fwereade: I wasn't expecting them only in the sense I had no idea we had that many branches ready to land already. :-) [15:20] fwereade: It's awesome to have them, though! [15:20] niemeyer: be careful what order you do them, some of them have >1 prerequisite so the diffs look larger than they really are [15:20] niemeyer: cool :) [15:20] fwereade: Ok.. is it well documented in the description/metadata of the MP? [15:20] niemeyer: should be [15:20] fwereade: Cool [15:20] fwereade: I'll find my way thne [15:20] then [15:21] Looks like that's my afternoon today [15:21] niemeyer: it's just that the MP can only understand a single prereq (AFAICT) [15:21] fwereade: Yeah, but if you documented, I'll find a way to get a reasonable diff [15:21] fwereade: I just won't be happy if I have to guess what are the pre-reqs :-) [15:21] niemeyer: cool :) [15:25] fwereade, that's a whole lot of branches, cool [15:25] jimbaker: cheers, sorry I was unwittingly hiding them all week :) [15:26] fwereade, do please take a look at the my branch expose-provider-ec2. i think it's ready for merging since i've addressed your and niemeyer's questions [15:27] jimbaker: just saw that had landed, will do [15:27] i will be on vacation next week, so i'd like to see this land soon, and not wait to austin [15:28] the requested changes were easy, the painful part was dealing with the refactoring that happened on trunk [15:45] Will get some lunch and bbiab [15:57] <_mup_> ensemble/states-with-principals r300 committed by kapil.thangavelu@canonical.com [15:57] <_mup_> security convience api for integration work. [16:10] <_mup_> Bug #818139 was filed: orchestra: can't terminate instances < https://launchpad.net/bugs/818139 > [16:11] I think I need to stop at the correct time today: so, later all, enjoy your weekends :) [16:11] fwereade: later man [16:11] (it's funny, these timezones make me feel like a slacker ;)) [16:11] ha! [16:11] it works the other way around too [16:11] fwereade, have a good weekend [16:11] we get in and you've got gobs of stuff there [16:12] haha, jolly good, as long as the pain is spread evenly :p [16:12] cheers [16:27] <_mup_> ensemble/states-with-principals r301 committed by kapil.thangavelu@canonical.com [16:27] <_mup_> incorporate the workaround for zk-770 fast auth directly into principal.attach methods. [17:08] jcastro, kim0: language check please... ensemble.ubuntu.com/Interfaces (and the mysql one too) [17:09] on it [17:10] m_3: I'm going to just make you an ensemble interface template [17:10] xclnt [17:12] jcastro: they'll be scripted if that makes any difference for the template [17:24] m_3: is writing a formula consumers really necessary ? mysql consumers list is gonna grow large quick [17:27] m_3: oh ok, so they'll just all output like the mysql one? [17:27] jcastro: yeah, that's the idea [17:27] kim0: nope, not necessary [17:28] oh ok, we won't need a template then [17:28] Yeah drop the consumers [17:28] kim0: it'd be nice though... let's say I'm writing a new formula that consumes mysql... I know where to copy from [17:28] If we want to auto-generate that from the repository we will, but don't manually maintain it [17:28] yeah, the info is in the repo [17:29] m_3: you might want to add CategoryInterface at the bottom of each so moin can group them all [17:29] was gonna try to not _manually_ maintain any of it [17:29] jcastro: thanks, I'll add [17:31] m_3: and <> at the top, so it adds the nice menu thing at the top, I've added a submenu for Interfaces [17:32] do I add the "Interfaces" page to CategoryCategory to get CategoryInterfaces to show up? [17:32] m_3: this is *way* too hard to understand. Fields for what? when do they get set (on join or on changed?) .. also does it wait for incoming stuff? [17:32] yeah [17:33] SpamapS: ok, I'll add more structure to it [17:34] * kim0 starts weekend .. partially afk [17:35] m_3: Hmm [17:36] m_3: Having a table like that at the top as a summary may be fine, but the mysql interface description is lacking the most important bits [17:36] m_3: Also, note that an interface is two-sided.. there's a client and a server [17:36] m_3: I'll do the organizing, as long as you have the CategoryInterface on each page we'll be good, I'll go find out how to add it to CategoryCategory [17:36] niemeyer: yeah, SpamapS was just sayign that [17:37] jcastro: cool [17:37] m_3: Who sets what, and how should the other side react [17:37] m_3: At which time, etc [17:37] niemeyer: understood [17:38] m_3: We really need some prose [17:38] niemeyer: I was actually thinking a picture [17:38] m_3: Also, the interface should be lowercased: "mysql" [17:38] niemeyer: not sure that's maintainable though [17:38] m_3: Pictures are hard to maintain over time [17:39] m_3: Also hard to debate on a mailing list [17:39] niemeyer: gotcha [17:41] m_3: The table is a nice touch, though, as a summary [17:44] first pass... I'll refine [17:45] Okay, reviews [17:50] m_3: to me the format is more hierarchical.. there are 4 events for any relation, and thus, any interface, and 2 participants, so potentially 8 tables. provider->joined,changed,departed,broken and the same for requirer [17:51] broken may not be useful at the interface level tho [17:54] niemeyer, thanks for that approval [17:56] SpamapS: yes, saw your email on it... that made a lot of sense [17:56] jimbaker: np [17:56] SpamapS: one thing that wasn't clear was what you meant by an interface repo as opposed to a formula repo [17:56] jimbaker: Looking at expose-provider-ec2 now [17:57] jimbaker: Trying to understand why we need machine_id in the provider [17:57] niemeyer, sure. the problem is we need to determine the security group associated with that machine [17:58] jimbaker: Machine.instance_id should uniquely identify that machine within the provider, and we have that information [17:58] niemeyer, the security group must be created before the machine is launched, so the machine_id seems appropriate for this sort of info [17:58] jimbaker: Why do we need two identifiers within the provider? [17:58] niemeyer, we only get the machine.instance_id upon launch [17:58] jimbaker: That's right.. hm [17:59] niemeyer, so we need some sort of identifier that exists prior to launch, and can be uniquely associated with the machine [17:59] niemeyer, this is the role of machine_id [17:59] jimbaker: It is, but I'm a bit unhappy with having to cross-reference.. [18:00] niemeyer, agreed [18:00] jimbaker: The machine in zk knows the instance id, and now the machine itself has a reference to the machine id [18:00] niemeyer, sure. but we still need to have the security group created [18:00] niemeyer, this is why i kept them separated in the original API [18:01] niemeyer, we can remove this ugliness once we stop using security groups [18:04] jimbaker: Yeah, let me ponder for a moment if there's a way to avoid having the machine id there [18:05] niemeyer, sounds good [18:05] jimbaker: Just so you understand, doing this means the provider cannot construct such an object anymore [18:05] jimbaker: Because it can't tell what's the id of the machine === daker is now known as daker_ [18:06] niemeyer, it cannot construct an immutable object, that's true [18:07] niemeyer, so maybe the better alternative is to back to the original API and pass machine_id separately from machine [18:07] niemeyer, however this will not be necessary once we remove the security group requirements [18:08] jimbaker: Sure, once we remove the code you're introducing the code being introduced stops being a problem ;-) [18:08] niemeyer, hah [18:09] niemeyer, given that the responsibilities for managing the firewall will also move to the machine agent, it does kill a lot of the code in the predecessor branches too :) [18:10] niemeyer, but at least the provider API remains the same, although that's a small part [18:11] m_3: I mean if we were going to put interfaces in machine readable format, we shouldn't put them in with the formulas. They' don't belong to any one formula [18:12] ok [18:14] <_mup_> ensemble/debug-log-relation-settings-changes r285 committed by jim.baker@canonical.com [18:14] <_mup_> Fix minor formatting nits [18:14] m_3: FWIW, there's no need to define the hook where the setting is made [18:15] m_3: It's fine for this to be implementation specific [18:15] m_3: Due to the way Ensemble works, formulas can wait until the other side provides the setting [18:16] I'm thinking language like... [18:16] When 'relation-joined', mysql: [18:16] provides: [18:16] accepts: [18:17] m_3: Yeah, but that's exactly the point.. we don't have to enforce that [18:17] When 'relation-changed', mysql: etc etc [18:17] m_3: It's fine for a formula to provide the setting at its own time [18:17] m_3: The other side will get told that the setting is now available [18:18] m_3: Let me think of an analogy.. hmm [18:18] niemeyer: so you would consider two different formulas setting the same parameters in different hooks to be using the same interface [18:18] niemeyer: and SpamapS would consider those to be two different interfaces? [18:18] m_3: If you come to visit me, you'll ring the bell, and I'll open the door [18:19] m_3: We don't have to agree at the exact time when this happens, because when you're ready you'll ring the bell, I'll listen to it, and will open the door [18:20] m_3: Yes, I'll consider the same interface, because formulas shouldn't be built on the expectation of the exact hook where something must happen [18:20] m_3: The relation settings are events.. if formula A has to provide a username to formula B, formula B can sit and wait until the username is available [18:21] ok, lemme put language around it and we can go over concrete examples [18:21] niemeyer: what you're saying is, define what to do when you receive a hook, not what to do inside a hook [18:23] SpamapS: Assuming you mean "when you receive a setting", right [18:23] <_mup_> ensemble/trunk r287 committed by jim.baker@canonical.com [18:23] <_mup_> merge debug-log-relation-settings-changes [r=hazmat,niemeyer][f=766317] [18:23] <_mup_> Log (at debug level) the relation setting changes that occur upon a [18:23] <_mup_> successful hook exit. [18:24] formula developers, rejoice! - you can now see all of your relation setting changes in the debug log [18:25] jimbaker: *\0/* [18:26] http://foss-boss.blogspot.com/2011/07/ubuntu-takes-ufos-to-cloud.html [18:26] m_3, the next step is to parse the resulting debug log to generate some sequence diagrams, http://sdedit.sourceforge.net/example/index.html [18:27] woo, so when can we get rid of those first two steps with branching people's formulas? [18:27] m_3, ok, a little bit out as a step... but definitely something that would be cool [18:29] <_mup_> ensemble/expose-provision-machines r294 committed by jim.baker@canonical.com [18:29] <_mup_> Merged trunk [18:31] jimbaker: that would actually be very useful... and damned sexy [18:32] <_mup_> ensemble/expose-provision-machines-reexpose r303 committed by jim.baker@canonical.com [18:32] <_mup_> Merged upstream expose-provision-machines [18:32] jcastro: dunno man... there's fluffy stuff associated with that... like the name for principia for instance [18:32] heh [18:33] end of july! [18:35] m_3, yeah, it would be a cool complement to the docs on interfaces being discussed [18:36] m_3, here's how my formula actually interacts with your formulas [18:36] one could see a riak ring in action - what info is being communicated as the ring is being modified with add-unit/remove-unit [18:37] one swim lane could be the ensemble administrator doing commands, the other swim lanes would be the unit agents [18:37] it'd be interesting to visually debug [18:38] <_mup_> ensemble/expose-provider-ec2 r311 committed by jim.baker@canonical.com [18:38] <_mup_> Merged upstream expose-provision-machines-reexpose [18:38] niemeyer, SpamapS: ensemble.ubuntu.com/Interfaces mysqlA vs mysqlB? [18:40] (general approach) [18:41] m_3: Hmm.. that seems to be going away from what I had in mind [18:42] m_3: What we need is paragraph of text for a human to read [18:43] niemeyer: oh sorry, yes, I understood that... filling in text is next [18:43] m_3: "The mysql interface enables a formula that requires a mysql database to have access to one. (...) The server side of such a relation should be able to provide database information in the `username` and `password` settings to the client side (...) When the client side detects the ..." [18:43] niemeyer: just wanted to get the presentation details straight first [18:44] m_3: Yeah, sorry.. I just mean that all of these tables and hook names just shouldn't be there [18:44] niemeyer: ok, I'll give y'all a little more polished version then [18:45] m_3: Try to think about this as a consumer [18:46] m_3: If you were just starting to use Ensemble, and went into a page about an interface you had to implement.. what would you like to read there? [18:47] niemeyer: right... this was just enough content there to support our email conversation about what information should be part of the interface... certainly not done [19:05] As a consumer I'd want tables of variables available to me, and a description of what they mean [19:06] I'd also want to know what I can assume about the service [19:07] I'd also want to see a list of other similar services so I could copy [19:07] Nah [19:07] if thats there, it needs to be generated [19:07] oh, yeah [19:10] Agreed [19:29] I'd also like a pony [19:31] SpamapS: how'd OSCon session go? [19:31] robbiew: *great* [19:32] robbiew: good response from all.. reasonably well attended [19:32] SpamapS: nice [19:32] robbiew: 2 guys came to the BoF later and I think enjoyed the demo [19:32] about to go to the wrap up plenary... [19:33] and then probably go to the airport and work from there till my flight leaves at 6 [19:33] ok === niemeyer_ is now known as niemeyer [20:08] jimbaker: Review delivered on expose-provider-ec2 [20:08] jimbaker: Please let me know what you thyink [20:08] jimbaker: I've covered the conversatino above there [20:09] jimbaker: I really think we should do something about it [20:09] jimbaker: But the suggestion is not too dramatic, I think [20:09] niemeyer, checking [20:09] jimbaker: Just the same thing, with a less magical API [20:09] Man.. and I didn't even get to William's 500 branches yt [20:09] yet [20:10] I'm half-depressed [20:12] niemeyer, thanks, it's definitely going to be what i work on in austin [20:13] jimbaker: Hmm? [20:13] but i'll try to take some stabs at it now [20:13] i'm on vacation next week [20:13] jimbaker: Oh, I didn't know that [20:14] niemeyer, sorry about that [20:14] jimbaker: LOL, you should be happy rather than sorry :-) [20:14] I think I'm taking a couple of months of holiday myself until 11.10 [20:14] * niemeyer looks at robbiew [20:14] niemeyer, well it should be good, my wife has family visiting us, and they're fun people [20:15] jimbaker: Nice [20:15] * robbiew pretends he didn't read that [20:24] * niemeyer => coffee break [21:09] Okaaaay [21:10] Globo (huge TV channel around here) just made my day by saying in the news that people should not click on any link that ends in a .php because it could be a _trap_! [21:17] niemeyer: save yourself by taking my floppy and clicking on the .exe that comes up. [21:18] hallyn: :-) [21:36] niemeyer: they're still blind to the dangers of .asp then? this is a disaster waiting to happen! [21:37] fwereade: Very true :-) [21:50] bcsaller: Given that you'll be looking at the local dev/LXC stuff, would be nice to have your reviews on William's branch that are up [22:00] niemeyer: I will [22:19] bcsaller: Thanks [22:19] Alright.. that's enough reviewing for me for the day.. [22:20] Have a good weekend all.. I'll step back and lay down for a moment [22:31] niemeyer, take care! [23:56] m_3: does your postgres formula mostly work?