[01:37] <_mup_> ensemble/expose-cleanup r337 committed by jim.baker@canonical.com [01:37] <_mup_> Test new securitygroup function [02:23] <_mup_> ensemble/expose-cleanup r338 committed by jim.baker@canonical.com [02:23] <_mup_> Merged trunk & resolved conflicts [03:03] <_mup_> ensemble/expose-cleanup r339 committed by jim.baker@canonical.com [03:03] <_mup_> Fixed merge issues [13:40] Greetings! [13:55] niemeyer, greetings [13:57] i do find the abbreviations to make reading the go branches a bit harder to read [13:57] it would help if the abbreviations where at least documented once [13:58] +type ifaceExpC struct { [13:58] is interface expander c? [13:59] the variable declaration for the type uses stringC which i assume to be string constant.. but thats a different usage than the preceding statement that uses C for ? [14:09] hazmat: Checker [14:09] hazmat: That convention was used on every single type within schema.go [14:10] hazmat: I can rename it if it feels so tricky [14:10] hazmat: It's akin to _state, though [14:10] niemeyer, its just that without that context it becomes harder to understand, i don't know that it needs renaming but a comment would be useful [14:11] hazmat: I'm happy to rename or to add the comment [14:11] hazmat: // The *C convention is used within schema.go for Checkers [14:11] ? [14:12] niemeyer, what's schema.M? [14:13] hazmat: That's documented [14:13] hazmat: [14:13] / All map types used in the schema package are of type M. [14:13] type M map[interface{}]interface{} [14:13] niemeyer, ic [14:14] hazmat: This is just a handy alias to this type that allows building maps as [14:14] hazmat: schema.M{"a": 1}, etc [14:14] hazmat: map[interface{}]interface{}{"a": 1} would work, but.. :) [14:15] indeeds its an improvement [14:15] hazmat: Yeah, this became clear in tests [14:15] hazmat: I wasn't using it before, but when I started to test the results in the formula work, got boring [14:16] niemeyer, is this sort of multi-file lookup for abbreviations common in go? [14:16] hazmat: multi-file lookup? [14:17] niemeyer, abbreviation used in one place, but needs to be tracked down to a different file to understand naming? [14:17] hazmat: Not a different file.. this is a package, and the type is defined and has meaning within the package itself [14:18] hazmat: schema.M is used for schemas only [14:18] hazmat: Just for comprehension, note that the file could be named foo.go [14:18] hazmat: The "schema" in "schema.M" comes from the package name, not the file name [14:19] hazmat: In other words, I wouldn't use schema.M in things not related to the schema itself [14:20] niemeyer, their not different packages? one is ensemble/go/schema and ensemble/go/formula? [14:20] hazmat: These are different packages, yeah [14:22] hazmat: In these terms it's quite similar to Python.. if you use ensemble.environment.Environment, you may go to ensemble.environment to see what's that [14:22] niemeyer, so in this case we have one package's abbreviation being used in another package, the question is if that sort of usage is common? [14:23] hazmat: The distinction, as I was pointing out, is that unlike Python packages are multi-file [14:23] hazmat: It's not an abbreviation.. it's a type [14:24] hazmat: and the reason why the type is used in a different file is because we're using _schema_ in the other package [14:24] niemeyer, the name itself is an abbreviation, its the name for a type [14:24] hazmat: It's not an abbreviation, it's a real type [14:24] hazmat: reflect.TypeOf(schema.M{}) != reflect.TypeOf(map[interface{}]interface{}{}) [14:25] "M" is not an abbreviation for MapType? [14:25] hazmat: Sorry, I see what you mean [14:25] hazmat: Yes [14:25] hazmat: Let me try to drive a parallel to show you what I mean [14:26] i think this would be much readable if we avoided cross package abbreviations, i'm just trying to understand if this usage is common in go [14:28] hazmat: Alright.. I'll put map[interface{}]interface{}{} back then [14:29] niemeyer, i don't think thats any better, but is there a better name for the type? [14:29] hazmat: schema.M? [14:29] hazmat: That was my take on it.. makes building the map easier [14:29] hazmat: I've used the same convention in other marshalling modules [14:30] hazmat: But I won't bikeshed on that.. if you have other suggestions, I can change [14:34] niemeyer, schema.SchemaMap ? [14:34] hazmat: Heh [14:34] hazmat: Duplicating the schema name won't make it handier or more readable in any way [14:35] hazmat: Also, what's the difference between schema.SchemaMap and schema.Map? [14:36] hazmat: I can rename schema.Map to ... [14:36] MapChecker [14:36] hazmat: schema.GenericMap [14:36] yeah.. that works as well, and M->Map [14:36] hazmat: I can do that as well, as long as you never complain again about naming things with _state.. ;-D [14:37] hazmat: Imagine how the schema definition would look like.. [14:37] niemeyer, sounds good as long as there no complaints about a -(_state) branch ;-) [14:37] hazmat: schema.MapChecker(schema.StringChecker(), schema.InterfaceChecker(....)) [14:38] yeah.. that's lame [14:38] hazmat: Ok, I'll do the name change [14:38] hazmat: But in a follow up branch if that's alright [14:38] Since it'll touch everything else [14:39] niemeyer, feel to change it a different name, i just think having a common entry point into a package be an abbreviation also feels like a violation of encapsulation [14:40] hazmat: I don't understand how that could be the case [14:40] hazmat: There's zero encapsulation involved [14:40] because it tends to require reading the relevant implementation to understand its use [14:40] hazmat: The only reason we use schema.M in formula is because we're dealing with the schema there [14:42] hazmat: I also don't think this is true.. you don't have to read the implementation to understand its use.. you have to read the documentation, which is a pretty general problem [14:42] hazmat: It's like saying we need to read Python's json.dumps prototype to understand how to call it [14:43] what it is and what its doing are fairly obvious by name alone, schema.M is not [14:43] hazmat: Agreed.. it's a convention to be aware of [14:43] so if your reading code that uses both, one is pretty clear without additional context and one is not [14:43] hazmat: The same convention is used in gobson and goyaml [14:44] Actually, I lie.. gobson only it seems [14:45] Anyway.. I was explaining the POV [14:45] I'm fine with renaming it [14:48] hazmat: and answering your very first and simple question, no, this is not a general convention, and if it becomes one it's my fault ;-0 [14:48] ;-) [14:50] niemeyer, thanks, good to know [14:52] <_mup_> ensemble/go-formulas r8 committed by gustavo@niemeyer.net [14:52] <_mup_> Renamed schema.M/L to schema.Map/ListType [14:55] <_mup_> Bug #836753 was filed: schema.M/L are not readable < https://launchpad.net/bugs/836753 > [15:16] fwereade: Gosh.. is all of that really needed for adding _http auth_? [15:38] http://ec2-107-20-64-136.compute-1.amazonaws.com:8080/job/tested%20PPA/42/console [15:39] So, other than the failure at the end because I forgot to quote the 'cd' ... first iteration of integration tests went well. [15:39] lp:~clint-fewbar/ensemble/jenkins-test-suite has the code under misc/jenkins [15:40] I need to break it up into 3 or 4 bash scripts.. but the interesting parts are wait4state and unit2machine .. which I think should become subcommands or at least helper args to some of the commands [15:40] wait4state especially would be best written as a zookeeper watch [15:45] One thing I was struggling with was whether there was a way to address yaml nodes similar to xpath for xml. [15:46] The way that wait4state works is pretty weird and crazy.. but it works for this specific job [15:47] SpamapS, taking a look [15:48] SpamapS, we should definitely make wait4state easier to do. as you said, it really should be using zk watch support [15:50] SpamapS, re some sort of xpath - really this is some sort of query syntax for working w/ python collections. there must be a number of packages out there to do just that [15:51] biab, i have to go to a quick doctor's appt [15:51] * SpamapS will have to vacate his house and go work at a co-working space in about an hour while the exterminators work. UGH [15:54] niemeyer: sorry, had to pop out -- and, as far as I can tell, yes it is [15:54] niemeyer: I looked pretty hard for alternatives I could piggyback on, but nothing seemed cleanly doable [15:58] fwereade: were you at least able to configure the webdav service to auth against cobbler, so the auth credentials can remain the same? [16:01] SpamapS: hm, I didn't think to try that -- if you don't specif separate cobbler credentials, it falls back to the orchestra ones [16:02] SpamapS: same model as the configurable storage-url -- if you don't include one, it assumes http://%(orchestra-server)s/webdav === daker is now known as daker_ [16:03] SpamapS: but I have no idea how to set up cobbler and webdav with a shared auth source [16:04] fwereade: I forget what our default is, but https://fedorahosted.org/cobbler/wiki/CustomizableAuthentication has a run down on the auth methods possible.. authn_passthru would be the simplest way to get this done. [16:04] fwereade: actually authn_configfile will also work because we can point the webdav service at the same config [16:05] SpamapS: easier than I thought then :) [16:08] niemeyer: https://code.launchpad.net/~fwereade/ensemble/webdav-unicode-paths/+merge/72579 -- did you mean to mark that as approved for merge, or should I get someone else to give it a look? [16:13] fwereade: how well does urllib2 handle https btw? [16:13] fwereade: is it going to squawk because of self signed certs or anything like that? [16:14] SpamapS: hm, I have no idea [16:15] fwereade: I'd say that initially it *should* complain with invalid certs, but that we should make it easy to turn that off given Orchestra's "backoffice" role. [16:15] SpamapS: certs for what? [16:15] SpamapS: wounds reasonable, but I'm not sure what context you're asking about [16:15] kirkland: for the webdav file storage [16:16] SpamapS: RoAkSoAx is already using orchestra + preseed to install SSL certs in deployed guests [16:16] SpamapS: for SSL protected rsyslog [16:16] SpamapS: you should be able to just force the webdav to use those same certs [16:16] Seeing as formulas run as root and have no cryptographic verification built in.. its fairly important to consider the security implications they carry. [16:16] SpamapS: or use the same mechanism to install additional certs [16:16] Yeah that sounds good. [16:17] Give the admin a way to easily pull that in to his local cert store. [16:18] I think by default it should be easy and just work w/o intervention. I also think that if the admin needs/wants to crank up the security, they should be able to. [16:19] (And I'm sure some people will argue that by default it should be high security, and admins should be able to turn off security) [16:24] adam_g, negronjl, lynxman: ping [16:25] niemeyer: pong [16:25] negronjl: Hey man [16:25] niemeyer: Happy Monday ( if there is such a thing ) :) [16:26] negronjl: Was just wondering about how you guys feel about how the package and code stability has been progressing [16:26] negronjl: Are you happy with it in general, or do you feel we've been breaking things too often somehow? [16:27] niemeyer: Althought I am very happy with the project in general, I would feel more confortable with some stable version of ensemble that, may not have all of the latest and greates features, but, it is stable. [16:27] negronjl: That's a bit different from the actual question.. I would like to have a stable version too :-) [16:27] niemeyer: I found myself with a very close call on NoSQL conference where I was scheduled to talk about Ensemble but, ensemble was broken [16:28] negronjl: The question is whether we've been breaking things often [16:28] niemeyer: not often no. [16:28] negronjl: Ok, cool.. I understand the desire for stability, and we're doing things to improve that [16:28] negronjl: I just want to make sure I understand the feeling [16:30] niemeyer: To clarify my earlier point. It would help if there is a version in ensemble that would be considered stable enough and we can somehow pin it ( a /stable ppa or so ) until another version becomes stable enough and can replace it. This would alleviate some of the issues that I experienced during NoSQL last week. [16:30] niemeyer: Just my thoughts. [16:31] negronjl: Sounds good.. I hope this is the version in Ubuntu [16:36] * SpamapS too [16:37] negronjl: to niemeyer's point, the version in Ubuntu is actually pretty stable. :) [16:53] yes that's what I was using and it was working fine [17:06] assuming whats in the archive is stable, it would good if nodes install straight from the archive by default, with the option to install a bleeding edge versin from ppa:ensemble/ppa (default now). [17:06] niemeyer: ^ [17:06] fwereade: I did, sorry [17:06] niemeyer: thanks, no worries [17:06] fwereade: Marked it now [17:06] fwereade: Sad about the auth complexity :( [17:06] niemeyer: cheers [17:06] niemeyer: me too :( [17:07] fwereade: It's _auth_, for dev's sake [17:07] niemeyer: I came to terms with it eventually, but... quite [17:08] adam_g: you mean the nodes install something else? Like, trunk? [17:09] ahasenack: i just bootstrapped a node using 0.5+bzr330-0ensemble1, and userdata sets up access to the ppa instead of installing out of the archive [17:10] adam_g: is that ppa "bleeding edge"? [17:12] ahasenack: honestly do not know how/when/why packages get built and pushed there, but installing from there is where ive personally hit show stopper bugs. [17:12] ahasenack, the ppa is a trunk daily build [17:12] ok [17:15] fwereade: Can't we use the same technique twisted.web._auth.digest uses, and rely on twisted.creds.credentials? [17:16] fwereade: It feels like you're reimplementing it [17:19] niemeyer: hm, I was put off by the privateness of the module, and assumed that something would end up breaking if I used a _kindaprivate module [17:19] fwereade: The _auth module has pretty much nothings inside it [17:19] hm, I thought that was where the meat of it was [17:19] niemeyer: let me check again [17:19] fwereade: You're allowed to look inside private modules! ;-0 [17:19] :-) [17:20] "_auth" ;-) [17:20] there's a bug open for it [17:20] it might be nice to ask therve or free [17:20] or on #twisted [17:21] hazmat: There's nothing interesting inside it [17:21] niemeyer: well, crap, somehow I missed cred.credentials [17:21] hazmat: Besides a pointer to twisted.cred.credentials [17:21] niemeyer: I, er, did look :/ [17:21] niemeyer: maybe I stopped looking at that point though :( [17:22] fwereade: Don't worry about it [17:23] fwereade: Maybe it's not even useful.. but certainly has very similar logic, so worth checking [17:23] niemeyer: definitely so, I'll give it a look [17:23] fwereade: In either case, the auth logic itself should be factored out of FileStorage [17:23] fwereade: and the file [17:23] fwereade: and put into its own module, and tested in isolation [17:23] niemeyer: it's not immediately apparent how to hook it up, but you're right, it's doing just the right thing [17:23] fwereade: This is a major missing piece in the http library, rather than part of the FileStorage problem [17:24] niemeyer: it didn't feel quite big enough to do that at the time, but you're right [17:24] cheers [17:24] as another way of getting basic auth into twisted web client ... http://code.activestate.com/recipes/525493-simple-crawler-using-twisted/ [17:25] hazmat: Not sure about what's the point there? [17:26] its a much simpler layering of basic auth onto twisted web client [17:26] hazmat: surely basic auth is barely worth bothering with? [17:27] hazmat: Yeah, I'm trying to understand what's your underlying suggestion.. is it that we should use client.HTTPDownload? [17:27] fwereade: Can we? [17:28] niemeyer: use basic auth? [17:28] fwereade, that's fair, its not secure over http, but over https its fine [17:28] hazmat: Use HTTPDownloader [17:28] Erm [17:28] fwereade: Use HTTPDownloader [17:29] niemeyer: looking at it, we probably can, the only reason I didn't use getPage is because apparently I can't get at the reply headers [17:30] fwereade: gotHeaders? [17:30] niemeyer: yes indeed, that's on HTTPDownloader [17:30] niemeyer: so, looking at it, we probably can [17:31] niemeyer: I saw things saying "if getPage isn't good enough, use Agent"... so I used Agent :/ [17:32] niemeyer: sounds like it's a solid "needs fixing" with a side helping of "lrn2google" [17:32] niemeyer: at least the code'll end up smaller :) [17:32] fwereade: Both approaches are surprisingly involved, to be honest [17:32] niemeyer: the creds stuff looks like it might be quite helpful, I'll look into that [17:33] fwereade: The approach referenced by hazmat seems slightly better because it's more encapsulated than the one in your branch [17:33] niemeyer: there are approaches I didn't consider, so I should definitely look into them [17:33] fwereade: It should be possible to merge your auth with the simpler approach [17:33] hazmat: Which is why I was asking what was the underlying suggestion [17:33] niemeyer: ah, I see [17:34] niemeyer: I think it's worth spending a little while trying to hook up the cred.credentials solution though, that would be best [17:34] fwereade: Sounds good.. I'm happy with any of these options [17:34] niemeyer: anyway, I'm sorry to cut you off, but the calls for suppertime are becoming quite insistent [17:35] niemeyer: I'll try to have something better tomorrow [17:35] fwereade: The only thing that we should necessarily do is to factor that problem auth of the FileStorage [17:35] s/auth of/out of/ [17:35] Contextual typo :) [17:47] SpamapS: This is sitting on our review queue for 2 weeks untouched: https://code.launchpad.net/~clint-fewbar/txaws/fix-s3-port/+merge/71289 [17:48] SpamapS: Can you please address the points for getting it merged, or put it back in Work In Progress otherwise? [17:50] niemeyer: I started working on that about 30 minutes ago actually. :) [17:50] SpamapS: WOohay [17:50] niemeyer: just writing some test code. I addressed the first point before Robert made it. :) [17:50] SpamapS: Great timing then :) [17:51] niemeyer: It seems that all of the tests in txaws are functional tests.. I see no mocking for AWS [17:53] SpamapS: I've tweaked tests there before [17:53] SpamapS: There are unittests [18:12] jimbaker: so.. security group cleanup... [18:13] SpamapS, yes, it's about ready to land [18:13] (in expose-cleanup branch) [18:14] jimbaker: http://ec2-107-20-64-136.compute-1.amazonaws.com:8080/job/tested%20PPA/43/console [18:14] jimbaker: is that a bug? [18:15] SpamapS, expose-cleanup fixes this issue [18:16] jimbaker: so I have to manually delete all the groups before I can re-use the environment? [18:16] what we see here is generally seen when doing an immediate bootstrap after shutdown - the instance originally associated with this group is still around (in 'shutting-down' state) and so cannot be deleted [18:16] SpamapS, once expose-cleanup lands in trunk, you won't have to anything [18:16] the other possibility is to just wait sufficiently long [18:17] jimbaker: the shutdown was 24 hours ago [18:17] I just wait it out [18:17] wow! [18:17] SpamapS, something is holding onto that security group then [18:17] usually cleans itself up after a few minutes [18:18] SpamapS, so expose-cleanup simply polls the instance state to see if they're terminated. if so, it can delete the associated security groups [18:18] or i should say, this polling is done in a revised shutdown_machines in ensemble.providers.ec2 [18:19] SpamapS, please check whether or not you can manually delete that security group [18:19] using the AWS console [18:19] or tool of choice [18:19] how can I see what instances are in a group w/ the cmdline tools? [18:20] SpamapS, i don't believe that's directly possible, but i could be wrong [18:20] i know it's in the reservation part of the instance description [18:21] hmm must have been transient [18:21] <_mup_> ensemble/expose-cleanup r340 committed by jim.baker@canonical.com [18:21] <_mup_> Merged trunk [18:22] its odd, my testing does shutdown+bootstrap pretty constantly.. maybe 10 seconds of lag between those two steps [18:23] 10s is rarely enough time... I usually let it sit for a minute between shutdown and the next bootstrap [18:23] SpamapS, that testing regimen is not going to work well until this cleanup occurs - it needs at least a couple of minutes in between for ec2 to properly cleanup [18:23] jimbaker: but it has worked maybe 20 times so far [18:23] * m_3 surprised [18:24] SpamapS, i'm also surprised [18:24] :) [18:24] sounds like ec2 was especially fast in its cleanup [18:24] I think the way I'll make it work is just to put each target in its own "environment" [18:24] SpamapS, that makes much more sense [18:24] good test of other things too :) [18:25] I need to find a way to generate and dump an environments.yaml into the test chroots anyway. :p [18:25] that will definitely isolate [18:34] <_mup_> ensemble/expose-cleanup r341 committed by jim.baker@canonical.com [18:34] <_mup_> PEP8 & PyFlakes [18:34] niemeyer: pong, sorry it was bank holiday here in the UK [18:35] lynxman: Hey [18:35] lynxman: No worries [18:35] hazmat, i briefly looked at autodoc, but there seems to be some problem in our docstrings which is causing it to crash when i run the sphinx build [18:35] lynxman: Was just wondering about how Ensemble has been treating you.. I've heard a few claims that you guys were facing problems with us introducing new bugs daily, but it sounds like it wasn't really the case [18:36] niemeyer: we had some stability issues in formulas indeed, but as soon as that happens we either ask here or open bug reports [18:37] lynxman: What kind of stability problem was that? Just want to make sure I'm aware [18:37] jimbaker, any tracebacks? [18:37] hazmat, sure, let me paste [18:38] hazmat, actually hold on that - i was going against my expose-cleanup branch, but i didn't want it to get mired in a bigger problem, so i reverted my sphinx conf changes [18:39] hazmat, just fairly simple - update conf.py, plus __init__.py (or use some third party gen scripts to do the walking) [18:40] niemeyer: in my case was just trying to make Ensemble work against openstack, but hazmat helped a lot on it so I didn't need to file any bugs [18:40] hazmat, docutils was crashing in its statemachine.py code w/ a recursion limit issue, so something was indigestible [18:40] lynxman: Ahh, ok [18:40] niemeyer: negronjl and iamfuzz are working on the cloudfoundry formulas which encountered more issues, which were raised on time [18:40] lynxman: Well.. OpenStack support is a minefield at the moment [18:40] so.. does ensemble make use of blueprints at all? [18:40] hazmat, anyway, i will recreate in a bit [18:40] SpamapS, not really bcsaller1 and i where experimenting with them [18:41] but pretty much zero [18:41] niemeyer: it kinda is, but so far works pretty okay :) [18:41] niemeyer, its not that bad [18:41] lynxman: That's good to know :-) [18:41] I was thinking of a suite of tools to make ensemble more command-line-useful ... waiting for states, outputting data about machines/services that is not yaml.. and exit codes to differentiate permanent and transient errrors. [18:41] niemeyer: as said, hazmat helped a great deal [18:42] niemeyer, the ec2 compatibility issue they have marked as a bug, although its unclear when they'll fix it.. but just as far as impact to the ec2 provider, its like a dozen lines [18:42] SpamapS, maybe "ensemble watch" [18:42] hazmat: Yeah, I'm more concerned in conceptual terms than in terms of the particular issue [18:42] not including tests of course [18:42] SpamapS, another thing that might be useful is to have a wait-for ensemble is ready command [18:42] hazmat: It doesn't feel like there's serious interest in preserving the compatibility [18:42] niemeyer, so they will fix ec2 bugs, but not s3 is what i got out of it [18:43] niemeyer, agreed about the overall compatibility.. they have a layer, it has tests, they won't break it, but if something doesn't work, we'll probably need to make it do so [18:43] SpamapS, there are now at least two places that do that last bit for the ec2 provider (via polling ec2_describe_instances); in particular ftests have this functionality [18:43] jimbaker: right, there's really a swath of changes that need a binding purpose.. this is what blueprints are for [18:44] m_3, i would assume this wait-for-ensemble command would be useful for you too [18:44] niemeyer, they have an ec2 compatibility bug for the security group work, that ensemble needs, its still marked as a bug as opposed to the s3 one. [18:44] https://bugs.launchpad.net/nova/+bug/829609 [18:44] <_mup_> Bug #829609: EC2 compatibility describe security group returns erroneous value for group ip permissions < https://launchpad.net/bugs/829609 > [18:44] jimbaker: actually I'd say that zookeeper is the system of record... so polling zk is the only sensible thing to do. [18:44] SpamapS, also in ftests [18:45] specifically a watch for our ZK nodes created by ensemble [18:45] hazmat: Good to know about their feeling about that, thanks [18:46] let me push what i have for that - not yet sufficiently robust, but at least it usually works [18:46] depending on the command probably an agent watch for unit or machine [18:47] <_mup_> ensemble/fix-functional-testing r334 committed by jim.baker@canonical.com [18:47] <_mup_> Better branch identification [18:49] SpamapS, i don't like ensemble.ftests.test_ec2_provider.EC2MachineTest.assert_node yet, but it's on the right path [18:49] note that we still need to poll ec2 first, then *watch* zk [18:50] jimbaker: well my point is that this is something to plan at a larger scale than solving each of these issues one by one. [18:50] jimbaker: there's a real use case for using ensemble in jenkins for instance. [18:51] jimbaker: you only need to ask the provider for the seed so you can find zookeeper. After that you wait for zookeeper, then you listen to zookeeper. [18:51] SpamapS, agreed with that approach [18:56] jimbaker: yes, something that gives us that functionality [18:58] ugh these test cases are really oddly organized [19:03] SpamapS, yeah.. you need an output capture from openstack effectively [19:03] hazmat: for what I did, I just need to verify that the port is put back into the generated urls actually. :) [19:04] SpamapS, that sounds much nicer ;-) [19:04] fixing the security group parsing needs a response output capture to verify the implementation [19:05] hazmat: right, that one at least is being discussed upstream as being fixed .. tho probably not in diablo. :( [19:13] hrm.. txaws's test cases that use venusian fail when it is installed .. looks like they were written to a different venusian api [19:14] http://paste.ubuntu.com/677409/ [19:31] <_mup_> ensemble/trunk r335 committed by jim.baker@canonical.com [19:31] <_mup_> merge expose-cleanup [r=niemeyer,fwereade][f=824219] [19:31] <_mup_> For EC2: shutting down machines removes machine security groups; [19:31] <_mup_> destroying the environment removes the environment security group. [19:32] jimbaker: Just double checking: have you tested it with a real interaction on _trunk_? [19:33] niemeyer, i will double check it one more time - it has worked every time, but i will check as pushed [19:33] jimbaker: Awesome, thanks a lot [19:33] SpamapS: txaws needs venusian 1.0a1 at least, it was just released [19:33] SpamapS: your opinion on this would also be welcome [19:36] niemeyer, looks good as pushed with bouncing repeatedly (ensemble bootstrap && ensemble shutdown && ...), this was the pathological case of what the cleanup now fixes [19:38] SpamapS, m_3 - but nothing like real usage. so please tell me if this doesn't fix the problem you've been seeing [19:38] jimbaker: will do [19:39] m_3, thanks! [19:39] so, I just realized a big problem [19:40] Ensemble by default pulls itself on deployed machines from the PPA [19:40] This means that if ensemble is being driven from, say, mac ports.. at revision 295 .. it may deploy incompatible versions of ensemble onto the machines. [19:41] I *think* have an answer for this.. but it may be a little bit weird... [19:42] SpamapS, there's an outstanding bug about this that i started looking into [19:42] bug 828147 [19:42] <_mup_> Bug #828147: Ensemble branch option needs to allow for distro pkg, ppa, and source branch install < https://launchpad.net/bugs/828147 > [19:43] SpamapS, it's not quite what you described just now, but it does allow for precise specification of what it should be on the target environment [19:45] jimbaker: it has to automatically choose itself though [19:45] SpamapS, exactly, it would be nice for an automagic option in ensemble-origin [19:45] I can't think of any time where I want to, by default, deploy something other than what I'm running. [19:45] unless we guarantee backward compatibility [19:46] SpamapS, we actually are planning to do just that for client access [19:46] jimbaker, we're not guaranteeing backward compatibility [19:46] there's now a protocol version that's stored in zk (in the topology node to be precise); it's checked for compatibility on every operation working w/ topology (which is most of them) [19:46] we're just detecting a conflict [19:47] hazmat, sorry i've reinterpreted the problem so it works like you said :) [19:47] we only guarantee backwards compatibility if the protocol version is compatible [19:48] this still allows for some wiggle room [19:49] SpamapS, the txaws tests run fine on natty fwiw [19:49] hazmat: yeah the tests run fine for me w/o venusian [19:49] on oneiric [19:51] * ahasenack is invisible [19:54] SpamapS, ping me when you've got tests pushed, i'll merge to my updated my branch (~hazmat/txaws/fix-s3-port-and-bucket-op/) and push for review [20:00] hazmat: Does that look good: [20:00] c.Assert(meta.Provides["server"], Equals, formula.Relation{Interface: "mysql"}) [20:00] ? [20:01] niemeyer, that looks great [20:01] hazmat: Awesome, I'll move forward with it then, thanks [20:03] hazmat: I pushed over an hour ago [20:03] SpamapS, awesome, good to know, i'll merge and push the additional fixes for review, thanks [20:03] does ensemble have to be used w/ EC2 or can you use it with, say, an in-house ESXi environment? [20:04] ah, wait, found the FAQs finally [20:04] OldSchool, no esxi support at the moment, ensemble supports pluggable providers, and we're working on some additional ones at the moment (physical machine, openstack, local dev), but no plans atm for esxi [20:04] hazmat: no, thank *you* :) [20:06] thank you [20:06] OldSchool, it shouldn't be too bad to add one, implementing a provider has gotten cleaned up quite a bit, and made easier, but its probably going to need community contribution.. if your interested folks around here be happy to help you out [20:07] sweet, I just heard about it from a Canonical rep on Friday after doing some tinkering w/ Puppet and Landscape [20:08] I only admin 4-5 Ubuntu boxes at work and they're pretty self-sufficient, I'm just always interested in automation, more curious for future projects [20:12] OldSchool, if you want to give ensemble a test drives, its probably to use ec2.. we'll be supporting a local development mode in the 11.10 release for ubuntu laptop/desktops which will also allow some experimentation and test driving [20:13] s/probably/probably best [20:22] awesome [20:22] I'll have to try out EC2 [20:26] thanks for your help, gotta run but I'm sure I'll be around === daker_ is now known as daker [21:34] argh, why does vim think 'description:' needs to be un-indented in yaml? [21:50] <_mup_> ensemble/go-formulas r9 committed by gustavo@niemeyer.net [21:50] <_mup_> Adding sample formula repository content for tests. [21:51] <_mup_> ensemble/go-formulas r10 committed by gustavo@niemeyer.net [21:51] <_mup_> Added support for pasing relations within the formula metadata. [21:51] <_mup_> This completes support for formula metadata parsing. [21:55] <_mup_> Bug #837027 was filed: Formula metadata parsing in Go must be completed < https://launchpad.net/bugs/837027 > [21:58] <_mup_> ensemble/stack-crack r329 committed by kapil.thangavelu@canonical.com [21:58] <_mup_> be flexible about int vs. str status codes, remove error test relying on error result response parsing. [22:04] <_mup_> ensemble/stack-crack r330 committed by kapil.thangavelu@canonical.com [22:04] <_mup_> remove the ec2 key name usage, to verify its nesc. [22:28] ensemble: environments [22:28] environments: [22:28] sample: [22:28] type: ec2 [22:28] control-bucket: ensemble-9cb5c6eea8334c9780078fe63f6ccdb0 [22:28] admin-secret: 06ae08af8e0f4dd79180401bc28824b0 [22:28] access-key: AKIAISCMJNNSSTGM4L2A [22:28] secret-key: PAWXMkTfIpfySaYuayzK3jnGOmOBBzodJ9ysyqPW [22:28] That does't work and this does... [22:28] ensemble: environments [22:28] environments: [22:28] sample: [22:28] type: ec2 [22:28] control-bucket: ensemble-9cb5c6eea8334c9780078fe63f6ccdb0 [22:28] admin-secret: 06ae08af8e0f4dd79180401bc28824b0 [22:28] access-key: AKIAISCMJNNSSTGM4L2A [22:28] secret-key: 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 [22:29] also any help on how to pre-seed a environments.yaml file before the bootstrap? [22:32] botchagalupe: Hey [22:33]