/srv/irclogs.ubuntu.com/2011/09/25/#juju.txt

_mup_juju/unit-with-addresses r395 committed by kapil.thangavelu@canonical.com04:44
_mup_unit pub/priv address retrival on ec2, orchestra, and lxc04:44
_mup_juju/unit-with-addresses r396 committed by kapil.thangavelu@canonical.com05:01
_mup_service unit state accessors/mutators for public/private addresses05:01
koolhead17hi all06:45
_mup_juju/unit-with-addresses r397 committed by kapil.thangavelu@canonical.com06:47
_mup_dummy unit address for tests06:47
koolhead17SpamapS: hey10:13
hazmatmornings11:45
koolhead17hi hazmat11:57
koolhead17although its evening for me :D11:57
koolhead17https://juju.ubuntu.com/docs/write-formula.html12:05
koolhead17now will i replace ensemle to juju12:05
koolhead17ensemble: formula == juju: charm12:06
=== niemeyer_ is now known as niemeyer
_mup_juju/go-charm-bits r12 committed by gustavo@niemeyer.net20:04
_mup_Improvements and fixes in charm directory packing:20:04
_mup_- Ignore hidden files as the Python version.20:04
_mup_- Use the new filepath.Walk interface.20:04
_mup_- Pack unix file mode into the charm bundle.20:04
_mup_The last change requires this Go CL:20:04
_mup_  http://codereview.appspot.com/5124044/20:04
_mup_Bug #859151 was filed: Improvements and fixes in charm directory packing <juju:In Progress by niemeyer> < https://launchpad.net/bugs/859151 >20:08
hazmatniemeyer, do you want another look at the placement stuff?20:21
hazmatniemeyer, also i think bcsaller addressed the review comments on the clone-lib20:22
niemeyerhazmat: Have you changed the placement stuff since we last talked?20:22
hazmatwe've basically got things working and factored out, just landing things into the queue and trunk in the right order20:22
hazmatniemeyer, yes.. its basically redone to use a provider provided list of supported policies, intersected with the cli option and environment option20:23
hazmatniemeyer, just pushed the latest20:23
niemeyerhazmat: Unless it'd make a difference for bcsaller (as in, he'd work on it now), I'll review it first thing on my morning tomorrow before he comes online20:23
niemeyerhazmat: Sure, I can check it out20:23
hazmatniemeyer, he'd work on it now.. he's pending on pushing another branch into review, but its got a two headed pre-req20:23
bcsallerI'll push the merge tip now, but I could really use the rest of the day off :)20:24
hazmatwith lxc-provider-config (which is held up on placement) and lxc-lib-clone20:24
hazmatso we need to get at least one of the pre-reqs in or the merge proposal diff is going to be messy20:24
hazmatand then i've got the status/ssh/debug-hooks working with that as a pre-requisite20:24
niemeyerbcsaller: Sure.. I imagined that was the case.. I'll review it tomorrow during my morning then20:25
hazmatie.. (provider-placement->lxc-provider-config && lxc-library-clone ) -> lxc-omega -> units-with-addresses -> local-cli-support20:25
niemeyerhazmat, bsaller: two-headed pre-reqs are not a big deal.. just mention the puzzle in the summary so that I can sort out the base20:26
hazmatniemeyer, cool20:26
hazmatbcsaller, cool, then if your comfortable with lxc-omega can you put that into review, i'll push the last two into the review queue, i've got some minor work to finish on the local-cli-support (debug-hooks tests, and a functional test round)20:27
hazmatit doesn't need to be in the review queue, but its just nice for pre-reqs20:27
hazmatits been interesting to see how bzr/lp work with concurrent dev20:28
hazmatlots of interconnected dev20:28
_mup_juju/trunk r361 committed by gustavo@niemeyer.net20:28
_mup_Added .dir and hooks/install in repository/dummy test data.20:28
_mup_This is just to sync up with the Go port.20:28
_mup_[trivial]20:28
_mup_Bug #859180 was filed: LXC driven local development story <juju:New> < https://launchpad.net/bugs/859180 >20:32
koolhead17hazmat: niemeyer hello20:39
hazmatkoolhead17, hello yes re replace in docs ensemble/juju formula/charm20:39
hazmatkoolhead17, canonical does require a contributor agreement though .. http://www.canonical.com/contributors20:40
koolhead17hazmat: you mean i cannot contribute to juju without signing that? :P20:41
koolhead17hazmat: also wordpress/hooks/db-relation-changed20:41
koolhead17can i use this in case of other charms too ? hostname=`curl http://169.254.169.254/latest/meta-data/public-hostname`20:42
koolhead17?20:42
hazmatkoolhead17, yeah.. that's how contrib agreements work.. actually they've become quite common in corporate or foundation backed projects (openstack, plone, etc) all use them20:42
koolhead17hazmat: will sign it once i start contributing, all this while i been only reporting bugs :D20:43
hazmatah.. those are welcome, but if your going to submit a patch to the core it will be needed20:43
koolhead17hazmat: point noted!!20:44
hazmatkoolhead17, charms don't count.... those are your own unless derived from an existing20:44
niemeyerhazmat: We can move forward with this given timing20:44
niemeyerhazmat: But I have a pretty bad feeling about the back and forth going on20:44
hazmatkoolhead17, a patch to the docs would count towards a core contrib though the way things are structured in the repo20:44
niemeyerhazmat: command line knows about provider, placement, and env; placement knows about env and provider; provider knows about env and placement20:46
niemeyerhazmat: and in the end we're solving an extremely simple problem, that right now would work fine with "placement = name" in the provider class20:47
hazmatniemeyer, i pushed into placement because, else we'd be duplicating the logic in the commands20:47
hazmatthe pick_policy stuff is just a factoring out of what the cli uses to do just that20:47
niemeyerhazmat: We don't need any logic in the commands either right now20:47
niemeyerhazmat: We'd be totally fine just hardcoding the placement in the provider20:48
hazmatniemeyer, then the user has no selection?20:48
hazmatniemeyer, yeah.. i could drop list_policies easily20:48
niemeyerhazmat: Yep.. that sounds fine right now20:48
niemeyerhazmat: I mean, giving the user no selection20:48
niemeyerhazmat: Each provider has only a single option that works20:48
hazmathmmm20:48
niemeyerhazmat: the --placement option is a lie, pretty much20:48
hazmati'd like to introduce new policies in the cycle20:49
hazmatniemeyer, only against local20:49
hazmata min/max machines policy if we can get lxc working on ec2 would be sweet for example20:49
niemeyerhazmat: I also feel bad with state.policy poking into provider.config20:49
niemeyerhazmat: It knows about the provider schema, when it shouldn't20:49
hazmatyeah..20:49
niemeyerSorry, state.placement20:50
hazmatniemeyer, with the provider not having selection anymore.. somebody has to poke at the config20:50
niemeyerhazmat: This is the default..20:50
hazmat?20:50
niemeyerhazmat: The first entry in the preferences list20:50
niemeyerhazmat: get_placement_policies()20:51
hazmatniemeyer, it is the default in the absence of user choice20:51
hazmator configuration20:51
niemeyerhazmat: If we're putting the user choice in the provider's configuration, the provider should validate the user choice, not state.placement20:51
hazmati'm crazy tired btw... i might be a little dense picking things up.. i've had 3hrs sleep over the last 4820:52
niemeyerhazmat: Ouch20:52
hazmatactually i exagerate its more like 3620:52
koolhead17hazmat: go sleep :D20:52
hazmatkoolhead17, much to do for the release20:52
koolhead17ooh. when is it happening?20:53
niemeyerhazmat: So, suggestion to clean things up:20:53
hazmatkoolhead17, well we're trying to get into the oneiric cycle so we need to be there with some testing time, at this point its monday20:53
niemeyer(btw, again, I'm fine with doing this later if you'd rather not work further on that)20:53
hazmatniemeyer, okay.. suggest away20:54
koolhead17waoo. ok20:54
niemeyer1) Remove the placement option from the command line entirely20:54
niemeyer2) Support it in the provider's configuration20:55
niemeyer3) Validate it just like we validate the rest of the provider configuration20:55
niemeyer4) Make Provider.get_placement_policy() a single item again, which is either the default preferred for the provider, or the placement config20:55
niemeyer5) Kill pick_policy()20:55
hazmatniemeyer, so basically revert to previous and drop the preference arg20:56
niemeyerhazmat: Well, you've also moved to support config in the env20:56
hazmatniemeyer, ? that support was already there (placement config in env yaml)20:58
niemeyerhazmat: I missed it then20:58
hazmatniemeyer, thats why the previous usage was doing things like serializing the env, so it could pick out the value on the cli20:58
hazmatactually i think i yanked that a few branches back20:59
niemeyerhazmat: Hmmm..20:59
niemeyerhazmat: Yeah, I'm pretty sure it wasn't in the branch I reviewed lsat20:59
niemeyerlast20:59
niemeyerhazmat: Either way, it looks like a good idea.. certainly more sensible than the command line option21:00
niemeyerhazmat: It will also avoid the wide distribution of knowledge going on21:00
niemeyerhazmat: and probably become tirvial21:00
niemeyertrivial21:00
hazmatso we'll rely on env yaml schema for validation.. yeah.. that sounds reasonable21:01
hazmatbcsaller, you still around.. just curious if you had any concern over dropping cli placement21:01
hazmatniemeyer, okay i'll have a look at reverting and simplifying after i finish up the cli support and do some end-to-end testing21:06
hazmati think i'm gonna have a nap first though21:06
niemeyerhazmat: Awesome, thanks a lot for that, and please do.. well deserved21:07
niemeyerhazmat: I'm pushing some additional tweaks to the Go bundling fixes21:07
niemeyerhazmat: It wasn't packing directories, now it will21:07
hazmatdoh..21:07
hazmatniemeyer, cool saw some of the commits floating by21:07
niemeyerhazmat: Not in the sense you probably think, though21:07
niemeyerhazmat: zip files work fine without intervening directories21:08
bcsallerhazmat: people have used placement as a kludge for somethings like doing deployments all to one machine apparently21:08
niemeyerhazmat: e.g. "foo/bar" is unpacked fine21:08
bcsallerbut I'm not really opposed21:08
niemeyerhazmat: but after a while they started to pack "foo/" as an entry too21:08
niemeyerhazmat: as a consequence the Go port wasn't handling empty directories21:08
hazmatbcsaller, but they could do that via env.yaml config just as well afaics21:08
niemeyerYeah21:08
bcsallerhazmat: yes, but if you can picture them having to change that file between deploys to get the effect they want then we have other work to do ;)21:09
_mup_juju/go-charm-bits r13 committed by gustavo@niemeyer.net21:09
_mup_Bundle directories into the zip as well, so that empty directories are21:09
_mup_handled properly.21:09
hazmatbcsaller, better to come up with a min/max placement to solve the issue i think21:10
hazmatbcsaller, afaics there's two usage modes.. fast deploy against local21:10
bcsallerhazmat: I think SpamapS and adam_g have both used it and would be better to ask than me. I don't strongly feel the need to keep it but I don't want to make it harder to do something people can already do21:10
hazmatactually that it... i don't think folks are doing it adhoc21:10
hazmatie per deploy21:10
bcsalleryou might be right21:11
bcsallerthe intention was to expand on the allowed models though21:11
hazmatif we had min/max placement, with some pre-booted background machines, and max cost of machines (max machines), i think people would use that in preference to the kludge usage21:11
bcsallerto pair a cpu bound and i/o bound set of services for example21:11
hazmatbcsaller, local placement doesn't do that21:12
hazmatalthough i see your point21:12
bcsallerhazmat: right, its not about local21:12
niemeyerDone!21:12
niemeyerDirs, modes, etc.. all works well.  Tested with "zip" as well to make sure everything is compatible.21:12
bcsallerniemeyer: that is such a satisfying word.. Done21:12
niemeyerI'll go outside for a while to check out a bit of day light..21:13
niemeyerbcsaller: Very true!21:13
hazmatbcsaller, hmmm.. maybe its magic pie in the sky.. but ideally juju could sample and do that for us, rebalancing units as needed21:13
hazmatto get max efficiency out machines21:13
hazmatcrazy talk21:13
bcsallerhazmat: our model has always been to build the primitive first and then think about controlling layers later21:14
hazmatbcsaller, but that sort of manual placement isn't easily encoded in a policy21:14
bcsallermanual or strategic ... agreed that modeling that is hard21:14
hazmatbcsaller, that's admin knowledge of machine usage and assigned units, with manual assignment to a particular machine when deploying21:14
bcsallerand the basis is usually cost-savings21:14
hazmator performance benefits21:15
bcsallerutilization21:15
hazmatie deploy the namenode on a machine with these characteristics21:15
bcsallerwe are not really prepared to assist on performance as we intend to isolate in VMs anyway21:15
hazmatbcsaller, its more a question of capacity and machine characterstics guided by usage constraint21:16
bcsallerand things are move during their lifetime, its gets complex, like machine 1 goes down and those 2 services end up where?21:16
hazmatwhen deploying a service21:16
hazmatbcsaller, volume management will open up new doors on this stuff21:16
bcsalleragreed, it will help21:16
hazmatits just not clear that placement helps here21:17
bcsallerbut the per thing is often about faster IPC which we don't naturally aid in21:17
hazmatbecause its either trying to describe machine constraints of capacity, or against current usage assuming sampling21:17
hazmatfor this scenario21:17
hazmathmm21:18
bcsallerhazmat: the placement strategies were supposed to be smart than they are now, but yeah, in their current form they don't allow that kind of reasoning21:18
hazmatcross-az placement21:18
hazmatis interesting here though21:18
hazmatdeploy these units... for this unit deploy in a different az21:18
bcsalleryeah21:18
bcsallerwhich the openstack stuff wants anyway21:18
hazmatbut we're overloading placement policies, we'll want to have multiple policies with that scenario given our current usage of placement21:19
hazmator overlapping responsibility21:19
bcsallersounds like the choice is grow or die, I'd say grow, but we need more info before we do that21:20
bcsallerok, off  for a while again21:21
hazmatyeah.. sleep time21:23
hazmatbbiab21:23

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!