/srv/irclogs.ubuntu.com/2012/10/01/#juju-dev.txt

=== philballew_ is now known as philballew
Guest7958fwereade: mornin'06:51
fwereadehi rog(?) ;p06:51
=== Guest7958 is now known as rogpeppe
rogpeppefwereade: not sure why it does that. surely noone else is using "rogpeppe"...06:54
fwereaderogpeppe, I *think* I saw your name change spontaneously at some point yesterday06:54
fwereaderogpeppe, I have a couple of CLs up, and I would appreciate your thoughts06:54
rogpeppefwereade: i'm having a look right now06:55
fwereaderogpeppe, sweet, tyvm06:55
rogpeppefwereade: i saw this:06:55
rogpeppe[17:25:19] [Notice] -NickServ- rog__!~rog@ool-45754a58.dyn.optonline.net has released your nickname.06:55
rogpeppe[17:25:20] *** You are now known as Guest7958.06:55
fwereaderogpeppe, omg, it's that nickserv dude again! he's almost as bad as peer!06:55
fwereade(sorry)06:56
TheMuemorning07:08
fwereadeheya TheMue07:37
TheMuehey, hey, fwereade07:38
TheMue;)07:38
rogpeppeTheMue: hiya08:04
TheMuehi roger08:04
rogpeppefwereade: ping08:35
fwereaderogpeppe, pong08:36
rogpeppefwereade: i'm looking at the filter code, and had a little question08:36
fwereaderogpeppe, go for it08:36
rogpeppefwereade: why only receive on the want* channels when there's an event to send?08:37
rogpeppefwereade: would it be a problem to always wait on those channels?08:37
fwereaderogpeppe, because receiving on the want channels will unblock the sending channel and we don't want them to send nils08:38
fwereaderogpeppe, and we can't guarantee they won't until we've received a change08:38
rogpeppefwereade: but aren't the sending channels themselves nil until an event arrives?08:38
fwereaderogpeppe, happily, all  the watchers guarantee us initial events, so it can't be blocked for a long time08:38
fwereaderogpeppe, the action of a receive on a want channel is to un-nil the sending channel08:39
rogpeppefwereade: is that necessary?08:40
rogpeppefwereade: erm, yes, i see08:40
fwereaderogpeppe, in the context of that code, the condition of wanting to send on an out chan is defined by it being non-nil08:40
fwereaderogpeppe, it means that want* calls can all block for a short time until the filter is ready to deal with them, but the filter should be guaranteed to either become ready or fail, assuming the watchers themselves work as advertised08:41
fwereaderogpeppe, and the failing will itself unblock the want calls08:42
fwereaderogpeppe, and hence, all is sunshine and puppies, I think08:43
rogpeppefwereade: i'm definitely starting to see puppies in the sun08:43
* fwereade feels cheered08:44
rogpeppefwereade: one other thing: i'm wondering if it's possible to make the dying status propagate without another step through the watcher08:49
fwereaderogpeppe, I thought about that and decided that having One True Path for the information was the clearest thing I could do08:50
fwereaderogpeppe, it would not be a problem to check-and-set the local life var, and close the channel immediately though08:50
rogpeppefwereade: yeah, i was wondering if there was an elegant way to do that08:51
fwereaderogpeppe, it just feels more complex and prone to embarrassing screwup ;)08:51
rogpeppefwereade: i do know what you mean08:51
rogpeppefwereade: i guess it's just my premature-optimisation head saying "5 more seconds! 5 more seconds!"08:51
fwereaderogpeppe, well, life could actually be a filter field and we could have a separate hey-the-unit-is-dying method08:51
fwereaderogpeppe, it's a reasonable concern, any number of crazy operations could be kicked off in 5 seconds08:52
rogpeppefwereade: i was mainly thinking about responsiveness, but that too, yeah08:52
fwereaderogpeppe, if the life-field-plus-method SGTY I can do that, it does feel more ruthlessly efficient in a pleasant sort of way08:52
fwereaderogpeppe, but then... I dunno08:53
fwereaderogpeppe, in terms of sheer simplicity of implementation it's a mere Sync away...08:54
rogpeppefwereade: true. i seem to remember it being considered bad practice to call Sync in a watcher though.08:55
rogpeppefwereade: but i definitely see the attraction08:55
fwereaderogpeppe, leave some sort of comment for discussion, I imagine niemeyer will have input too ;)08:56
rogpeppefwereade: i'm sure of it :-)08:56
fwereaderogpeppe, crack-check on another idea08:57
fwereaderogpeppe, move most of cmd/jujuc/server -- ie the commands, HookContext, and associated types -- into worker/uniter/hook08:58
rogpeppefwereade: +108:58
rogpeppefwereade: it's part of the worker after all, right?08:59
fwereaderogpeppe, I'm quite certain I will do more with the idea but I feel I can't start until I've moved them and absoerbed the impact08:59
fwereaderogpeppe, I have an intuition the the commands themselves may want to move somewhere else08:59
fwereaderogpeppe, but yeah08:59
rogpeppefwereade: let me just have a quick look at cmd/jujuc/server again09:00
rogpeppefwereade: this is where i come up against the "what things are actually intended to be exported?" thing09:01
rogpeppefwereade: looking at http://go.pkgdoc.org/launchpad.net/juju-core/cmd/jujuc/server - which functions are actually part of the package API and which are just there for testing purposes?09:02
fwereaderogpeppe, some of RelationContext's methods may not technically need to be exported09:02
fwereaderogpeppe, and if you want to get technical about it we *could* make all the commands internal-only, but I think that actually ossifies the package09:02
rogpeppefwereade: yeah, i was wondering about NewClosePortCommand and friends09:03
rogpeppefwereade: i was trying to work out when we'd need them09:03
fwereaderogpeppe, ATM the only client is NewCommand or whatever it is09:03
rogpeppefwereade: i'm not sure what you mean by "ossifies" actually09:03
fwereaderogpeppe, adds to the burden of moving parts of it as it becomes more apparent that they're different packages09:04
rogpeppefwereade: ok, so everything else other than NewCommand could be private?09:04
fwereaderogpeppe, and HookContext and most if not all its methods09:04
rogpeppefwereade: i don't think that's a substantial problem - renaming is easy09:04
fwereaderogpeppe, and RelationContext and several of its methods09:05
rogpeppefwereade: this is why i think we should have a more rigorous approach to private/public naming. the important parts of the API are obscured by all the bits that could be private.09:06
rogpeppefwereade: but regardless of that, i think it would sit well under worker/uniter09:10
Arammoin.09:11
fwereaderogpeppe, sorry, got a phone call09:16
rogpeppefwereade: np09:16
rogpeppefwereade: you've got a filter review09:17
fwereaderogpeppe, tyvm09:17
rogpeppefwereade: what does it mean for a relation to be "broken" rather than "departed" (looking at the comment "TODO don't stop until all relations broken.")09:21
rogpeppe?09:21
fwereaderogpeppe, has executed the -broken hook (and left the scope)09:21
fwereaderogpeppe, ie when that relation will never again be touched by that unit09:21
rogpeppefwereade: ah, so "broken" for a relation is a kinda edge-triggered thing, rather than a steady-state broken like the unit broken thing?09:22
fwereaderogpeppe, yeah, it's an awful name09:22
rogpeppefwereade: it certainly confused me :-)09:22
fwereaderogpeppe, I'm not sure there's anyone it doesn't09:23
rogpeppefwereade: so the "broken" in this context is like a connection breaking - it doesn't imply anything going wrong at all?09:24
fwereaderogpeppe, yeah, exactly09:24
rogpeppejeeze09:24
rogpeppefwereade: that's something the docs are going to need to be very careful about...09:24
rogpeppefwereade: just looking at modeContext - is there ever a mode that would *not* want to use modeContext?09:28
rogpeppedavecheney: hiya09:28
fwereaderogpeppe, basicaly no09:29
rogpeppefwereade: why not factor it out into the loop that's calling the modes then?09:29
davecheneyrogpeppe: hey09:29
fwereaderogpeppe, because the bits that need to know the name (which include the error annotation) can't be used there, because you can't turn a Mode into a name09:30
rogpeppefwereade: hmm, the mode name might be a problem09:30
rogpeppefwereade: you could always do something like this, i suppose: http://paste.ubuntu.com/1253570/09:39
rogpeppefwereade: i do feel that modeContext feels a bit like a hack in the midst of nice clean code.09:41
fwereaderogpeppe, I feel that modeContext enables a fair amount of that code's cleanliness09:41
rogpeppefwereade: if you could get the name from a mode, then you could still have that cleanliness without modeContext AFAICS09:42
fwereaderogpeppe, although, hmm, actually the non-name bits would probably be better if they were stuck in the main loop09:42
rogpeppefwereade: i think so09:42
fwereaderogpeppe, sgtm09:43
rogpeppewoo hoo, just got a 50 pound royalty payment.09:49
fwereaderogpeppe, nice!09:55
fwereaderogpeppe, what for?09:55
rogpeppefwereade:  a tune i wrote some time ago09:55
rogpeppefwereade: someone put it on their album09:55
fwereaderogpeppe, awesome :D09:56
TheMuerogpeppe: +109:56
rogpeppefwereade: it'll at least pay for my next bow re-hair...09:57
rogpeppefwereade: (which is long overdue as it happens)09:57
fwereaderogpeppe, excellent :)09:57
TheMuerogpeppe: where can we d/l your first album? ;)09:57
rogpeppeTheMue: when i record it. that's predicted to be in NaN years.09:58
TheMuerogpeppe: ok, i plan to get old, but not so old.:(09:59
TheMuerogpeppe: different topic. anything special to know when doing ec2 tests (beside setting the environment with the keys)?10:00
rogpeppeTheMue: you should set the test timeout, because it takes a while10:00
rogpeppeTheMue: here's the script i use (i call it "livetest"):10:01
rogpeppego test -amazon -test.timeout 2h -gocheck.vv $* >[2=1] | timestamp10:01
rogpeppefor bash, it would be:10:01
rogpeppego test -amazon -test.timeout 2h -gocheck.vv "$@" 2>&1 | timestamp10:01
TheMuerogpeppe: great, thx10:02
rogpeppei find the timestamp bit very useful, as i can glance over at the running test and see if it's hanging up unreasonably or progressing normally10:02
TheMuerogpeppe: if i do a "go test" in environ/ec2 and get an error that "juju-sample" is not found, what did i wrong?10:58
rogpeppeTheMue: hmm, good question, i'll have a look11:06
rogpeppeTheMue: could you paste the output of go test -gocheck.vv ?11:07
TheMuerogpeppe: sure, one moment11:07
TheMuerogpeppe: http://paste.ubuntu.com/1253687/11:09
rogpeppeTheMue: is this running live tests? on trunk?11:10
TheMuerogpeppe: it's a fresh branch of trunk, just to get sure it's non of my changes11:10
rogpeppeTheMue: this happen even with you're not running with the -amazon flag?11:17
rogpeppes/happen/happens/11:17
rogpeppes/with/when/ :-)11:18
TheMuerogpeppe: yes, just a play go test11:18
TheMuerogpeppe: ah, tried it now with -amazon, looks different11:19
rogpeppeTheMue: i'm wondering if you've got an old copy of the goamz package11:19
TheMuerogpeppe: i'll force an update, to get sure11:20
rogpeppeTheMue: my version of goamz is on revision 1311:20
rogpeppeTheMue: and trunk environs/ec2 passes all tests for me11:21
rogpeppeTheMue: (N.B. go get -u doesn't work with bzr repositories)11:21
TheMuerogpeppe: uuh, i've got the rev 1111:22
TheMuerogpeppe: thx, i'll update now and test again11:22
rogpeppeTheMue: that should fix your problem11:22
Aramrogpeppe: what GUI tool do you use for diff?11:22
rogpeppeAram: qbzr11:22
rogpeppeAram: it's not as good as codereview, but much more immediate :-)11:23
Aramcan it help with merges?11:23
rogpeppeAram: good morning, BTW!11:23
Aram(as in, is it a three way diff tool?)11:23
Arammoin.11:23
rogpeppeAram: no. there is a tool around that does, but i can never remember its name11:23
AramI'm fine with raw diffs when I need a diff, but merge conflicts really confuse me.11:24
Aramso I'm searching for a better tool.11:24
TheMuerogpeppe: thx a lot, you've been right, now the rev 13 runs fine11:25
rogpeppeTheMue: cool11:25
TheMueAram: moin from here too11:25
Aramhi.11:25
rogpeppefwereade: stupid bzr question: if i've got a full revision id, how can i make a branch from it?11:34
fwereaderogpeppe, sorry, i have no idea11:34
rogpeppefwereade: thanks11:34
rogpeppeah! done it!11:35
rogpeppemramm: morning!11:57
mrammmorning11:58
mrammHow goes?11:58
TheMuemramm: hiya11:58
Aramhi.11:58
rogpeppemramm: pretty good, just about to propose the final piece of the upgrade logic (the --bump-version flag on upgrade-juju)11:58
mrammawesome12:00
Arammeh, I'm using p4merge, all these open alternatives suck.12:01
rogpeppeAram: i usually find that apart from very occasional cases, doing it inline is fine12:02
rogpeppefwereade: relatively trivial: https://codereview.appspot.com/659305312:02
Aramrogpeppe: my brain is simply not adapted for such a task.12:03
Aramso I have to do it three times before it's right.12:03
fwereaderogpeppe, sorry, just having lunch, I'll do it when I come back12:04
rogpeppeAram: i found it hard to start with, but then worked out a technique that seems to work12:04
rogpeppeAram: i edit the "this" parts by copying and pasting from the "source" parts.12:05
rogpeppeAram: then i delete the source parts12:05
rogpeppefwereade: np, enjoy!12:05
niemeyerGood morning juju masters!12:42
TheMueniemeyer: heya12:43
niemeyerHmmm.. why am I getting a BADSIG on us.achive12:56
niemeyerarchive12:56
TheMueniemeyer: time for a short question?12:57
niemeyerTheMue: Sure12:59
TheMueniemeyer: fine, thx12:59
TheMueniemeyer: when set-up a new machine we set the juju-group and a machine-individual group13:00
TheMueniemeyer: in ec213:00
niemeyerTheMue: Hmm.. do we?13:01
TheMueniemeyer: i hope i describe it right, try to make it better understandable13:02
TheMueniemeyer: there's a setUpGroup() for a machine-id. and there a group with the group name ("juju-xyz") and a machine group ("juju-xyz-(machineid)") is set-up13:03
TheMueniemeyer: now i removed the machine group, so all machines share only one group and the ports are opened/closed on it per instance13:04
niemeyerTheMue: Ah, okay, you're talking about *security* groups in EC2..13:04
TheMueniemeyer: sorry, yes, missed to tell more about the context13:04
niemeyerTheMue: Ok13:06
TheMueniemeyer: so now the question: shell the IP permissions for the juju-group stay the same (ssh for all plus all ports inside the group)?13:06
TheMueshall13:06
niemeyerTheMue: Yeah, sounds ok13:08
niemeyerTheMue: Btw,13:08
TheMueniemeyer: ok13:08
TheMueniemeyer: yes?13:08
niemeyerTheMue: I'm wondering if we should have that as an option rather than the default behavior13:09
niemeyerTheMue: If we keep the firewaller behaving as-is, would it be feasible to have this behavior switchable within the env only?13:09
TheMueniemeyer: would be fine to me as i prefer to start with as few open ports as possible13:09
TheMueniemeyer: let's make it in two CLs. i start with the current default behavior for the juju-group and then, after your lgtm, look how we can make this switchable. ok?13:13
niemeyerTheMue: Hmmm.. you mean breaking what's working so that we can then see how to fix it?13:13
TheMueniemeyer: it seems i did not get your idea right.13:15
TheMueniemeyer: could you please tell me what you mean by "switchable"?13:15
niemeyerTheMue: I mean being able to select between current behavior and a new behavior13:15
TheMueniemeyer: oh, understand (hopefully).13:16
TheMueniemeyer: i reduced it to the default ports, but you talk about the groups, don't you?13:17
niemeyerTheMue: I'm not sure about what "reducing to the default ports" means13:17
niemeyerTheMue: And yes, I think we've both been talking about security groups13:17
TheMueniemeyer: I thought about which ports to be open by default.13:18
niemeyerTheMue: I don't think we want to change that13:18
niemeyerTheMue: Do we?13:18
TheMueniemeyer: no, but after your first sentence "i'm wondering …" i thought so.13:19
niemeyerTheMue: Ah, ok.. no, I'm talking about the whole change13:20
TheMueniemeyer: yes, now i got it. sorry i got it wrong the first place.13:20
niemeyerTheMue: It's all good13:20
TheMueniemeyer: the firewaller relies on the interface for opening and closing ports. so the change imho can be done without changing the firewaller.13:21
TheMueniemeyer: but the current implementation opens and closes on the machine group13:22
TheMueniemeyer: and in future on the juju group13:22
TheMueniemeyer: so i think this can be done configurable13:22
niemeyerTheMue: Right.. so the first CL would be adding such a setting, without any further changes13:23
TheMueniemeyer: ok, and i keep my current branch as a reference for the follow-up ;)13:25
TheMueniemeyer: another question about security groups13:25
niemeyerTheMue: Yeah, nothing is lost probably13:25
TheMueniemeyer: if i change a ip permission for the juju group, this does not effect all instances, doesn't it?13:26
niemeyerTheMue: If you change a security group, it affects all instances to which the security group is attached13:28
TheMueniemeyer: oh, so instead of a machine security group a service based security group could make more sense, or do i get it wrong?13:29
niemeyerTheMue: Security groups can only be attached to a machine when the machine is created13:30
TheMue_Sh…, disconnected13:32
niemeyerTheMue: Security groups can only be attached to a machine when the machine is created13:33
=== TheMue_ is now known as TheMue
TheMueniemeyer: Thx, that has been the missing link to the chosen model. Just reading a lot about it to get a better understanding.13:36
niemeyerTheMue: np13:38
rogpeppeTheMue, niemeyer: do you think it still makes sense to have two security groups? i'm thinking that we can make do with only one now.13:46
rogpeppeniemeyer: --bump-version is back, BTW: https://codereview.appspot.com/6593053/13:47
niemeyerrogpeppe: It depends a bit on the implementation13:47
niemeyerrogpeppe: Would be best to have one, but given that it is switchable, I don't know what will end up being easiest13:47
niemeyerTheMue: ^13:47
niemeyerrogpeppe: Cheers13:47
rogpeppeniemeyer: ah, i see.13:48
rogpeppeniemeyer: yeah, it depends. if you want it switchable at runtime, there's no other option.13:48
TheMuerogpeppe: they need something like gosecuritygroups, more lightweighted than real groups. ;)13:51
rogpeppeTheMue: all they need is to be able to add security groups after launch13:51
rogpeppes/add sec/modify sec/13:51
TheMuerogpeppe: yes, that would help a lot13:52
niemeyerrogpeppe: I'm not considering much runtime at this point13:52
niemeyerrogpeppe, TheMue: I'm hoping we can keep the core issue in mind and have that as a trivial yet nice addition soon13:53
rogpeppeniemeyer: it's interesting actually - we'd need to store the current mode in the environment somewhere - something we don't do anywhere. maybe an additional field in the bucket.13:54
niemeyerrogpeppe: We don't need to do anything we're not doing currently. Just a trivial setting that defines the intended behavior13:54
rogpeppeniemeyer: i mean, we'd need to do that regardless of switching at environment-creation-time or run-time.13:54
rogpeppeniemeyer: StartInstance needs to know whether it needs to create a new security group or not.13:55
niemeyerrogpeppe: StartInstance is in the environment, that has the setting to define its behavior13:56
rogpeppeniemeyer: yes, assuming noone calls StartInstance from the client side, where the setting might be different from the one that the environment was bootstrapped with.13:57
niemeyerrogpeppe: If we randomly use different settings client side and server side, we surely will have many issues13:58
rogpeppeniemeyer: ok. i actually thought it would work pretty well currently with just the bare minimum of settings on client's side (after bootstrap of course)13:59
niemeyerrogpeppe: default-series?14:00
rogpeppeniemeyer: interesting. tbh i have to say i'm not entirely clear on where and how default-series is used.14:01
niemeyerrogpeppe: Okay, so before we dive away, TheMue issue is trivial one as it stands. We must consistently use the environment settings stored in the environment no matter what.14:02
rogpeppeniemeyer: +114:02
TheMueniemeyer: +114:02
TheMuelunchtime14:05
rogpeppeniemeyer: FWIW in the only occurrence of default series being used that i can find, it's taken from the state env config, not the local config. i imagine there are other places where it will come in though.14:05
niemeyerTheMue: Enjoy!14:06
niemeyerrogpeppe: That seems to agree with what was just stated above14:06
rogpeppeniemeyer: i probably misunderstood you, sorry. i thought you were saying that things could be mucked up by a client using different env config settings from what we bootstrapped with. i don't *think* we can, currently.14:08
niemeyerrogpeppe: They cannot because we're doing exactly what I suggested, which is consistently using the environment settings stored in the environment.14:09
niemeyerrogpeppe: This is not a coincidence.14:09
rogpeppeniemeyer: ah, i see. someone *could* still muck things up by calling StartInstance directly, but none of our stuff will do that.14:13
niemeyerrogpeppe: Of course, someone can screw up things arbitrarily by doing arbitrary things..14:14
niemeyerrogpeppe: We give people all the tools to pull an environment configuration from a mailing list and running wild..14:15
niemeyerrogpeppe: Just sent a question to the CL14:16
rogpeppeniemeyer: i'm not sure what "interacting properly with the --version option" would be14:19
rogpeppeniemeyer: currently the --version flag does not influence which version is uploaded14:20
rogpeppeniemeyer: it seems like you might be suggesting that it should14:20
rogpeppeniemeyer: tbh, i'm thinking that the tool upload functionality might be best off as an entirely separate subcommand14:23
niemeyerrogpeppe: Hmm14:23
niemeyerrogpeppe: Yeah, you're right14:24
niemeyerrogpeppe: --version is about selecting which version to pick14:24
niemeyerand if we're uploading we don't really have a choice14:24
niemeyerrogpeppe: We should conflict these options explicitly then, I suppose?14:24
rogpeppeniemeyer: we could do.14:24
rogpeppeniemeyer: though i don't mind override semantics either14:25
niemeyerrogpeppe: It's bogus I think.. it will pretend to be a different version than it actually is14:25
rogpeppeniemeyer: i'm not sure that will happen.14:26
rogpeppeniemeyer: s/will/can/14:27
niemeyerrogpeppe: What does --version 10.11.12 --upload-tools --bump-version mean?14:27
rogpeppeniemeyer: the given version is ignored, the tools are uploaded with a bumped version if necessary14:28
niemeyerrogpeppe: There you go..14:28
rogpeppeniemeyer: i'm not sure what you mean by "pretending to be a different version".14:28
rogpeppeniemeyer: what's pretending?14:28
niemeyerrogpeppe: Nothing is pretending, because you've ignored the option entirely14:28
rogpeppeniemeyer: you can also do: --version 10.11.12 --upload-tools14:29
niemeyerrogpeppe: Let's just error when the option means nothing14:29
rogpeppeniemeyer: ok.14:29
niemeyerrogpeppe: Unrelated to the current CL, though.. LGTM14:29
rogpeppeniemeyer: thanks14:29
rogpeppeniemeyer: apart from that last wrinkle (and major-version upgades, of course), that completes upgrading for the time being, i think.14:35
niemeyerrogpeppe: Woohay!14:35
rogpeppeniemeyer: i'm now moving on to fixing a few bugs that have been assigned to me (in particular some of the sporadic live test failures). other suggestions welcome.14:36
niemeyerrogpeppe: We have a pretty high/critical one on the pipeline that could see your attention, actually14:37
niemeyerrogpeppe: I've originally invited Aram to look at it, but on a second thought you probably have more context to do it quickly14:37
rogpeppeniemeyer: ok. let me at it!14:38
niemeyerrogpeppe: We're not game to start kicking out the ssh proxying hackery14:38
niemeyers/not/now14:38
rogpeppeniemeyer: woo. do we think that go.net ssh is up to the task?14:39
niemeyerrogpeppe: We don't need ssh at all, hopefully14:39
rogpeppeniemeyer: even better!14:39
niemeyerrogpeppe: The idea is to connect straight onto mongo14:40
rogpeppeniemeyer: what shall we use for crypto?14:40
niemeyerrogpeppe: It'll take a few steps to get there, though14:40
niemeyerrogpeppe: Mongo can talk SSL14:40
rogpeppeniemeyer: cool.14:40
niemeyerrogpeppe: mgo can't right now, but I'll fix that before you get there, I'm hoping14:40
niemeyerrogpeppe: As a very first step, we have to introduce the idea of a connection password for the agents14:41
rogpeppeniemeyer: where does the password come from?14:42
niemeyerrogpeppe: The agent passwords are automatically generated14:43
niemeyerrogpeppe: For the machine agent, we have to hand the password off during the machine creation, so that the new agent can connect once it comes up14:43
niemeyerrogpeppe: Unfortunately, we have to do a little trick once the agent comes up the first time, and replace the agent password with a new one that is created locally14:44
niemeyerrogpeppe: Because the metadata that we send to the machine is visible to anyone within the machine itself14:44
rogpeppeniemeyer: perhaps we could have a new agent that's responsible for exchanging one-time passwords for persistent passwords14:44
rogpeppe(thinking aloud)14:45
niemeyerrogpeppe: It's significantly simpler than that14:45
niemeyerrogpeppe: It doesn't even have to be a one-time password14:45
niemeyerrogpeppe: It's just a "Oh, is that the password I got during creation, okay.. here is a new one I want to use from now on."14:46
niemeyerrogpeppe: As long as we do that before the machiner goes on to create units, it's all good14:46
rogpeppeniemeyer: how do you get the new one?14:46
niemeyerrogpeppe: Just create one and change it14:47
rogpeppeniemeyer: ah, does SSL have provision for changing passwords remotely?14:48
niemeyerrogpeppe: Huh.. wires crossed there14:48
niemeyerrogpeppe: Forget SSL for the moment14:48
niemeyerrogpeppe: This is juju state logic14:49
niemeyerrogpeppe: and mongo auth14:49
rogpeppeniemeyer: ok.14:49
rogpeppeniemeyer: are we planning to store passwords inside the state?14:50
niemeyerrogpeppe: Nope14:51
rogpeppeniemeyer: sorry, ignore my bleating. i'll shut up until you've explained :-)14:51
niemeyerrogpeppe: No, that's it really14:51
niemeyerrogpeppe: That's the first step14:51
rogpeppeniemeyer: could you outline the next steps, so i know where we're heading?14:51
niemeyerrogpeppe: How far? :-)14:52
rogpeppeniemeyer: until we've got equivalent functionality to today14:52
rogpeppeniemeyer: but via direct connection to mongo14:52
niemeyerrogpeppe: We can't drop SSH just yet, but that means we can then introduce SSL more or less easily, and dro pit14:52
niemeyerrogpeppe: This is the tricky bit for doing so14:53
niemeyerrogpeppe: After that we need some means to define the admin password14:53
niemeyerrogpeppe: Which is what that old admin-secret meant14:53
niemeyerrogpeppe: So that we can connect straight to mongo14:54
rogpeppeniemeyer: mongo's authorization is password-based, presumably14:54
niemeyerrogpeppe: and then we just enable SSL14:54
niemeyerrogpeppe: Yes, user/pass14:54
niemeyerrogpeppe: In a future universe, we'll then introduce an HTTPS API to which everyone will talk to14:55
niemeyerrogpeppe: Which maps more or less well to that whole model14:55
rogpeppeniemeyer: and there's a mongo capability to add users and passwords, i guess14:55
niemeyerrogpeppe: Yeah, that's already well mapped onto mgo14:55
rogpeppeniemeyer: so we'd have one user per agent, each with its own password14:55
niemeyerrogpeppe: Exactly14:55
rogpeppeniemeyer: out of interest, which part of the API do you use to do that?14:56
rogpeppeniemeyer: oh, i see14:57
niemeyerrogpeppe: You mean in mgo?14:57
niemeyerrogpeppe: http://go.pkgdoc.org/labix.org/v2/mgo#Database14:57
* rogpeppe grepped for password, not user14:57
rogpeppeniemeyer: so can anyone change any user?14:58
niemeyerrogpeppe: No.. we have to put some thinking on how that will work14:59
rogpeppeniemeyer: it seems like the model we've got here is something like this: http://paste.ubuntu.com/1254028/15:04
niemeyerrogpeppe: As you spotted above, we'd have one user per agent, each with its own password15:06
rogpeppeniemeyer: i'm not quite sure how the MA would pass the password securely to other agents. maybe we should just put it in a file.15:06
rogpeppeniemeyer: after all, if you're root you can get the password anyway.15:07
niemeyerrogpeppe: We probably need it in a file either way, since we must know it after it changes15:07
rogpeppeniemeyer: good point.15:07
rogpeppeniemeyer: ok, if the above sketch looks right to you, i'm happy.15:08
niemeyerrogpeppe: It doesn't because of what I mentioned above15:09
niemeyerrogpeppe: As you spotted above, we'd have one user per agent, each with its own password15:09
niemeyerrogpeppe: The sketch makes it feel like the password provided is the machine password15:09
rogpeppeniemeyer: ok, so the MA creates a new mongo user for each agent it starts15:10
rogpeppeniemeyer: and so, presumably, does the unit agent when it starts subordinates?15:10
niemeyerrogpeppe: Yeah, that sounds right15:10
niemeyerrogpeppe: This is a bit silly right now, if you think about it, because all those users have the same permissions15:11
niemeyerrogpeppe: But it starts to make more sense if we consider that situation won't be like that soon15:11
rogpeppeniemeyer: yeah, each agent could potentially delegate permissions.15:11
niemeyerrogpeppe: Btw, when you say "the MA creates a new mongo user", it's not just a mongo user we're talking about.. these details should be abstracted behind our own API15:12
rogpeppeniemeyer: yeah, because the MA knows nothing about mongo15:12
niemeyerRight15:12
rogpeppeniemeyer: would this be sufficient? state.State.AddUser(user, pass string) error15:13
niemeyerrogpeppe: No..15:14
rogpeppeniemeyer: and an extra field in state.Info15:14
niemeyerrogpeppe: We have to think through the relationship between user and entity15:14
rogpeppeniemeyer: a "capability" bitmask or set?15:15
niemeyerrogpeppe: Sorry, I just meant that we don't want dangling users around without association to an entity, and we don't want to have to manipulate the needs of authentication and the needs of entity creation/etc independently15:16
niemeyerrogpeppe: It should be clear from the API how these things take palce15:17
niemeyerplace15:17
rogpeppeniemeyer: ok. by "entity" you mean some type or value in the state?15:17
rogpeppeniemeyer: or an agent?15:17
niemeyerrogpeppe: We've been using entity to refer to unit/service/machine15:18
niemeyerrogpeppe: But in this case it's really the subset of those that have an agent ATM15:19
niemeyerrogpeppe: So unit/machine15:19
rogpeppeniemeyer: so the Machine entity is kinda superior to the Unit entity, because an MA can create users for UAs but not vice versa?15:20
niemeyerrogpeppe: We won't have the low-level restrictions in place today.. we just need a proper API that makes sense in such a context15:21
rogpeppeniemeyer: that's what i'm trying to imagine15:22
niemeyerrogpeppe: We don't need to "create users" in general15:22
niemeyerrogpeppe: We have proper unique keys for the agents15:22
niemeyerrogpeppe: We shouldn't have to be manipulating users and passwords in our own API externally to the state model15:22
niemeyerrogpeppe: This would add an implicit API on top of our own API that is probalby not necessary15:23
rogpeppeniemeyer: what about something like this? http://paste.ubuntu.com/1254071/15:26
niemeyerrogpeppe: Yeah, that looks nice15:27
rogpeppeniemeyer: and then a new "Password" field in state.Info15:28
rogpeppeniemeyer: (which in actual fact incorporates both the mongodb username and password)15:28
niemeyerrogpeppe: I think we can be a bit more explicit: // SetPassword returns a new random password the agent responsible for X should use to communicate with the state servers. Previous passwords are invalidated.15:29
rogpeppeniemeyer: NewPassword?15:30
niemeyerrogpeppe: NewFoo in general returns a new foo, without further consequences15:30
rogpeppeniemeyer: yeah15:30
niemeyerrogpeppe: ReplacePassword maybe15:31
rogpeppeniemeyer: or ChangePassword15:31
niemeyerrogpeppe: SOunds good15:31
niemeyerrogpeppe: ChangePassword() (auth string, err error)15:33
niemeyerrogpeppe: So we can have Auth in state.Info rather than password, as it may contain the username too15:34
niemeyerrogpeppe: Although we never expose those details out of the state itself15:34
rogpeppeniemeyer: sounds good15:34
niemeyerrogpeppe: Hmm.. there's a small detail here, though.. we may need to split that up a bit into separate methods15:35
rogpeppeniemeyer: i was wondering about this: http://paste.ubuntu.com/1254085/15:35
niemeyerrogpeppe: Ah, no.. we should send the new password..15:36
niemeyerrogpeppe: ChangePassword(password) (auth string, err error)15:36
niemeyerrogpeppe: Otherwise there's a race15:36
niemeyerrogpeppe: We need to persist the new password before we ask to change, otherwise the agent might be locked out15:37
rogpeppeniemeyer: ah, interesting, yeah15:37
niemeyerrogpeppe: Regarding the past, I'd prefer an explicit API instead15:37
niemeyerrogpeppe: unit.ChangePassword.. machine.ChangePassword15:37
rogpeppeniemeyer: ok15:37
niemeyerrogpeppe: No need to guess where it makes sense and where it doesn't15:38
rogpeppeniemeyer: sounds good15:38
rogpeppeniemeyer: the nice thing about passing in a password is that the method becomes idempotent.15:39
niemeyerrogpeppe: Indeed15:39
rogpeppeniemeyer: i think SetPassword makes better sense now.15:40
niemeyerrogpeppe: +115:40
rogpeppe// SetPassword sets the password the agent responsible for the machine15:40
rogpeppe// should use to communicate with the state servers.  Previous passwords15:40
rogpeppe// are invalidated.15:40
rogpeppefunc (m *Machine) ChangePassword() (password string, err error)15:40
niemeyerfunc (m *Machine) SetPassword(password string) (auth string, err error)15:41
niemeyerI suppose15:41
rogpeppeniemeyer: yeah c&p error15:41
niemeyerfwereade: I mid-way through the filter review.. the conversation here was good, so I didn't make much progress, but will continue after lunch15:41
fwereadeniemeyer, no worries15:42
niemeyerrogpeppe: I'll step out for lunch and bbiab15:42
rogpeppeniemeyer: ok cool15:42
fwereadeniemeyer, I may be off soon but will try to pop back on later15:42
niemeyerfwereade: Sounds good, thanks for the nice progress15:42
* niemeyer => lunch15:42
fwereadeniemeyer, seems to be going ok :)15:42
fwereadeniemeyer, cheers15:42
TheMueniemeyer: https://codereview.appspot.com/6596051/ as the first step is in16:14
TheMuecu later16:14
niemeyermramm: Do we have a call in 20 mins?16:41
mrammniemeyer: Yea the Cloud Consistency Call16:42
mrammniemeyer: we will be going over the certified public cloud initiative, and some server stuff16:42
niemeyermramm: 'k16:42
mrammniemeyer: not sure that your attendance is required, if you want to skip I can ping you if something important comes up16:43
niemeyermramm: Sounds good, I'll be around then16:43
rogpeppeniemeyer: just wondering what's the best thing to do about removing users. i'm thinking that SetPassword("") removes the user for an entity (i can't see that we'll ever want to actually set a blank password)16:51
niemeyerrogpeppe: Why would we want to remove a user in such a fashion?16:52
rogpeppeniemeyer: if a unit gets removed, i think it makes sense to remove the unit's mgo user too16:52
niemeyerrogpeppe: Exactly! :)16:52
rogpeppeniemeyer: erk16:53
rogpeppeniemeyer: of course.16:53
niemeyerrogpeppe: Btw, over lunch I was thinking on our interface.. we'll stumble on problems regarding SetPassword returning auth, for the same reason that we need to pass the password in instead of just using the result16:54
niemeyerrogpeppe: I suggest this instead:16:54
niemeyerfunc (m *Machine) SetPassword(password string) error16:54
niemeyerstruct Info { .....; Principal interface{ SetPassword(string) }; Password string; }16:54
rogpeppeniemeyer: interesting16:55
rogpeppeniemeyer: i don't quite see why SetPassword is a problem though, assuming it's deterministic16:56
niemeyerrogpeppe: The only change is that the auth result was nonsense16:56
rogpeppeniemeyer: the Info change above is a pretty major thing.16:57
niemeyerrogpeppe: Because?16:57
rogpeppeniemeyer: currently it's a very simple set of info.16:57
niemeyerrogpeppe: Ah, you're right.. we have a chicken and egg16:58
niemeyerrogpeppe: Well, we'll need to save the principal as a string then16:58
rogpeppeniemeyer: alternatively we could just have a Principal method on Machine and Unit16:59
niemeyerrogpeppe: For?17:00
rogpeppeniemeyer: we need to get the user name and password from somewhere.17:00
niemeyerrogpeppe: That's not an alternative to the issue above, since we still must tell it what the principal is, and we can't get a password from state17:01
rogpeppeniemeyer: what about Machine.Auth(password) ?17:01
niemeyerrogpeppe: Sorry, feels like we're going backwards in our conclusions17:01
rogpeppeniemeyer: i'm not sure i've concluded anything yet17:02
niemeyerrogpeppe: We already have machine.SetPassword(password).. Auth(password) is no better17:02
rogpeppeniemeyer: sorry, i'm thinking *in addition* to SetPassword17:02
niemeyerrogpeppe: I'm a bit lost17:02
niemeyerrogpeppe: What problem are you solving?17:02
rogpeppeniemeyer: so we'd have SetPassword(password) error; and Auth(password) (auth string, err error)17:02
niemeyerrogpeppe: Why?17:02
rogpeppeniemeyer: so that we can generate the password, save it to disk, then if we crash we can regenerate the required auth info from the password only.17:03
niemeyerrogpeppe: All of that seems unrelated to Machine.Auth(password)17:03
rogpeppeniemeyer: Auth is maybe a bad name. AuthInfo, perhaps.17:04
niemeyerrogpeppe: Seems like a complex way to do something simple17:05
rogpeppeniemeyer: what's your suggestion?17:06
niemeyerrogpeppe: The same.. Principal string17:06
rogpeppeniemeyer: where does that come from?17:06
niemeyerrogpeppe: We already have a nice key on PathKey.. we can rename it to something more sensible and reuse17:08
rogpeppeniemeyer: i was going to suggest that17:08
rogpeppeniemeyer: i still think that AgentName is a reasonable name for that method.17:08
rogpeppeniemeyer: BTW there's one thing we haven't talked about17:09
rogpeppeniemeyer: what does the client do?17:09
rogpeppeniemeyer: in my implementation so far, i've implemented State.SetPassword17:10
niemeyerrogpeppe: +1 on AgentName.. I can't come up with anything better right now17:10
rogpeppeniemeyer:17:10
rogpeppe// SetPassword sets the password the administrator17:10
rogpeppe// should use to communicate with the state servers.17:10
rogpeppeniemeyer: but actually maybe we don't care much about changing the admin password17:11
rogpeppeniemeyer: and we could hardwire "admin" as a principal name17:11
niemeyerrogpeppe: The thing I dislike about AgentName is that it blocks adding the same interface to other things we'll want to identify in the same manner17:11
rogpeppeniemeyer: you mean, if we have principals that don't map directly to state objects?17:13
niemeyerrogpeppe: No, I mean that the concept of having a unique and intelligible key for entities is very useful17:13
niemeyerrogpeppe: EntityName perhaps17:14
rogpeppeniemeyer: i wrote that, then deleted it17:14
rogpeppeniemeyer: but it's not too bad, despite being a nasty mouthful.17:14
niemeyerrogpeppe: This means we can have "service-foobar" too17:14
rogpeppeniemeyer: +1 to EntityName17:15
rogpeppeniemeyer: or even just Name...17:15
niemeyerrogpeppe: -1, unless we refactor further17:15
niemeyerrogpeppe: We already have that in some places17:15
rogpeppeniemeyer: oh yeah, we already use it17:15
rogpeppeniemeyer: darn17:15
niemeyerrogpeppe: It's not a bad idea, though17:16
niemeyerrogpeppe: We might move to Id() in the other fields17:16
rogpeppeniemeyer: yeah, Id would work well for how unit.Name etc is used currently.17:17
rogpeppeniemeyer: and it would fit well with Machine.Id too17:17
niemeyerrogpeppe: unit.Id(), service.Id(), and then we have Name() with the full-blown name17:17
rogpeppeniemeyer: +17:17
rogpeppe117:17
rogpeppe!17:17
niemeyer:-)17:17
rogpeppeniemeyer: ok, that shouldn't be hard to change.17:18
niemeyerrogpeppe: We have to do that sooner rather than later, or we'll be stuck with the database schema17:18
rogpeppeniemeyer: i'll do it now, if you think it's good17:18
niemeyerrogpeppe: It sounds like a good move17:19
rogpeppeniemeyer: what do you think about the client-initiated connection? Info.Principal == "admin" ?17:19
niemeyerrogpeppe: Yeah17:19
niemeyerrogpeppe: Your idea above regarding "admin" sounds good17:19
rogpeppeniemeyer: that we have State.SetPassword ?17:20
niemeyerrogpeppe: No, that we can hardcode the "admin" user for the moment17:20
rogpeppeniemeyer: ok, sounds good17:20
niemeyerrogpeppe: and that we can do without changing its password for a while17:20
niemeyerrogpeppe: I think, but not sure17:20
rogpeppeniemeyer: then "admin-password" as an environment setting makes total sense17:20
niemeyerrogpeppe: Either way, SetAdminPassword would be ok17:20
rogpeppeniemeyer: which would be a nice change :-)17:21
niemeyerrobbiew, mramm: Where is the meeting taking place today?17:27
robbiewniemeyer: what meeting? cloud consistency?17:28
niemeyerrobbiew: Yeah17:28
robbiewniemeyer: meh..you can skip it, unless you wanna hear an update on charm collection17:28
niemeyerrobbiew: Yeah, that's what I'm curious about17:28
robbiewconf id 80589 6323817:28
niemeyerrobbiew: Cheers17:29
niemeyerHoly crap17:29
niemeyer17 people..17:29
=== philballew_ is now known as philballew
rogpeppeniemeyer: first stage of Name change: https://codereview.appspot.com/6585053/17:57
niemeyerrogpeppe: Looking17:57
rogpeppeniemeyer: i'm off for the evening. see ya tomorrow...17:58
rogpeppenight all!17:58
niemeyerrogpeppe: Cool, thanks for the quick turnaround17:58
niemeyerrogpeppe: Have a good evening17:58
rogpeppeniemeyer: np17:58
rogpeppeniemeyer: static typing FTW17:58
niemeyerrogpeppe: Indeed :)17:58
niemeyerI'll have a walk outside and come back shortly for more reviews19:20
fwereadeniemeyer, very briefly: my issue with using the uniter's existing unit/service is just that I don't want to share them with the filter goroutine19:22
fwereadeniemeyer, so I rather think I have to pay the cost of a refresh anyway19:23
fwereadeniemeyer, from that perspective, passing a single new watcher into the new goroutine, and leaving it to reconstruct everything it required from there, seemed rather neat19:24
fwereadeniemeyer, will be on and off, if you have a quick answer just dump it in the channel and I'll see it soon enough19:25
niemeyerfwereade: I half agree20:35
niemeyerfwereade: If you want to not share the service, that's certainly reasonable20:36
niemeyerfwereade: But the points in the review still feel valid.. you can easily pick the service up from state when starting the filter20:36
niemeyerfwereade: and watch it upfront20:36
niemeyerfwereade: Or do I misunderstand your point?20:39

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!