/srv/irclogs.ubuntu.com/2012/07/06/#juju-dev.txt

Aramgood morning.04:15
fwereadeyay, I has power07:46
fwereadegoing out to get some coffee and a pastry, still recovering from broken fridge :/08:15
fwereadeheya TheMue09:02
TheMueHi fwereade09:02
TheMueHmm, I hope my line gets more stable when the new provider has set it up.09:03
TheMueJust seen, need a new watcher.09:15
fwereadeTheMue, sorry, what watcher is that now? :)09:44
TheMuefwereade: The ServiceWatcher for added or removed services. It's needed by the firewall.09:45
fwereadeTheMue, ah, cool09:45
TheMuefwereade: Thx to your watcher changes it's pretty simple now.09:46
fwereadeTheMue, I kinda suspect that about 50% of juju will actually just be watchers of one sort or another by the time we're done :)09:46
fwereadeTheMue, awesome09:46
TheMuefwereade: Yes, we're a kind of event-driven09:46
fwereadeTheMue, yeah, totally09:47
TheMuefwereade: So one architecture alternative could have been that all those watchers publish their changes to a kind of bus where interested listeners subscribe to those events.09:49
fwereadeTheMue, not sure, there are a number of subtle ordering interactions in a couple of places09:50
fwereadeTheMue, we only want to watch X in between a specific pair of events on Y, for example09:51
fwereadeTheMue, and we don't want to just watch ALL THE CHANGES all the time09:51
fwereadeTheMue, I guess it could be done09:51
TheMuefwereade: That's depending on the subscription mechanism, would be possible09:52
fwereadeTheMue, but I think unwatching X at the right time, in the example above, would be a bit fiddly09:52
fwereadeTheMue, offhand, do you know whether we currently have any mechanism for putting *valid* environment configs into /environment?09:54
fwereadeTheMue, I know we can poo an arbitrary map[string]interface{} in but I don't think there's yet any way to push a real env config implemented in real code09:54
TheMuefwereade: No, that doesn't sound real got.09:57
TheMuefwereade: The environment fields depend on the chosen environment?09:58
fwereadeTheMue, yeah09:58
fwereadeTheMue, I'm implementing bits and pieces in that area right now09:58
fwereadeTheMue, I'm just trying to figure out whether I will break things *any worse than they currently are* if I follow my preferred path09:59
TheMuefwereade: So I would setup a type hierarchy per environment and implement a Validate() on each type working top-down.09:59
fwereadeTheMue, whoa, hierarchy?>09:59
TheMuefwereade: I'm doing so for my private RSS and Atom implementation.09:59
TheMuefwereade: And that could be marshaled to e.g. JSON inside the node.09:59
fwereadeTheMue, we already have code for validating them09:59
fwereadeTheMue, NewConfig :)09:59
TheMuefwereade: Just see it, very generic.10:01
niemeyerMorning!10:14
fwereadeniemeyer, heyhey10:19
niemeyerfwereade: Heya!10:19
fwereadeniemeyer, how's it going?10:20
niemeyerfwereade: Great10:20
niemeyerfwereade: A little bit on the short side in terms of sleep, but hopefully will fix that tomorrow :)10:20
fwereadeniemeyer, awesome; thanks again for the conversation yesterday, everything seems to be lining up really neatly now10:20
niemeyerfwereade: My pleasure!  Appreciated too.10:20
fwereadeniemeyer, I have a couple of CLs that should be short/sweet: https://codereview.appspot.com/6345071/ and https://codereview.appspot.com/6351068/10:21
TheMueniemeyer: Morning10:23
niemeyerTheMue: Heya10:23
niemeyerfwereade: Looking10:23
niemeyerfwereade: https://codereview.appspot.com/6345071/ done10:39
fwereadeniemeyer, cheers10:43
niemeyerfwereade: I'm not sure about the other one.. what's the benefit?10:43
niemeyerAram: ping10:52
TheMueHmm, sh**, have to reboot. Back in a moment.10:54
fwereadeniemeyer, sorry, was making lunch11:06
fwereadeniemeyer, benefit is that we can make State.EnvironConfig return an environs.EnvironConfig11:06
fwereadeniemeyer, which really feels like the Right Thing now I've started implementing it11:07
fwereadeniemeyer, (but incorporating the move in the forthcoming CL for EnvironConfig felt a bit much)11:12
niemeyerfwereade: How about moving EnvironConfig to state instead?11:12
fwereadeniemeyer, don't think so offhand -- we also need NewConfig, and IMO that's unquestionably part of environs11:13
fwereadeniemeyer, besides, the environs are meant to not depend on state, right?11:13
niemeyerfwereade: Why do we need NewConfig on State?11:13
niemeyerfwereade: Or is the state supposed to not depend on environs? ;-)11:14
fwereadeniemeyer, because if we try to do environs.NewConfig from state we get an import cycle11:14
niemeyerfwereade: Why do we *need* NewConfig on state?11:14
niemeyerfwereade:11:15
niemeyer[niemeyer@gopher ..juju-core/state]% grep -r environs *11:15
niemeyer[niemeyer@gopher ..juju-core/state]%11:15
fwereade_niemeyer, sorry, don't know what happened there11:16
fwereade_niemeyer, because State.EnvironConfig wants to return an EnvironConfig, and all it has is a map[string]interface{}11:16
niemeyerfwereade_: Hmm.. something else seems wrong then11:17
niemeyerThinking11:17
niemeyerfwereade_: Not sure if you've seen this, btw:11:18
niemeyerniemeyer> fwereade:11:18
niemeyer<niemeyer> [niemeyer@gopher ..juju-core/state]% grep -r environs *11:18
niemeyer<niemeyer> [niemeyer@gopher ..juju-core/state]%11:18
fwereade_niemeyer, yeah, tautologically so; environs was importing from state11:18
niemeyerfwereade_: Yes, which is a point worth bringing into the discussion11:19
niemeyerfwereade_: You say it's uncontroversial. It would be if we were simply taking a dependency down, but we're inverting a dependency11:20
niemeyerfwereade_: Some of the purpose of the Conn, for example, is precisely to integrate an environ with a state11:20
fwereade_niemeyer, environs is responsible for creating every known (non-test) instance of the types it imports from state11:20
fwereade_niemeyer, from a certain perspective, environs is the only "source" of those types11:21
fwereade_niemeyer, you were once very firm that environs must not depend on state -- do you recall something specific that changed your perspective?11:22
fwereade_niemeyer, another thought; if InstanceId were a type, that would be another one that would naturally live in environs and be used by state11:23
niemeyerfwereade_: That's not a relevant point.. the only purpose of state.Info is to provide it onto the state package as an argument to Dial. Of course it won't be *created* by the state package.11:24
fwereade_niemeyer, fair point indeed11:26
fwereade_niemeyer, the heart of it is that if we want State.EnvironConfig to return an environs.EnvironConfig, or even to be able to check the sanity of environment config changes, state needs to use environs11:29
TheMueSo, new ServicesWatcher for review is in.11:30
niemeyerfwereade_: I'm not sure we do.. let's look at the overall issue without an established dogma to make sure we're doing the right thing11:30
niemeyerfwereade_: Here is an idea..11:32
niemeyerfwereade_: Configuration always comes in the form of keys, right?11:33
fwereade_niemeyer, yes11:33
niemeyerfwereade_: In state, in environments.yaml, and from the cmdline11:33
fwereade_niemeyer, agreed11:33
niemeyerfwereade_: We actually have infrastructure to deal with that information as keys even, in the form of the schema package11:34
fwereade_niemeyer, right...11:34
niemeyerfwereade_: So, it's at least interesting that we chose to transform that information onto a type so early on11:34
niemeyerfwereade_: What if..11:35
niemeyerfwereade_: EnvironsConfig was not an interface..11:35
niemeyerfwereade_: EnvironConfig, that is11:35
niemeyerfwereade_: But rather, *state.EnvironConfig11:35
niemeyer?11:35
fwereade_niemeyer, how will that end up being noticeably different from the python-style data-bag?11:36
niemeyerfwereade_: I don't know what's the underlying statement you're making with that. The configuration of an environment *is* a data bag.11:37
niemeyerfwereade_: And this is quite bogus:11:38
niemeyer/ EnvironConfig represents an environment's configuration.11:38
niemeyertype EnvironConfig interface {11:38
niemeyer        // Open opens the environment and returns it.11:38
niemeyer        Open() (Environ, error)11:38
niemeyer}11:38
niemeyerfwereade_: This is just wrong..11:38
fwereade_niemeyer, ok, we followed this approach in python, and we ended up in a totally ridiculous situation, in which keys were added and used willy-nilly without even being used in the schema11:38
niemeyerfwereade_: The interface of an environment configuration cannot tell us anything about the configuration, but *opens* the environment11:39
niemeyerwtf?11:39
niemeyerfwereade_: Well, we don't want that to happen for sure11:39
niemeyerfwereade_: But that's a separate issue11:40
niemeyerfwereade_: and one that you just helped solving, in fact ;)11:40
fwereade_niemeyer, well, it is a big advantage of turning things into actual types as early as we can :)11:40
niemeyerfwereade_: I'm not sure.. that's exactly why I started with this:11:41
fwereade_niemeyer, how is the configuration of an environment anyone's business but the environ itself? there are, it is true, some common properties, but they can be added to the EnvironConfig interface as we feel the need for them11:41
niemeyer<niemeyer> fwereade_: Here is an idea..11:41
niemeyer<niemeyer> fwereade_: Configuration always comes in the form of keys, right?11:41
niemeyer<niemeyer> fwereade_: In state, in environments.yaml, and from the cmdline11:41
niemeyer<niemeyer> fwereade_: We actually have infrastructure to deal with that information as keys even, in the form of the schema package11:41
niemeyer<niemeyer> fwereade_: So, it's at least interesting that we chose to transform that information onto a type so early on11:41
niemeyerfwereade_: Aren't you trying to inject a dependency on environs onto the state just so you can get an EnvironConfig from it?11:42
niemeyerfwereade_: Clearly it seems to be someone else's business too11:42
niemeyerfwereade_: cmdline, state, config11:43
fwereade_niemeyer, well, the state has to store EnvironConfigs somehow, right?11:43
niemeyerfwereade_: The configuration of an environment has well known fields, and then other flexible fields11:44
fwereade_niemeyer, sure, I would like at some stage to split the common fields out into their own type, but that's orthogonal11:44
niemeyerfwereade_: What I'm suggesting is that we, actually, realize the EnvironConfig into exactly what it is11:44
niemeyerfwereade_: We can have methods on it11:44
niemeyerfwereade_: And can also have a Specific method, for example that returns all the unknown fields11:46
niemeyerfwereade_: (*without* the well known ones.. we handle those internally)11:46
fwereade_niemeyer, and how are we meant to have any sanity-checking on those unknown fields?11:46
niemeyerfwereade_: That's the environ's business to do11:47
fwereade_niemeyer, ok, so State.Set/UpdateEnvironConfig should just poo whatever it's given into /environment and damn the consequences?11:48
* niemeyer waits until fwereade_ relaxes again11:48
* fwereade_ is sorry, but this feels like a pretty fundamental change in direction of which I had no warning11:49
TheMuefwereade_: Who, in the sense of which part of the code, knows what unknown fields are valid?11:49
fwereade_TheMue, something in environs11:49
niemeyerfwereade_: I think you're being slightly unfair11:50
niemeyerfwereade_: You're the one changing the direction of things11:50
niemeyerfwereade_: This is in state, today:11:50
niemeyer/ EnvironConfig returns the current configuration of the environment.11:50
niemeyerfunc (s *State) EnvironConfig() (*ConfigNode, error) {11:50
niemeyer        return readConfigNode(s.zk, zkEnvironmentPath)11:50
niemeyer}11:50
niemeyerfwereade_: We're "pooing" stuff with "damned consequences" in the current code base11:50
fwereade_niemeyer, yes, and I feel it's a serious problem11:51
TheMuefwereade_: If it's in environs it doesn't seem unknown to me. environs only should handle well knows fields. If we have a need for more we should define a validator function type so that the provider/user of those fields could also provide a validator.11:51
fwereade_TheMue, so every time we create a State we pass in an EnvironConfigValidator?11:51
niemeyerfwereade_: Yes, so let's run through this with less emotion, ok? I'm not changing the direction of anything, and I'm not taking you by surprise on anything11:51
fwereade_niemeyer, you were very very clear in orlando that environs must not depend upon state11:51
TheMuefwereade_: Only when changing the content of the EnvironConfig.11:52
niemeyerfwereade_: and it does not depend on state.. it depends on data11:52
niemeyerfwereade_: We can put EnvironsConfig onto  the foobar package if you want11:52
fwereade_niemeyer, seeing environs using state, I thought "oh, that's just a mistake that has lain dormant because we hadn;t yet implemented enough to be aware of the problems"11:52
niemeyerfwereade_: I didn't say in Orlando "let's make state depend on environs"11:53
niemeyerfwereade_: Changing 6 by 12/2 won't help11:53
fwereade_niemeyer, ok, so your position is that state and environ should not know about one another at all?11:54
niemeyerfwereade_: Yes11:54
fwereade_niemeyer, but state should hold environment configuration?11:54
niemeyerfwereade_: Re-read that question again, and think through it in the broader sense of things for a moment11:55
niemeyerfwereade_: What *is* the state?11:55
fwereade_niemeyer, a description of the desired state of the environment?11:55
TheMuefwereade_: Is it something that represents a state or is it something we only store convenient in ZK?11:56
niemeyerfwereade_: Exactly.. it's the information about the whole environment11:56
fwereade_TheMue, AFAICT it absolutely represents state... not sure what you're getting at11:56
niemeyerfwereade_: We can put the type elsewhere11:57
* TheMue asks to reflect this topic from a distance.11:57
niemeyerfwereade_: But the state will necessarily have to *hold the state configuration*!11:57
niemeyerErm11:57
niemeyerfwereade_: But the state will necessarily have to *hold the environment configuration*!11:57
niemeyerfwereade_: There's simply no option for it to not do it11:57
niemeyerfwereade_: It does today, already11:57
fwereade_niemeyer, yes, this is at the heart of my conviction that the state package is a natural consumer of the environs package11:58
niemeyerfwereade_: That's the fundamental disagreement11:58
niemeyerfwereade_: The environs package *talks to the environment*11:58
niemeyerfwereade_: There's zero reason for the state package to talk to the environment11:58
fwereade_niemeyer, which abstracts away the differing features of various environs such that they can be used by other code without them having to worry about the differences between them11:58
TheMuefwereade_: My understanding has been that other packages (like environ) just *use* state.11:58
niemeyerfwereade_: The right thing for us to do is to break the dependency, both ways11:58
fwereade_niemeyer, ok, fine as far as it goes, wrt the types moved in the CL11:59
fwereade_niemeyer, but it seems to me that it is valuable for the state to be able to validate its environ config somehow12:00
TheMueIMHO environs has to read the configuration from state and instantiate the according "factory" (dislike the word) which then uses the config data to do the real setup.12:00
niemeyerfwereade_: It's valuable to validate the environ config somehow, agreed12:00
niemeyerfwereade_: The state doesn't have to do that, though12:01
fwereade_niemeyer, and I don't see how it can do that without using code from the environs package12:01
niemeyerfwereade_: The state package doesn't have to do that12:01
TheMueniemeyer: Validation has to be provided by the various environs, EnvironConfig only provide a mechanism.12:02
niemeyerTheMue: You can assume I understand that12:02
fwereade_niemeyer, I would prefer that it be *impossible* to set a broken environment config via the state package12:02
TheMueniemeyer: Fine12:03
fwereade_niemeyer, I guess we could also pass a ConfigValidator into state.Open, or something..?12:03
fwereade_niemeyer, hmm, ok, actually, the fundamental thing I don't understand is whystate  should not depend on environs12:04
TheMuefwereade_: IMHO only when writing the config.12:04
fwereade_niemeyer, would you expand on that a little?12:04
niemeyerfwereade_: It feels like a slightly abstract goal to me. There are very few places the system gets unreliable configuration from the outside world.12:04
fwereade_TheMue, so it's not worth checking validity on the way out? only on the way in?12:05
niemeyerfwereade_: On the way out?12:05
TheMuefwereade_: If it only can be written well validated, reading could be assumed as ok, yes.12:05
niemeyerTheMue: +112:06
niemeyerfwereade_: We're not checking anything else for brokeness every time we load from state.. (?)12:06
fwereade_niemeyer, aren't we? in general I think we return either errors or valid instances, rather than timebombs without errors12:08
TheMueExactly, only when during a following operation something weird is discovered an error is returned. This indeed can happen in operations depending on the config too (e.g. by changing it concurrently while it's already loaded by another part).12:08
niemeyerfwereade_: No.. if you get "poo (*&@(*#&(!" as a service name.. do we validate that?12:09
niemeyerfwereade_: This feels like a total departure12:09
fwereade_niemeyer, yeah, probably12:09
fwereade_niemeyer, could we return to "why should state not use environs" please? I think there's something I'm missing there12:09
TheMueErrors always may and can happen, so it's more useful to keep our software handling those errors well.12:10
TheMuefwereade_: It's a dependency thingy. environs uses state, state uses environs? Can't say exactly why, but I dislike this behavior.12:11
fwereade_TheMue, that's unpossible :)12:11
niemeyerfwereade_: Because it's much easier to keep things on track that way. The environs package and the specific implementations of environs have the goal of talking to an IaaS provider.12:11
niemeyerfwereade_: The state package has no business there.12:12
niemeyerfwereade_: If we start breaking that line, something is going wrong12:12
niemeyerfwereade_: That said,12:12
niemeyerfwereade_: I wouldn't be opposed to, for example,12:12
fwereade_niemeyer, I agree with that bit unquestionably :)12:12
niemeyerfwereade_: Introducing the environs/config package12:13
niemeyerfwereade_: and importing *that* from state12:13
fwereade_niemeyer, it's the other direction I don't see a fundamental problem with, because the state describes the environ and it seems odd to stipulate that it cannot know anything about the environ12:13
fwereade_niemeyer, ah-ha!12:13
fwereade_niemeyer, I can totally live with that12:13
niemeyerfwereade_: Woohay middle ground ;012:13
niemeyer;)12:13
fwereade_:D12:14
TheMue:D12:14
fwereade_niemeyer, so the Open stuff stays in environs, and all the other config stuff moves?12:15
niemeyerfwereade_: Yeah, we'll have to figure how to handle Open12:16
niemeyerfwereade_: I never liked the idea of EnvironConfig handling Open, fwiw12:16
niemeyerfwereade_: I think EnvironProvider.Open sounds sane..12:17
fwereade_niemeyer, that sounds pretty plausible to me12:17
fwereade_niemeyer, I'll kick it around and see how it turns out12:17
niemeyerfwereade_: Btw, I have a bikesheddy desire of renaming env to envs12:17
niemeyerErm12:18
niemeyerfwereade_: Btw, I have a bikesheddy desire of renaming environs to envs12:18
niemeyerand Environ to Env12:18
niemeyer:-)12:18
niemeyerMaybe some day12:18
fwereade_niemeyer, +1 but it feels like a distraction right now ;)12:18
niemeyerfwereade_: It is12:18
fwereade_niemeyer, other issue -- should we keep Info and AssignemntPolicy in state, and consider environs' usage of same to be an acceptable purity break?12:19
fwereade_niemeyer, if not, where do they go?12:20
niemeyerfwereade_: I think both of these belong in state12:20
niemeyerfwereade_: Inherently12:20
niemeyerfwereade_: Info we've talked about12:20
niemeyerfwereade_: AssignmentPolicy is logic fully implemented in state12:20
fwereade_niemeyer, yeah, I'm ok with it, just wanted to confirm12:20
* fwereade_ breaks out the chainsaw and goes to work on environs :)12:21
niemeyerfwereade_: ROTFL12:21
TheMuefwereade_: Be careful, don't forget the earmuffs and safety glasses.12:24
TheMueSo folks, will continue tomorrow. Have to do some final preparations before the birthday guests are coming.13:07
niemeyerTheMue: Oh, is it your birthday today?13:15
niemeyerfwereade_: Oh, btw, I was going to ask you earlier about mstate vs. the test reorganization14:29
niemeyerfwereade_: Have you had a chance to look at that?14:29
Aramniemeyer: pong, oops, different computer.14:51
Aramniemeyer: thanks for the reviews.14:51
Aramreviewing the reviews.14:51
Aramgreat, thanks.14:52
Aramniemeyer: I can answer the last question, I am duplicating william's changes.14:58
fwereade_niemeyer, yeah, I started talking to Aram and he was already on it :)14:59
fwereade_(so it was a short conversation :))15:03
niemeyerAram, fwereade_: Thanks!15:16
* fwereade_ straightens up, bloody to the elbows15:59
fwereade_niemeyer, https://codereview.appspot.com/6355077 :)15:59
niemeyerfwereade_: Awesome!15:59
niemeyerfwereade_: I'll have a look right after lunch15:59
fwereade_niemeyer, cool, enjoy :)15:59
niemeyerfwereade_: Thanks!16:00
niemeyerfwereade_: The CL is not quite what I had in mind17:54
niemeyerfwereade_: Just thinking through about pros/cons now17:55
niemeyerfwereade_: https://gist.github.com/306169718:05
niemeyerfwereade_: Do we need anything else?18:05
niemeyerAram: ping18:38
Arampong niemeyer.18:39
niemeyerAram: Yo18:40
Aramhey.18:40
niemeyerAram: How're you doing there.. are you working or relaxing?18:40
niemeyerAram: I was going to invite you for a call, but I know it's late18:40
niemeyerAram: So it can wait if that's the case18:40
Aramyeah, I'm working, I've had some errands to do today, but I'm compensating this evening/night.18:41
niemeyerAram: Awesome, can we have a quick sync up call then?18:42
Aramyes, yes, skype?18:42
niemeyerAram: G+?18:44
Aramok18:44
niemeyerAram: Inviting18:44
niemeyerAram: https://codereview.appspot.com/6356060/diff2/2001:6002/mstate/service.go18:47
niemeyerAram: unit.Life()18:49
niemeyerAram: Remove only Life() == Dead18:50
niemeyerAram: SetLife..18:50
Aramhttps://codereview.appspot.com/6356060/diff2/2001:6002/mstate/state.go?column_width=8018:51

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!