| Aram | morning. | 06:08 |
|---|---|---|
| rog | fwereade: mornin'! | 07:02 |
| rog | fwereade: ping | 07:13 |
| fwereade | rog, heyhey! | 07:15 |
| fwereade | rog, nice holiday? | 07:15 |
| rog | fwereade: lovely, thanks, although the weather... left something to be desired | 07:15 |
| rog | fwereade: i'd forgotten just how lovely and wild cornwall actually is | 07:16 |
| fwereade | rog, ha, yes, I have been hearing such reports | 07:16 |
| fwereade | rog, lovely | 07:16 |
| rog | fwereade: we had no problem finding places to park up "al fresco" | 07:16 |
| rog | fwereade: i'm a bit sad though 'cos i managed to splash sea water on my phone on the last day and it's died | 07:17 |
| rog | fwereade: losing some nice photos with it | 07:17 |
| fwereade | rog, aw, bad luck :( | 07:17 |
| rog | fwereade: and probably other stuff too that i haven't thought about. not to mention 300 quid's worth of phone :-( | 07:17 |
| rog | fwereade: annoying thing is that all the stuff's probably still in there ok, i just can't get at it! | 07:18 |
| fwereade | rog, quite so :( | 07:18 |
| rog | fwereade: anyway, i was looking at environs/config | 07:18 |
| fwereade | rog, well, I guess once it's dead you can take it to bits ;p | 07:18 |
| fwereade | rog, ah, yes | 07:19 |
| rog | fwereade: true, but i'll probably just find a flash chip - not easy to hook it up to anything | 07:19 |
| rog | fwereade: there was a reason why parsing was separate from opening | 07:19 |
| rog | fwereade: (gustavo seems keen to remove the distinction) | 07:20 |
| fwereade | rog, cool, go on? | 07:21 |
| fwereade | rog, wait I don't think that's what he wants | 07:21 |
| rog | fwereade: no? | 07:21 |
| fwereade | rog, getting a Config is still separate from getting an Environ and I expect it to remain so | 07:21 |
| rog | fwereade: it looks to me like he wanted validaton to take place at Open time | 07:21 |
| rog | " | 07:22 |
| rog | The translation onto a struct type, though, and the further validation | 07:22 |
| rog | of the configuration itself should still be done in the same style | 07:22 |
| rog | done today, except inside EnvironProvider.Open. | 07:23 |
| rog | " | 07:23 |
| rog | fwereade: but perhaps i'm misunderstanding | 07:23 |
| fwereade | rog, heh, perhaps I was too | 07:23 |
| fwereade | rog, because I'm not sure what he means by "further" there | 07:24 |
| rog | fwereade: i think he's suggesting that *all* the current validation take place in EnvironProvider.Open | 07:24 |
| rog | fwereade: which means that if you have an environs config with several sections, each section will only be validated if you open it. | 07:25 |
| rog | fwereade: i dunno, perhaps that's actually ok. | 07:25 |
| fwereade | rog, I think that aspect particularly is in fact a win | 07:26 |
| rog | fwereade: oh? | 07:26 |
| fwereade | rog, I am still not 100% sold on the rest of it, but I am soldiering through and hoping all will become clear | 07:26 |
| fwereade | rog, just because I have ballsed up the configuration of environment X I should not lose access to environment Y | 07:27 |
| rog | fwereade: hmm, if that's what we want, then gustavo's comments make sense. although i'm not sure there's any need for a config type at all in that case. | 07:28 |
| rog | fwereade: we can just use a map | 07:29 |
| rog | fwereade: and i'm not sure when we'd ever use ValidateConfig | 07:29 |
| fwereade | rog, ConfigSet.Get, env-set, secrets upload | 07:30 |
| fwereade | rog, but, yes, I understand your reaction, the provider-internal struct types now seem pretty pointless | 07:32 |
| rog | fwereade: could you expand on the above ( ConfigSet.Get, env-set, secrets upload) | 07:32 |
| fwereade | rog, (and ValidateConfig does kinda assume that you can get a Config from an Environ in the first place) | 07:32 |
| fwereade | rog, those are the times we want to change environment config | 07:33 |
| fwereade | rog, but, well, I am not really in a position to give a robust defence of these choices | 07:34 |
| rog | fwereade: we can easily change a config by setting elements in the map, i think. | 07:34 |
| fwereade | rog, yeah, but those changes need to be validated somehow | 07:35 |
| rog | fwereade: that's what EnvironProvider.SetConfig would do, no? | 07:35 |
| rog | fwereade: ah, before we put the new config into the state? | 07:36 |
| fwereade | rog, yeah, the idea is that the SetConfig is too late | 07:36 |
| fwereade | rog, I dunno, this API does feel like it will be more awkward in some situations and less so in others | 07:37 |
| fwereade | rog, I thought it was great having the EnvironConfig interface | 07:38 |
| rog | fwereade: has any of the code for env-set or secrets upload already been written? | 07:38 |
| fwereade | rog, not really | 07:38 |
| fwereade | rog, I had the bulk of it written (and, I thought, quite nice), but... | 07:39 |
| rog | fwereade: is the idea behind env-set that you can change any of the environ config settings live | 07:39 |
| rog | ? | 07:39 |
| fwereade | rog, some of them | 07:39 |
| fwereade | rog, name, type, ec2 region etc must not change | 07:39 |
| rog | fwereade: oh, and what's ConfigSet.Get ? | 07:40 |
| fwereade | rog, ConfigSet is EnvironsConfig or whatever the environments.yaml-wrapper was called | 07:40 |
| fwereade | rog, ie the point at which configs enter the system | 07:41 |
| rog | fwereade: i see. i'm not sure how that relates to ValidateConfig though. | 07:42 |
| fwereade | rog, ValidateConfig will be called as part of it (with nil for a first argument) | 07:42 |
| rog | fwereade: hmm, i realise i don't understand how ValidateConfig is supposed to work | 07:44 |
| fwereade | rog, I dunno, it really doesn't feel to me like it's been thought through properly, but I'm experiencing notable internal conflict between just-implement-what-was-asked-for and but-I-think-it-hasn't-been-thought-through-fully | 07:44 |
| rog | fwereade: what's the "new" argument? | 07:44 |
| fwereade | rog, it always validates new, and returns something based on it, with possible tweaks (ensure region is set; fix the authorized-keys-path malarkey) | 07:45 |
| fwereade | rog, if old is not nil, it also checks need-to-be-immutable fields against old | 07:45 |
| rog | ah | 07:46 |
| fwereade | rog, it still doesn't feel quite like the right fit to me | 07:47 |
| * rog looks through code | 07:48 | |
| TheMue | Morning btw | 07:48 |
| fwereade | rog, but I do not in general have niemeyer's foresight, so I am trying to reserve judgment and follow the spirit of what he seeks | 07:48 |
| rog | TheMue: yo! | 07:49 |
| fwereade | rog, if you wish to take it up with him I will observe thediscussion with great interest, though | 07:49 |
| fwereade | TheMue, heyhey | 07:49 |
| TheMue | rog: The two weeks are already over? | 07:49 |
| rog | fwereade: AFAICS your environs/config proposal didn't have equivalent functionality to ValidateConfig, or am i missing something? | 07:49 |
| * TheMue thought it only has been one so far. | 07:49 | |
| rog | TheMue: yes, in the blink of an eye! | 07:50 |
| fwereade | rog, no it didn't -- I was going to add a ValidateReplacement method to the EnvironConfig interface | 07:50 |
| fwereade | rog, I felt that having providers only ever emit validated EnvironConfigs was a great feature | 07:51 |
| fwereade | rog, now any bozo can create a *Config from whatever junk they have lying around in their scope | 07:51 |
| Aram | morning rog. | 07:52 |
| Aram | already back? :). | 07:52 |
| rog | Aram: hiya | 07:52 |
| fwereade | rog, hopefully it will be apparent enough that this is not actually a very helpful thing to do | 07:52 |
| rog | fwereade: with niemeyer's proposal, i really don't see what advantage the config type has over just using a map. | 07:52 |
| fwereade | rog, but it seems reasonably sane to have SetConfig call ValidateConfig under the covers anyway so we shouldn't e able to screw it up too badly | 07:53 |
| fwereade | rog, I am not the right person to give helpful insight into the advantages of niemeyer's proposal | 07:54 |
| fwereade | rog, perhaps it will suddenly click with me | 07:54 |
| rog | fwereade: hmm, it does seem a bit like churn for the sake of it | 07:54 |
| fwereade | rog, but it doesn't feel to me like the simplest or most expedient way to go about it | 07:54 |
| fwereade | rog, well, what I *originally* wanted to was to have State.EnvironConfig and associated methods return and use actual EnvironConfigs | 07:55 |
| fwereade | rog, and it's perfectly easy to do this, and to have real instances which I know for sure are valid, in every possible circumstance, bcause they're backed by a real struct type that can only be created by code inside that specific environ | 07:57 |
| rog | fwereade: currently, there's no way of turning an EnvironConfig back into a map, i think. | 07:58 |
| rog | fwereade: wouldn't that be a disadvantage of the above? | 07:58 |
| fwereade | rog, yeah, but we need that one way or another and it's trivial | 07:58 |
| fwereade | rog, the EnvironConfig itself can be responsible for producing a map that will unpack back to an identical instance | 07:59 |
| rog | fwereade: it's just one more thing to get consistent (every time you add an environ field, you have to add code to go in the opposite direction) | 07:59 |
| rog | fwereade: in gustavo's proposal, the translation is always one way, and i think that could work ok. | 08:00 |
| rog | fwereade: network probs? | 08:20 |
| fwereade | rog, apparently so | 08:20 |
| fwereade | rog, did I miss anything? | 08:20 |
| rog | fwereade: last two things i said: | 08:20 |
| rog | [08:59:29] <rog> fwereade: it's just one more thing to get consistent (every time you add an environ field, you have to add code to go in the opposite direction) | 08:20 |
| rog | [09:00:09] <rog> fwereade: in gustavo's proposal, the translation is always one way, and i think that could work ok. | 08:20 |
| rog | fwereade: last thing i saw from you: [08:59:23] <fwereade> rog, the EnvironConfig itself can be responsible for producing a map that will unpack back to an identical instance | 08:21 |
| fwereade | rog, ha, our timelines are somewhat divergent | 08:21 |
| fwereade | <fwereade> rog, yeah, but we need that one way or another and it's trivial | 08:23 |
| fwereade | rog, the EnvironConfig itself can be responsible for producing a map that will unpack back to an identical instance | 08:23 |
| fwereade | <fwereade> rog, don't we have to do that regardless? | 08:23 |
| fwereade | rog, they're stored in /environment as maps, so we have to have some way to turn a config into a map, like it or now | 08:23 |
| fwereade | rog, I think it would be a really good thing if each env's idiosyncratic config-handling code were isolated behind a Config interface | 08:24 |
| fwereade | rog, it seems to me that different static types for configs with very different behaviour is a good thing because we don't have to smoosh the config-handling code across various bits of the relevant Environ and EnvironProvider types | 08:24 |
| fwereade | rog, as it is the only opportunity we have to turn it into a struct type is at SetConfig time, at which point its usefulness as a struct seems to me to be somewhat limited | 08:24 |
| * fwereade at this point looked at the code again and remembered something else, and went off along a new train of thought | 08:24 | |
| fwereade | rog, and now I'm not sure what to do about environments without secrets | 08:25 |
| fwereade | rog, offhand it seems to me that an environ without secrets is not intrinsically an invalid thing | 08:25 |
| fwereade | rog, ie you don't need to have secrets locally if the env already has them | 08:25 |
| fwereade | rog, but maybe that's a case we don't care about | 08:25 |
| fwereade | rog, hmm, do we have any rational basis for expecting the control-bucket to be public? | 08:25 |
| rog | fwereade: do we expect it to be public? | 08:25 |
| fwereade | rog, I didn't think so | 08:26 |
| fwereade | rog, so that then means we *do* need auth to even get a StateInfo out of an environ | 08:26 |
| fwereade | rog, and an Environ is not actually valid or useful in any way without its secrets | 08:27 |
| fwereade | rog, this is inconvenient wrt possible future needs (let someone manipulate your env without giving them your AWS keys? can't be done) | 08:28 |
| rog | fwereade: agreed. | 08:28 |
| fwereade | rog, but maybe I should just not worry about that at the moment | 08:28 |
| fwereade | rog, however this breaks my mental model a little | 08:28 |
| rog | fwereade: there's no particular reason that it must be the *same* secret, i suppose | 08:29 |
| fwereade | rog, I had been thinking the right thing to do was to allow an environ to exist without secrets, because I don't like the idea of an environ config delivered by the system that is not actually valid | 08:29 |
| fwereade | rog, true, but that's still a bunch of stuff to manage | 08:29 |
| rog | fwereade: when are we creating a partially valid environ? | 08:30 |
| fwereade | rog, state.Initialize seems sensible | 08:30 |
| fwereade | rog, IMO we should be pushing everything in the config that isn't a secret at bootstrap time | 08:30 |
| rog | fwereade: why's that? | 08:30 |
| fwereade | rog, consider default-series | 08:31 |
| rog | fwereade: i have a suspicion that that might make several things more complex without much benefit, but please persuade me otherwise! | 08:31 |
| rog | fwereade: go on | 08:31 |
| fwereade | rog, if we start an env with a given default-series, and then change our environments config, it is IMO unhelpful to suddenly swap out the environment's default-series justbecause we're doing our first explicit deploy | 08:32 |
| rog | fwereade: that's true of the environ secrets too. | 08:33 |
| rog | fwereade: i think it might be more reasonable just to document that the environ config gets frozen at first deploy | 08:33 |
| fwereade | rog, but there is a purpose to that and it doesn't lead to apparent random inconsistencies assuming the keys are validin the same situations | 08:34 |
| fwereade | rog, so, env-set shouldn;t work until we've deployed? | 08:34 |
| rog | fwereade: then we don't need to have anything that distinguishes secret from non-secret | 08:34 |
| rog | fwereade: hmm. | 08:34 |
| rog | fwereade: actually, i think env-set should work, but it should freeze the environment first | 08:35 |
| rog | fwereade: so we could say that the first command *after* bootstrap freezes the env | 08:35 |
| fwereade | rog, ok, actually, that works nicely | 08:35 |
| fwereade | rog, kinda :/ | 08:35 |
| * fwereade arranges thoughts | 08:35 | |
| fwereade | rog, default-series in particular seems to me to act oddly there | 08:36 |
| fwereade | rog, as an example of an env setting that has a tanglible effect on the bootstrap machine | 08:36 |
| rog | fwereade: because it affects which bootstrap machine is chosen? | 08:37 |
| fwereade | rog, and that it seems somewhat surprising to overwrite that arbitrarily, givn that the expectation want to set up, AIUI, is that environments.yaml settings will not be pushed by random operations | 08:38 |
| fwereade | rog, and that changes to environments.yaml will not in general have any effect on a running environment | 08:38 |
| rog | fwereade: i think that it's ok to say that bootstrap is special, and that environments.yaml is frozen on the first command after bootstrsap | 08:38 |
| rog | fwereade: and that's a reasonable and comprehensible model | 08:39 |
| fwereade | rog, I would prefer not to encourage the idea that it's sane or meaningful to alter environments.yaml after bootstrap | 08:40 |
| rog | fwereade: otherwise people might well say "why is default-series different from aws secret keys?" | 08:40 |
| rog | fwereade: i agree, but we do have the reality that we *have* to push some of the environ config after bootstrap | 08:41 |
| fwereade | rog, because the secrets are unique in that we need them to do *anything* with the env, even get the bootstrap machine's id from the storage | 08:41 |
| fwereade | rog, valid secrets are a prerequisite ofr doing *anything*; and so, transparently pushing known-valid secrets when none already exist in state prevents users from having to actually think about it at all | 08:42 |
| rog | fwereade: am i right in thinking there's only one secret key that we need to push? | 08:42 |
| rog | fwereade: (for ec2, at any rate) | 08:43 |
| fwereade | rog, I guess the access key id is not technically a secret | 08:43 |
| rog | fwereade: yeah | 08:43 |
| rog | fwereade: here's a possible (maybe crackful) thought | 08:43 |
| rog | fwereade: outside of environs, we just use map[string]interface{} | 08:44 |
| rog | fwereade: then | 08:44 |
| rog | fwereade: we provide EnvironsProvider.ValidateConfig(old, new map[string]interface{}) error | 08:45 |
| rog | fwereade: and we say that any key containing the string "secret" is secret | 08:45 |
| rog | fwereade: so it's trivial for provider-agnostic logic to filter out secret keys | 08:45 |
| rog | fwereade: of course, this assumes that other provider types use the string "secret" in their secret keys... | 08:46 |
| fwereade | rog, I have had thoughts roughly along those lines | 08:46 |
| fwereade | rog, not specifically including the secret stuff | 08:47 |
| * fwereade reserves judgment | 08:47 | |
| rog | fwereade: if we go this way, then the environs/config package isn't required at all, i *think* | 08:47 |
| rog | fwereade: oh no, it is | 08:47 |
| rog | fwereade: to avoid the loop | 08:47 |
| rog | fwereade: except... | 08:47 |
| rog | fwereade: there's no need for state to call ValidateConfig | 08:48 |
| rog | fwereade: so there's no need for a cyclic dependency. | 08:48 |
| rog | fwereade: so no need for environs/config, i think. | 08:48 |
| fwereade | rog, I think I may just be overly paranoid about people dumping random crap into /environments | 08:49 |
| fwereade | rog, this is at the heart of my desire to have the state methods that manipulate environ configs work with actual environ configs | 08:49 |
| rog | fwereade: we can still check that! | 08:50 |
| fwereade | rog, but not within state | 08:50 |
| rog | fwereade: state is internal | 08:50 |
| rog | fwereade: we still have control over where the environ config gets into the system | 08:50 |
| rog | fwereade: env-set | 08:50 |
| rog | fwereade: environs.Open | 08:51 |
| rog | etc | 08:51 |
| rog | fwereade: and those places can do the checking | 08:51 |
| fwereade | rog, so we have a bunch of state methods that validate what they're given, and one that pushes all responsibility for validation onto the various clients | 08:51 |
| rog | fwereade: state methods validate what they can. | 08:51 |
| rog | fwereade: we can state explicitly that the environment pushed into the state is not validated by the state | 08:52 |
| rog | fwereade: that may even be a good thing actually | 08:52 |
| fwereade | rog, ofc we can, doesn't stop it feeling sloppy to me | 08:53 |
| fwereade | rog, we may as well say that state is internal and so there's no need to validate anything we put into it | 08:53 |
| fwereade | rog, we can always trust the client code we write to do the right thing, after all | 08:53 |
| rog | fwereade: it's a trade-off - i *think* that it's not worth creating a new package for | 08:53 |
| * fwereade maintains a studiously straight face | 08:54 | |
| rog | fwereade: there are lots of ways we can create invalid content inside state, i thinl | 08:54 |
| rog | s/thinl/think/ | 08:54 |
| fwereade | rog, like what? | 08:55 |
| rog | fwereade: like we could do OpenPort("tpc", 45) | 08:56 |
| fwereade | rog, instructive example | 08:57 |
| * fwereade ponders | 08:57 | |
| fwereade | rog, I think the distinction there is that there is no global source of truth we can use to validate against the set of protocols we know about elsewhere in the system | 08:59 |
| fwereade | rog, in the environment case we can do so, and it doesn't feel like much work to me | 08:59 |
| fwereade | rog, but look, this is not an especially productive conversation because I keep falling back to describing my own perspective on the problem | 09:00 |
| fwereade | rog, which is to all intents and purposes irrelevant because you the perspective that is most relevant is niemeyer's, and my internal niemeyer sim is not an outstanding predictor of the real niemeyer's thought | 09:01 |
| rog | fwereade: it seems to me that there are several levels of verification we have on the state | 09:01 |
| rog | :-) | 09:01 |
| fwereade | rog, I think I do have to step back and sketch everything out again :) | 09:02 |
| rog | fwereade: i'm looking at niemeyer's paste, and it seems to me that his environs/config is trivial | 09:02 |
| rog | fwereade: and not really worth it | 09:02 |
| rog | fwereade: each of the different methods on config.Config just return a particular field of the map | 09:03 |
| fwereade | rog, look, yes, I am somewhat sympathetic to that point of view; if the prevailing opinion is that the content of /environment is a free-for-all and we can handle that then map[string]interface{} is perfectly sufficient | 09:04 |
| fwereade | rog, however at the moment I am somewhat without compass and it has become apparent that I need to chart the differences between my perspective and niemeyer's in some detail so I can actually proceed usefully | 09:05 |
| rog | fwereade: i'm sympathetic to this point of view of niemeyer's: " | 09:05 |
| rog | There are very | 09:05 |
| rog | few places through which configuration get into the system, and the | 09:05 |
| rog | configuration should be validated right there. | 09:05 |
| rog | " | 09:05 |
| rog | fwereade: so the content of /environment is only a free-for-all if we treat it as such | 09:05 |
| fwereade | rog, my concern is that if we can then at some stage we will | 09:05 |
| rog | fwereade: and placing the validation outside of state means that everything else becomes simpler. | 09:06 |
| rog | fwereade: (the fewer places with registered global state the better, i think) | 09:06 |
| rog | fwereade: the idea behind State isn't, i think, that we design it such that it can express *exactly* the things we can do in juju, but that it's a nice convenient abstraction layer that expresses *at least* the things we can do in juju. | 09:08 |
| rog | fwereade: when we start making things more complex so that we can get more verification in State, i think we're going a little too far. | 09:09 |
| rog | fwereade: it's a convenience for our own use, and i don't think we need to be bound by it. | 09:09 |
| fwereade | rog, I just don't see how requiring explicit validation outside of Config objects *actually* makes things simpler | 09:09 |
| fwereade | rog, it just means we need to do lets of external messing about with maps and typecasting | 09:09 |
| rog | fwereade: it means we don't need a new package with separate global registration | 09:09 |
| fwereade | rog, if that's the only objection then that can be worked around | 09:10 |
| rog | fwereade: really? i thought that was inherent to the cyclic-dependency-breaking nature of environs/config. | 09:11 |
| fwereade | rog, oh wait, maybe not | 09:11 |
| fwereade | rog, no, it means we need a provider to get a config and there's no way around it | 09:11 |
| rog | fwereade: yup | 09:12 |
| rog | fwereade: i'm not sure where you think all the messing around with maps and typecasting will go | 09:12 |
| fwereade | rog, my perspective just clicked around a notch | 09:14 |
| rog | :-) | 09:14 |
| fwereade | rog, I can restrict the messing around to only the bits that are directly concerned with the struct type | 09:15 |
| rog | fwereade: not sure i understand | 09:15 |
| fwereade | rog, ok, this perspectives STM to assume that ValidateConfig will directly construct struct types from its *Config (or map) params | 09:16 |
| fwereade | rog, will manipulate those as required | 09:16 |
| fwereade | rog, and then return a fully validated one | 09:16 |
| fwereade | rog, as a new *Config (or map) | 09:17 |
| rog | fwereade: i don't think there's any need to return a validated config. | 09:17 |
| rog | fwereade: is there? | 09:17 |
| rog | fwereade: isn't the "new" argument supposed to be valid? | 09:17 |
| fwereade | rog, how are we meant to know that? | 09:17 |
| rog | fwereade: isn't that what ValidateConfig checks? | 09:18 |
| fwereade | rog, yeah, a valid user-specified config may still require manipulation | 09:18 |
| fwereade | rog, (1) region must never change at runtime, so we should bake in the value in case an update changes the default region | 09:18 |
| rog | fwereade: that's why we pass in the old config, no? | 09:19 |
| fwereade | rog, (2) authorized-keys-path should not exist on the server -- if it does it's evidence of total crazkfulness on our part | 09:19 |
| fwereade | rog, and we should therefore remove it and sub in a correct authorized-keys setting | 09:19 |
| rog | fwereade: how does the provider know that it's on a server or not? | 09:20 |
| fwereade | rog, right! it shouldn't have to -- it shouldn't even see anything that could indicate a difference | 09:20 |
| fwereade | rog, the actual configuration setting is authorized-keys -- authorized-keys-path is a convenient alternate way of setting it | 09:21 |
| fwereade | rog, the environment should not have to care even slightly about what filesystem it happens to be running on, or even I guess whether it is ;) | 09:22 |
| rog | hmm | 09:22 |
| rog | fwereade: i see. what we get out of environments.yaml isn't what we put into the state. | 09:23 |
| rog | yuck | 09:23 |
| fwereade | rog, yeah, I have this total conviction that they are two distinct things and we could make our lives easier by distinguishing them in code | 09:24 |
| fwereade | rog, but that has not received a lot of traction | 09:24 |
| fwereade | rog, to tweak what you said: what we get out of environments.yaml isn't what fwereade personally thinks we put into the state. | 09:25 |
| fwereade | rog, I guess we could just have the field and not bother to look at it when it might not be valid, but that seems pretty crackful itself | 09:25 |
| fwereade | to me | 09:25 |
| rog | fwereade: mutating authorized-keys-path makes sense. | 09:26 |
| rog | fwereade: this *was* my idea for how ValidateConfig would work: http://paste.ubuntu.com/1082463/ | 09:27 |
| fwereade | rog, yeah, I think it needs a little more | 09:27 |
| rog | fwereade: the problem with making client-side and server-side config two different things is that they are really very similar! | 09:28 |
| fwereade | rog, sure | 09:28 |
| rog | fwereade: hmm, against my previous inclination, perhaps we should just have Environ.Config to return a configuration map from an opened environ | 09:29 |
| fwereade | rog, but if we had struct types we could just serialize/deserialize the struct types with yaml for storage, and *only* ever let valid struct types into the system, and only bother with the fancy stuff when loading a user config | 09:30 |
| fwereade | rog, yeah, that feels to me like a sensible thing | 09:30 |
| rog | fwereade: that means we'd need two versions of the deserializing code, which seems somewhat like overkill to me. | 09:32 |
| rog | fwereade: also, we can't deserialize directly from a config code, i think. | 09:33 |
| rog | s/code/node/ | 09:33 |
| fwereade | rog, one which just blats the yaml direct into a struct, and one that contains all the pre-validation and tweaking etc? | 09:33 |
| fwereade | rog, oh? why not? | 09:33 |
| fwereade | rog, I would imagine we could just blat it straight into a *whateverConfig | 09:34 |
| rog | fwereade: because yaml.Unmarshal takes a []byte | 09:34 |
| rog | fwereade: not a map[string]interface{} | 09:34 |
| fwereade | rog, yes, this also depends on the separate global config registry so that we can have the configs directly responsible for what is stored | 09:35 |
| fwereade | rog, which I know you don;t like | 09:35 |
| rog | fwereade: the *only* reason that i made config parsing separate from config opening was so that a config file would get validated completely. | 09:36 |
| rog | fwereade: and actually, that's still possible with ValidateConfig | 09:36 |
| fwereade | rog, I don't actually see the benefit of refusing to work with env X because env Y is broken | 09:36 |
| rog | fwereade: you may well be right. in which case i'm very happy to live without passing around configurations as structs. | 09:39 |
| rog | fwereade: i'm happy to have state never call into environs | 09:39 |
| fwereade | rog, yeah, I know :) | 09:39 |
| rog | fwereade: it seems like a reasonable separation of concerns to me | 09:40 |
| rog | fwereade: BTW does state actually need to know anything that goes on inside the environ config at all? | 09:41 |
| rog | fwereade: for instance, does the state need to know that environ name, default series or type? | 09:42 |
| fwereade | rog, only if we care about its validity -- and I still think we do/should but I think I have been pretty comprehensivelyoverruled on that one | 09:42 |
| rog | s/that/the/ | 09:42 |
| fwereade | rog, certainly default-series | 09:42 |
| rog | fwereade: of course we care about its validity... | 09:42 |
| fwereade | rog, yes, but it has been argued that we can trust the clients | 09:42 |
| rog | fwereade: they're our clients, so can we trust ourselves? | 09:42 |
| fwereade | rog, and therefore at that specific point we merely assume validity, rather than worrying about it | 09:43 |
| rog | fwereade: yea | 09:43 |
| rog | h | 09:43 |
| rog | fwereade: what does state use default-series for? | 09:43 |
| fwereade | rog, IMO no, of course we can't trust ourselves ;p | 09:43 |
| fwereade | rog, deploy command | 09:43 |
| fwereade | rog, and the only sane place to get it is from state | 09:43 |
| rog | fwereade: the deploy command isn't in state, is it? | 09:43 |
| fwereade | rog, how is it going to get the default series *except* from state? | 09:44 |
| rog | fwereade: i'm not suggesting that the default series isn't store inside state, | 09:44 |
| rog | stored | 09:44 |
| rog | fwereade: but that state has no reason to get default-series (or any environ config attribute) itself | 09:45 |
| rog | fwereade: i.e. for the purposes of the state package, the environ config is opaque | 09:45 |
| fwereade | rog, well there was talk of a state.DefaultSeries a while back but I think that would indeed want reevaluation in the light of the current direction | 09:45 |
| fwereade | rog, it all comes down to how much we trust ourselves, and I have found that *not* trusting myself is frequently a winning strategy | 09:47 |
| fwereade | rog, I am happy to follow the collective wisdom on this point but I have not successfully internalised the attitude that the decision follows from | 09:48 |
| rog | fwereade: it's a trade-off as always IMHO. i've also found that keeping things as simple as possible is good. | 09:48 |
| rog | fwereade: it's also a matter of how easy it is to verify the trust | 09:49 |
| rog | fwereade: in this case, i think there are only a very few places that need verification. | 09:49 |
| fwereade | rog, yeah, I know -- I'm just saying it's a tradeoff I would make differently, and that makes it harder because my complexity-predicting heuristics lead me in a different direction to that I am consciously aiming for | 09:51 |
| rog | fwereade: here's a representation of my current thinking: http://paste.ubuntu.com/1082500/ | 10:04 |
| rog | fwereade: oops, EnvironConfig should have gone there | 10:05 |
| fwereade | rog, was wondering about that | 10:05 |
| fwereade | rog, not implausible, I'm just trying to think through the consequences of a slightly different shape | 10:06 |
| rog | davecheney: hiya! | 10:44 |
| Aram | what's the difference between Test and TestPackage in Gocheck? | 10:53 |
| Aram | s/in/for/ | 10:57 |
| davecheney | rog: howdy mate | 11:11 |
| davecheney | had a good holiday ? | 11:11 |
| rog | davecheney: that i have | 11:11 |
| rog | davecheney: just trying to get up to speed with all the stuff that's happened while i've been gone... | 11:11 |
| rog | davecheney: how're the agents these days? | 11:12 |
| rog | davecheney: did you manage to get that branch working? | 11:12 |
| rog | Aram: i don't understand your question | 11:16 |
| Aram | rog: the function used to integrated gocheck tests into gotest sometimes is named Test, sometimes TestPackage. | 11:17 |
| Aram | any particular meaning to that? | 11:17 |
| rog | Aram: it looks like Test is standard | 11:18 |
| rog | Aram: but it doesn't matter too much | 11:19 |
| Aram | ok. | 11:19 |
| davecheney | rog: yup, it's waiting on final approval now | 11:20 |
| rog | davecheney: yay! | 11:20 |
| rog | davecheney: so the machine agent is actually starting new machines now? | 11:21 |
| rog | davecheney: i mean the PA of course | 11:21 |
| davecheney | rog: no | 11:21 |
| davecheney | sadly not | 11:21 |
| davecheney | we're still stuck at the 'who pushes the secrets' problem | 11:22 |
| rog | davecheney: ah, of course. | 11:22 |
| niemeyer | Gooood Mondays! | 13:07 |
| Aram | hey hey | 13:09 |
| rog | niemeyer: yo! | 13:14 |
| niemeyer | rog: Welcome back! | 13:18 |
| rog | niemeyer: i was discussing the newly proposed environs/config package with fwereade earlier | 13:19 |
| rog | niemeyer: i think i'm broadly in agreement with your take on it | 13:20 |
| niemeyer | rog: That's great.. I think we're all in agreement then, phew | 13:20 |
| rog | niemeyer: i'm not sure that the actual package pulls its weight though | 13:20 |
| niemeyer | rog: The package is needed for the state package to not have a dependency on environs | 13:21 |
| rog | niemeyer: i was wondering about something like this: http://paste.ubuntu.com/1082729/ | 13:21 |
| rog | niemeyer: i'm not sure | 13:21 |
| rog | niemeyer: i think the main reason for the dependency was so that state could call back into environs to verify the config | 13:22 |
| rog | niemeyer: but your proposal does away with that | 13:22 |
| rog | niemeyer: and hence, i *think*, the need for a new package | 13:22 |
| niemeyer | rog: Yes, and state still wants to return a Config | 13:22 |
| niemeyer | rog: Despite the fact it won't be validating | 13:22 |
| rog | niemeyer: it could just return a map[string]interface{} | 13:22 |
| niemeyer | rog: Yes, it could be less nice, but it can be nicer too | 13:23 |
| rog | niemeyer: which can then be validated outside state | 13:23 |
| niemeyer | rog: There's no need to validate what's coming out of state | 13:23 |
| niemeyer | rog: It was validated when going in | 13:23 |
| rog | niemeyer: oh yes, that's true | 13:23 |
| rog | niemeyer: if we *do* have a config package, i also wondered if it was really worth hiding the map. | 13:25 |
| rog | type Config map[string]interface{} | 13:25 |
| rog | might be more transparent | 13:25 |
| niemeyer | rog: I think it is, because we don't want code to be fiddling with the map in principle | 13:25 |
| rog | niemeyer: ok, fair enough | 13:26 |
| niemeyer | rog: Nor using stock methods (DefaultSeries) from the map | 13:26 |
| rog | niemeyer: i don't mind m["default-series"].(string) too much | 13:26 |
| rog | niemeyer: but i see your point too | 13:27 |
| niemeyer | rog: I do mind, because it's just as easy to write m["default-seres"].(string) | 13:28 |
| rog | niemeyer: presumably we'd also need config.New(attrs map[string]interface{}) (*Config, error) too? | 13:29 |
| rog | niemeyer: with a config package, i might suggest something like this: http://paste.ubuntu.com/1082741/ | 13:30 |
| niemeyer | rog: Yeah, that sounds good | 13:30 |
| niemeyer | rog: Yeah, that looks mostly ok.. the Validate method needs to return a new config, though, because the proposal is to have config being immutable for now | 13:31 |
| rog | niemeyer: why does validate need to return a new config? | 13:32 |
| rog | niemeyer: isn't the new config already valid? | 13:32 |
| niemeyer | rog: The transition from -keys-path to -keys will be done there | 13:32 |
| rog | niemeyer: i thought that could be done by Environ.Config | 13:32 |
| niemeyer | rog: Doing SetConfig/Config so we can get a proper configuration is awkward.. validation sounds like the proper place to do it | 13:33 |
| rog | niemeyer: it seems like Validate is doing more than just validation here. | 13:34 |
| niemeyer | Validate \Val"i*date\, v. t. [See {Valid}.] | 13:35 |
| niemeyer | To confirm; to render valid; to give legal force to. | 13:35 |
| rog | niemeyer: and i'm not sure that we would ever need to do SetConfig then Config to get a valid configuration - wouldn't we always Open the environ first. | 13:35 |
| rog | ? | 13:35 |
| niemeyer | That's validation. | 13:35 |
| niemeyer | rog: Not necessarily.. we can change the configuration without going through a re-open cycle | 13:36 |
| rog | niemeyer: the "?" was to be added to the end of my sentence... | 13:36 |
| niemeyer | rog: Sorry, ok, that's the answer htne :) | 13:36 |
| rog | niemeyer: so Validate is the same as Open(old) followed by SetConfig(new) followed by Config? | 13:38 |
| niemeyer | rog: No.. validate is definitely not the same as Open | 13:39 |
| rog | niemeyer: i'm trying to get a handle on the different checks performed by Validate vs Open | 13:39 |
| Aram | man, how annoying the misinformation on #go-nuts can be. "regexp needs an overhaul", "yes, it's very bad on the shootout", etc. | 13:40 |
| rog | niemeyer: because, presumably Validate does some checks that are currently performed by Open, such as opening the ssh provider key | 13:40 |
| niemeyer | rog: What does opening a key mean? | 13:41 |
| niemeyer | Aram: Ouch | 13:41 |
| rog | niemeyer: i mean opening the authorized-keys-path and reading it, so that it can transmute it into authorized-keys | 13:42 |
| niemeyer | rog: Well, yes, I just mentioned that above | 13:42 |
| niemeyer | rog: Another one is the validation of whether a region can change, for example, and the hardcoding of the region name | 13:53 |
| niemeyer | rog: region: "" => region: "us-east-1" | 13:53 |
| rog | niemeyer: something like this, perhaps? http://paste.ubuntu.com/1082778/ | 13:54 |
| niemeyer | rog: Yeah, except it should be called Validate as we've been agreeing on | 13:54 |
| rog | oops, yeah | 13:54 |
| rog | niemeyer: as for secrets, how about when the environment is initially uploaded, all keys containing the string "secret" are excised. | 13:55 |
| rog | ? | 13:55 |
| rog | niemeyer: alternatively... | 13:55 |
| niemeyer | rog: That's a great idea, but it's unfortunately too late | 13:56 |
| rog | niemeyer: we don't upload the environment at all until the first command after bootstrap. | 13:56 |
| niemeyer | rog: Too late as in we have a configuration format right now that people are used to | 13:56 |
| rog | niemeyer: are there secret keys in other providers that are called different things? | 13:56 |
| rog | niemeyer: i guess so. | 13:56 |
| rog | niemeyer: i should look at orchestra etc | 13:57 |
| rog | niemeyer: another alternative: http://paste.ubuntu.com/1082789/ | 13:58 |
| niemeyer | rog: That doesn't seem like a good place to do this | 13:59 |
| rog | niemeyer: where are you thinking is a good place? | 13:59 |
| niemeyer | rog: We should look at the problems we have relative to secrets, and try to address those, but for now it sounds like we're good to go with the first change | 13:59 |
| rog | niemeyer: sounds good | 14:02 |
| niemeyer | rog: When you have a moment would you mind to file your flight details in the sprint page? | 15:44 |
| niemeyer | rog: Otherwise you may miss a room :-) | 15:45 |
| rog | niemeyer: ok, will do | 15:45 |
| rog | niemeyer: hmm, i'm not allowed to view the page, it seems | 15:46 |
| niemeyer | rog: You have to login | 15:46 |
| rog | niemeyer: i was logged in. perhaps i was logged in as the wrong user... will try again. sadly my phone is dead, with all my login details with it :-( | 15:47 |
| niemeyer | rog: Everyone in Canonical must necessarily have access to this wiki | 15:47 |
| niemeyer | rog: I don't think you can have two users there even | 15:48 |
| rog | niemeyer: ah, yes, i've now logged in. apparently i've got two valid logins. | 15:48 |
| niemeyer | rog: I doubt it | 15:48 |
| niemeyer | rog: This is associated to your Launchpad account, and it's tightly controlled | 15:48 |
| rog | niemeyer: there was a mix up. i thought they'd sorted it out, but it seems not. | 15:49 |
| rog | niemeyer: i've got two valid launchpad ids | 15:49 |
| rog | niemeyer: but only one is associated with the right canonical perms | 15:50 |
| niemeyer | rog: Well, apparently one of them doesn't work? :-) | 15:50 |
| rog | niemeyer: it works for some things... | 15:50 |
| niemeyer | rog: Sure, it can work for anything that is not the internal wiki :) | 15:50 |
| rog | niemeyer: this was the post i got from someone in admin: https://pastebin.canonical.com/69677/ | 15:52 |
| rog | niemeyer: it probably not surprising i was confused | 15:53 |
| rog | s/it/it's/ | 15:53 |
| niemeyer | rog: Indeed | 15:53 |
| niemeyer | Sounds messed up | 15:53 |
| rog | niemeyer: yeah. i'm not sure i really understand the underlying auth model. | 15:54 |
| rog | niemeyer: done. | 15:59 |
| niemeyer | rog: Cheers | 15:59 |
| rog | TheMue: hey, seems like we're sharing flights! | 15:59 |
| TheMue | *click* | 16:01 |
| TheMue | rog: Nice, should try to get seats together. | 16:02 |
| TheMue | niemeyer: How free are you for reviews today? With https://codereview.appspot.com/6345072/ in I would propose my firewall tomorrow for a first look. Right now I'm doing the tests, all service related ones are green, now I start with the machin related ones. | 16:03 |
| niemeyer | TheMue: I was just about to ping you to talk about your eod email the other day and see how things are going | 16:04 |
| niemeyer | TheMue: But I decided not to because lunch is waiting | 16:04 |
| niemeyer | TheMue: I'm definitely available for reviews + talk on it | 16:04 |
| niemeyer | TheMue: In ~1h | 16:05 |
| TheMue | niemeyer: Fine, the watcher in the review above should be simple. | 16:05 |
| niemeyer | TheMue: Super, let's catch up in a moment then | 16:06 |
| niemeyer | bbs | 16:06 |
| TheMue | ok | 16:06 |
| rog | niemeyer: what was the name of that logging web service you mentioned a while back? | 16:53 |
| niemeyer | rog: https://papertrailapp.com/ | 17:01 |
| rog | niemeyer: thanks. i'd been grepping the logs and googling "logging as a service" but failed to dig it up | 17:02 |
| niemeyer | rog: No worries | 17:02 |
| niemeyer | TheMue: Yo | 17:27 |
| niemeyer | TheMue: ping? | 17:36 |
| * rog is off for the day. i hope to get something more done than just catching up tomorrow! | 17:36 | |
| niemeyer | rog: Have a good one | 17:39 |
| TheMue | niemeyer: pong | 17:42 |
| niemeyer | TheMue: Hey | 17:42 |
| niemeyer | TheMue: Going over your CL now.. do you want to catch up something around it? | 17:42 |
| TheMue | niemeyer: No, it's only needed for the one containing the firewall. That one is almost done (some tests missing) and I would like to propose is tomorrow morning. | 17:44 |
| niemeyer | TheMue: Super | 17:45 |
| TheMue | niemeyer: I've needed some time to get behind it, but it's nice to see in the tests how it fits together. | 17:46 |
| niemeyer | TheMue: Neat! | 17:48 |
| niemeyer | TheMue: That's always a good sign | 17:48 |
| niemeyer | TheMue: I thought the watcher tests had been moved to a nicer location (out of state_test.go) with fwereade_'s modification? | 18:27 |
| TheMue | niemeyer: It has been watcher_test.go before. William moved them to the types where you retrieve them from. And This one is retrieved directly from State, | 18:30 |
| niemeyer | TheMue: I see.. there's WatchMachines there indeed | 18:32 |
| niemeyer | TheMue: Might be good to at least put them next to each other | 18:32 |
| niemeyer | TheMue: Perhaps pulling WatchMachines down to the end | 18:32 |
| TheMue | niemeyer: Geed idea. | 18:33 |
| niemeyer | TheMue: The use of idx in the test is a bit awkward | 18:34 |
| niemeyer | TheMue: Well.. I guess it's fine | 18:34 |
| niemeyer | TheMue: Bothers me a little that it only works for a single change, but I guess it's ok | 18:34 |
| niemeyer | TheMue: LGTM with doc suggestions only | 18:35 |
| TheMue | niemeyer: Cheers | 18:36 |
| TheMue | niemeyer: Will do it tomorrow, wife want's me now. ;) | 18:37 |
| niemeyer | TheMue: Enjoy man | 18:37 |
| TheMue | niemeyer: Thx, cu. | 18:37 |
| niemeyer | TheMue: Cheers | 18:37 |
| niemeyer | Will have a walk outside.. back later | 20:24 |
| niemeyer | davecheney: Morning! | 22:13 |
| davecheney | niemeyer: howdy | 22:41 |
| davecheney | just on the phone | 22:41 |
| niemeyer | davecheney: About // TODO(rogerpeppe) change upstart.Conf.Cmd to []string so that | 22:50 |
| niemeyer | davecheney: This is going in with this CL, apparently? | 22:50 |
| niemeyer | davecheney: Ah, you've based off of his branch | 22:50 |
| davecheney | niemeyer: yes | 22:50 |
| davecheney | that is somethign rog identified, i guess he is going to look at it | 22:51 |
| davecheney | but I can do so if you like | 22:51 |
| niemeyer | davecheney: Please just drop it.. that code is fine. THere's no quoting necessary for it to work correctly | 22:51 |
| davecheney | understood | 22:52 |
| niemeyer | davecheney: Actually, hold on | 22:52 |
| niemeyer | davecheney: The %q there is bogus.. that's probably what it is referring to | 22:52 |
| niemeyer | davecheney: I suggest using something simpler, such as "'%s'" | 22:53 |
| niemeyer | davecheney: No address should contain single quotes in them, so this should be fine | 22:53 |
| niemeyer | davecheney: It's also not something we're getting through user input, which might turn it into an issue | 22:54 |
| niemeyer | davecheney: Sent a review on https://codereview.appspot.com/6347044 | 22:59 |
| davecheney | thanks, still on the phone | 22:59 |
| davecheney | will check in a sec | 22:59 |
| niemeyer | davecheney: Cheers.. there just a couple of minors from the previous where we seem to misunderstand each other. Otherwise we can move on | 23:00 |
| davecheney | awesome, i'll review those now | 23:00 |
| niemeyer | and a plain LGTM on the other one.. good stuff. | 23:03 |
| davecheney | niemeyer: in related news, are you able to add mramm to the golang gophers | 23:04 |
| davecheney | and also juju-dev on lp | 23:04 |
| davecheney | i noticed he was missing from those groups ? | 23:04 |
| niemeyer | davecheney: You mean ~juju? | 23:05 |
| niemeyer | (there's no juju-dev, I think) | 23:05 |
| davecheney | niemeyer: yes, i think I do, he should probably be in the same groups I am in so he can receive moar email | 23:06 |
| niemeyer | Done adding to ~juju | 23:06 |
| niemeyer | davecheney: Haha :) | 23:06 |
| davecheney | niemeyer: just submitting my fix branch that fixes a bug on lp[ | 23:06 |
| davecheney | this will be interesting to see how the integration works | 23:06 |
| niemeyer | davecheney: hm? bug in lp? | 23:06 |
| niemeyer | Which lp? | 23:07 |
| davecheney | no no, a bug that was registered to lp:goamz project | 23:07 |
| davecheney | actually no, the bug is on juju-core | 23:07 |
| davecheney | i have another branch for the goamz work | 23:07 |
| davecheney | what i meant is this is the first time submitting a -bug via lbox | 23:08 |
| davecheney | niemeyer: in related news, i had a quick go at using gocov on juju-core | 23:08 |
| davecheney | it works, but is quite buggy | 23:08 |
| davecheney | niemeyer: this was the goamz bug, https://bugs.launchpad.net/goamz/+bug/1021515 | 23:10 |
| niemeyer | davecheney: Ah, I see.. your first message got here corrupted (it said "lp[") | 23:10 |
| niemeyer | davecheney: As one of the tasks for the sprint, I'd like to try to get the lifecycle logic correct in mstate, state, and the provisioning agent (which is our most complete agent to date) | 23:24 |
| niemeyer | davecheney: making it consider alive/dying/dead semantics | 23:24 |
| niemeyer | davecheney: Just something to boil up in your mind meanwhile | 23:25 |
| davecheney | niemeyer: yes, i have been pondering on that since last week | 23:36 |
| niemeyer | Phew.. review queue empty again | 23:37 |
| davecheney | nice work | 23:39 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!