[00:08] niemeyer: howdy [00:08] sorry i'm a bit slow off the mark today [00:08] m_3 and I are checking out a coworking space in the city today [00:09] niemeyer: short status report, got ec2 local tests working with a provisoiner working last night [00:09] but there are a few problems that need to be solved before I can propose it [00:09] davecheney: Heya [00:09] davecheney: Neat, sounds good [00:09] niemeyer: the main one is we ask ec2test for the DNS name of machine/0 [00:09] whcih returns something liek i-3.example.com [00:10] which breaks the local test because it then uses that hostname in the stateinfo passed to anyone who wants it [00:11] so I need to add some kind of hook to either ec2test to say 'always return localhost' as the dns name, or to be able to override it if the test is running in local mode [00:11] niemeyer: the second problem is ec2 is hard coded to expect the zkport to be 2181 [00:11] but that is simpler to fix [00:11] i'll propose a fix real soon now [00:12] davecheney: Hmm [00:12] davecheney: Wait, you mean you're putting a worker within the ec2 tests? [00:13] niemeyer: by worker, do you mean provisioner ? [00:13] davecheney: a juju-core/worker [00:14] niemeyer: yes, the tests that roger left bootstrap an environment then try to do a state.AddMachine [00:15] davecheney: Within the *ec2* tests? [00:15] davecheney: I may be wrong, and will be happy to look at it, but it doesn't sound entirely right [00:15] niemeyer: yes, roger extended jujutest/livetest to start a machine in the provisioned environment [00:15] hang on, will get a link [00:21] niemeyer: sorry, m_3 just turned up [00:21] https://codereview.appspot.com/6347044/ << not a proposal [00:21] davecheney: No worries, I'll actually have to step out for dinner [00:21] this is a grab bag of rogers stuff and my additions [00:22] niemeyer: the key change is https://codereview.appspot.com/6347044/patch/10001/11012 [00:22] specifically, if hasProvisioner then testprovisioner [00:58] davecheney: I'm indeed a bit lost.. in a sentence or two, what is this testing? [01:05] niemeyer: the changes to jujutest test that after bootstrapping, a provisioner is running, and it can start an instance with state.AddMachine [01:06] davecheney: I can't see how we could possibly be testing that without a functional test [01:06] davecheney: How can we be testing that after bootstrap a provisioner is running? This is entirely dependent on details of ec2 itself [01:08] yes, i haven't got that to work in the ec2 amazon tests yet because of the push secrets problem [01:08] this only works when we make the local provisioning agent in process for local_test.go [01:08] but in theory, once the secrets are pushed (faux juju deploy) the test will also pass on the real ec2 [01:56] davecheney: The point is that there's nothing being tested there [01:57] davecheney: The live real tests can do that on real EC2 [01:57] davecheney: If we fake the precise point we want to test, what's the purpose of the test? [01:58] niemeyer: this test should (ha!) work on the real ec2 as well [01:59] so at the moment all I can prove is the test case works when we fake out all the moving parts behind it [02:00] that is, localtest continues to work with a test plan that expects a provisioner [02:00] once the pushing of secrets is fixed, we can remove the 'hasprovisioner' field as it will always be true [02:02] davecheney: I understand, but this doesn't seem to be going in a good direction [02:02] davecheney: All this faking isn't buying us stability or anything [02:02] davecheney: It's messing up the code instead [02:03] niemeyer: should I put this on hold until we know how to fix the pushing of secrets and can do a real test in a real ec2 ? [02:03] davecheney: No, I think we should fix the problems we have to fix for getting the real EC2 running [02:04] niemeyer: cool, that means resolving the secrets problem [02:04] davecheney: The secrets problem is resolved in deploy [02:04] davecheney: There's no *real* problem.. we just need to do the same thing we used to do [02:05] davecheney: Sorry, I'm kind of stating that with the belief I know what's going on, and maybe I'm wrong [02:05] davecheney: So please don't take that at face value.. feel free to bring other points [02:05] niemeyer: thats cool [02:05] the problem i think I'm solving is [02:06] 1. get boostrapping working -- which I believe is done, juju bootstrap works [02:06] 2. prove it with tests [02:06] 2 is tricky, and this CL is part way towads solving it [02:06] davecheney: There are also several bits in the same CL that should be broken down so we can debate about them (and integrate them!) in isolation [02:07] davecheney: e.g. there's a fix for cloudinit, etc [02:07] niemeyer: yes, i'll peel off those bits [02:07] just at the moment i'm taking another pass at the s3 location constraint proposal [02:08] because I want to commit that before telling everyone to please go get -u [02:08] then i'll start to pull apart this CL [02:08] I find this, for instance, unexpected: [02:08] 87 err = t.seedSecrets(env) [02:08] 88 c.Assert(err, IsNil) [02:08] 89 if t.HasProvisioner { [02:08] 90 t.testProvisioning(c, st) [02:08] 91 } [02:10] davecheney: This is in TestBootstrap [02:10] yes, all environments should have a provisioner, but we can't do that yet [02:10] sorry, all providers [02:11] davecheney: So why would we be seeding secrets if we're testing bootstrap? [02:11] niemeyer: right, i understand [02:11] I will make a new test case [02:11] davecheney: In fact, not only seeding secrets, but hacking the environment configuration arbitrarily [02:12] niemeyer: agreed, this is to work around the fact that bootstrap doesn't push the complete environment configuration into the state [02:12] if it did, then this wouldn't be a problem [02:12] davecheney: But this is TestBootstrap! :) [02:13] davecheney: It does not, and it should not [02:14] understood, i'll rework it [02:14] davecheney: If the idea is to test a real ec2 deployment on live tests, we should call fwereade_'s conn.Deploy [02:14] davecheney: and let it do the work [02:15] niemeyer: i'll try that [02:17] davecheney: There's another issue with the CL approach as well that is good to keep in mind [02:17] davecheney: One of the goals of jujutest is for it to be reusable across providers [02:17] yes [02:18] davecheney: So we have to watch out for provider-specific behavior there.. in some cases we may need to hook onto the provider [02:18] davecheney: Via a callback or whatever [02:18] davecheney: But ideally we should be able to take these tests, plug on the OpenStack or Compute Engine backend, and have them working [02:19] niemeyer: yes [02:19] just checking what abuse i've done to jujutest [02:20] davecheney: The configuration is the only thing that stands out to me [02:20] niemeyer: the complication is the ec2 environ has two modes [02:20] local fake ec2 mode and amazon mode [02:21] supporting that is hard [02:21] for example, fake ec2test server gives back DNS names which are impossible [02:21] ie, i-3.example.com [02:21] which can't be used as value StateInfo addresses [02:22] davecheney: Right, but the question is where do we draw the line [02:23] davecheney: I don't think it'd pay off if we simulated the *whole* provider [02:23] davecheney: How do we actually deploy a machine? [02:23] davecheney: How do we get unit agents running and interacting? [02:23] davecheney: Faking out all of that stuff would have us with white hair by then [02:24] and as you say, doesn't actually test anything [02:24] davecheney: It could test, the interaction between the pieces, but indeed not the *deployment* of the pieces, since that's all faked indeed [02:24] Meaning it could be entirely broken and we'd think it's alright [02:25] davecheney: So, to begin with, I'd be happy with [02:25] it's really about continuing the illusion of the ec2 provider in local mode [02:25] if !t.CanOpenState { return } [02:25] davecheney: At the top of the test that checks the real provisioning agent with conn.Deploy [02:25] davecheney: run that with real Amazon only [02:26] davecheney: This means we're testing stuff *for real*, cloud-init, running of commands, upstart script, and Inc. [02:26] niemeyer: ok, i'll try that [02:26] davecheney: Of course, we won't be able to run that all the time [02:26] davecheney: But for that we have the unit tests, closer to the implementations themselves [02:27] davecheney: As soon as we have stuff working like that, I'll be happy to setup an integration test somewhere that does the boring work for us [02:27] davecheney: I mean, integration test runner [02:28] davecheney: LiveTest is already an integration test [02:28] LiveSuite, that is [02:30] niemeyer: thank you for your time this evening [02:31] davecheney: Sure thing, I'm glad to have that stuff moving forward.. we're about to have something real working, and that's quite exciting [02:31] i'm going to take a break on this CL and work on the s3 location one for a little [02:31] niemeyer: it's been working for over a week [02:31] actually, no [02:31] thta is not true [02:31] once we have deploy it works [02:32] also, ... value *net.OpError = &net.OpError{Op:"remote error", Net:"", Addr:net.Addr(nil), Err:0x28} ("remote error: handshake failure") [02:32] fu amazon [02:33] Yeah, Amazon has been going through some bad times [02:34] niemeyer: changing gears [02:34] I'd like to change the s3 tests to hit a number of regions [02:34] not just USEast [02:34] is there anyting in juju-core that I can crib for reusing a test ? [02:35] davecheney: Should be easy to have a setting on the suite type on s3i_test.go, and instantiating a couple of them with different values at the Suite(&S{..}) call time [02:35] I'm thinking jujutest/tests.go [02:35] niemeyer: yes, that is what I want to do [02:35] davecheney: But if it's s3, s3i_test.go is the thing [10:04] TheMue, fwereade: Mornings! [10:05] niemeyer: Morning [10:05] niemeyer: You're up early today. [10:05] TheMue: Yeah, couldn't sleep for whatever reason [10:07] niemeyer: So used the time for a relaxed start with a good breakfast [10:08] TheMue: Yeah, exactly [10:24] niemeyer, heyhey [10:27] fwereade: Heya [10:40] fwereade, TheMue: How're things going guys? Good progress there? [10:40] niemeyer: Think so, yes. Setting up my tests. [10:40] niemeyer, not too bad, getting default-series handling right is a touch fiddly [10:41] niemeyer, had been hoping to chat to dave this morning, but missed him [10:42] fwereade: Aw [10:42] fwereade: What's the trickiness around it? [10:43] niemeyer, at the heart of it is now we initialize state [10:43] niemeyer, how [10:44] niemeyer, the current Initialize does basically nothing [10:44] niemeyer, what I would like to do is (among the other stuff) to put in a complete environ config, excluding only secrets, at bootstrap time [10:45] fwereade: That sounds great [10:45] niemeyer, but that has distressingly far-reaching consequences [10:45] fwereade: Oh? [10:45] niemeyer, I'm hoping that I will be able to see a way to break this down into 2 changes with hindsight [10:46] niemeyer, well, the whole bootstrap pipeline needs to change to accept that complete environ config [10:46] fwereade: How so? [10:46] fwereade: Bootstrap() is in the environ, and the environ knows its own config [10:47] niemeyer, cloud-init and initzk [10:47] fwereade: So on that side at least, we're good [10:47] fwereade: Right, on the other side we have to load it [10:47] fwereade: A base64 option to initzk should do the deal [10:48] niemeyer, yeah, that's basically done [10:48] niemeyer, also, we need to be able to figure out which bits are secret and which aren't [10:49] niemeyer, and ATM it feels like we can't sanely get away without some sort of distinction between a loosely-specified user config and a strict real config... but we'll see how we go there [10:50] fwereade: The only thing that knows about that is the environ [10:50] niemeyer, (I feel it is a profoundly bad thing to have the remote components inferring anything at all about their config, and they should not even accept junk like authorized-keys-path) [10:50] fwereade: Handly enough, when sending the environ itself can filter out secrets [10:50] fwereade: given that this is an implementation detail behind Bootstrap [10:51] niemeyer, sure; but I think Conn will also need to get the secrets so it update state on first access [10:52] niemeyer, none of these things are intrinsically painful, but I have yet to see how to turn it into a very clean set of CLs [10:52] fwereade: I don't think it needs to know those details [10:52] fwereade: (so far) [10:52] fwereade: It just has to send the config it has [10:52] fwereade: So we can make the operation be "send initial config if never done before" rather than "send secrets" [10:54] niemeyer, hmmmmm, so maybe we shouldn't set anything in /environment on bootstrap at all? [10:54] niemeyer, I would be a lot more comfortable if an initialized state implied valid data, even in the /environment node, eschewing only secrets [10:55] fwereade: Your idea of sending an initial configuration when we have none sounds good, but it doesn't solve the entire problem, so we can postpone it for the moment I guess [10:55] fwereade: These two options do not conflict with each other, though [10:56] fwereade: "Bootstrap with config-secrets" & "send initial config if never done before" can be friends :) [10:56] Sorry [10:56] fwereade: "Bootstrap with config secrets" & "send initial config if never done before" can be friends :) [10:56] niemeyer, consider environment-specific settings that should not change, such as the control bucket [10:57] niemeyer, hm, we'll be broken anyway if that changes, won;t be able to find the bootstrap machine anyway [10:57] * fwereade ponders [10:58] niemeyer, I guess it just feels kinda icky to ever *overwrite* env settings without checking them [10:59] niemeyer, it feels smarter to send public settings OAOO, and then to update them with secrets alone OAOO [11:00] fwereade: Okay, I see your point, agreed [11:01] fwereade: This can be solved in a somewhat clean way, I suppose, if we introduce ConfigSecrets in the Environ interface, returning []string [11:01] niemeyer, anyway, it is progressing nicely enough, and I'll know better how to split it all up cleanly before too long [11:01] niemeyer, yeah, something like that [11:02] fwereade: I'd be happy with that [11:13] Lunchtime [11:14] TheMue: Enjoy! [11:24] hey all. [11:25] Aram: Morning [11:31] Aram, heyhey [11:33] niemeyer, I just had a thought... what would happen if we made State.EnvironConfig() return an environs.EnvironConfig? allthe consequences I can currently see are positive ones [11:34] niemeyer, we wouldn't be able to casually update it as we currently do, thanks to it currently being a ConfigNode [11:34] fwereade: How do we Write it? [11:35] fwereade: Stat.SetEnvironConfig? [11:35] State [11:35] niemeyer, I had been thinking of a distinct State.UpdateConfigSettings(map[string]interface{}) [11:35] sorry UpdateEnvironConfig [11:35] fwereade: Does it replace or does it append? [11:36] fwereade: I suppose it replaces, so Set would be better.. anyway, +1 [11:36] niemeyer, cool, thanks [11:36] niemeyer, I had originally been thinking of appending, but that's by the by [12:13] fwereade: That'd mean we'd need an extra interface for removing keys, and it'd be different from the interface we have on Environ for the same purpose [12:14] niemeyer, true [12:14] niemeyer, not sure when we actually want to do that though [12:15] fwereade: Hmm.. all the time? [12:15] fwereade: I mean.. what other way would a config be updated? [12:15] fwereade: Maybe you have something else in mind that I've missed [12:15] niemeyer, I mean I don't know when we delete keys from an environ config [12:16] niemeyer, I guess I'm missing something? [12:16] fwereade: We surely are able to have missing keys, right? [12:16] fwereade: Maybe we shouldn't allow that, or should say that missing keys and a key set to "" are the same? [12:17] fwereade: Either way, the effect is the same.. Set would replace [12:17] niemeyer, IMO we shouldn't really ever have an /environment with missing keys (that aren't secrets) [12:18] niemeyer, we should do all our fuzzy guessing at defaults OAOO, when we load the config [12:18] fwereade: I don't think that's the case today, and it doesn't feel sane to force the user to have all possible keys, even those not wanted, set [12:18] niemeyer, having missing keys in the provider config has always been an irritation in the python [12:19] fwereade: In which sense? [12:19] niemeyer, self.config.get("region", DEFAULT_REGION) all over the place, occasional self.config["region"]s that blow up at unexpected times [12:19] niemeyer, it's just ugly [12:20] niemeyer, and I don't see why we shouldn't construct complete configs with real values as soon as they enter the system [12:22] fwereade: Hmm [12:22] fwereade: This is bogus indeed [12:23] niemeyer, yeah... I was somewhat upset when I saw we'd copied those flaws over [12:23] fwereade: The point, though, is that some values may be actually missing. You seem to be implying what I said above regarding having unset keys behaving as "", though [12:23] fwereade: Not all values have defaults [12:24] fwereade: In some, the "unset" value is signal [12:24] niemeyer, hmm, yes; I don't see any of those in go atm though [12:24] fwereade: That's not relevant.. the system is far from complete. We're talking about what we want to happen in those cases. [12:25] fwereade: We also can't default to "", because our config is typed [12:26] niemeyer, fair point, ok, but consider a setting that *does* have a significant effect, like ec2-uri [12:27] fwereade: Yep, good example [12:27] fwereade: What's your feeling about it? [12:28] niemeyer, that an unset ec2-uri is a really bad signal for "not really EC2, modify behaviour appropriately" :) [12:28] niemeyer, it's all very well in a user config [12:28] niemeyer, but internally I would far rather have "ec2-uri" always containing a real value, and another "this-is-not-really-ec2" value that controls the different behaviour [12:29] fwereade: I'd rather not, to be honest [12:29] fwereade: Or maybe I do.. hmm [12:29] fwereade: digesting, just a sec [12:29] niemeyer, it does feel like squishing far too much subtle meaning into a single value [12:30] fwereade: I don't understand that.. it's the endpoint we're talking to.. how's that subtle? [12:31] niemeyer, that bit is fine... but let me find a code ref [12:32] niemeyer, ok, its impact is limited, because I got annoyed and did this: [12:32] @property [12:32] def using_amazon(self): [12:32] return "ec2-uri" not in self.config [12:32] niemeyer, and even that is not heavily used [12:33] fwereade: This is wrong, actually [12:33] fwereade: Because it may be set to an endpoint in Amazon [12:33] niemeyer, it's wrong if someone sets ec2-uri explicitly [12:33] fwereade: I don't understand this particular issue [12:33] fwereade: Why!? [12:33] fwereade: It certainly wasn't the intention when I put it there [12:33] niemeyer, that is what AFAIK it has always been used for [12:34] niemeyer, I always considered it somewhat nasty [12:34] fwereade: The goal of this setting is to send the endpoint manually, as its name indicates [12:34] fwereade: I don't get the leap on how it can be something else than that [12:34] niemeyer, so, yes, it would be *even* better if we didn't have this ugly and frequently-incorrect overloading of meaning [12:35] niemeyer, I would guess that someone doing non-ec2 ec2 provider work made an unhelpful assumption [12:35] niemeyer, I stipulate that a this-is-not-amazon setting would be superior in every way [12:35] fwereade: Yeah, that sounds like the heart of the issue we've been brainstorming on [12:36] fwereade: If we need such a setting, +1 [12:36] niemeyer, but then, assuming we fix that [12:36] fwereade: Do we need such a setting? [12:36] niemeyer, possibly, and possibly not, I would have to look through a bit more carefully [12:37] niemeyer, but, ok, consider a correct ec2-uri setting [12:38] fwereade: Okay [12:39] niemeyer, how do we gain by leaving that as "", when we could just as easily infer the real value *only* when parsing user configs, and ensure that the contents of /environment are always correct and need no further substitutions? [12:39] fwereade: Because when a user questions for the configuration of their environment, I'd rather tell them what they've set [12:40] fwereade: We also lose the ability to do changes [12:40] fwereade: It's fine to adapt values we've set ourselves.. it's not fine to hack user-given values in general [12:42] niemeyer, I feel that argument might not apply all that well to omitted settings... don't they imply "figure it out for me", rather than "I wish to communicate with the fairyland EC2 which has no endpoint?" ;) [12:42] fwereade: Exactly [12:43] fwereade: What I mean is that "figure out for me" does not mean "figure out for me just when I bootstrap" [12:44] niemeyer, ok, I've caught up, I think I agree there :) [12:44] niemeyer, so there are indeed some settings that can reasonably be left unset [12:44] niemeyer, and still have a valid config [12:46] niemeyer, how does authorized-keys fit in here? [12:46] niemeyer, I'm pretty certain that should be an environment setting [12:47] niemeyer, and I'm also pretty sure that existence of authorized-keys-*path* in the /environment node indicates that we've screwed up pretty badly [12:47] fwereade: Agreed on both [12:47] fwereade: keys-path should be translated to keys as we do today [12:48] niemeyer, ok; would you agree that an authorizedKeysPath field on environConfig is an unwarranted temptation to foolishness? [12:51] * niemeyer looks [12:51] fwereade: Which environConfig? [12:52] niemeyer, ec2 [12:53] fwereade: You mean providerConfig? [12:53] niemeyer, er, sorry, yes [12:54] fwereade: np, just to make sure we're on the same page [12:54] fwereade: Do you have alternative suggestions? [12:55] niemeyer, essentially, that we have 2 paths for config parsing -- fuzzy user configs and strict internal configs [12:56] niemeyer, strict configs can have empty values as discussed above ofc, but we do need to do slightly different work when interpreting user configs and when checking for validity of internal ones [12:56] fwereade: Feels like a lot of code and work for a one-off field [12:57] niemeyer, similar considerations IMO apply to region [12:58] fwereade: Hm.. why? [12:58] fwereade: The path is special because we load from the filesystem on one side, and don't care about the other side [12:58] niemeyer, because region is an env property that should be immutable [12:58] fwereade: The region doesn't feel similar [12:58] niemeyer, the "what if the default changes" argument applies in the opposite direction [12:59] fwereade: The path being changed is irrelevant [12:59] fwereade: A region being changed can be caught up on SetConfig and refused [12:59] fwereade: Seem simple [12:59] Seems [12:59] niemeyer, I was referring to the "changing defaults" argument you made wrt ec2-uri [13:00] fwereade: Erm.. I'm even more confused now [13:00] niemeyer, if we change the ec2-uri default it is a bad thing to have baked it in from the start [13:00] niemeyer, if we change the region default it is a bad thing NOT to have baked it in from the start [13:00] fwereade: I'm not suggesting we change anything right now.. [13:01] fwereade: I was pointing out reasoning why having unset fields matters [13:01] fwereade: The region may be hardcoded yeah, because it can't be changed [13:02] niemeyer, ok, and that then implies again that if we get an /environment without a region, we have screwed up, just as we have if we get an /environment with an authorized-keys-path [13:03] niemeyer, and hence is IMO support for the idea that user configs and internal configs are different enough that we want separate paths for interpreting them [13:04] fwereade: Sorry, I don't see the relation between these two points. [13:04] niemeyer, both should translate to the same type [13:04] if region == "" { region = "us-east-1" } [13:04] fwereade: Why are we discussing this? :) [13:05] fwereade: The path case is special.. the region case is trivial [13:07] niemeyer, because I think that interpreting an /environment config in exactly the same way as one from environments.yaml is sloppy and risks bugginess [13:08] fwereade: I've been trying to demonstrate that this is not the case.. preventing a region from changing is if oldRegion != newRegion && oldRegion was set { return error } [13:08] fwereade: We don't need a new code path for these three lines [13:08] fwereade: Do I misunderstand? [13:09] niemeyer, since you're talking about changing regions at runtime, I think so [13:09] fwereade: Okay, so please bear with me while I adapt to reality [13:10] niemeyer, earlier you pointed out that some settings should remain empty, and not be baked in at parse time, on the basis that we may wish to change the defaults in future and having them baked into the env config would therefore be bad [13:10] niemeyer, agreed? [13:10] fwereade: Can we please focus on the specific issue of region being changed? [13:10] fwereade: Otherwise we'll derail I think [13:11] niemeyer, this is directly relevant to region changes [13:11] fwereade: Yeah, so how is region changed difficult to do with current infra? [13:11] niemeyer, do you agree with how I characterised your position above? [13:11] fwereade: THat's the core point [13:13] fwereade: Okay, I guess you do want to review the points made? [13:13] niemeyer, I am laser-focussed right at the moment, because I think it will help our communication [13:14] fwereade: Ok [13:14] niemeyer, do you agree with my statement that " earlier you pointed out that some settings should remain empty, and not be baked in at parse time, on the basis that we may wish to change the defaults in future and having them baked into the env config would therefore be bad" [13:14] fwereade: More precisely, [13:15] fwereade: So the point I made is that having unset values allows us to change the values *when that's wanted*, and I also said that it makes sense to allow values to be unset because some settings have unset values naturally, and we can't have a value like "" for unset because we have typed values in the configuration, and also that we want to show users the configuration that they've made [13:18] niemeyer, maybe we should skip sideways a little -- what is the distinction between a "naturally unset value" and an unset value that indicates that some default substitution should be made? [13:18] niemeyer, can you give me an example of the first kind? [13:19] fwereade: A naturally value is any value that is unset, and should remain as so because we want to know it's unset [13:19] niemeyer, but I can think of two cases where failing to bake them in will kill environments (even if one of them is historical) [13:20] niemeyer, that is, will kill environments if we ever change the default settings [13:20] fwereade: I don't understand.. suppose we have a new "environment-description" field [13:21] fwereade: By default we call it "No description for this environment." [13:21] fwereade: We've made a typo.. now we want to fix it [13:21] fwereade: Bingo.. we change the code so that the unset value now returns the proper description. [13:22] niemeyer, ok, that is an example where there are distinct benefits to deferring the substitution rather than doing so at parse time [13:22] fwereade: If the user has put his own description, though, we don't want to touch that. [13:22] niemeyer, I am aware of these examples [13:22] niemeyer, you already explained it perfectly wrt ec2-uri [13:22] fwereade: Yeah.. so can we go back to region? [13:22] niemeyer, absolutely! what happens when we change the default ec2 region and it has been left unset? [13:22] fwereade: We don't change it [13:23] niemeyer, never ever? [13:24] fwereade: How can we possibly change the region of a running environment? [13:24] niemeyer, well, yeah, that would be crackful [13:24] niemeyer, how can we possibly predict thatus-east-1 will be the appropriate default region for ever and ever? [13:25] fwereade: We can't, we shouldn't, and as far as we know, we don't.. !? [13:25] niemeyer, well, if we allow region to remain unset, how can we prevent a default region change from causing a *runtime* region change when that env is upgraded? [13:26] fwereade: Why don't we set the region? [13:26] Jul 05 10:04:14 if region == "" { region = "us-east-1" } [13:27] niemeyer, well, we do; but that's a setting that the user did not make [13:27] niemeyer, you seemed to be arguing that we should not do that [13:27] fwereade: So the point I made is that having unset values allows us to change the values *when that's wanted* [13:28] niemeyer, agreed and understood [13:28] Jul 05 10:04:14 if region == "" { region = "us-east-1" } [13:28] fwereade: I said that :) [13:28] niemeyer, you *also* argued, and reiterated, that we should not show the user settings that they did not set [13:28] fwereade: True [13:29] niemeyer, these positions are in conflict [13:29] fwereade: The region case is fine, though [13:29] fwereade: and I've been saying that for a while [13:29] Jul 05 10:01:50 fwereade: The region may be hardcoded yeah, because it can't be changed [13:30] fwereade: That said, there's a bit missing it seems, which is why I was trying to focus on how we do that in practice and avoid side-tracking [13:31] niemeyer, I have shown you 2 ways in which a user config is a stupid /environment config -- authorized-keys-path and region -- and I can name more (acquired-mgmt-class and available-mgmt-class, off the top of my head) [13:31] fwereade: It doesn't look like there's a place for us where we get an Environ to intercept a configuration that is about to be put in SetConfig, in a way that it has both the old and the new config [13:32] fwereade: Sorry, that path has proven unhelpful already.. I've been repeatedly pointing out that region is easy to solve, and you've failed to provide any facts for why it's not [13:32] fwereade: If there's a reason why region is hard to solve, please tell me as I do want to understand [13:32] fwereade: But focus on it.. no backtracking [13:33] niemeyer, region is not hard to solve in the small [13:34] niemeyer, the fact is that certain values, one of which is region, can be safely omitted from environments.yaml but should not be omitted in /environment [13:34] niemeyer, and that certain values that can be set in environments.yaml *should* be omitted in /environment [13:36] niemeyer, I am arguing that there is a meaningful distinction between parsing the two environment configuration styles, and that using the same code to do so is going to bite us one day [13:37] fwereade: I find much more likely that duplicating the same configuration parsing with different semantics will bite us. [13:37] fwereade: If we have problems, let's address them [13:38]