[00:16] mgz_: thanks for your help [00:16] this is working a treat [00:16] davecheney: ace [00:16] a few small issues and i'll be able to land this branch [00:16] thumper: a pint of grog for mgz_ ! [00:21] awesome [00:43] hmm.... [00:43] I'm seeing state/leadership package sometimes timeout after 20 minutes [00:43] * thumper smells race [00:43] but my --race doesn't work [00:59] it's totally possible to have logical races without memory races [01:03] sure [01:57] thumper: why discard that PR about server->controller? is that rename not happening anymore? [01:57] axw: it is, but I'm going to retarget it against the new 'controller-rename' feature branch [01:57] as I feel that I'm going to break CI [01:57] thumper: okey dokey [01:57] especially as I change some of the command stuff [01:58] and remove the feature falg [01:58] so I thought I retarget it all [01:58] hopefully a very short lived feature branch [01:58] as conflicts against master are going to be a biatch [01:59] thumper: did you see my latest email about multi-env vs. azure? [02:00] I think so [02:00] * thumper goes to look again [02:01] thumper: did not get a reply. just curious about how we're supposed to do create/destroy when an env has resources shared between machines [02:01] setup/teardown, a la bootstrap/destroy [02:01] like what? [02:01] thumper: in the case of azure: a resource group, subnet, network security group, storage account [02:02] thumper: each env will have its own everything, except for a single virtual network that all envs are connected to [02:02] ok... [02:03] well, shouldn't we just hook into the create environment, and destroy environment endpoints? [02:03] when we destroy a controller, we take down all the hosted environments first [02:03] axw: waigani has been refactoring all our destruction code so it is nicer and more async [02:03] thumper: yeah, but atm EnvironProvider just has "PrepareForCreateEnvironment" - I'm using htis but it doesn't feel quite right [02:03] thumper: sounds good [02:04] you should use the create environment, not the prepare [02:05] thumper: there is no "create environment" in EnvironProvider, at least not on master. maybe in a feature branch? [02:05] where are you looking? [02:05] hmm... [02:05] thumper: https://github.com/juju/juju/blob/master/environs/interface.go [02:05] * thumper thinks... [02:05] * thumper looks [02:07] axw: actually, there is currently no function we call at all in the provider when creating a new environment [02:07] before there was never anything to execute [02:09] hmm... [02:09] thumper: so I think we need a CreateEnvironment as well as a PrepareForCreateEnvironment [02:09] I think you may be right [02:09] * thumper greps for the Prepare... [02:09] thumper: PrepareForCreateEnvironment is called (twice), and it is usable [02:09] hmmm [02:10] yeah, checking for valid Config [02:10] thumper: i.e. I've got multi-env working in azure [02:11] axw: it feels better to have a separate CreateEnvironment call [02:11] that we call just once [02:11] thumper: +1 [02:12] thumper: also, it's not obvious why Prepare is called twice (once before assigning UUID, once after) - would be good to have better docs on that [02:12] I believe it is the two step call we do [02:12] which we are looking to fix [02:12] the first is to ask for a config skeleton [02:13] and then the validation when the user passes it back [02:13] but to be honest, I'm not entirely clear [02:13] we are wanting to make the juju cli do the same as jem [02:13] and have all providers supply the configschema [02:13] oh... [02:13] can you make the new azure provider supply the configschema? [02:14] not sure if the old one did [02:14] thumper: it didn't. I'll add it to the list. [02:37] axw: here is that branch retargeted to the controller-rename branch http://reviews.vapour.ws/r/3053/ [02:54] axw: small one if you have a moment https://github.com/juju/charmrepo/pull/36 === natefinch-afk is now known as natefinch [03:44] thumper: http://reviews.vapour.ws/r/3054/ [03:44] thumper: adds "no tail" support to state.LogTailer [03:44] * thumper looks [03:48] menn0: why do we need to call tailer.Stop when we said "don't tail" ? [03:50] thumper: you don't ... [03:51] thumper: it's just in case the tailer isn't working during tests [03:51] thumper: like when I wrote the test but NoTail wasn't implemented yet [03:51] thumper: happy to reemove it if you think it's confusing [03:51] perhaps just a comment would be enough [03:52] * thumper sighs [03:52] state/leadership is failing for me again... [04:01] thumper: i've responded to the review [04:06] wallyworld: reviewed [04:06] ty === Ursinha-afk_ is now known as Ursinha === Ursinha is now known as Guest82331 === psivaa_ is now known as psivaa === Guest82331 is now known as Ursinha === Ursinha is now known as ursula1234 === ursula1234 is now known as Ursinha [04:22] waigani: http://reviews.vapour.ws/r/3056/ [04:22] waigani: it is the earlier branch retargetted and many of the missed systems renamed too [04:30] Bug #1454466 changed: Deployment times out waiting for relation convergence - nvp-transport-node in installing state [04:30] [04:33] Bug #1454466 opened: Deployment times out waiting for relation convergence - nvp-transport-node in installing state [04:33] [04:36] Bug #1454466 changed: Deployment times out waiting for relation convergence - nvp-transport-node in installing state [04:36] [04:39] thumper: shipit with a question about "controller environment" [04:59] thumper: menn0 axw http://reviews.vapour.ws/r/3057/ [05:01] davecheney: shipit [05:02] axw: that was awfully fast! [05:02] boom [05:02] menn0: awfully easy review :) [05:02] -2700 lines -> ship it! [05:02] heh [05:02] I think axw has a bot that just automatically replies with ship it [05:02] ;) [05:03] in seriousness, I wish we had one for back/forwardports [05:04] or a way to just not post to RB [05:05] axw: sorry, one more https://github.com/juju/charmrepo/pull/37 [05:08] wallyworld: you should probably use yaml.v1, it's in dependencies.tsv [05:09] axw: ah, yeah, thanks === bodie__ is now known as bodie_ [08:11] ericsnow: ping? [09:00] axw, I think you're probably right re manifold-in-different-package, but I'm really reluctant for some reason [09:01] axw, possibly because so many others are not yet sufficiently separate from their worker packages as to cleanly accommodate that? [10:02] dimitern, dooferlad: standup? [11:10] fwereade_: sorry, I was away from IRC. I'm of two minds. On the one hand, it's obviously inherently tied to the worker, but on the other, it has a separate role to play (policy over starting the worker, and connecting the worker to other things). I think having it in a separate package just keeps the responsibilities clearer [11:31] axw, yeah, I think you're right. subpackage is probably not *really* the right place, but close enough for now [12:16] perrito666: thanks [12:18] voidspace: yw [12:18] * perrito666 overslept, rainy day, fresh drapes, impossible not to [12:21] cherylj: hi, could you please take a look at http://reviews.vapour.ws/r/3063/ when you have time? [13:26] fwereade_: I realized late that alexisb moved our meeting time (relative to my tz at least). I need to take the dog out, but will try to be back for the meeting === niedbalski_ is now known as niedbalski [13:45] jam, we dropped but I am here if you want to chat post dog walk [13:45] Hello, I am trying to build the packages from source, how can i download the dependency packages individually instead using the makefile ? [13:47] alexisb: hiya. I don't have anything specific. You dropped the meeting for today? [13:47] jam, fwereade_ and I met for a moment and then dropped [13:50] alexisb: you realize you really have nothing to say to eachother without me ? :) [13:51] jam, so very true ;) [13:58] frankban: did you have any write up on using the new bundle support that can go out into the release notes for 1.26-alpha1? [14:11] rogpeppe, do you recall why EnvironObserver has a mutex, and replaces its environ, instead of doing a SetConfig? environs are meant to be goroutine-safe... [14:12] rogpeppe, (I know it's from, what, 20 months ago :)) [14:19] cherylj: no I don't but I can get it prepared [14:19] frankban: that would be great. Just enough to tell people it's there and how they can use it. [14:21] cherylj: cool, Makyo ^^^ would you like to prepare something about bundle deployment in core? [14:22] cherylj: will you have time to look at the branch today? [14:25] fwereade_: i'll have a look [14:25] rogpeppe, cheers [14:28] fwereade_: even if an Environ itself is goroutine-safe, a worker still doesn't want it changing underneath itself with no warning [14:32] frankban: yes, I can. I see you have two ship it reviews already. Did you still want me to take a look? [14:33] cherylj: yes please, I need a review from a core developer [14:33] frankban: ah, ok. Will look at it this morning [14:34] cherylj: ty! [14:41] rogpeppe, sorry, why not? [14:42] rogpeppe, isn't that the point of it being goroutine-safe, that someone updating the config should be no cause for concern? [14:42] bother [14:47] rogpeppe, sorry, bad timing before [14:48] fwereade_: np [14:48] rogpeppe, what's the problem with the environ config being updated under a worker? I rather thought that was the point of the boroutine-safety? [14:49] fwereade_: if i'm writing a worker, i might be doing something on the basis of one attribute, but another attribute might change underfoot so it's inconsistent [14:49] fwereade_: i think it's better if the config is treated as immutable [14:50] Bug #1513084 opened: 1.20 cannot upgrade to 1.26-alpha1: run.socket: no such file or directory <1.20> [14:50] rogpeppe, that sounds like a problem with the Config method alone then? [14:51] rogpeppe, (also: if a worker's depending on env config values from the env, not from state, isn't it already confused?) [14:53] fwereade_: if that's an issue, why are we creating the env at all? [14:53] rogpeppe, to call methods that do something to the substrate [14:53] fwereade_: ok, so those methods will depend on env config values in the env, right? [14:54] rogpeppe, yes, but that's the env's problem, not the client's [14:54] fwereade_: so should the worker rely on two sources of truth for the config values from the state - the env and another watcher? [14:56] fwereade_: currently a worker needs only to use the EnvironObserver, which gives it the config values from state (and an environment that can use them) [14:56] Bug #1513096 opened: invalid agent version in environment configuration: "1.26-alpha1" [14:57] rogpeppe, what's the overlap between worker-relevant values and env-relevant values? [14:57] fwereade_: i don't know. but i wouldn't want to rely on there being none ever [14:58] fwereade_: having a single place to watch seems like a reasonable thing to me [14:58] rogpeppe, in practice, what the EnvironObserver is used for is to create an environ, hand it over to instancepoller, and never update it [14:58] rogpeppe, the instancepoller doesn't remotely care about what wwe're doing in the background, it just wants its instancegetter to work [14:59] fwereade_: i'm slightly surpised the provisioner worker doesn't use it [14:59] rogpeppe, what would the point be? if it doesn't update the environ we still have to thread env-watching through every worker [15:00] rogpeppe, if it simplified its clients, then it'd be great, but as it is there's no reason to switch implementations to use that afaics [15:03] fwereade_: presumably the provision *does* update its Environ when the environ config changes? [15:03] fwereade_: ah yes, it does [15:05] rogpeppe, yeah -- and that one is also sl. different in that it has an interest in actual juju-model config as opposed to just the substrate config [15:05] rogpeppe, but I'm pretty sure that if it's paying attention to the substrate config it's DIW somewhere [15:06] fwereade_: "dead in water" "doing it wrong" ? [15:06] rogpeppe, the latter was my intent :) [15:06] fwereade_: it feels a bit racy to me [15:08] rogpeppe, only if the environ itself is, surely? [15:08] fwereade_: not really. [15:08] fwereade_: i don't like state changing underfoot [15:09] fwereade_: i mean, it's *probably* ok [15:10] rogpeppe, I feel like the biggest worry is that few environs have ever been actually used from multiple goroutines, so their safety is not battle-tested [15:11] fwereade_: there should be at least one cross-environment test that's designed to trigger the race detector in such cases [15:13] rogpeppe, sure, but the number of ways to subtly break safety tends to overwhelm any but exhaustive/invasive testing, and I don't think we really have anything sophisticated enough [15:13] fwereade_: ah, i was being confused by the configObserver - it's only there for testing [15:13] fwereade_: if a provider doesn't simply use a mutex, it's probably doing it wrong [15:14] fwereade_: and that simple mutex logic can be easily tested with the race detector [15:14] rogpeppe, there's another way to approach it that I quite liked, but forget the details of [15:14] fwereade_: i tend to just make a test like: go setConfig(); go getConfig(); go setConfig(); go getConfig() [15:15] fwereade_: which will trigger a failure in the race detector if the appropriate mutexes aren't used [15:15] rogpeppe, right, but that's just 2 methods and still rather luck-dependent [15:16] fwereade_: true that it's only 2 methods (it should probably be all of 'em), but i don't think it's that luck dependent [15:16] rogpeppe, I accept it'll probably fail most of the time, but I doubt it'll be 100% :) [15:16] fwereade_: why wouldn't it be? [15:18] rogpeppe, it might happen to run them all serially by sheer coincidence, and never happen to flag the issue -- is it smarter than that? I thought it erred in fvaour of false negatives [15:18] fwereade_: even if it runs them all serially, the race detector will flag it [15:19] * fwereade_ scrubs that bad data out of his brain then [15:19] fwereade_: of course you can get false negatives if the code never actually accesses the memory in question [15:19] rogpeppe, cool [15:19] fwereade_: it's a pretty good tool [15:19] rogpeppe, yeah, but that's not the case here [15:19] fwereade_: i'm a big fan of race-detector-oriented tests [15:20] rogpeppe, indeed, I have a deep and abiding love for the race detector [15:20] Bug #1511822 changed: imports github.com/juju/juju/workload/api/internal/client: use of internal package not allowed [15:20] Bug #1513096 changed: invalid agent version in environment configuration: "1.26-alpha1" [15:20] rogpeppe, ok, so, some go-spam-every-method tests would be good in general [15:26] fwereade_: so presumably your motivation for the initial question was wondering why the instance poller aggregator isn't seeing env config changes? [15:27] rogpeppe, not so much, actually, I came across that later -- I was more thinking of running a separate EnvironObserver that *could* be shared by the various workers that use environs [15:28] fwereade_: well, the current one can be in fact [15:29] fwereade_: you could do the SetConfig locally in each worker as desired [15:29] rogpeppe, don't think so? there's only one Environ in the observer [15:29] fwereade_: each worker would have its own Environ [15:30] rogpeppe, that's what I'm trying to get away from [15:30] fwereade_: for reasons of efficiency? [15:31] rogpeppe, more that the env-watching is unnecessary extra complexity in a bunch of different workers [15:31] rogpeppe, and yeah, there's some efficiency improvement as well [15:32] rogpeppe, and it opens the door to us being able to actually manager rate limits sanely, instead of having a herd of N environs stampeding to talk to the same endpoint from a single agent [15:32] fwereade_: the difficulty is that it's all bound up with environ config and that smears a bunch of semi-related attributes together [15:34] fwereade_: i'm toying with the thought that workers shouldn't ever look at the environ config directly at all [15:34] rogpeppe, I do agree that env config is a mess; but AFAICS the potential for harm comes from people using the environ config -- ha, yeah [15:34] rogpeppe, can you think of any legitimate uses for .Config() for a client? :) [15:34] fwereade_: but instead have the parameters passed in and derived from the config [15:34] rogpeppe, yeah, absolutely [15:35] fwereade_: then you'd probably need to restart the worker when config relevant to it changed [15:35] rogpeppe, we just want to pass in something that knows how to ask for instances, and which isn't all whiny and needy about being updated by the same client who wants instances [15:35] rogpeppe, still not really understanding why not to SetConfig and be done with it? [15:36] rogpeppe, in the observer, I mean [15:36] fwereade_: SetConfig and use chan struct{} ? [15:36] rogpeppe, no, I don't want the clients to have to care about environ config at all [15:37] fwereade_: that's ok iff there's no overlap between the attrs used by the worker vs those used by the provider [15:38] fwereade_: i wonder what it would take to split 'em [15:38] rogpeppe, well, at least in the instancepoller case, it doesn't care about env config at all [15:38] fwereade_: i'm more thinking of the provider case actually [15:38] rogpeppe, in the provisioner case all I can think of is, uh, instance-reaping mode, whatever it's called [15:38] sorry [15:38] provisioner [15:38] fwereade_: yeah [15:39] rogpeppe, which the provisioner shouldn't really know is to do with the env config anyway, it shoudl just be watching/asking for reaping mode [15:43] fwereade_: one way to be more sure that the workers aren't doing anything untoward might be to make the Environ with Config and SetConfig methods that return an error or panic [15:52] rogpeppe, mm, yeah, I will poke around in that direction and see what happens [15:53] fwereade_: that wouldn't be compatible with logic doing type inspection of the Environ, but i would hope that nothing is doing that anyway (and i've always considered it seriously misguided in the places we *are* doing it) [15:53] s/hope that nothing/hope that nothing in the workers/ [15:55] rogpeppe, yeah [16:04] gc.ErrorMatches is horrible and bad, and anyone who uses it should feel bad. [16:05] natefinch, gtg out, would still be interested to hear more on this -- I'd say that *overbroad* ErrorMatches~s are the usual problem? [16:06] fwereade_: the only time the *text* of an error message matters, is if it is shown to a user... which 99% of the time it is not [16:06] fwereade_: we use it as a shorthand for "I expect this specific error to be returned" [16:07] fwereade_: which is not what is actually being tested [17:11] natefinch: i disagree. i think it's really helpful to see what our errors look like [17:13] natefinch: the alternative is often just asserting that the error is non-nil, which isn't very helpful (i have discovered lots of places where we generated a really stupid or malformed error which we never knew about because of that) [17:15] rogpeppe: the problem is that asserting the error message asserts a ton of information you don't actually care about. Like how certain types serialize into strings, like how many levels of abstraction there are between you and the origin of the error [17:16] natefinch: yes, i agree. that can be problematic too. but i think the upside of seeing what your error messages actually look like outweighs that. [17:16] natefinch: i often finish an ErrorMatches string with .* [17:17] natefinch: i've found the error string thing really helpful recently - i found one error that was mentioning the same URL 3 times in the same error message. [17:18] natefinch: crafting decent error messages is important for the user experience [17:18] natefinch: and the tests are usually the only place *we* will ever see them [17:19] natefinch: if it hadn't been for ErrorMatches, we wouldn't have made the recent fix to the charm error messages ("charm or bundle not found" vs "entity not found") caught when ian was reviewing our juju-core changes in response to a charm dep update [17:21] rogpeppe: sure, there are times when we show errors to users, and in those cases, error text matters.. but we use ErrorMatches *everywhere* in places we know the user will never see the message (logs don't really count). [17:22] natefinch: logs definitely count [17:22] rogpeppe: not down to the letter [17:22] natefinch: so what would you do? just check that the error isn't nil? [17:23] rogpeppe: no, check what you actually care about, that we detected a specific type of error and failed in an expected way. Use a custom error type or something. [17:23] natefinch: another issue i found recently was where an API test wasn't actually testing what it thought it was - the error was for an entirely different reason. [17:24] natefinch: i don't think we should need a custom error type or value for every kind of error we want to test for [17:24] rogpeppe: the thing that brought this up for me was that a test started failing on master because I added a level of indirection, so the error message had an extra section of "while you were doing foo:" [17:24] natefinch: yes, that's definitely the down side [17:24] natefinch: it does make for more churm [17:24] natefinch: but i still think it's worth it [17:25] natefinch: (and i have thought about this quite a bit when making similar changes) [17:25] rogpeppe: and the problem is that no one is actually thinking about what the error message says or means. For example, to fix that test, I just copied what the new text is from the error and pasted it into the regex. That's not really making our code or tests better. [17:26] rogpeppe: imagine if we decided to change how tags print out, so now they print out as "tag: foo/0" ... we'd have thousands of broken tests. [17:26] natefinch: when you did that, i hope you asked yourself the question "does this error accurately and reasonably represent the error that's being tested for?" [17:27] natefinch: yes we would [17:27] natefinch: but if we changed the representation of tag.String we'd have lots of other issues too [17:29] rogpeppe: but there would be 1000 failing tests that are failing because they're testing something they don't actually care about. certainly there are places that care what tag's String() function outputs... but that's not the only tests that would fial. [17:29] natefinch: yes, it's a tradeoff [17:30] natefinch: but i don't see a better alternative [17:30] rogpeppe: checking for specific error types (a la os.IsNotExist and errors.IsNotFound) [17:31] ericsnow: so every different error message should have its own error type? [17:31] rogpeppe: no, but they shouldn't all be effectively bare strings either [17:31] rogpeppe: arguably one for each equivalence class of error [17:32] ericsnow: how do you decide that? [17:32] rogpeppe: through bikeshedding [17:32] ericsnow: and adding hundreds of different error types is a serious maintenance burden [17:33] rogpeppe: unlike hard-coding checks for error strings ;) [17:33] ericsnow: at least that's only in the tests and easily changed [17:33] ericsnow: the problem with having error types is that they're a burden on every developer and need to be maintained as part of the API [17:34] ericsnow: but often a given error might only have meaning within the context of a given implementation [17:34] rogpeppe: I'd argue that the errors you get *are* part of the API [17:34] ericsnow: i agree totally [17:34] ericsnow: and i think that any API should think very hard about the set of error types it exposes [17:34] rogpeppe: definitely [17:35] ericsnow: which doesn't mesh well with tests that want to test internal functionality [17:36] rogpeppe: agreed, though often having to test internal functionality is a code smell [17:37] rogpeppe: regardless, as you said there are tradeoffs [17:37] ericsnow: if you're aiming for high coverage, you have to do that in practice [17:37] ericsnow: (and i think aiming for high test coverage is worthwhile) [17:37] rogpeppe: I'll leave it to katco to expound the higher plane that is dependency injection :) [17:38] rogpeppe: and I couldn't agree more about the importance of high test coverage [17:38] quick, look over there! [17:38] * rogpeppe looks at the pretty birdy [17:38] I'm with roger on internal tests. I think they greatly increase the confidence you can have in your tests. They make writing tests immensely faster, and they prevent you from mangling your exported API just to support testing. [17:39] does anyone remember what the environmental var is for tabular status? [17:41] katco: maybe JUJU_CLI_VERSION=2 ? [17:42] katco: (haven't tried it, just scanned the code) [17:42] that works [17:42] rogpeppe: ty [17:42] katco: np === ericsnow is now known as ericsnow_afk [18:08] Bug #1513165 opened: Containers registered with MAAS use wrong name [18:17] Bug #1513165 changed: Containers registered with MAAS use wrong name [18:19] katco: got a sec? bug 1512399 [18:19] Bug #1512399: ERROR environment destruction failed: destroying storage: listing volumes: Get https://x.x.x.x:8776/v2//volumes/detail: local error: record overflow [18:20] why does the autogenerated cinder stuff have "https://cinder.example.com" in it at all? can't it just have the path? [18:20] Bug #1513165 opened: Containers registered with MAAS use wrong name [18:20] issue is the calling code overrides the host, but not the scheme [18:21] so, it breaks if the endpoint is not https [18:23] Bug #1513165 changed: Containers registered with MAAS use wrong name [18:26] Bug #1513165 opened: Containers registered with MAAS use wrong name [18:32] katco: "Pull request successfully merged and closed. You’re all set—the natefinch:assign-worker branch can be safely deleted." [18:41] mgz_, cherylj has been looking at lp 1512399 [18:41] alexisb: it's easy to fix, it's just katco's autogenration code needs changing to take an endpoint rather than do this weird overriding [18:42] wanted to check with her before hacking it [18:42] mgz_, ack [18:42] I will stay out of it, thanks [18:42] alexisb: no worries [18:46] mgz_: natefinch: sorry was eating lunch [18:46] natefinch: grats... feel like we should throw a party or something :p [18:46] katco: right? :) [18:46] mgz_: i believe axw has already modified the auto-generated code, so further modification is probably ok [18:47] ...that's more like not okay then, unless he pull requested back to your repo? [18:47] oh, or you mean he modified the generated file? [18:48] mgz_: yes i think so [18:48] yeah, seems like that... mehp, that make updating more annoying [18:50] looks trivial [18:50] mgz_: fwiw he asked first. we were both busy and i think the cinder project made some modifications that broke auto-generation w/o looking into it [18:50] beautiful, this is what mongorestore says when file not found: don't know what to do with file [18:50] mgz_: but yes, because of the layering approach, it should be very trivial :) [18:50] katco: anyway, the code just needs to build urls in a sane way [18:51] mgz_: should just have to modify protocol here: https://github.com/go-goose/goose/blob/v1/cinder/client.go#L25-L33 [18:51] an endpoint is not just a host, it's a scheme+host+path that you then append a path to [18:51] mgz_: still, just need to modify that 1 function [18:52] mgz_: and that's not even auto-generated code. that's goose [18:52] katco: just + req.URL.Scheme = endpoint.Scheme in SetEndpointFn works for the bug as reported, but that code is still wrong [18:52] mgz_: modify the path in the same place [18:52] the auto gen code should append paths to the endpoint [18:53] and having hardcodes fake urls is ugly. [19:07] katco, ericsnow_afk, wwitzel3: the tech debt bug to fix the tests: perrito666: nice error message [19:07] lol wrong copy [19:07] http://reviews.vapour.ws/r/3068/ [19:13] wwitzel3: why does the package_test.go file need a +build go1.3? [19:13] the card said it did? [19:14] I foobar'd my go install trying to install 1.2 so I could test the build flags [19:15] so I'm playing with that right now, when I threw it at the bot through, it still didn't work, so I missed some [19:15] natefinch: feel free to give it a try if you have 1.2 [19:15] wwitzel3: install go from source, it's easy and easy to switch versions [19:16] (switching versions is just git checkout go1.2 and then make.bash) [19:17] * natefinch just switched from 1.4 to 1.2 in like 10 seconds) [19:21] wwitzel3: every file that imports github.com/lxc/lxd is going to have to be +build go1.3 ... and every file that imports anything those files [19:24] natefinch: so if the file that imports lxd is go1.3, all the files in the lxd package still have to have it? [19:24] ahahahahahahahahahaha .... [19:24] oh shit [19:25] wwitzel3: go 1.2 doesn't have a "go1.3" build tag implemented [19:25] waity wait... no, that should still work [19:25] nevermind, sorry === ericsnow_afk is now known as ericsnow [19:25] screwing myself up [19:26] wwitzel3: if you do go install ./... it'll independently try to build that package regardless of whether anything else imports it. [19:32] wwitzel3: it's looking like probably all of the provider/lxd files will need // +build go1.3 [19:32] great [19:33] wwitzel3: I have most of the changes on my local machine... haven't run tests which will also need it. [19:35] > Directory: /home/nate/src/github.com/juju/juju/provider/lxd [19:35] > Command: /home/nate/go/bin/go test ./... [19:35] > Output: [19:35] ? github.com/juju/juju/provider/lxd [no test files] [19:35] > Elapsed: 0.131s [19:35] > Result: Success [19:35] Success! [19:35] cherylj: PTAL: http://reviews.vapour.ws/r/3067/ [19:36] wwitzel3: running a full compile of test code now, but I think I got everything [19:37] hmm..... [19:37] I bet just having github.com/lxc/lxd in dependencies.tsv is going to be a non-starter [19:37] katco: ^ [19:48] wwitzel3: I'm getting some wacky compile errors in environs/bootstrap [19:48] a bunch of these: [19:48] ./bootstrap_test.go:79: cannot use env (type *bootstrapEnviron) as type environs.Environ in function argument: [19:48] *bootstrapEnviron does not implement environs.Environ (wrong type for Bootstrap method) [19:48] have Bootstrap(environs.BootstrapContext, environs.BootstrapParams) (string, string, environs.BootstrapFinalizer, error) [19:48] want Bootstrap(environs.BootstrapContext, environs.BootstrapParams) (*environs.BootstrapResult, error) [19:49] ericsnow: can you give me a quick explanation of what the bug was about? [19:49] cherylj: Go 1.5 disallows packages named "internal" from being used outside the package tree they are in [19:50] oic [19:50] it's a feature! ... just not one we intended to be using ;) [19:56] wwitzel3: I pushed a bunch of my changes here : https://github.com/natefinch/juju/tree/lxd-provider-flags [20:04] ericsnow: I thought I should share the hapiness with you http://reports.vapour.ws/releases/3261 [20:04] perrito666: nice! :) [20:04] old restore is no more [20:04] and CI is now using the new one which is faster [20:05] yay [20:05] and, works flawlessly on HA [20:19] can someone please review this to unblock me [20:19] https://github.com/juju/utils/pull/170 [20:19] it's fairly uncontraversial [20:21] davecheney: well, I disagree [20:21] but fine [20:22] the reason it's stalled is because this bullshit change-it-everywhere-or-nowhere problem [20:22] there's no reason that our deps shouldn't change apart from people want to keep lock-step on them [20:22] which is then... why are they even seperate github projects [20:23] and this is not going to be different when the big-bang v2 change has happened [20:24] people wanting to get a new fix in utils for a stable branch are going to cope [20:24] there's no utils 1.24 or 1.25 at present [20:27] mgz_: yaml.v1/2 is only renferenced from one file in juju/utils [20:27] called trivial.go [20:27] which contains one function [20:27] WriteYAML [20:27] i'm going to move that function back into juju/juju where it is used [20:27] and end this madness [20:27] and yes, i agree taht this upgrade everything at once thing is bullshit [20:27] but the project is called utils! lets just stuff everything in there :) [20:28] i hold that as prima facie evidence that putting version numbers in your import paths is a mistake [20:28] i agree, creating a package called utils is bad enough [20:28] then a repository called utils is asking for it [20:28] at atlassian to try to counteract this we created a jar called 'bucket' [20:29] so people would feel embaressed about using classes from it [20:29] it didn't work, we ended up with bucket2 [20:29] davecheney: the problem is not version numbers in your import paths, its that we don't care about backwards compatibility in those branches, and we use godeps in addition for some BS busywork on top of it just for kicks [20:30] objection! [20:30] this is an unrelated argument [20:30] yaml.v1 and yaml.v2 _ARE_ different [20:30] it says so right in their name [20:30] davecheney: yes [20:30] the problem is one we have created for oursleves where we _WANT_ to think of them as the same [20:31] so there is no argument about backward compatability [20:31] if you want them to be backwards compativle, they'd have the same major version number [20:31] according to the rules of gopkg.in [20:31] davecheney: sorry, I was thinking more of our own repos, charm.v5 etc [20:31] don't even get me started on v6-unstable [20:32] lol [20:32] yes [20:33] and yes, i have no idea why we use godeps _and_ gopkg.in versioned import paths [20:33] the fact that we have to do both says that neither of those alone is a workable solutin [20:34] davecheney: honestly, the problem s till seems to be godeps. that code doesn't expose yaml objects through its api, and I presume the yaml output is valid whether it's v1 or v2 of goyaml. [20:34] katco: I'm going to murder the autogen-ness of this cinder stuff. The current upstream wadl doesn't make working code, and if we're not syncing we're not gaining anything for the pain of using this. [20:34] mgz_: that's fine [20:34] davecheney: so why do we even care if some helper function uses yaml.v2 instead of v1? [20:34] natefinch: beacuse we have tests in juju that look for a precise string match [20:34] katco: I pr'd some changes I needed to test, diff I get is http://paste.ubuntu.com/13105359 [20:35] v2 changes the string form, and handling of empty keys [20:35] davecheney: well I think I know where the problem lies :/ [20:36] you get $0 for pointing out that our tests are fragile [20:36] aww [20:36] * natefinch was ranting about gc.ErrorMatches earlier [20:36] natefinch: the problem is not godeps, we have lots of code that depends on how json is handled, and it's part of our api stability contract [20:37] which we're already not good on. [20:38] mgz_: ....and there were tests that were failing because we changed how we handled stuff in this function....? [20:39] natefinch: in this case, how yaml is rendered to the client [20:39] not in dave's current case, that's just an error message change, but the charm stuff is scary [20:40] there is also that yaml.v2 changes the handling of fields/map keys with empty values [20:41] wow, worker/meterstatus/state.go: [20:41] 47: return errors.Trace(utils.WriteYaml(f.path, st)) [20:42] oh [20:42] sorry [20:42] that writeyaml doesn't return yaml [20:42] it returns an error if it couldn't write [20:42] eheh [20:42] review please: http://reviews.vapour.ws/r/3061 [20:43] * perrito666 ponders buying a paper book after some time [20:43] thumper: wow, utils.WriteYaml has a huge concurrency failbomb [20:44] prep := path + ".preparing" [20:44] f, err := os.OpenFile(prep, os.O_WRONLY|os.O_CREATE|os.O_SYNC, 0644) [20:44] davecheney: did you actually read the DK book about go? is it good? [20:45] perrito666: nope, both my physical and ebook copies are still on order [20:45] but it has been reviewed by the whole go team [20:45] so if you were looking for _the_ word on go [20:45] you could not do better than that book [20:45] * perrito666 sends to sprint hotel [20:48] davecheney: yeah, that writeyaml function is pretty awful [20:48] davecheney: we have an atomic writefile somewhere [20:52] (probably in a utils package somewhere ;) [21:01] menn0, waigani: boring rename branch http://reviews.vapour.ws/r/3056/ [21:02] thumper: swap you: http://reviews.vapour.ws/r/3061/ [21:02] davecheney: I bet many of our tests have certain concurrency expectations [21:02] waigani: looking [21:02] thumper: woops, I reviewed that one yesterday, but didn't publish it [21:14] wwitzel3: my branch now compiles (including tests) on go 1.2 [21:15] natefinch: nice, we should just use that then? [21:15] wwitzel3: yeah. probably. I'll PR it [21:15] natefinch: make the PR and I'll review :) [21:17] wwitzel3: http://reviews.vapour.ws/r/3070/ [21:19] ericsnow, katco: super simple review to support go 1.2 in the lxd provider branch ^ [21:19] waigani: "controller environment" vs "controller", I do think that it is worthwhile talking about both. Mark wanted the initial environment / state server environment to be called the "controller environment". [21:19] I have tried to do that in the places we talked about the other [21:20] the controller itself encompasses the whole thing (in my mind at least) [21:20] meaning the environment, and the machines and bits in that env [21:21] waigani: but I do take your point, and we should perhaps think a bit more on our language [21:21] at least grepping for "controller environment" shouldn't be too hard [21:22] natefinch: if you rebase against my lxd-fix-local-remote branch, you don't need to touch the instance and container/factory packages [21:23] ericsnow: is that branch landing soon? [21:25] natefinch: not before we get the 1.3+ support we need [21:25] ericsnow: well, this PR is so we don't need 1.3+ support [21:25] )ish) [21:25] natefinch: just to trick the merge bot? [21:26] natefinch: strike that :) [21:27] natefinch: regardless, feel free to strip all the LXD-as-a-container code now rather than waiting for any of my code to land [21:28] natefinch: how is this different from wwitzel3 's pr? http://reviews.vapour.ws/r/3064/ [21:28] thumper: right. It's really about the concept we want to sell. "controller environment vs an environment in a controller" or just "controller vs environment". Either is fine, but we should consciously chose one and be consistent. [21:28] katco: mine works ;) [21:28] yeah [21:28] lol [21:28] natefinch: also, I'm pretty sure we don't import github/lxc/lxd anywhere under provider/lxd so why is the build constraint needed there? [21:29] ericsnow: transitive dependencies... since we're not building all the lxdclient code, the code that uses *that* code now won't compile [21:29] waigani: the tricky bit of this concept is that we have a controller that hosts environments, but also the controller environment that is the environment that contains the API server bits [21:29] yep [21:30] waigani: so I think I'm doing to defer changing these comments for now :) [21:30] okay, sure [21:30] natefinch: meh, I meant solve that a different way but that would be worse than sprinkling the build constraints all over :) [21:32] ericsnow: you could make fake implementations of the lxdclient stuff for go1.2... but I think this actually might be ore clear and less real work. Really didn't take long, and it'll be trivial to undo (even if it does touch a lot of files... it's just one line per file). [21:32] natefinch: right; not worth it [21:36] thumper: But I do think it's worth thinking about. AFAICS, fact that a controller is an environ is implementation detail to support the API - as you say. It shouldn't be confusing the language we use to describe the overall design. [21:36] except it shows up whenever we talk about "all environments" [21:36] and environment commands work on the controller [21:37] I definitely agree that it is worth thinking about [21:44] thumper: I think it comes down to implementation details vs design. Are those details leaking into the design? If so, do we separate them (e.g. never speak of a "controller environment", rename env cmds on the controller to controller cmds) or do we update the design (e.g. always talk about "controller environment"). [21:45] katco, ericsnow: I think we need to talk about dependency injection for the purpose of testing, re: http://reviews.vapour.ws/r/3021/#comment18995 [21:45] I don't think so... I think it depends a lot on context [21:45] you are either talking about the controller environment, and dealing with environment *things* [21:45] natefinch: it's been queued for awhile. i have half an hour if everyone else does [21:45] or you are talking about the controller in a hosting capacity [21:45] the controller environment doesn't host environments [21:46] katco: I have 15 minutes, which is probably not going to be enough. Tomorrow? [21:46] but talking about environment things on the controller doesn't make sense really either [21:46] natefinch: tomorrow is meeting day for me =/ [21:46] katco: I'm willing to sacrifice my 1:1 time to talk about this. I think it's worthy. [21:47] natefinch: we can do that if everyone is available [21:47] thumper: ah okay. In a similar way you could talk about a controller machine? [21:47] I'm not sure you would... [21:48] you may talk about an API server machine [21:48] but a controller machine doesn't really make sense [21:48] so what do we call what we use to call the state server machine? [21:48] a machine in the controller environment is ok... [21:48] katco, natefinch, wwitzel3: I'm game [21:49] personally I tend to still call them state server machines [21:49] ericsnow: natefinch: wwitzel3: invite in the mail [21:49] :) [21:49] haha [21:49] thumper, waigani: FWIW, I think that "talking out loud" about the new names for things is a very good way to nail down gaps in the vocabulary, and exposes places where names may be misleading. [21:50] like state server machine :) [21:50] agreed [21:50] definitely think it needs a new name :) [21:50] like: Bob [21:50] lol [21:50] +1 ship it [21:50] I used to use Eric all the time, from Eric the Viking [21:50] but I feel I have to stop that now that we have ericsnow [21:51] way to mess it up ericsnow [21:51] mwahaha [21:51] ericsnow: how about you change your name? [21:51] perhaps "snowman" === ericsnow is now known as bobsnow === bobsnow is now known as ericsnow [21:51] ericsnow: for this branch: http://reviews.vapour.ws/r/3067/ Did you just do a rename for internal to private? [21:51] nah :) [21:51] cherylj: yep, that's it [21:53] snowman sorta sounds like the codename for a mobster [21:53] ericsnow: it's just showing up in a not-clear way in RB and github. Like it shows that you modified something in internal/client/unitfacade.go, but it doesn't show that you deleted any of the old files [21:53] natefinch: haha [21:54] cherylj: yeah, GH is confused and RB isn't helpful when it comes to renames [21:54] I heard he snowed someone just for looking at him wrong. [21:54] bleh. fun. === natefinch is now known as natefinch-af === natefinch-af is now known as natefinch-afk [22:08] cherylj: in case it helps, I rebased that PR so GH is showing the changes correctly now [22:10] ah yeah, that helps [22:10] ericsnow: there were just a couple things [22:10] cherylj: thanks [22:12] Bug #1513236 opened: Cannot build trusty armhf with go1.2 on from master [22:25] who loves shitty rename branches? http://reviews.vapour.ws/r/3072/diff/# [22:26] katco: FYI, I have the payloads-into-master patch up [22:29] * ericsnow raises hand [22:30] ericsnow: nice :) [22:48] ericsnow: go ahead and merge your payloads branch into master [22:48] katco: k [23:01] katco: alas, master is blocked [23:01] ericsnow: doh [23:10] katco: ericsnow this may very well be an upstream bug [23:10] looking at the ci dashboard [23:10] davecheney: k [23:10] no builds for the crypto repo have run on arm for months [23:10] and the go team turned off build faiure notifications a while back [23:11] so yeah, there is that [23:11] let me quickly check something [23:11] davecheney: thanks [23:12] builds on go 1.5 [23:12] we might have to chalk this one up to go 1.2 being unsupported [23:12] katco: please unblock the build [23:12] we cannot fix this [23:12] it is blocked on other work which has been scheduled [23:13] but has not reasonable ETA that can be used to unblock the build