[00:57] anyone know how the vlan work is going on, particularly testing against maas? [00:57] bigjools: dimiter is leading that AFAIK [00:58] bigjools: but, no, I have no idea [00:58] thumper: ok thanks. I had not heard anything and, well, it's kinda late to make changes now, [00:59] * thumper nods [00:59] AH HA! [00:59] oh, no mwhudson here [00:59] i've figured out why some tests fail to compile on gccgo [00:59] davecheney: mwhudson is on leave for the start of this week [00:59] * davecheney works on smaller repro [01:01] nah, it's a bug when compiing a package for test that has both internal, package agent, and external, package agent_test suites [01:01] well, that is my hypothesis [01:01] now to make a smaller repro [01:01] ah... [01:02] it's really hard to pin down [01:02] delete either bootstrap_test.go -or- package_test.go in juju-core/agent [01:03] and the test binary builds [01:03] the common thing between them is they are both external test files [01:45] http://paste.ubuntu.com/7149183/ [01:45] http://paste.ubuntu.com/7149202/ [01:45] thumper: strong correlation between duplicate symbols and those with coverage [01:45] i need to take a walk and think about this [02:29] * thumper goes to make a coffee [02:40] YES! [02:40] i have a reproductoin [02:41] * davecheney blushes [02:54] thumper: https://code.google.com/p/go/issues/detail?id=7627 [02:54] victorious [02:56] \o/ [03:11] thumper: one-liner (excluding test) https://codereview.appspot.com/79620043/ [03:11] including, two-liner ;p [03:29] axw: and I assume you have tested locally? [03:29] thumper: yup [03:30] thumper: modified sudoers to prevent setting env vars, and ensured it failed then worked [03:30] axw: I almost have a test for the mongo service cleanup [03:30] thumper: cool [03:30] cool [03:32] yikes, I chewed through 7GB yesterday installing MAAS [03:32] forgot to tell it to not download the Internet [03:35] axw: yikes, was any of that unmetered ? [03:35] from a local mirror ? [03:36] davecheney: nope. but it's okay, I use very little of my quota [03:36] ow. [03:36] suggested daily usage: 24G ;) [03:41] daily? [03:41] geez === vladk|offline is now known as vladk === Ursinha is now known as Ursinha-afk === Ursinha-afk is now known as Ursinha [03:45] axw: store test failed :-( [03:45] le sigh [03:45] thanks [03:46] fucking kick that thing out [03:46] why do we keep having this discussion [03:46] month after month [04:00] davecheney: because we like punishing ourselves [04:07] thumper: is this the right room for an argument [04:07] davecheney: it depends [04:08] argument about what? [04:08] thumper: doing things many times to keep seeing if they hurt [04:15] axw: just pushing the test branch now [04:16] thumper: thanks [04:16] I'll go and help make dinner, and come back to check on the review a little later [04:16] after I have proposed that is [04:16] axw: and yours merged \o/ [04:17] thumper: I'm going to block SIGINT right up in cmd/juju/bootstrap.go - but now I need to make it possible to cancel tools upload [04:17] thumper: hooray [04:17] hmm... [04:18] cos otherwise you could cancel during upload and you'll also have a broken .jenv file [04:19] why do we create an empty .jenv so early? [04:19] ^ i have asked myself that question. [04:19] axw: don't forget you can't unhook a signal handler [04:20] also, sorry for butting in. Just saw something that made me extremely happy. [04:20] axw: https://codereview.appspot.com/79690043 [04:20] davecheney: don't need to in this case, but you can ...? os/signal.Stop [04:20] thumper lazyPower: yeah I kinda think we should not write it out until things are working [04:21] however we do want to still catch the interrupt so we can clean up [04:21] I dont think its a problem if there's a cleanup on exception. Thats the part that gets me, it leaves traces behind that prevent normal users from bootstrapping and they dont understand why they have to delete it. [04:22] especially now we tell them they can use ^C to undo a partial boostrap [04:22] the inverse of that - the more I think about it - is what exception is ok to remove it? I can see someone trying to bootstrap an env and wiping out their jenv of a working deployment.. [04:23] which is exponentially worse [04:23] lazyPower: i don' think it works like that [04:23] if .jenv exits [04:23] lazyPower: .jenv is only removed if created for the bootstrap [04:23] exists [04:23] then the environment is bootstrapped [04:23] what davecheney said [04:23] cogito ego bootstrap [04:23] axw: I haven't done the 'remove the rsyslog file if it exists', perhaps I should add that... [04:23] any way, dinner cooking time [04:24] thumper: would be good. I will review what you've got so far anyway. [04:24] later === thumper is now known as thumper-cooking [04:25] ahhh ok. talking about completely different phases of the bootstrap process. [04:25] Some day i'll dig through the juju-core source and try to make sense of this "go" thing you guys are having so much fun with === vladk is now known as vladk|offline [07:49] mornin' all [07:57] jam1: i'm not quite sure how your Worker.Running suggestion solves the problem [07:58] jam1: AFAICS there's a dependency between two workers - we must only start the API worker after the localstorage worker is ready. [08:00] morning [08:00] dimitern: hiya [08:01] rogpeppe, jam is proposing a mechanism for communicating that dependency [08:01] fwereade: right [08:01] rogpeppe, jam, I'm not really that keen though -- do we have use cases for that that *aren't* localstorage? because localstorage is a bit of a hack [08:01] rogpeppe: right now, we don't have a way to detect that the local storage worker is ready [08:01] fwereade: we shouldn't start the environ provisioner until the API server is ready [08:01] we do it [08:02] and just bounce [08:02] firewaller, etc. [08:02] we have a "startAfterUpgrader" [08:02] that is our hack to handle doing upgrades before running the other workers. [08:02] but it isn't generic in any fashion. [08:02] fwereade: in that particular case we want to start after the other one is stopped [08:02] i don't really see how the proposed mechanism would work for communicating the dependency, though i'm probably just being stupid here [08:02] but it is still a case of "don't start all Workers simultaneously and hope for the best" [08:03] rogpeppe: so it doesn't actually do the dependency [08:03] but right now, we don't even have visibility [08:03] rogpeppe: we know when something has been "started" but not when it would actually be able to respond to requests [08:03] jam: i don't see how you'd use the mechanism in this case to achieve the desired goal [08:04] rogpeppe: so in my first cut, it would be that Bootstrap can wait until all services claim to be Running() [08:04] jam: Bootstrap? [08:04] rogpeppe: so it doesn't actually handle dependencies, except that all dependencies must be met if everything is running. [08:04] rogpeppe: "juju bootstrap" would only return once we determine all services have gotten to Running() [08:04] so we could add an API for "are you Running()" [08:04] which reports that info [08:05] bootstrap starts, ssh's in, sets everything up, then waits to connect to the API, and waits for Running() to return true [08:05] Client.AllServicesRunning() or whatever. [08:05] jam, I worry that that would be both tricky and brittle [08:05] jam: oh, i see, this is just about bootstrap [08:05] fwereade: worse than the race conditions we have today ? [08:05] rogpeppe: well, *today* we have a clear race condition [08:06] but we have a general pattern [08:06] jam: but this could be a problem at other times too - any time the machine agent bounces [08:06] of assuming that "if I start X, it is ready" [08:06] which isn't true. [08:06] jam, I don't think that's ever really a sane assumption to make, and I'm not sure that waiting for self-reported readiness makes it much better [08:07] fwereade: so alternatives are that we put retries at all places where we think something might be starting/restarting/etc. [08:07] which is a lot of whack-a-mole (from what I can tell) [08:08] jam: alternatively we could add explicit support to localstorage for determining when that it's running [08:09] rogpeppe: sure, but we have the same thing with provisioner and api worker [08:09] so it isn't *that* special of a case [08:09] jam: do we? [08:09] rogpeppe: environ provisioner bounces until the API server is up [08:09] which is "ok" I guess [08:09] its better than when we restarted the whole thnig [08:09] jam: i don't think that's user-visible [08:09] so you kept racing until it won [08:10] rogpeppe: note, I'm not strictly suggesting that this is high priority, but it does feel like we are missing a pattern of being able to express and take action based on dependencies [08:10] jam: i see how your suggestion allows bootstrap itself to wait until everything has started, but i don't see how it could be used in a more general way [08:11] rogpeppe: so if you wanted to express "don't start the API Server until local storage is ready" [08:11] how do you know when local storage is ready [08:11] if it doesn't give a way to tell you ? [08:11] similarly [08:11] don't start env provisioner until api server is ready [08:11] you can do it bespoke each time [08:11] so the env provisioner just tries to connect, and bounces until it succeeds [08:12] the api server would try to connect to port 8040 and bounce until it can [08:12] etc. [08:12] jam: i think i would add a mechanism (not in worker.Runner) to individual workers to allow them to express readiness [08:13] jam: part of the problem is that there's not a good definition of "readiness" - for example the API server could come up and then promply go down again [08:14] rogpeppe: that is true, being able to handle transient failures is nice, though having the common cases just work is a good thing, too. === vladk|offline is now known as vladk [08:14] we *know* about start-the-world [08:14] it happens at least as often as things dying out of band [08:14] jam: but i think we could work around that in an approximate way, by saying that we'll start as soon as we know it's been up very recently [08:14] good morning [08:14] morning vladk [08:18] rogpeppe: so it is entirely possible to fix the current race by just adding a retry in AddCharm if Put fails, or even put it in the local Storage implementation that retries a Put() that fails. [08:18] I was bringing up the larger discussion because we've had a couple cases where workers depend on eachother [08:18] in ways that it feels would be good to actually model. [08:18] jam: yeah, i think it's a good discussion to bring up [08:19] jam: i'm just putting together a possible way that it might be done [08:20] rogpeppe: so "add to individual workers" doesn't seem that different from having some workers implement a Running() method. [08:20] (or Ready(), Started(), whatever else we would want to call it) [08:20] it is still difficult to define the "I want X to wait for Y" [08:20] jam: i haven't yet seen how that method allows dependencies between workers [08:20] rogpeppe: necessary but not sufficient. [08:21] as in, you have to know when something is ready before you could react to it. [08:21] jam: because workers don't have access to the Worker types created for other workers [08:21] jam, rogpeppe: I am in general in favour of the idea that workers should themselves wait for their dependencies, rather than complicating the decisions about when to start workers [08:21] an alternative is a generic event bus and things starting up fire events that others can chose to listen to. [08:21] fwereade: in a Poll and bounce method, or in a "wait for an event" style ? [08:22] fwereade: i'm not sure about that [08:22] jam, I think it's probably situational [08:22] fwereade: i don't think that, for example, the API server should know about the possibility of a localstorage worker that it maybe has to wait for [08:22] fwereade, hey [08:22] rogpeppe: right, I would want the localstorage to indicate the API server should wait, rather than the other way around. [08:23] rogpeppe, well we shouldn't have a localstorage worker anyway, which is why I'm nervous about this -- the motivating case doesn't seem like a good argument for anything [08:23] dimitern, heyhey [08:23] fwereade: don't we end up with *something* listening even if we have gridfs storage? [08:23] fwereade, re your comment on https://codereview.appspot.com/79250044/diff/20001/state/compat_test.go - do you mean readNetworks should return []string{nil}, []string{nil}, nil (networks and error) when mgo reports NotFound ? [08:24] jam, certainly we do, but doesn't that want to be part of the API server, like the charm put api etc? [08:24] dimitern, I think so, yes [08:24] fwereade: we have UploadTools and AddCharm that now both sit on a more standard HTTP POST URL, presumably object storage would also be just-another HTTP PUT/POST/etc [08:24] jam, yeah, that's my thought [08:25] jam, although probably not even that -- all we need to expose is GET, surely? [08:25] fwereade: well, fwiw, it seems really strange to have the API server be POSTing back to itself. [08:25] fwereade, I fail to see the point in that though [08:25] jam, fwereade: here's how i see that it might work: http://paste.ubuntu.com/7150179/ [08:25] jam, yeah, exactly [08:25] fwereade: so localstorage today is a way to make local look like other providres. [08:25] fwereade, what if we do want to know if there are no networks set and there are nil ones set [08:26] s/know if/distinguish between/ [08:26] rogpeppe: anyway, I don't mean to distract us from the rest of our work too much. I'm not sure that fixing this bug is worthwile yet anyway [08:26] actually, this is a little better: http://paste.ubuntu.com/7150180/ [08:26] dimitern, restate use case please? my model has been hat we shouldn't be able to tell the difference between a machine/service started with no networks set, and one created before it was possible to set networks [08:26] dimitern, when do we need to distinguish those cases? because that's basically what that NotFound does for us, and I think it's a disadvantage [08:27] jam: yes, i don't think i mind a little instability at bootstrap time *that* much [08:27] rogpeppe, honestly I do [08:27] fwereade, hmm.. ok then [08:28] rogpeppe, bootstrap weirdness is disproportionately painful for new users [08:28] fwereade, mgo.ErrNotFound will be ignored in readNetworks [08:28] fwereade: yeah, consider me convinced [08:28] dimitern, plus an explanatory comment, and we're good [08:28] fwereade: so this *particular* race requires you to be able to download a charm faster than the localstorage can start [08:29] jam, rogpeppe: so what's the actual rate we're seeing [08:30] fwereade: but I'm perfectly happy that if this takes precedence over other work, fixing it with retry code in LocalStorage.Put/Get is the most expedient fix [08:30] fwereade: CI has to do a retry loop in their local provider bootstrap && deploy test [08:30] fwereade: I've never seen it in the wild, and had to put a Sleep() in the code to trigger it myself [08:30] fwereade, something in the line of "We treat the non-existence of networks for a service or machine as a valid case and return nil error" ? [08:31] dimitern, "because pre-1.17.7" [08:31] dimitern, but yeah [08:31] fwereade, ok, cheers [08:32] jam, if it's really only hurting CI that that's a bit less of a stress for me -- and makes me all the more reluctant to build a general worker-dependency mechanism vs just implementing in-env storage [08:32] fwereade: so if you are using the local provider on a machine with low latency to launchpad and slow CPU, you could probably trigger it, that would generally be exercising the local provider on Canonistack [08:33] fwereade: the worker-dependency was never the fix for the bug [08:33] it was a "we have this pattern that seemed to come up that we aren't really handling" [08:33] jam, ok, I see [08:35] fwereade: it seems the outcome of the discussion is that Retry loops handle more cases [08:35] jam, that fits my instinct too [08:35] fwereade: the one thing I *did* think we also wanted [08:35] was a way to have Bootstrap know when things are actually ready [08:36] we can trivially hook into something like "workersStarted" and some sort of global flag the API server can see. [08:36] that Bootstrap could then request "have you started all workers" [08:36] though functionally that is probably the same as "is the API Server up" [08:36] jam, that's definitely interesting [08:37] jam, I'm a little suspicious of the quantity of gubbins we'd need but I'm not against the idea ;) [08:38] fwereade: so I already added a "workersStarted" channel to have the test suite be able to wait until it knew the workers were actually running, but I was personally aware that "started != running" [08:38] and we have no channel today to signal that [08:39] jam: i think it would be more like "are all workers currently up" than "have you started all workers" [08:41] rogpeppe: sure, but we *do* have code today that knows when all workers were started, we *don't* have a way to poll all the workers to know if they are running [08:41] (running as in able to respond to requests) [08:42] we could certainly phrase it in "is everything running" [08:42] and just return the best answer we have today [08:42] which is everything was started [08:42] and is not currently Dead [08:43] jam: i can only think of two workers like that - localstorage and API [08:43] rogpeppe, zero one infinity ;) [08:43] jam: none of the others respond to external requests AFAICS [08:43] fwereade: :) [08:44] fwereade: istm that it would be more appropriate to make those two special cases than to change the whole worker mechanism around them [08:44] hence my suggestion above [08:45] jam: what did you think of that as an idea, BTW? [08:46] rogpeppe, I'm not sure that what jam proposes is as much of a change as you seem to think [08:46] rogpeppe, sorry, I can't parse the important parts out of that paste [08:46] fwereade: ah, sorry, i'll trim [08:47] (jam, rogpeppe: there *is* some legitimate fuzziness in between "zero one infinity" and "three strikes and you refactor") [08:47] fwereade: in fact, it's easier than that - the important lines all contain the string "localStorageReady" [08:48] fwereade: Ithink it is essentially "NewWorkerReady" is a way to signal, you pass it into LocalStorage if you create it, otherwise just set ready immediately, and API Server waits on it. [08:49] jam: yeah. NewDependentWorker provides a way to gate the starting of one worker on the readiness of one or more ready signals [08:50] if the dependency graph got more complex, you could make it more table-driven [08:50] jam, rogpeppe: that does seem like a reasonable cut at it, but I do worry about that sort of mechanism's sanity in the face of restarting workers [08:51] fwereade: i think it can work ok, as far as it's possible to make it work at all [08:51] fwereade: in particular, once a worker has started, you have no idea that it remains up for any period of time [08:51] rogpeppe, you speak eloquently of the core of my objections :) [08:52] fwereade: but you have to do something some time... [08:52] fwereade: so the idea behind NewDependentWorker is that it waits for readiness and then starts the worker. [08:52] fwereade: but readiness is not set-once [08:52] fwereade: when a worker goes down, it should reset its ready flag [08:53] rogpeppe, and more mechanism creeps in to iteratively refine the quality of the guess... but we still need to be prepared for it failing to work [08:53] fwereade: another possibility i suppose is that if the thing you're dependent on goes down, then you should go down too [08:54] fwereade: sure, it can certainly fail to work - we should be prepared for workers to fail at any time [08:55] fwereade: but if we can't take a worker's readiness as a signal that we can start its dependents, then i'm not sure we can *ever* start its dependents :-) [08:56] rogpeppe, if the dependents can be resilient in the face of the occasional absences of heir dependencies, I think we sidestep the whole issue [08:57] fwereade: that's a reasonable point [08:57] fwereade: though i hate arbitrary retries :-) [08:58] rogpeppe, intellectually, so do I, but they often quite useful in practice [08:59] fwereade: you've certainly convinced me that this is the best solution at this time to this problem [08:59] * fwereade is happy then :) [08:59] * rogpeppe reluctantly throws away the nascent DependentWorker implementation [09:01] we should probably just use gridfs for storage, so there's no need for another service (though I realise worker dependency is bigger than this one issue) [09:02] fwereade: unfortunately goose has a very nice retrying HTTP Client that is not factored into a 3rd party lib, and is slightly Openstack specific (because of the Retry-after: header not being everywhere) [09:03] though I don't think the goose one retries if it fails to connect completel [09:03] completely [09:03] vs getting a Retry response [09:14] axw: +1 [09:17] mgz, hey [09:21] mgz, you should really set up some alerts on the (other) machine you're using irc to notify you when there's a message :) [09:23] axw, +100 to that [09:23] fwereade, one last look at https://codereview.appspot.com/79250044/ before I submit it for landing? [09:24] * fwereade looks [09:26] dimitern, LGTM [09:27] fwereade, thanks! [09:30] dimitern, would you take a look at https://codereview.appspot.com/79730043/ please? mostly moves, only significant change is using uniter/charm.Bundle instead of charm.Bundle [09:30] fwereade, sure, looking [09:30] dimitern, and abit of preparatory test gubbins that will be used in a followup [09:44] hello [09:45] fwereade, reviewed [09:45] hey wwitzel3 [10:07] dimitern: I have to miss the standup today, do you care to crack the Whip ? [10:07] jam1, sure, my pleasure :) [10:10] rogpeppe: did you push your changes from last night? [10:10] dimitern, reproposing [10:11] wwitzel3: checking [10:11] fwereade, will look when it appears [10:11] dimitern, cheers [10:12] good morning everyone [10:12] wwitzel3: pushed [10:13] voidspace: hiya [10:13] voidspace: hope you're feeling a bit better today [10:13] hi perrito666 [10:13] perrito666: hiya [10:13] hey voidspace , you getting along better today? [10:15] hey guys [10:15] thanks [10:16] rogpeppe: does that build for you? [10:16] wwitzel3: rogpeppe: definitely better - I wouldn't recommend coming to visit but I got a good night's sleep [10:16] wwitzel3: no, it's WIP [10:16] which makes a lot of difference :-) [10:16] voidspace: yeah [10:16] rogpeppe: k, just checking :) [10:16] fwereade, LGTM [10:18] rogpeppe: want to join the standup hangout, I'd like to talk about some of the changes? [10:18] wwitzel3: joining [10:18] rogpeppe: I didn't get back from dinner early enough last night to catch you :) [10:19] wwitzel3: i'm there [10:19] rogpeppe: I don't see you .. [10:19] rogpeppe: these hangouts are a PITA [10:20] wwitzel3: i've joined the usual juju-core standup hangout, right? [10:20] rogpeppe: yeah, same here :/ [10:20] wwitzel3: https://plus.google.com/hangouts/_/calendar/Y2Fub25pY2FsLmNvbV9pYTY3ZTFhN2hqbTFlNnMzcjJsaWQ5bmhzNEBncm91cC5jYWxlbmRhci5nb29nbGUuY29t.mf0d8r5pfb44m16v9b2n5i29ig [10:28] morning core devs: I am trying to track down the issue described in bug 1277445. It seems to be a juju-core provisioning issue, could you please take a look? [10:28] https://bugs.launchpad.net/ubuntu-advantage/+bug/1277445 [10:38] frankban, looking [10:38] dimitern: thanks [10:45] fwereade, mgz, standup [10:54] frankban, I think you're on the right track for that bug [10:54] frankban, wallyworld is the person most likely to have immediate insight, but I can talk about it myself in a bit, I think [10:54] fwereade: cool thanks. should this be assigned to juju-core? [10:55] frankban, yes, I think it should [10:55] fwereade: cool thanks [10:56] fwereade: uhm... Proprietary bugs cannot affect multiple projects. do you have a team to which assign this bug? [10:57] i can look at the bug a little later [10:58] frankban, maybe assign it to wallyworld then :) [10:58] fwereade, wallyworld: thank you! [10:58] * wallyworld has no idea off hand about root cause but will try and figure it out [11:10] voidspace: separate hangout? (unless you particularly want to follow the topics) [11:13] rogpeppe: sure, you start one and I'll grab coffee [11:13] wwitzel3, natefinch: i've pushed my WIP branch to lp:~rogpeppe/juju-core/030-MA-HA/ [11:13] voidspace: k [11:13] rogpeppe: ok, thank you [11:14] wwitzel3: did it work? I'm Z. Nate Finch noew, but of course I'm still all the way right for myself [11:14] natefinch: you're still in the same spot, you probably have to rejoin [11:15] wwitzel3: I did. Boo. [11:16] wwitzel3: wonder if making it an initial instead of a name was a mistake [11:16] voidspace: https://plus.google.com/hangouts/_/76cpijifqckugj18ncu09812oc?authuser=1&hl=en [11:28] rogpeppe: currently failing to join, trying again [11:30] natefinch: ready for my call? [11:32] wwitzel3: not exactly. Tuesday & Thursday, Lily, my older daughter, goes to preschool, and 7:30-8:30 is where I help get her up and dressed etc. Around 8:30 I'll be able to call, but I'll have Zoë, the baby, until about 9:15 or so, when my wife gets back from dropping off Lily. [11:32] wwitzel3: and now you know more than you ever wanted to know about my weekly schedule :) [11:33] natefinch: ok, I will look in to the upgrade stuff we discussed during standup then [11:33] wwitzel3: cool [11:33] and just ping me when you're ready, I've merged rogers latest code and pushed to my branch [11:34] natefinch: I need more nate, I need to know it all. :) [11:34] but the cameras will be installed next week, so we'll be cool then. [11:34] lol [11:35] jam: haha... we make boring TV, it's all poopy diapers and finger paints :) [11:35] (and hopefully not vice versa) [11:36] bbiab [11:36] natefinch: poopy paints and finger diapers ? Doesn't sound too bad [11:36] lol [11:38] * perrito666 discover a whole new world by having hangout in a completely different device than work [11:44] wwitzel3, natefinch, rogpeppe: so how is HA looking? Did we manage to sort out actually bring up a replica cluster? [11:45] jam: we actually got EnsureMongoServer working yesterday [11:45] rogpeppe: I thought Wayne's summary is that you still needed keyfile to have everything work together, but that would be a follow up patch [11:45] jam: we still do need keyfile, yes [11:46] jam: there should probably be an API to allow the machine agent to retrieve the keyfile data [11:46] jam: i've been considering whether we could just use a hash of the state server private key [11:47] rogpeppe: it is just a big password, right? I'd rather it just be a unique string that we hand to all the state server agents, otherwise there is bits of "why is it the hash(X)" rather than just "this is the string" [11:48] jam: if we use a hash of some existing data, we don't need any new API entry points [11:49] jam: and i can't *really* see why one might want to give a server the private key but not the keyfile [11:49] jam: or vice versa [11:49] jam: but... [11:49] rogpeppe: while that is True, I'm guessing we don't have 100% the API we need for everything today, do we? In which case we are changing the API anyway. [11:49] jam: i don't mind much either way. i was just wondering if we could save ourselves some work [11:50] jam: that's true, and i think that reminds me of some tickets we don't have on the board [11:50] jam: we need API entry points for retrieving the API server private key [11:51] rogpeppe: so I would be fine extending that API to be "give me the stuff I need to be a state server" and it hands the private key and the keyfile content and anything else we find is relevant. [11:51] jam: sgtm [11:52] jam: +1 [12:02] * perrito666 wonders why our ssh command doesn't prepend ubuntu@ to the host [12:11] perrito666: it does in cmd/juju/ssh.go [12:12] jam: I was talking about the utils one, ssh.Command [12:17] perrito666: because the utils/ssh one is meant to be less opinionated than that [12:18] i.e. it should allow ssh'ing to hosts as other users - that is required for the manual provider [12:23] axw: I see, thank you for the info :D [12:23] perrito666: nps [12:34] morning all [12:35] axw: ping [12:35] rogpeppe: yo [12:36] axw: if you wanted to deliberately inject a bad environ config into the state (for a test), how would you go about it? [12:36] axw: i am being thwarted by the Policy [12:36] rogpeppe: reopen state with a nil policy [12:36] axw: doh! of course. [12:36] axw: ta [12:36] rogpeppe: if you grep for state.Policy(nil) you'll find an example somewhere [12:36] nps [12:37] axw: presumably you could just use nil rather than state.Policy(nil) [12:37] axw: although it *was* useful to grep for :-) [12:38] rogpeppe: yeah that's just for documentation [13:00] fairly trivial review, anyone? (just moving code) https://codereview.appspot.com/76890049 [13:03] rogpeppe: lgtm [13:03] mgz: ta! === Ursinha is now known as Ursinha-afk [13:15] rogpeppe: natefinch: did you guys get the EnsureMongoServer patch put up for review? [13:15] jam1: i pushed the changes i'd made, but it wasn't anything like ready for review [13:16] jam1: i'm currently working with michael on getting the instance-ids published in the environ [13:17]