[08:10] davecheney, fwereade: mornin' [08:10] rogpeppe, davecheney, heyhey [08:15] TheMue, morning [08:16] fwereade: hiya [08:17] rogpeppe, re https://codereview.appspot.com/6819115/ I am concerned that ServerCertAndKey is fundamentally part of the environment config, but that it's not being treated as such... comments? [08:17] fwereade: it's actually not part of the environment config, and it's important that it doesn't go in it [08:17] rogpeppe, jolly good -- please expand? [08:18] fwereade: it's private information passed particularly to the state server only [08:18] fwereade: you don't need any of that information in order to connect to the state server [08:18] rogpeppe, I though we were meant to be able to switch on state-serveriness for arbitrary machines [08:19] fwereade: we want to be able to start new state servers, but i think that's a slightly different thing [08:19] rogpeppe, (and I don't think that applies -- what about authorized-keys, which is similarly broken, but at least we have a route to unbreak it because it's in the env config) [08:19] rogpeppe, for that matter, what about ec2 keys? they also go in the env config [08:19] fwereade: authorized-keys is different - it only has public keys [08:20] rogpeppe, I don't see how that's a consideration -- isn't env config access going to be restricted? [08:21] fwereade: yeah, and that's an interesting issue too - we may actually need to provide different levels of env config access [08:21] rogpeppe, I've been saying this for months, AIUI niemeyer will magically fix it all :/ [08:22] fwereade: oh that's good then :-) [08:22] rogpeppe, but he was so keen to slap down the segregation suggestion I made in oakland -- like *weirdly* keen -- that I just haven't bothered to keep on about it [08:22] fwereade: anyway, the certificate is for the eyes of that state server only - it may or may not be shared by other state servers [08:23] fwereade: unlike the ec2 keys which are global to whoever needs them [08:24] rogpeppe, ah, ok, I thought it was an env-global certificate... maybe I don't understand what's going on at all here [08:24] fwereade: also, the ec2 keys can be passed with the other secret keys, after bootstrap, but the server certificate and key need to be passed earlier [08:25] fwereade: the thing that's env-global is the root CA certificate [08:25] fwereade: that's the thing that can authorize servers [08:25] rogpeppe, then isn't that needed by everything that needs the ec2 keys? [08:25] fwereade: also, if the state server wants to spawn new state servers, it can pass its key to newly spawned instances [08:26] fwereade: i don't *think* so [08:27] morning dimitern [08:27] fwereade: well... it depends how we spawn new state servers [08:27] rogpeppe, ah ok -- it's attached to the machine not the instance, and it's only the client that needs it so it can add machines, maybe? [08:27] TheMue: morning :) [08:27] TheMue: hi [08:27] rogpeppe, yeah, I'm very unclear on this, I probably should have been following along more [08:27] dimitern, heyhey [08:27] rogpeppe: moin, didn't wanted to interrupt you ;) [08:28] fwereade, rogpeppe, hey guys [08:28] dimitern: hiya [08:28] dimitern, fwiw, this is the blueprint mentioned last night [08:28] dimitern, https://blueprints.launchpad.net/ubuntu/+spec/servercloud-r-juju-resource-map [08:28] fwereade: ah, 10x! I'll take a look [08:28] dimitern, don't expect to actually implement anything directly related to it, but bear it in mind :) [08:28] dimitern, thanks :) [08:29] fwereade: you don't need the state-server cert to add new machines [08:29] rogpeppe, can we step back a bit so I get a fresh overview of the whole plan? [08:29] fwereade: sgtm [08:30] rogpeppe, AIUI, the end purpose of this work is to allow us to verify the identities of machines and units -- how inaccurate am I so far? [08:30] fwereade: pretty inaccurate, tbh :-) [08:30] rogpeppe, sweet, I am about to learn something :) [08:31] fwereade: machines and units identify themselves with a password. they're the client side. this is the server side, which identifies itself with a tls certificate, like a normal https web server [08:33] fwereade: if it was about allowing us to verify the identities of machines and units, we *definitely* wouldn't have an environment-global cert anyway [08:33] rogpeppe, wait, who's going to be allowed to look in env config anyway? [08:34] rogpeppe, I'm approaching this from a perspective in which generally the env config is not accessible [08:34] fwereade: anyone that can access the state and has the right authorisation [08:36] rogpeppe, (fwiw, I thought we were doing the thing mentioned in the 3rd bullet at http://www.openssh.org/txt/release-5.4 which STM to imply that we can actually use a root CA to verify SSH keys) [08:36] rogpeppe, yeah, but the "right authorization" needs to be finely gradated, doesn't it? [08:37] rogpeppe, just because you can see one part of it doesn't mean you should be able to see it all [08:37] fwereade: something in me says it's wrong to have a private key in the state (i'm fairly reluctant even to pass it over the network, but i don't see a decent alternative) [08:37] fwereade: i agree. there are different parts of the state appropriate for different roles [08:38] fwereade: it may well be the case that we need to put the state server private key in the state (i now realise) [08:38] rogpeppe, well, yeah -- isn't it going to also be hanging around forever, accessible to anything deployed on a machine that runs a state server? [08:38] fwereade: no; at least not eventually [08:38] rogpeppe, ie anything that can access the metadata service on that machine? [08:39] rogpeppe, I thought stuff in cloud-init stayed available forever, effectively [08:39] fwereade: the plan is to change the state server cert when we first connect [08:39] rogpeppe, ah! ok cool [08:39] fwereade: similarly to admin-password [08:39] fwereade: we're skipping that bit for the time being though [08:39] rogpeppe, ok, consider that objection sidelined [08:40] fwereade: anyway, even if we *do* have the state server cert in the environment, we'll still need to pass it in cloudinit.Config, i think [08:41] rogpeppe, yeah, that's the only channel we have, and it's fine if we're replacing things [08:41] fwereade: exactly. the state server needs to know its key before there *is* a state [08:41] rogpeppe, huh, I just thought [08:42] rogpeppe, but I can't articulate, bah [08:42] fwereade: once we've bootstrapped a state server, the key can probably be passed in the state. in fact, it probably doesn't need to be part of the environment config, as it's independent of the environment [08:43] rogpeppe, ok, I am confused: we pass up a cert+key, but those have to be replaced because they must be considered compromised by virtue of being in cloud-init [08:43] fwereade: yeah. go on. [08:43] rogpeppe, how can we end up with a replacement that is not specific to the env that generated it? [08:44] fwereade: it's just part of the state. [08:44] rogpeppe, but the state is not independent of the environment, is it? [08:44] fwereade: it doesn't interact with the environs providers at all [08:45] fwereade: ok, sorry, "independent of the environment" i meant in a fairly specific way [08:45] fwereade: i meant "independent of environs/*" [08:45] fwereade: or perhaps "independent of Environ" [08:45] rogpeppe, hmm, ok, that still doesn't quite sit right with me but I'll take it on trust (until I suddenly derail another conversation when I figure out my objection ;)) [08:46] fwereade: anyway, this is to do with high availability stuff, which we haven't figured out yet entirely, but i know at least that we need to pass a server cert and key in the initial cloudinit [08:47] fwereade: which is what this CL does [08:48] fwereade: i am about to change it though, so that the cert and key are in different fields. and i'm probably going to add another field, RootCACert [08:50] rogpeppe, ok, that does sound sane to me, given the basic assumptions above [08:51] rogpeppe, my brain still itches, though; I may well keep on hassling you about it ;) [08:51] fwereade: please do :-) [08:51] rogpeppe, not right now though ;) [08:54] So, made another mint tea, still barking too much, damn cold. [09:09] popping out for a mo, bbs [10:05] mgz, dimitern: I hope your weekends went well [10:12] jam: oh, yeah, yours? [10:12] pretty good overall, went to the beach, did some work around the house, etc. [10:13] jam: so, I'm done with the nova stuff in the client, just the swift api left, and I'm looking into writing table based tests for everything [11:29] Good morning [11:30] o/ [12:19] is there anyone working on an openstack provider for juju-core? [12:26] sidnei: Yeah, work on it is starting just now [12:26] awzm [12:26] just 'go got' juju-core :) [12:27] niemeyer: yo! [12:28] niemeyer: hey :) [12:28] sidnei: Superb :) [12:28] rogpeppe, dimitern: Heyas! [12:29] is there a wiki page on how to hack on it? [12:29] niemeyer, ^ [12:31] sidnei: Not yet [12:31] sidnei, https://codereview.appspot.com/6816114/ and https://codereview.appspot.com/6817113/ are not yet merged but may cover much of what you seek [12:31] sidnei: Best is to ask questions here (and perhaps build such a page :) [12:31] niemeyer, heyhey :) [12:32] fwereade: Oh, wow.. I'm out-of-date :-) [12:32] fwereade: Heya [12:32] sidnei, (those two links contain README and CONTRIBUTING files :)) [12:32] niemeyer, nice holiday? [12:32] fwereade: Yeah, very.. hmm.. distracting :) [12:33] niemeyer, in a good way, I hope :) [12:33] fwereade: Yeah, a bit exhausting at times, but was certainly fun [12:34] fwereade: We've been doing some home improvements lately, and last week was the peak [12:35] fwereade: With plaster installation.. plaster is both the most amazing thing and something I don't ever want to see again (but I said that before..) [12:36] haha [12:36] niemeyer: plastering is an amazingly skilled job [12:43] rogpeppe: Agreed [13:15] fwereade: those docs look nice, dave is rocking it out. [13:16] FYI all it's a national holiday here in the US so lots of folks will be out for the day [13:17] mramm, cool, thanks, and np :) [13:24] fwereade, thanks! [13:25] sidnei, np :) [13:25] * fwereade cheerfully takes credit ;p [13:25] mramm: Enjoy [13:26] fwereade: I'm reading lp:~fwereade/+junk/juju-braindump now [13:26] fwereade: you get full credit for that one! [13:26] niemeyer: thanks! [13:26] mramm, :) [13:28] it's a bit unclear from the contributing doc how tests from the branch get run [13:29] sidnei, when you run `go test launchpad.net/juju-core/...`, all the tests will be run; if that dir contains some particular branch, the tests from that branch will be run [13:29] sidnei, does that address the confusion? [13:30] nope :) so bzr branch lp:juju-core trunk; cd trunk; go test launchpad.net/juju-core/... will run the tests from the trunk branch? [13:31] sidnei, sorry, nope -- the branch has to be checked out to $GOPATH/src/launchpad.net/juju-core [13:31] sidnei, hence the utility of cobzr [13:31] sidnei, which lets you use just the one filesystem location for the N branches you're working on [13:32] i see. so that 'branching' section needs some clarification [13:34] fwereade: Reading your braindump too. Reads very good. [13:34] sidnei, ah, yes, I see -- do you think it will be clear if we just add a note to the beginning of that section stating that you're expected to be in the go got juju-core directory to run these commands? [13:34] TheMue, cool [13:35] fwereade: And especially your side notes like "… and be left for ages.". *lol* [13:35] fwereade, yes, and probably removing the mention of 'bzr branch lp:juju-core' completely, and just using 'bzr checkout -b' to make a branch from the go-got directory [13:36] sidnei, isn't 'bzr branch lp:juju-core` exactly what should be done there though? [13:36] fwereade, nope, because go get already did that [13:36] sidnei, it's assuming you're using cobzr, which does clever things with the branch command [13:37] well, the document starts with installing cobzr ;) [13:40] sidnei, good point re checkout, indeed, sorry I misunderstood [13:41] fwereade: Glossary brings me memories.. I hope it's less controversial this time around.. ;) [13:42] fwereade: ("I don't wanna manage the wiki!") [13:42] niemeyer, really? ha, I think I missed that [13:42] niemeyer, lolo [13:45] mramm, TheMue, other readers: whoops, I forgot to add the glossary [13:45] it's there now [13:45] fwereade: Great, thanks. [13:52] fwereade, should probably include a list of dependent packages too, running the tests complains that 'zip' and 'mongod' are not available for example. [13:53] sidnei, I could swear that were covered somewhere [13:55] sidnei, ah, they're in the readme, but zip is not mentioned (and nor is 'git' for that matter) -- thanks! [13:56] * niemeyer goes for lunch [13:57] sidnei, ah, and maybe both of those are covered by 'build-essential' [13:57] build-essential the meta package? pretty sure that doesn't install mongodb-server :) [13:57] sidnei, the README has `sudo apt-get install mongodb build-essential bzr` [13:58] * TheMue steps out, bbl [13:58] sidnei, if that's inaccurate, though, ofc we want to know :) [13:58] i see [14:00] so zip and git-core need to be added there [14:01] fwereade, next up: http://paste.ubuntu.com/1353151/ [14:01] fwereade, there's this warning when running the tests, but the command it tells me to run doesn't work [14:02] sidnei, ha, yeah, I grew used to that as a minor annoyance a while ago [14:02] sidnei, it never crossed my figure-out-what-the-problem-is threshold though [14:03] rogpeppe, are you familiar with the `go test -i foo/...` thing? [14:03] bzr [14:03] oops [14:04] fwereade: you can't do that AFAIK [14:04] fwereade: it's a bit annoying [14:04] maybe that warning can be removed/silenced? [14:04] sidnei: just go install foo/... before running the tests [14:04] usually go test -i works [14:04] (never seen it not working here) [14:04] yeah, you can do 'go test -i' but not 'go test -i package' [14:05] it's a bug. it may well have an issue for it. [14:06] neither works here, only 'go test' without the '-i' [14:07] -i didn't work for me either [14:07] ah, this was issue 3896, but it's fixed in tip [14:08] dimitern: you have to be in the directory that has failed [14:08] * Aram only runs tip [14:08] https://code.google.com/p/go/issues/detail?id=3896 [14:08] rogpeppe: yes, both in the dir, and out, specifying full path [14:09] Aram: i try to run against 1.0.3 mostly because i don't want us to be incompatible with that because of later-fixed bugs [14:09] Aram: although currently i'm running against tip because i've got a few Go CLs pending [14:10] rogpeppe, btw, re https://codereview.appspot.com/6811091/diff/1/worker/firewaller/firewaller.go#newcode546 [14:10] sidnei: you can always just ignore that warning [14:10] rogpeppe: I run tip because lbox requires it, I'm making sure it will work with future Go versions (we already found problems this way) and I am sure most people run Go 1.0.3 so we will be compatible with that :-). [14:10] i can yes, just thinking about the next guy that gets the warning :) [14:10] rogpeppe, it's not my code, but it looks to me like it's trying to make sure that ports and change aren't using the same underlying array [14:10] oh, and I like to play with exp/types [14:11] fwereade: they won't, even with my suggested change [14:12] sidnei: yeah, it's annoying. there are quite a few things which are fixed in tip but not 1.0.3. nothing crucial, but little things like that. [14:14] rogpeppe, can you describe the mechanism that prevents that? I think I'm a bit slow today [14:14] fwereade: we're appending changes to ports, not assigning the changes slice to the ports slice. [14:15] fwereade: currently it reallocates the ports slice each time. that's unnecessary - all we need to do is copy the elements, and potentially reuse the old ports slice. [14:15] fwereade: it's minor stuff tbh [14:15] rogpeppe, ok, and appending to the (empty) slice is safe because append knows that someone else is using the following slots, and copes automatically? [14:16] rogpeppe, I had clearly misunderstood that then [14:16] rogpeppe: fwereade: I'd also appreciate an eye on https://codereview.appspot.com/6820112/ and https://codereview.appspot.com/6814108/ when you have some time. [14:16] fwereade: append only reallocates if the capacity of the slice isn't already large enough [14:16] rogpeppe, ok, that was what I thought [14:17] rogpeppe, so... GAAAH sorry [14:17] rogpeppe, forget I said anything ;p [14:17] fwereade: what was that? [14:17] fwereade: someone was talking [14:18] rogpeppe, no idea :) [14:18] Aram, I shall get right on those, sorry to have neglected them [14:24] oh, BTS, Y U DON'T UNDERSTAND ME [14:24] they are awful === TheMue_ is now known as TheMue [14:27] Aram: BTS? [14:27] rogpeppe: our travelling agency. [14:28] Aram: i think i must use a different one [14:28] I see. [14:39] * Aram is back in a couple of hours or so [14:43] Aram: you gone yet? [16:18] niemeyer: ping [16:18] rogpeppe: pongus [16:18] niemeyer: i'm just in the process of writing a test that does *yet another* setup and teardown of a fake home directory [16:18] niemeyer: and wondering if there's room for something like a HomeSuite [16:19] rogpeppe: Isn't that just a SetupFakeHome function? [16:20] niemeyer: well, yes, but there needs to be a teardown function too [16:20] rogpeppe: Well, I guess we have to undo the var [16:20] Yeah [16:20] rogpeppe: Sounds reasonable [16:20] niemeyer: cool [16:20] rogpeppe: FakeHomeSuite [16:20] niemeyer: +1 [16:21] niemeyer: in juju-core/testing [16:21] rogpeppe: Yeah [17:23] Aram: My evaluation code so far is at lp:~themue/+junk/golxc. Real tests are missing, only a main using the implementation. It's corrently no rocket science, but opposite to the Python code it covers no Juju aspects, it's just a base package for later usage in the providers. [17:52] ssh is too clever for its own damn good [17:52] env [17:54] rogpeppe: rofl [17:55] rogpeppe: how comes? [17:55] TheMue: i tell it that $HOME is one thing, and it still finds out my $HOME via some other, as yet unknown, mechanism [17:55] TheMue: i suspect it's doing a getpwuid [17:56] rogpeppe: And how do you set $HOME? [17:56] TheMue: i'm trying to get the state tests to pass irrespective of what the user's home directory contains [17:56] TheMue: os.Setenv("HOME", ...) [17:57] TheMue: you can try it yourself: cd juju-core/state; chmod 0 $HOME/.ssh; go test [17:57] rogpeppe: On the remote site? [17:57] TheMue: no, in the test code [17:58] rogpeppe: Yes, test code. I'm only trying to get the mechanism. [17:59] TheMue: i'd be interested to know if the above commands succeed or fail for you, actually [18:00] rogpeppe: One moment. [18:00] rogpeppe: Yeah, ssh won't respect $HOME [18:01] niemeyer: pity [18:01] niemeyer: thanks for the confirmation though [18:02] rogpeppe: ssh_test fails here too [18:02] TheMue: yeah, i think we're gonna have to live with it [18:03] pwd [18:29] i've run out of time. gotta go. see y'all tomorrow. [18:31] cu === bcsaller1 is now known as bcsaller [23:13] https://codereview.appspot.com/6817113/ [23:13] ^ has two LGTM, but waiting for any final comments [23:13] i am not a strong wordsmith, i don't expect to get it 100% right on the first go