[00:27] wallyworld__: taking the dog for a walk in the hope she'll go outside... [00:27] ok [01:20] geez luise [01:21] she has great bladder control holding it for inside [01:21] although this time, she did go outside [01:21] thankfully [01:21] only after we had got back, and I refused to take her inside [01:25] wallyworld__: wow some of our code is wonderfully inefficient [01:42] thumper: which bit? [01:42] wallyworld__: the provisioner, that I'm currently refactoring [01:42] ah. good thing you're onto it then [01:43] I'm busy teasing apart dependencies [01:48] i find not having generics leads to a *lot* of code duplication [01:50] I really wish our state methods would return interfaces rather than real structs [01:50] makes mocking and testing a PITA [01:52] oh yes +100 to that one [01:53] wallyworld__: I've made interfaces for the provisioner on all things that I can [01:53] so the new bits take interfaces not structs [01:54] \o/ [03:21] wallyworld__: OMFG [03:29] bigjools: ok, can have that talk now [03:41] thumper: what? [03:41] wallyworld__: my day of refactoring passed the tests first time (once I fixed all the compile errors) https://codereview.appspot.com/9937046 [03:42] cool, i'll look [04:29] wallyworld__: need to talk to you, and it's about work.... [04:29] shit eh [04:29] it could be [04:30] I could just call you but reception is shit so get on a hangout will ya [04:30] i thought since you asked you'd create one, but i will [04:31] https://plus.google.com/hangouts/_/d3f48db1cccf0d24b0573a02f3a46f709af109a6 === thumper is now known as thumper-cooking === tasdomas_afk is now known as tasdomas [06:54] mornin' all [07:02] rogpeppe1: heya [07:02] TheMue: hiya === thumper-cooking is now known as thumper === TheRealMue is now known as TheMue [08:48] TheMue, ping [08:55] fwereade__: pong [08:56] fwereade__: just wanted to ping roger ;) but will do it afterwards [09:01] fwereade__: ah, it seems i found what i wanted to ask [09:02] TheMue, would you please send thumper a link to your lxc work so he can maybe look through it tonight? [09:03] fwereade__: sure, it's a very interesting topic [09:03] fwereade__: will do immediately [09:04] fwereade__: btw, watcher with test is working [09:23] mgz: hey [09:29] dimitern: hey [09:29] mgz: how can we go about pairing on implementing the deployer stuff? [09:30] I'm open to suggestions [09:31] one option is using my vm with go stuff on and screen or something, plus voice [09:37] mgz: not sure i see the process will go - can you expand a bit? [09:40] rogpeppe1: responed to https://codereview.appspot.com/10044043/ [09:44] trying to remember what jam and I did exactly, basically you just ssh into my canonistack box then run screen with a couple of flags, then we try to not type at the same time :) [09:49] mgz: I see, well we can try it out [09:49] just as soon as I finish responding to my reviews [09:52] sure [09:59] rogpeppe1: responded to this as well https://codereview.appspot.com/10026044/ [10:01] looking for reviews on this: https://codereview.appspot.com/9937045/ [10:02] fwereade__: ^^ [10:06] mgz: i'm afraid we have to base the deployer stuff on one of my CLs, so until it lands maybe we can discuss it first [10:07] that sounds good [10:07] but hopefully it won't change much, we can just use it as a prereq before it lands actually [10:08] what's the branch? [10:08] lp:~dimitern/juju-core/055-api-machiner-subpackages [10:09] assuming we take the same path and develop it first server-side as a subpackage [10:09] that branch introduces some interfaces and decouples the root api object from the machiner facade [10:10] fwereade__: ping [10:10] mgz: g+? [10:10] dimitern, pong [10:10] fwereade__: do you have some time to g+ about the general direction of the deployer facade? [10:11] dimitern, give me 5 mins [10:11] fwereade__: sure [10:12] dimitern: can do, but mumble would work as well for this [10:13] mgz: sgtm, if fwereade__ won't mind [10:13] ah, g+ then if we want him too [10:13] mgz: in the mean time have a look at worker/deployer/deployer.go [10:14] mgz: i have extracted a list of state calls used by it: http://paste.ubuntu.com/5738348/ [10:14] ah, crap, but I need to reboot into chromeos for that... [10:14] mgz: oh, sorry about that [10:15] fwereade__: can you install mumble quickly? [10:19] mgz, dimitern, ok, I am here, I need the bathroom, I will try to get mumble up in a sec [10:20] mgz, fwereade__: we can meet in the Blue room on mumble (Root -> Cloud Engineering -> Blue) [10:25] cheers [10:26] fwereade__: if you go to settings in mumble and enable push to talk + set the shortcut to something (i use numlock) and hold it while speaking I found it works best [10:26] fwereade__: because the noise/silence detection sucks in mumble [10:32] mgz, bug 1188126 [10:32] https://bugs.launchpad.net/juju-core/+bug/1188126 [10:37] *: any known problems with the .../worker tests? they timed out here when doing test ./... [10:40] jamespage: thanks! [10:51] rogpeppe1: ping [10:51] TheMue: pong [10:52] rogpeppe1: just performing tests including the worker package. had a timeout, tests ran too long. [10:52] TheMue: can you reproduce the problem? [10:52] rogpeppe1: now I took TestOneWorkerStart as a first one to see where it happens [10:52] rogpeppe1: and already this hangs [10:53] rogpeppe1: yes, reproducable [10:54] TheMue: this is with trunk? [10:55] rogpeppe1: had done a pull before and run 1253 (plus my change which is something added, but not used so far) [10:55] rogpeppe1: will go into trunk and try there too, to get sure [10:55] TheMue: if you get a stack trace from the hanging test (with ctrl-\), what does it print? [10:57] rogpeppe1: one moment [10:58] rogpeppe1: strange [10:59] rogpeppe1: the .\... timed out on the shell and from inside the editor [10:59] rogpeppe1: now i'm only in that package and is passes [10:59] * TheMue shakes his head [11:00] TheMue: how do you know that the worker package is the one that's failing? [11:00] rogpeppe1: *** Test killed: ran too long. [11:00] FAIL launchpad.net/juju-core/worker 600.005s [11:01] rogpeppe1: and then i started the worker tests from inside the editor with the same result (the output above has been on the shell) [11:02] TheMue: can you kill -QUIT the worker tests that are hanging within the editor, please. [11:02] TheMue: so we can find out what they're hanging [11:02] s/what/where/ [11:03] rogpeppe1: btw, just started it in the shell again and it hangs [11:03] rogpeppe1: will paste you the output [11:03] TheMue: thanks [11:04] rogpeppe1: http://paste.ubuntu.com/5738426/ [11:04] rogpeppe1: carmen just called me for lunch, biab [11:05] TheMue: oh dammit [11:05] rogpeppe1: https://codereview.appspot.com/10044043/ - need some help on this please [11:06] TheMue: when you get the chance, please run go test -c; then run the resulting binary and kill that [11:06] TheMue: then we won't be seeing two stack traces interleaved [11:06] dimitern: looking [11:07] dimitern: sorry for not responding earlier - i was in a call [11:07] rogpeppe1: np [11:08] rogpeppe1: i'm really bad at phrasing doc comments and will appreciate suggestions like for the Tagger [11:31] dimitern, wallyworld: hi, hangout? [11:32] ok [11:32] dimitern: responded === danilos_ is now known as danilos [11:32] danilos_: uh, yes, forgot we're doing g+ now [11:32] rogpeppe1: cheers [11:34] dimitern, https://codereview.appspot.com/9876043/ [11:40] rogpeppe1: can we agree to disagree on the apiserver machiner_test? :) [11:41] rogpeppe1: and it's value [11:41] (that it's valueable) [11:42] dimitern: is there anything there that can't be tested just as easily through the client API interface? [11:42] dimitern: i'm not saying the tests themselves aren't valuable [11:42] rogpeppe1: the client-side test is a different one [11:42] rogpeppe1: this is a unit test [11:42] rogpeppe1: the client one is an integration test [11:43] rogpeppe1: and they tests different things [11:43] dimitern: i see the tests in apiserver/machiner and i think that i would like to verify that all of that stuff works from the client all the way to the server [11:43] rogpeppe1: not quite [11:44] dimitern: i think by testing everything through the client, we get the best of both worlds [11:44] rogpeppe1: the client-side doesn't need to care about the correctness of bulk operations: handling parameters and results, as well as permission checks [11:44] dimitern: surely it does [11:45] rogpeppe1: no it doesn't, because the interface as exposed is not about that [11:45] rogpeppe1: permissions yes, but not bulk ops [11:45] dimitern: you mean because we don't expose the bulk operations currently as part of the machiner client API? [11:45] rogpeppe1: and the permissions are already tested at server side [11:45] rogpeppe1: yeah [11:46] dimitern: this feels wrong to me [11:46] dimitern: all the way along we have implemented client functionality that gives us access to the operations supported by the server [11:46] rogpeppe1: client-side tests are about how you use the interface as a client, not so much how the transport layer works [11:46] dimitern: and this is breaking that correspondence [11:47] rogpeppe1: don't you agree unit tests are a good thing? [11:47] dimitern: i think more test coverage is a good thing [11:48] dimitern: and AFAICS by putting the tests on the server side only, we get less test coverage [11:48] dimitern: whereas if we do it at the client side, we get to test the full path almost for free [11:48] rogpeppe1: exactly [11:48] rogpeppe1: and also testing at the right place [11:48] rogpeppe1: no, they are client-side tests in the follow-up [11:49] rogpeppe1: testing end-to-end only is not sufficient imho [11:49] dimitern: really? why not? [11:49] dimitern: what can you test on the server side that you cannot test end to end? [11:49] rogpeppe1: and incurrs the overhead of starting a server when you need only to test the server-part in isolation [11:50] dimitern: starting a server takes microseconds [11:50] rogpeppe1: as i already mentioned, bulk operations, parameters handling, permissions, results [11:50] dimitern: we should have client entry points for the bulk operations. i'm not sure how the rest aren't (or can't) be tested with end-to-end tests [11:51] dimitern: that's how it's all going to be used, after all [11:51] rogpeppe1: we don't need client-side bulk entry points for the machiner yet [11:51] rogpeppe1: but we need to support them at transport level, as agreed [11:51] dimitern: i think we should have them anyway, so we can test them end to end [11:52] dimitern: it's not much code, as fwereade__ keeps on reassuring me [11:52] rogpeppe1: ok, i think we're in a vicious circle here [11:52] rogpeppe1: we need to test both sides [11:52] rogpeppe1, dimitern, damn, I meant to address this before [11:52] rogpeppe1, dimitern, shall we have a quick g+? [11:52] fwereade__: sure [11:52] fwereade__: what? [11:53] fwereade__: ok [11:53] rogpeppe1: http://paste.ubuntu.com/5738483/ <= this one is by calling the test binary [11:53] fwereade__: shall I start one or you will? [11:54] TheMue: thanks. that's more useful. [11:54] dimitern, would you, just finishing an email [11:54] fwereade__: sure [11:54] rogpeppe1: great [11:55] rogpeppe1, fwereade__: https://plus.google.com/hangouts/_/1842432ac5d47d4b006b6679b49cb3d836c8163f?authuser=0&hl=en [12:28] When using the goose Nova binding, is the client expected to provide an imageRef themselves? [12:29] I have a similar set of bindings and I'm at a loss at how to combat the ever-changing list of images. [12:30] *: one CL for review: https://codereview.appspot.com/10078043 === BradCrittenden is now known as bac === gary_poster|away is now known as gary_poster [13:50] rogpeppe3: can you sketch out an idea of how requireMachiner should look like? [13:51] rogpeppe3: not sure about the check "is machine" [13:51] dimitern: http://paste.ubuntu.com/5738805/ [13:52] oops [13:52] dimitern: s/requireAgent/requireMachine/ [13:52] rogpeppe3: ah, ok, so it's simpler than i thought [13:52] dimitern: not too bad, eh? [13:52] rogpeppe3: will add it in machiner.new [13:53] dimitern: cool [13:54] rogpeppe3: indeed [13:55] rogpeppe3: and then there's this: https://codereview.appspot.com/10026044/ [13:56] dimitern: FWIW i think we could just have api.Machine [13:56] dimitern: and have it talk to whatever facade is appropriate [13:57] rogpeppe3: not really, because machiner.Machine and provisioner.Machine, etc. are different [13:57] dimitern: we could quite easily keep them compatible [13:57] rogpeppe3: i think the point of the separation is not making them compatible [13:58] dimitern: ah. i thought the reason was security. [13:58] rogpeppe3: that's the main part of it [13:59] rogpeppe3: not having a method you can call is the best security, isn't it? [13:59] dimitern: on the server side, yes. [13:59] rogpeppe3: and will catch issues like this at compile time === rogpeppe3 is now known as rogpeppe [13:59] oh well.. kanban time [14:00] dimitern: agreed. it depends how much code we're willing to write for that. === wedgwood_away is now known as wedgwood [14:44] time to reboot. this machine is becoming unusable. === BradCrittenden is now known as bac_ [14:58] rogpeppe: so wrt unifying ServerError and ServerErrorToParams [14:59] rogpeppe: the only thing that needs wrapping is the param to conn.Serve in apiserver.go [15:00] dimitern: i'm not sure i understand [15:00] rogpeppe: i'd still rather have separate helpers, but if you insist, I'll create a serverError there locally which will call common.ServerError and converting the result to error [15:00] dimitern: why can't serverError serve both purposes? [15:01] dimitern: you'll need to type-convert the result, but i don't think that's a big issue [15:01] rogpeppe: because conn.Server needs func (error) error [15:01] rogpeppe: and in all other places I need *params.Error, not error [15:02] dimitern: serverError(err).(*params.Error) ? [15:02] rogpeppe: it's not a method call, it's a callback [15:02] rogpeppe: so I need to pass something that takes error, calls common.ServerError internally and returns error [15:03] dimitern: i mean: keep serverError as it is (but make it always return *params.Error for non-nil errors). then do the above if you want a params.Error [15:04] dimitern: then there's no need for two separate error types (api.Error and params.Error) [15:04] dimitern: or for the somewhat awkwardly named ServerErrorToParams [15:04] rogpeppe: it has to be something like: err1 := common.ServerError(err); if err1 != nil { return err1.(error) } [15:05] dimitern: in all the cases you're currently using ServerErrorToParams, you've got that err != nil check anyway [15:05] rogpeppe: just a sec, let me paste it [15:06] Hi folks — quick question if you don't mind. Is this test failure normal? http://paste.ubuntu.com/5738993/ [15:06] It looks a bit like the machine's speed just happens to be barely out of the test's tolerance range or something. [15:06] dimitern: alternatively do as you suggested above: make a local closure in side serveConn [15:07] jtv: i haven't seen that failure before, but that test is quite new [15:08] * rogpeppe hates timing-sensitive tests [15:08] rogpeppe hates with reason. [15:09] jtv: i think the test just needs more wiggle room [15:09] Heh. [15:09] jtv: feel free to propose a change that ups it to 200ms or so [15:09] rogpeppe: http://paste.ubuntu.com/5739005/ something like this [15:09] rogpeppe: I'll try to get other tests running first though... not feeling very lucky today! === racedo` is now known as racedo [15:10] dimitern: that looks ok. i'm not sure i'd bother with the separate func for serverError though - just inline it in the call to conn.Serve [15:11] rogpeppe: that's nasty :) but might do [15:11] dimitern: oh, one other thing: [15:11] rogpeppe: or just make it serverError := func(error) error { ... } [15:11] dimitern: that's fine too [15:12] dimitern: you'll need to check if the error returns an error code [15:13] rogpeppe: why? [15:13] dimitern: and use that error code if o [15:13] so [15:13] rogpeppe: not sure i get you [15:13] rogpeppe: you mean in common.ServerError at the end? [15:13] dimitern: yeah === bac_ is now known as bac [15:14] dimitern: it means that API server code can return coded errors if they want [15:14] rogpeppe: I'll look into it (hopefully some tests will fail if I mess it up :) [15:15] dimitern: good point. i'm not sure that functionality *is* specifically tested. you'll probably want to add a test. [15:16] dimitern: ah, actually that functionality is tested [15:16] rogpeppe: so then it becomes: http://paste.ubuntu.com/5739032/ [15:16] dimitern: and the test will indeed fail [15:16] dimitern: not quite [15:17] rogpeppe: that's a direct translation of what happens when servererrortoparams is removed [15:17] rogpeppe: and integrated into servererror [15:18] dimitern: code will never be non-empty on line 17 [15:18] rogpeppe: how did it work before? [15:19] dimitern: it returned the error unwrapped [15:20] dimitern: i think you want something more like this: http://paste.ubuntu.com/5739044/ [15:21] dimitern: oops [15:21] dimitern: this: http://paste.ubuntu.com/5739046/ [15:21] dimitern: oops, wrong still [15:21] dimitern: better i think: http://paste.ubuntu.com/5739049/ [15:24] rogpeppe: ok, sgtm [15:42] Build failure in trunk as well? ../../../go/src/launchpad.net/juju-core/environs/openstack/provider.go:592: undefined: identity.AuthKeyPair [15:42] I wonder why that didn't affect my testing. [15:45] jtv: just the normal "you need to pull lp:goose" thing? trunk builds for me [15:46] mgz: I haven't been on juju for a while... what is the "normal you-need-to-pull-lp:goose" thing? Mind you this is a completely fresh "go get launchpad.net/juju-core/..." from scratch. [15:48] completely fresh? I'd double check your latest revs in launchpad.net/goose [15:48] because your error strongly suggests you have an older goose [15:48] which won't cook as well [15:49] mgz: yes, completely fresh — cloud instance [15:49] I'm on r95 "Add juju-tools keystone endpoint and fix tests" [15:50] Oh, of course I just discarded my instance. The compile error is on a different system. Sigh. [15:51] mgz: ah... I'm on a much older revision — "go get launchpad.net/goose" didn't update it apparently. [15:52] indeed not [15:53] mgz: apparently the branch URL has changed... how do I fix that? [15:54] hmm, my 2-factor auth is no longer working. any suggestions anyone? [15:54] `bzr pull --remember lp:goose` [15:54] rogpeppe: use your carefully prepared backup [15:54] mgz: the keyfob itself is working fine [15:55] mgz: but i see "The password is invalid" when I try to log in with it [15:55] mgz: hmm, maybe i messed up "Do not activate your authentication device when not needed. Extra activations will get your device out of sync with the server and lock you out of your account." [15:55] mgz: that updated it, thanks! I thought I'd run a "go get -u launchpad.net/juju-core/..." which I assumed would update my dependencies. [15:55] if you only have one device regiestered, and it's desynced, you need to ping webops in #webops [15:55] rogpeppe: ^ [15:56] jtv: go get is not very smart, and bzr recording resolved locations messes it up [15:56] mgz: thanks, i'll try that. hopefully that's the problem. [15:56] mgz: i didn't know the devices were synced. that seems highly fragile to me. [15:57] timebased auth is much nicer [15:58] but this way is simple, and easily fixable if you have a backup (generally just remove and readd the device) [15:58] mgz: yeah. or challenge response can work ok too. [15:58] mgz: readd ? [15:58] that's not really yubikey accessible though [15:59] ^re-add [15:59] generate a new seed, register device with it, start on fresh sequence [16:00] mgz: ah i see [16:00] mgz: if i'd had any idea that pressing the button one two many times would be a problem, i would have been a bit more careful [16:00] s/two/too/ [16:02] mgz: are you sure #webops is the place? perhaps #admin or something might be better? [16:03] rogpeppe, web0ps do 2fa resets, and they hangout in #web0ps [16:04] gnuoy: tis with "0" (zero) or upper-case "O"? [16:05] dimitern, sorry I am in the habit of not typing the word as it causes them (and I am one) to be pinged [16:05] gnuoy: so web0ps is the magic word, good to know :) [16:06] dimitern, sorry, no. It is webops and #webops [16:06] gnuoy: ah, ok [16:06] (on the internal server, not freenode, too) [16:06] gnuoy: now i got you :) sorry [16:07] dimitern, np, sorry for causing the confusion :) [16:09] jtv: that's a very good skeleton you have there [16:09] a second review of https://codereview.appspot.com/9698047/ would be good, it's almost trivial [16:09] jtv: will you develop most of it like the maas provider? [16:09] fwereade__: i'm on it [16:10] dimitern: so far it's been _exactly_ like the maas provider. :) [16:10] TheMue, ping === BradCrittenden is now known as bac [16:14] jtv: :) i meant in a separate branch [16:15] dimitern: ah, no, we won't be doing that this time. Too much integration pain. [16:15] That's why I put this up for review now. [16:15] * jtv → eod [16:16] jtv: ah ok [16:18] rogpeppe: https://codereview.appspot.com/10044043/ all done I think [16:20] rogpeppe: if you think it's worth an lgtm now i'd like to land it [16:20] dimitern: i'm looking now [16:33] dimitern: reviewed [16:33] danilos, ping [16:34] rogpeppe: cheers [16:37] rogpeppe: are you sure you want to leak info by reporting CodeNotFound when id != "" and we haven't checked the user is authorized? [16:38] dimitern: i don't think i'm suggesting that. i'm suggesting returning errBadId in that case [16:38] rogpeppe: istm it good to always first check for authorization [16:38] dimitern: which doesn't leak any info at all [16:38] rogpeppe: hmm.. [16:38] rogpeppe: ok, i'll go with this, but seems fishy [16:39] dimitern: when/if we start using the ids in this case, we will check first [16:39] dimitern: in that case, we'll probably pass the id into machiner.New [16:39] rogpeppe: yeah, fair enough [16:47] rogpeppe: if we lose the ErrNotLoggedIn returned from RequireMachiner, we'll need to have IsLoggedIn() bool in the Authorizer and check that separately [16:48] dimitern: hmm, i wonder if a simple ErrPerm would be good enough in that case [16:48] rogpeppe: that'll change the login_test [16:48] rogpeppe: and complicate things [16:49] rogpeppe: so I prefer to have a separate check in root.Machiner() to return ErrNotLoggedIn as well as now [16:49] dimitern: that seems reasonable actually [16:49] rogpeppe: ok [16:50] dimitern: then it's easy to check that all the root methods check for logged-in-edness [16:50] dimitern: and it's up to the individual facades to decide on further checks [16:50] rogpeppe: exactly === tasdomas is now known as tasdomas_afk [17:21] rogpeppe: last time for today, i promise :) https://codereview.appspot.com/10026044/ - you already reviewed it once, I fixed it as suggested, and the version arg is gone now [17:22] dimitern: looking [17:22] dimitern: oh darn, you've merged trunk [17:22] rogpeppe: what? [17:22] dimitern: i'm seeing lots more diffs than i should [17:23] dimitern: between patchset 4 and 5 [17:23] rogpeppe: when i proposed it the diff was screwed, so i repropsed [17:24] rogpeppe: i haven't merged trunk [17:24] dimitern: ah, sorry, i thought it was the other CL again! [17:24] rogpeppe: np [17:28] dimitern: can't we just delete the id argument to api.State.Machiner, as we discussed? [17:29] dimitern: reviewed [17:32] rogpeppe: not unless we need to test it [17:33] rogpeppe: and i'd like to have a test for id != "" [17:33] dimitern: you could use Call directly for that [17:33] dimitern: that's the kind of thing it's there for [17:33] rogpeppe: that's seems wrong [17:34] dimitern: it seems wrong to me to clutter the user-facing client API with meaningless parameters [17:34] dimitern: you could put another method in export_test.go if you don't want to use Call [17:34] rogpeppe: well, ok, it's a compromise I can live with for now :) [17:34] dimitern: in fact there are quite a few methods we should probably test with non-nil ids (the Client entry points, for example) [17:35] rogpeppe: agreed [17:35] dimitern: so a table-driven test using Call could work ok there [17:35] rogpeppe: in the apiserver you mean? [17:36] dimitern: i was thinking in the api client, but you could do server tests instead, yes [17:37] rogpeppe: that'll be better I think [17:37] rogpeppe: so that we don't need to expose them all the way to the client [17:38] dimitern: if you use Call, you wouldn't have to [17:38] dimitern: but i'm ok either way. this is trivial functionality and not actually important in any way that i can really think of. [17:39] rogpeppe: I can add a state_test.go in the api client and test with call [17:39] rogpeppe: then just drop it and add a todo instead? [17:39] rogpeppe: to do it in a follow-up [17:39] dimitern: seems ok to me [17:41] rogpeppe: ok then [17:54] I'm off to pack [17:54] good night all! [17:59] why do I need to add the "local:" prefix when deploying a charm from my disk? Isn't --repository /foo/bar/ specific enough? [18:49] andreas__, --repository just overrides the value of $JUJU_REPOSITORY, which might always be set, but doesn't always imply "look in a local repo" unless specified via the charm schema [18:51] andreas__, I see your point, but I think it's a bit cleaner to have --repository and JUJU_REPOSITORY match in usage as well as meaning, rather than layer on extra cleverness in --repository that doesn't really fir with the env var [19:10] fwereade__: --repository /foo/bar charm-name has surprising results [19:18] g'night all === BradCrittenden is now known as bac [21:10] morning [21:12] morning [21:12] /afternoon [21:12] :) [21:35] * thumper is now heads up after email reading [21:35] hmm... coffee [22:35] thumper, want to chat about the provisioner briefly? [22:46] thumper, short version: yes, the machines watcher can be expected to deliver an initial event that represents the complete set of known machines [22:48] thumper, subsequent events delivered are relative to that base "change" [22:48] thumper, if any machine enters the set, you'll be informed [22:48] thumper, if any machine in the set becomes dying, you'll be informed [22:49] thumper, if any machine in the set is removed *or becomes Dead*, you'll be informed, and will receive no further notifications for that id [22:50] thumper, calling AllMachines just raises questions of raciness that are *probably* ok but not trivial to analyse [22:53] thumper, and it seems unnecessary since we have this type sitting there that's designed to send precisely the information necessary to maintain a consistent view of the set of machines in play over the lifetime of the watcher === wedgwood is now known as wedgwood_away [23:07] fwereade__: oh hai [23:07] fwereade__: was taking the dog for a walk [23:09] fwereade__: given this knowledge then, I'll refactor a little... [23:09] thumper, nice weather for it? === wedgwood_away is now known as wedgwood [23:14] fwereade__: is actually, sunny and cool (well cold) [23:14] but nice [23:15] given what you have said about the watcher [23:15] I think we can get rid of the "all machines" bit [23:15] thumper, I'm starting to think wistfully of that sort of wather [23:15] thumper, cool, I'm glad that makes sense [23:15] thumper, I think it's a strict simplification [23:15] yeah [23:15] shouldn't be too hard [23:15] rogpeppe had a lot of good changes for loggo too [23:16] but includes breaking API [23:16] thumper, awesome [23:16] thumper, bah [23:16] so would need to land branches in parallel [23:16] because we don't have library dependencies :) [23:16] * fwereade__ sighs wistfully [23:17] perhaps soon [23:17] maybe === wedgwood is now known as wedgwood_away [23:23] fwereade__: i was going top ask you about the machineContainer stuff but it's very late for you [23:23] wallyworld, I responded to your CL I think, saying essentially machineContainer is good but quibble quibble quibble bikeshed bikeshed [23:24] :) [23:24] fwereade__: yeah, saw that thanks. question though - if we have fast regex queries, then there's really no need for the children slice [23:24] wallyworld, I've realised there is, I think [23:25] ok. i notice you also nixed by surrogate key :-) [23:25] wallyworld, when we're able to watch an index of documents we care about, in a single document, we get to avoid watching a whole collection [23:25] wallyworld, grep for globalKey in state [23:26] ok [23:26] wallyworld, no objection to the concept, but would prefer to maintain convention when context does not specifically indicate otherwise [23:26] davechaney around? [23:26] thanks for the input. will push up something today [23:27] fwereade__: also, i'll think about the txrevno - i do think it is required since the documwent has to be loaded, modified, and saved again [23:27] and that is not atomic [23:28] i think settings uses it for the same reason [23:29] thumper: you ok to +1 that clean flag branch? [23:29] wallyworld, the model is more for a client to make exactly the document change it needs, asserting only the minimal conditions that must pertain to maintain consistency in state [23:29] wallyworld, settings is crazy [23:30] wallyworld: I can take a look [23:30] fwereade__: ok. i'll have to think about it some more. i can't tight now see how we can do it without the possibility of two threads adding a container and stomping on each other [23:31] thumper: i didn't change anything, just answered your question [23:31] wallyworld: I'd have to page in the information, so can't +1 just yet [23:31] sure [23:31] hence needing to look at it again :) [23:31] my brain holds only so much in active state :) [23:34] wallyworld, so, briefly, there are mongo ops like $addToSet that we can use in transactions [23:35] ok, will look into that [23:35] thanks [23:47] wallyworld, yeah, $addToSet and $pull are used on the Principals and Subordinates fields that this is exactly analogous to [23:48] fwereade__: i also have the issue that the machineContainers doc might not exist yet [23:48] so i need to create sometimes and update via $addToSet other times [23:49] so there's still a potential race condition [23:50] with Principals, the doc always exists [23:50] wallyworld: I'm out for a bit, back hopefully in 30-40 minutes [23:50] wallyworld, for simplicity you could just create it alongside the machine doc, earlier in the ops list, and remove it later in its removal ops, such that the existence of a machine document always implies the existence of the containers doc? === thumper is now known as thumper-afk [23:51] fwereade__: thought about that but seems very wasteful since the machine may not even have a container added and we may hav lots of machines? [23:51] wallyworld, if you ever get a mongo NotFound from such an ancillary document, you can infallibly infer that the parent document was removed [23:52] wallyworld, they're tiny documents [23:52] ok. i'll create one regardless when a machine is created. it feels very dirty to me to have to do that though [23:55] wallyworld, I infer that given http://docs.mongodb.org/manual/faq/developers/#how-do-i-optimize-storage-use-for-small-documents -- in which it recommends the possibility of noticeably reducing overhead by using shorter field names -- that we're not going to be paying a significantly greater storage cost by moving som of those field names to a distinct document with a clear purpose [23:56] ok. thanks. i'll read that faq also [23:56] wallyworld, I would very much prefer not to go down the route of mangling our field names for the db's convenience until I get evidence that it's a source of trouble for us [23:57] ;p [23:57] i never suggested we mangle field names i don't think [23:57] wallyworld, no, I wasn't trying to imply you were [23:58] ah ok :-) [23:59] wallyworld, I'm saying that if the few bytes per document to store the string keys, which will indeed be somewhat repetitive, are a significant enough factor to recommend tweaking, the additional db-imposed overhead is going to be of a similar order of magnitude for that sort of change to be effective