/srv/irclogs.ubuntu.com/2013/06/26/#juju-dev.txt

wallyworld_thumper: TestSimple also does it's own cleanup. i think it's just to check that the EnsureDead/Stop etc methods work00:00
thumperok00:01
bigjoolswallyworld_: is the openstack provider configuring a cloud-drive ?01:33
bigjoolsor config drive01:33
wallyworld_no01:33
wallyworld_the instance just uses the storage attached to the instance01:33
bigjoolswhat format is that storage?01:34
wallyworld_it depends on the instance01:34
bigjoolsI got pointed at this: http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/sources/configdrive/README.rst01:34
wallyworld_larger instances use emphmeral disks01:34
wallyworld_perhaps that's a feature of cloud-init the provider doesn't yet support01:35
wallyworld_not that i am aware of anyway01:36
bigjoolshow do you get your user data into openstack?01:36
wallyworld_magic01:36
bigjoolscan you wave your wand on my azure stuff01:36
wallyworld_i think it is passed via cloud-init somehow, but need to check that01:36
bigjoolsyes, no shit :)01:36
* wallyworld_ waves his stick01:37
wallyworld_sorry, not sure of the exact mechanism01:37
wallyworld_it's been ages since i looked at that code01:37
bigjoolsmy question is how does cloud-init get it.  normally you set a url for the kernel and the userdata is in provider storage at that url01:37
bigjoolsbut we can't set kernel params in azure01:37
bigjoolshence the cloud-drive thing - which I thought openstack used as well01:38
wallyworld_bigjools: justed checked - openstack uses params passed when starting an instance to hold the user data01:39
wallyworld_there's a userdata field in the struct01:39
bigjoolsah so it's direct01:39
wallyworld_so it's not via cloud-init01:39
bigjoolscloud-init picks them up eventually01:39
wallyworld_server, err = e.nova().RunServer(nova.RunServerOpts{01:40
wallyworld_Name:               e.machineFullName(scfg.machineId),01:40
wallyworld_FlavorId:           spec.InstanceType.Id,01:40
wallyworld_ImageId:            spec.Image.Id,01:40
wallyworld_UserData:           userData,01:40
wallyworld_SecurityGroupNames: groupNames,01:40
wallyworld_})01:40
wallyworld_i'm not sure of the internals01:40
bigjoolsyou have a userData() func01:40
wallyworld_yes, but i'm not sure how the instance works internally01:41
bigjoolswhich pretty much answers my question now01:41
bigjoolsit renders to []byte01:41
bigjoolsI need to render to a "clouddrive"01:42
bigjoolsconfigdrive, even01:42
wallyworld_wave your wand and it will happen01:43
thumperoh FFS!!!!02:09
thumperwhy do we insist on making testing things so fucking hard?02:10
* bigjools wonders why there's two cloudinit packages02:12
bigjoolsthumper: if you want to take a break from fucking hard, can I have a pre-imp for something that is only moderately hard02:17
thumperbigjools sure02:25
bigjoolsthumper: gimme 2 mins and I'll call you02:26
* bigjools prods thumper to answer02:29
=== tasdomas_afk is now known as tasdomas
davecheney 05:02
davecheneyGoogle Drive05:02
davecheneyThe app is currently unreachable05:02
davecheneyshitter05:02
wallyworld_thumper: for your todo list (tomorrow maybe) https://codereview.appspot.com/10534043/ and https://codereview.appspot.com/10447045/05:02
wallyworld_davecheney: they must have used some microsoft code in there somewhere05:03
davecheneywallyworld_: http://instantrimshot.com/classic/?sound=rimshot05:04
* wallyworld_ laughs05:05
jamthumper: poke05:30
jamI'm trying to sort out what an Envorin is, and why he knows a guy named 'eric' very intimately. (testing/environ_test.go)05:31
jamI guess the goal is to just have a known home to compare against. Though we should probably s/Envorin/Environ/05:32
jamrogpeppe: I just found that "go1.1" is a build constraint. If you wanted to write your Sliceof code in 2 modules and have a '// +build go1.1' and a '// +build !go1.1'06:23
jamI don't think it is worth divergence just yet06:23
jambut we do have a mechanism to take advantage of features when available06:24
=== tasdomas is now known as tasdomas_afk
=== tasdomas_afk is now known as tasdomas
rogpeppejam: that doesn't really solve my problem - my workaround for the SliceOf code is to have every module register the slice types - the go1.1 constraint would poison lots of packages.07:24
jamsure07:25
rogpeppejam: mornin' BTW!07:25
jamI just hadn't seen it before07:25
jammorning07:25
rogpeppei've had this out for review all week and i've still only had one review of it. if i'm to get the API stuff in, i'd really like another review, please! https://codereview.appspot.com/10259049/07:28
rogpeppeto whomsoever it may concern :-)07:28
jamrogpeppe: to be honest, I've gone to look at it at least 3 times, it is just big enough and involves enough stuff I have trouble consuming it. I will certainly try again.07:29
rogpeppejam: sorry about that - there are probably a couple of trivial things that could be split off (the params.Life definitions for example) but i couldn't think of a way of splitting up the main change to agent.go and machine.go07:31
jamrogpeppe: probably a lot of it is my unfamiliarity with parts of the internals, so I can't always grasp *why* a change is being made until I dig into it for a bit.07:32
rogpeppejam: well, it would be nice for the future if you acquire some familiarity with this part of the code, so please feel free to ask anything about what's going on - we could have chat about it if you want.07:33
jamrogpeppe: so why did you need to add the worker.Worker.Kill implementation for the api server? It wasn't a worker before but should be? It was just useful for testing?07:33
rogpeppejam: it wasn't a worker before but now it is - rather than having explicit run loops in three places, the CL changes things to use worker.Runner in more places07:34
rogpeppejam: so in the machiner we've got the top level Runner that runs two workers; one of them runs any tasks that connect to the state; the other runs any tasks that connect to the API. each of them creates a runner themselves to run the tasks (so the runners are nested)07:35
jamrogpeppe: and the presumably we iterate until one of those isn't necessary.07:36
jamHowever, I would have thought that we would have tasks that couldn't quite be all API while we transition07:36
jamso we could implement *an* API call and start using it07:36
jamrather than having to finish off "everything a task might possibly need" before it can start using API.07:36
jamToo much chance of split brain?07:37
rogpeppejam: i thought that too originally, but i think the idea is to transition an entire agent at a time07:37
rogpeppes/agent/worker07:37
jamrogpeppe: Workers returns a worker that runs the unit agent workers. ???07:37
jamwhy is the method plural, but the result singular07:37
jamAnd the doc string doesn't seem to explain much.07:38
rogpeppejam: which method?07:38
jamrogpeppe: https://codereview.appspot.com/10259049/patch/5001/600607:38
rogpeppejam: ah, in unit.go07:38
jamIf it 'runs' it, I would have thought it would be a Runner07:38
jambut a Runner is also a Worker?07:39
rogpeppejam: the unit agent has two levels of runner too07:39
rogpeppejam: yes07:39
rogpeppejam: that's an important point07:39
jamFor the docstring, I think it is trivial to see that Workers returns a worker, so focusing more about what the thing it returns will be used for might be helpful07:40
rogpeppejam: the Workers method is probably badly named07:40
jamI also wouldn't make the method plural when it returns a single object, even if that object wraps multiple others.07:40
rogpeppejam: yeah, it's kind of a single object that encapsulates all the workers, but i agree07:41
jamrogpeppe: MetaWorker ?07:41
rogpeppejam: TopLevelWorker?07:42
rogpeppejam: StateConnectingWorkerRunningOtherWorkers ?07:42
jam// MetaWorker returns the Worker responsible for managing all individial workers that this Agent needs.07:42
jamJust calling it Worker would eliminate the confusion on the returned type not being a slice07:42
jamrogpeppe: so the thing you return isn't actually a runner, it is a newCloseWorker() which wraps a runner.07:43
rogpeppejam: StateWorker might be good enough07:43
rogpeppejam: yeah07:43
jamrogpeppe: so do Runners actually implement the Worker api?07:44
rogpeppejam: that's so we close the state connection eventually07:44
rogpeppejam: yes07:44
rogpeppejam: (all it requires is Kill and Wait)07:44
jamrogpeppe: why do you start the runner in cmd/jujud/machine/Init() rather than in Run07:47
rogpeppejam: that's so the tests can kill it07:47
jamrogpeppe: I can see that as a reason to have it as an attribute, but Init seems more like setup stuff-without-starting anything, no?07:48
rogpeppejam: because we're not using our own tomb now - we're leaching off the Runner's liveness07:48
rogpeppejam: yeah, i take your point.07:49
rogpeppejam: even though the runner is empty, calling Init without Run will leak07:49
jamrogpeppe: so you already have a method called StateWorker in MachineAgent code. Though perhaps the functionality is similiar enough that name sharing is a good thing?07:51
rogpeppejam: yes, that's what i was thinking07:52
rogpeppejam: and when we transition the unit agent, we'll probably have an APIWorker *and* a StateWorker in there until we can finally delete the StateWorker07:53
jamrogpeppe: I would *really* like to call this thing the StateRunner instead, though.07:53
jamThat makes it clear the thing isn't actually doing anything on its own.07:53
rogpeppejam: yeah, seems like a good idea07:53
jamBut it is managing all the tasks that want access to State.07:54
jamrogpeppe: It also seems odd that APIWorker (maybe called APIRunner) has a side effect of starting the StateRunner.07:55
jamRather than a sepacate07:55
jamMachineAgent.StartAPIRunner() and MachineAgent.MaybeStartStateRunner ?07:55
jamrogpeppe: naming aside, calling agent.APIFoo() and getting a runner, but having a side effect of mutating agent.runner seems a bad idea.07:56
rogpeppejam: unfortunately it needs to happen like that, because in general we can't start the state worker until we've connected to the API07:56
jamcertainly when reading the code it wasn't immediately obvious that "a.runner" isn't the same thing as the "runner" you are building up.07:57
jamrogpeppe: you can check and refuse if the API side isn't running yet07:57
jamand callers need to call them sequentially.07:57
jamor we have a StartRunners call that does both07:57
dimiternmorning all!07:57
jamwelcome back dimitern07:57
rogpeppedimitern: yo!07:57
dimiternthanks! :)07:58
rogpeppejam: we can't even know if we *need* to run a state server until we've connected to the API07:58
rogpeppes/state server/state worker/07:58
rogpeppejam: so there has to be some communication between the two07:58
rogpeppejam: i know it might seem awkward, but i haven't been able to think of a more elegant way of solving the problem07:59
jamrogpeppe: again, you can always start the state worker after the API. I'm objecting to having a side effect of "create this runner for me", also starting and forcibly instantiating another runner that isn't actually the one I asked for.07:59
jamrogpeppe: I'm just asking to move the code into another function that more clearly signals that it is potentially starting 2 things.08:00
jamrogpeppe: If I'm understanding the code, the block from "m := entity.(*machineagent.Machine)" until the end of if needsStateWorker could be done after you return the newCloseWorker08:01
jamwhich means it could be in a different function.08:01
jamyou need the 'entity', but you could expose that.08:01
rogpeppejam: i'm not sure it can be08:02
rogpeppejam: in the future, the information on how to connect to the state will be held in the API08:02
rogpeppejam: so we'll have to pass some information that we've obtained from the API into StateWorker08:03
jamrogpeppe: so after we have the API runner up, we ask the API if we also need to start the StateWorker.08:03
rogpeppejam: yes08:03
rogpeppejam: that's what we do08:03
jamrogpeppe: The point is you have a function labeled "Give me an APIWorker", that should be straightforward. But it has this side effect of maybe starting a StateWorker08:03
jamwhich is nowhere in the contract we've defined08:03
jamit doesn't even activate the APIWorker (doesn't add it to a.runner)08:04
jamso it seems doubly strange that it would add a different worker to a.runner.08:04
rogpeppejam: unfortunately one of the things we have to do after connecting to the API is maybe start a new state worker08:04
jamrogpeppe: so I'm just asking you to have a function called "StartWorkers" which connects to the API, creates an APIWorker, and then checks to see if it needs to start a state worker (and does so)08:04
rogpeppejam: how can it do that? - it won't have an API connection08:05
jamrogpeppe: you just created the API connection08:05
jamhow does it not have one?08:05
rogpeppejam: creating an API worker doesn't create the API connection until some time later08:06
jamrogpeppe: then how is your code doing it today?08:06
rogpeppejam: the only time we know we have an API connection is inside APIWorker itself08:06
jamif it doesn't have the connection?08:06
rogpeppejam: so you'd be happy if StartWorkers was called from within APIWorker?08:07
jamrogpeppe: wrong way around08:07
rogpeppejam: but if StartWorkers adds APIWorker to the top level runner, how can it get access to the API connection that only APIWorker has access to?08:08
rogpeppejam: perhaps i might understand better what you're suggesting if you paste some pseudocode08:10
jamrogpeppe: rough sketch: http://paste.ubuntu.com/5800860/08:10
jamrogpeppe: yeah, I was working on that08:11
rogpeppejam: what calls StartWorkers?08:11
jamrogpeppe: right now it is an immediate replacement for whatever called APIWorker08:12
rogpeppejam: currently APIWorker is called by the top level Runner08:14
rogpeppejam: which is important because it gives us our top level run loop (retrying on failure)08:14
jamrogpeppe: so MachineAgent is also a Worker?08:15
rogpeppejam: no08:16
rogpeppejam: machine.go:87,9008:16
jamrogpeppe: is there a reason we couldn't just have Workers() here, which returns a slice of Workers, which might be 1 for just the API or 2 for both the API and State worker if we find we need it.08:17
rogpeppejam: that's how APIWorker gets called08:17
jamThen we don't have to touch a.runner08:17
rogpeppejam: shall we G+ this - it seems like a bit of higher bandwidth might be useful08:18
rogpeppejam: i'm not sure what you mean with your slice suggestion. are you suggesting that MachineAgent.Run calls Workers directly?08:20
jamrogpeppe: just a sec, digesting a bit.08:21
rogpeppejam: the difficulty is that eventually we *do* need two independent workers in the top level a.runner, but we only know whether to run the state worker *after* the first API connection has been made, which happens to be exactly what the other runner has to do as a first step. i suppose we could have another worker function which connects to the API and whose sole purpose is to add the other runners to the top-level runner08:24
rogpeppejam: but that's more code and not necessarily any clearer08:24
rogpeppejam: and there's also the point that at some point we might have to *remove* the state worker (if the machine jobs change), which will probably require the API worker to mess with the top-level runner in a similar way08:26
dimiternjam: so now with the go-bot do I need to pull from a different location? bzr pull --overwrite --remember lp:juju-core ?08:28
rogpeppejam: would something like this help matters? http://paste.ubuntu.com/5800883/08:29
rogpeppedimitern:  if you have any existing proposals, they need to be reproposed08:30
rogpeppedimitern: but the pull location is still the same08:30
dimiternrogpeppe: no, save for the one mgz_ took over08:30
rogpeppedimitern: did you have a good holiday, BTW?08:30
dimiternrogpeppe: oh yeah, relaxed - just what I needed08:31
rogpeppedimitern: are you officially back now? (i thought you were back tomorrow)08:31
jamdimitern: it depends how you got your branch. If you got it from 'bzr branch lp:juju-core' that still points at the right branch, if you use 'go get launchpad.net/juju-core' that points to the wrong one (because it pre-resolves the full branch URL). you can always just do it and not worry, because it won't be the wrong thing to do (bzr pull --remember lp:juju-core)08:31
dimiternrogpeppe: i'm back today08:31
rogpeppedimitern: cool!08:31
rogpeppedimitern: we get a day and a half of overlap before i'm away...08:32
jamrogpeppe: thinking a different way, what do we gain by not starting the StateWorker ?08:32
dimiternrogpeppe: yeah08:32
dimiternjam: will bzr info tell me that?08:32
rogpeppejam: if we do start the StateWorker, how does it know how to connect to the state?08:32
jamrogpeppe: the fundamental bit for me is that it feels ugly to have APIWorker (create an api worker for me to start tasks on) have a side effect of touching the top level runner. We can manage the distrust by having 'Here be dragons' comments, but I'm trying to sort out if we can decouple it.08:33
jamdimitern: it can08:33
rogpeppejam: i definitely see your point, but i *think* this is a fundamental causality issue08:33
rogpeppejam: only when we connect to the API can we work out if (and how) to start the state worker08:34
rogpeppejam: and the APIWorker is the only place that connects to the API08:34
jamrogpeppe: well today we can just use a.Conf.Conf as you are already doing, no?08:34
rogpeppejam: currently we can, but in the future we won't be able to08:34
jamAnd when we are at the point where API knows better how to connect to State, won't we be at the point where we refuse to let anything but the API server itself connect to state?08:34
jamand the API server won't be connecting to itself.08:35
dimiternjam: i have this: http://paste.ubuntu.com/5800889/08:35
jamdimitern: parent branch: bzr+ssh://bazaar.launchpad.net/+branch/juju-core/08:35
jamthat is always "lp:juju-core" even when we point that at a different branch08:35
rogpeppejam: in the future, we will only know whether to run an API server by connecting to an API server08:35
dimiternjam: so it's fine then?08:35
jamdimitern: 'go get' would end with: parent branch: http://launchpad.net/~juju/juju-core/trunk/ (which would be the old wrong branch)08:36
jamdimitern: yes, you are fine08:36
rogpeppejam: i do indeed hope that we get to the stage where the API server is the only thing that needs a state connection08:36
dimiternjam: cheers08:36
jamdimitern: note though that juju-core now has a bunch more external dependencies08:36
jamlp:gwacl and lp:golxc come to mind08:36
jamwhich also needs 'apt-get install libcurl4-openssl-dev'08:36
jamand a branch from github08:37
rogpeppe[09:35:02] <jam> and the API server won't be connecting to itself.08:37
jamthough 'go get launchpad.net/juju-core' will sort some of that out for you08:37
rogpeppejam: it will be connecting to other instances of itself08:37
dimiternjam: yeah, just discovered these and i'm go getting them08:37
jamrogpeppe: how does the first one start?08:37
rogpeppejam: bootstrap08:37
rogpeppejam: that's the "0" case at the top of the MachineAgent.Run08:37
rogpeppejam: that's the only time we can't first connect to an API server.08:38
jamrogpeppe: in which case, I would offer that the task which actually runs an API server should be the thing starting the StateWorker and not APIWorker08:38
rogpeppejam: the StateWorker *is* the thing that runs an API server08:39
jamanyway, I don't really want to argue it for too long, but having a "Create me one of these" have a side effect of creating a second one and mutating the state of the caller is hard to keep track off and should be well guarged.08:39
rogpeppejam: i agree with your discomfort, but i haven't seen a decent alternative yet08:40
rvbaHi jam, I replied to your comment on lp https://code.launchpad.net/~rvb/juju-core/az-public-storage/+merge/171251/comments/38223608:40
jamrogpeppe: the ordering *could* be, "connect to the API, poll it for the jobs to run, one of those jobs will start up, and ask the agent to start another worker, which then gets a task to run the API server"08:40
jamrvba: I didn't see anything from jtv on the MP, sorry if I missed it.08:42
rogpeppejam: so rather than adding to the MachineAgent runner directly, we start another worker that does the same thing?08:42
jamrvba: to be fair, I still don't see anything looking back again.08:42
rvbajam: https://codereview.appspot.com/10541044/08:42
jamrvba: ugh, split brain between LP and Reitveld. Reitveld mirrors most comments back into Launchpad, except when the Reitveld identity isn't known to LP, then those messages just get dropped by LP08:43
jamrvba: if you look here: https://code.launchpad.net/~rvb/juju-core/az-public-storage/+merge/17125108:43
jamit *also* means that JTV's message isn't in my email folder08:43
jambecause Rietveld only sends messages for things which you've already commented on.08:43
jamsorry about that.08:43
rogpeppejam: in the end, starting the state worker *has* to be a consequence of connecting to the API. hmm, one possibility to make things easier to understand:08:44
jamis it possible for jtv to register his alternative email with LP?08:44
rogpeppejam: we could explicitly pass the top level runner into the APIWorker08:44
rogpeppejam: so that it's more obvious that APIWorker can control it08:44
rogpeppejam: would that be better for you?08:45
jtvOh hi guys...  I'll try to register that email then.08:47
rogpeppejam: something like this: http://paste.ubuntu.com/5800919/08:50
jamrogpeppe: I would be happy if the api was "add yourself to my runner", and then we just end up adding 2 things to the runner. rather than "create something and I'll take care of registering it" which then has a side effect of creating and registering an "unrelated" thing.08:50
dimiternmgz_: ping08:51
rogpeppejam: the second thing has to be added to the runner as a side-effect of running the first thing. unless we have some entirely different code which connects to the API for the first time, starts *another* worker that connects to the API and a state worker if appropriate.08:51
dimiternfwereade: ping08:52
jamrvba: I just sent an email to juju-dev about the config naming question. I actually prefer your method, but we were explicitly asked to do the common-names thing. Which is why I proxied it to you.08:52
fwereadedimitern, pong08:53
jamrogpeppe: the issue is that APIWorker isn't *running* the worker yet, right?08:53
fwereadedimitern, welcome back :)08:53
dimiternfwereade: thanks :)08:53
rogpeppejam: which worker?08:53
dimiternfwereade: i was thinking to pick up the deployer API stuff08:53
jamit seems odd to have the StateWorker registered with the top level runner before the API Worker has been registered08:53
jamcalling APIWorker() creates a worker object08:53
jambut hasn't added it to the topLevelRunner yet08:53
rvbajam: okay, thanks for starting the discussion.08:53
dimiternfwereade: as agreed before, if mgz_ haven't started on it08:53
fwereadedimitern, +1but speak to danilos -- he's about to go away but hasn't yet and has, AIUI, been looking into it08:54
fwereadedimitern, I'm not up to date on where he is with it though08:54
jamdimitern: right, danilos has started to do some of the infrastructure.08:54
rogpeppejam: erm, i don't quite understand. you don't actually add workers to runners, you add a function that starts a worker.08:54
jamand has 3 days of overlap with you to hand it off.08:54
dimiternfwereade, jam: ok08:54
dimiterndanilos__: hey08:54
rogpeppejam: the function that calls APIWorker is added to the top level runner immediately08:55
rogpeppejam: machine.go:8708:56
dimiternjam: i saw the mail about the objectives - what's the deadline for that?08:57
jamrogpeppe: so we have some more naming confusion.08:57
jamStartWorker doesn't start anything08:58
jamit registers something that you'll want to start later, right?08:58
rogpeppejam: no, it will call the function immediately08:58
jamdimitern: officially the end of this week. With official recognition that it is likely we'll miss it by a bit.08:58
jamdimitern: though it doesn't have to take super long. You can do it in an hour or so. You have a lot of other people's objectives you can crib from.08:59
dimiternjam: which ones should I take as a template - your or tim's ?08:59
rogpeppejam: the slight tension is that the name StartWorker implies only a single worker, but actually there's a sequential succession of workers08:59
rogpeppejam: each one started some time after the last one has quit, assuming it didn't quit with a fatal error09:00
jamdimitern: actually, you're officially under Tim now09:00
jamaccording to directory.canonical.com09:00
jamI wash my hands of you :)09:00
dimiternjam: ok :) \o/09:00
jamrvba: sorry if my comments came across as a bit attacking. It certainly wasn't meant that way. I only commented because of the naming thing, and then I was surprised about the 1 review bit.09:01
rvbajam: no worries ;)09:02
jamrogpeppe: so it feels like the 'right' fix is to change the StartWorkers api so that the function you call can optionally return more workers that you might want to start. However, your last paste seems reasonable enough for now, and has sufficient Here Be Dragons to avoid people getting lost as to why it is happening.09:03
rogpeppejam: thanks. i'm not sure how changing the StartWorker API would help really. it seems like it would make the Runner API more complex for no particular gain. if you want another worker, just adding it to the runner directly seems like a reasonable way to do it.09:08
fwereadeTheMue, config-get --all LGTM09:08
jamrogpeppe: runner.StartWorkers(..., func()) it doesn't feel like that func() should know what runner it is being attached to, so that it can be arbitrarily called by some other runner.09:08
jamrogpeppe: it is a 'do we have a singleton per process', imagine writing tests cases for this09:09
jamthat want to start their own runner, and add this thing to them.09:09
jamthat should get added to whatever runner they created09:09
jamrogpeppe: anyway, the bit you wrote is 'ok', it doesn't feel 'right', but it is acceptable09:09
jamyour pastebin does at least let you set which one you want it to add any future work to.09:09
rogpeppejam: yeah, it's pragmatic code - it's not beautifully regular or modular, but it encapsulates the task in hand and the scope is limited09:10
thumpernight all09:11
jamrogpeppe: yeah, I think 1 exception is ok when well documented, if we end up with 2 we should wonder, and if we have 5 exceptions, then we probably have the design wrong.09:12
rogpeppejam: when we do multi-tenant, i think we will probably be doing a lot of runner-manipulating - there will probably be a worker who's sole responsibility is to add and remove other workers from the API runner.09:14
rogpeppejam: at least, that's my wand-wavy plan09:14
TheMuefwereade: thx09:19
TheMuefwereade: and also thx for the better doc ;)09:20
rogpeppejam: this is, i think, something more like you were suggesting. i don't think it's a great improvement: http://paste.ubuntu.com/5800976/09:23
jamrogpeppe: I'm pretty sure line 32 is a.APIWorker09:24
rogpeppejam: indeed it is09:25
jamrogpeppe: so the idea is you don't need the for{} in your original code because topLevelRunner handles that?09:26
rogpeppejam: yes09:26
jamrogpeppe: I'll also note in your current proposal of "how do we know what to connect to", there is no actual connection of a.StateWorker() to anything the API returns09:30
rogpeppejam: not currently09:30
jamthough I realize you want to put something there.09:30
rogpeppejam: eventually the state server addresses will be accessible through the API09:31
jamrogpeppe: I think changing the signature of StartWorker such that the function you pass in can take the runner as an argument, allowing it to add units would be a good idea. Especially under your proposal that there will be a worker which does a lot of start/stopping of stuff in its parent runner (rather than as children of itself).09:32
jamHowever, your old pastebin is my current favorite for the time being.09:32
jam(09:32
jamhttp://paste.ubuntu.com/5800919/09:33
rogpeppejam: we *could* do that, but i think i'm happy having a runner as a closure variable too - there's a certain purity in having a function with zero args09:34
jamrogpeppe: as long as we never have to do something like migrate the workers to another runner, etc09:34
jamit *seems* like workers shouldn't know what runner they are running inside09:34
jamwhich the closure breaks that09:34
rogpeppejam: if we do, the code is small and easy to change09:34
jammgz_: did your patch finally land?09:36
rogpeppejam: how about this? http://paste.ubuntu.com/5800999/09:37
jamrogpeppe: it is pretty much equivalent to http://paste.ubuntu.com/5800919/ for me.09:38
jamrogpeppe: you still have to wrap that in a closure09:38
rogpeppejam: ok09:38
jamthat closure still has to save a.runner as a const in its closure09:38
jametc.09:38
rogpeppejam: i was trying to make it so that the worker doesn't know what runner it's running in09:39
jamIt is probably slightly better at having APIWorker not know the internals of what a StateWorker is09:39
jamrogpeppe: sure, but passing that in is the same thing.09:39
rogpeppejam: that knowledge is held outside (in MachineAgent.Run)09:39
jamrogpeppe: so if a runner passes itself into the thing it is calling, then the thing it is calling doesn't "know" where it is running, it is "told" where it is running.09:40
jamwhich means if another runner got that task, it would just run on the other runner.09:40
jamin this case, the closure has to *know*, which is true of either form of your code.09:40
rogpeppejam: that assumes that we always want to add the state worker to the same runner that's running the API worker.09:41
jamrogpeppe: if given both, I probably like http://paste.ubuntu.com/5800999/ more, it just doesn't solve the specific issue I was having troubles with.09:41
jamHaving nested runners that can start other runners that *aren't* nested underneath them is also a little bit confusing09:42
rogpeppejam: the startStateAPI func thing works out quite nicely actually: http://paste.ubuntu.com/5801013/09:44
rogpeppejam: i agree it's a bit confusing, but the whole look-at-api-then-connect-to-state-but-not-if-we-are-bootstrapping thing is a little inherently confusing, i think09:46
rogpeppejam: and i *think* that the code is just a reflection of that fundamental awkwardness09:46
mgz_jam: yup09:50
jamfwereade, rogpeppe: so what was decided on package naming vs tasks. Specifically, what package should I put upgrader in? Its own as state/apiserver/upgrader/upgrader.go, or sharing machine as state/apiserver/machine/upgrader.go? (I get the impression it may be run by machine or unit, so it should be its own thing)09:57
rogpeppejam: it should be in its own package09:57
fwereadejam, rogpeppe: this is kinda the problem with the "let's segment-by-agent" scheme09:58
rogpeppejam: this CL shows where i'm aiming https://codereview.appspot.com/10494043/09:58
fwereaderogpeppe, would you remind me what your objection was to putting common test infrastructure in its own package?09:58
fwereaderogpeppe, (iirc that was the main factor in your decision -- I hope I'm not misrepresenting?)09:59
jtvrogpeppe: does this correctly reflect your notes on concurrency hazards in provider implementations?  https://codereview.appspot.com/1060204310:00
rogpeppefwereade: that was part of it. the main thing was to try to keep packages from proliferating wildly.10:00
rogpeppejtv: will look in a mo10:01
fwereaderogpeppe, small packages with clear purposes are usually considered a good thing10:01
jtvthx10:01
rogpeppefwereade: the way we're going, we'll have apiserver/machine, apiserver/machiner and apiserver/machineagent10:01
fwereaderogpeppe, all the more so in go, surely, considering that the only encapsulation boundaries are at packages edges10:02
rogpeppefwereade: i think it's reasonable to gather the things that will only ever run in one agent inside a package for that agent10:02
* jam goes and hides in a hole to get actual coding done10:02
jamwill emerge around standup time10:02
rogpeppefwereade: i'm happy to encapsulate by type as well as by package10:03
fwereaderogpeppe, that's all very well in theory but when not enforced by language or ultra-string convention it tends to degrade ;)10:03
fwereaderogpeppe, I think that the details of exactly what runs where will be more fluid than you anticipate10:04
fwereaderogpeppe, and I would prefer not to impede our flexibility by signalling that the most important feature of, say, upgrader, is which agent runs it10:04
fwereaderogpeppe, we have a distributed system with a bunch of responsibilities10:05
rogpeppefwereade: i would definitely put reusable components inside their own packages10:05
rogpeppefwereade: so the upgrader would get its own package10:05
rogpeppefwereade: one mo, i'll paste a couple of sketches10:06
fwereaderogpeppe, and uniter?10:06
fwereaderogpeppe, I'll probably want to run some of those in machine agents at some point10:06
rogpeppefwereade: it would go into apiserver/unit10:06
rogpeppefwereade: really?10:07
fwereaderogpeppe, remove-unit --force10:07
fwereaderogpeppe, I'm not going to try a transaction to clean up the whole unit state10:07
fwereaderogpeppe, revoking the original unit's access and running a sandboxed uniter with a fake charm would work just fine though10:07
rogpeppefwereade: woah10:07
rogpeppefwereade: that seems a bit... heavyweight10:08
rogpeppefwereade: interesting idea though10:08
fwereaderogpeppe, I only thought of it relatively recently, but it seemed like a possible end-run around a lot of the diffculty10:09
fwereaderogpeppe, it would definitely require that the uniter be decomposed a little but that's definitely not a bad thing10:11
rogpeppefwereade: it's already decomposing, arf arf :-)10:12
rogpeppefwereade: it sounds like an interesting approach10:12
fwereaderogpeppe, I'd love to be able to drop most of the giant integrationy tests there and be able to run detailed unit tests on all the modes, for example10:12
rogpeppefwereade: using the unit agent for that purpose would presumably only require a small subset of the full uniter API?10:13
fwereaderogpeppe, yeah, that was the thought10:13
rogpeppefwereade: so it might have its own API facade anyway?10:13
rogpeppefwereade: ok, so if we go with the "packages for everything" approach, this is how i see the machine package looking: http://paste.ubuntu.com/5801079/10:15
fwereaderogpeppe, that hadn't been my thought in particular -- allowing access to that facade for force-dying units on ManageState machines connections seems maybe plausible10:15
fwereaderogpeppe, sorry, the machine agent implementation?10:16
rogpeppefwereade: what does cleaning up the unit state actually involve? is it that complex that it's a great help to have whole uniter around for it?10:16
fwereaderogpeppe, basically a load of scope-leaving10:17
fwereaderogpeppe, it may indeed not involve the whole thing, that's why I mention decomposing it10:18
rogpeppefwereade: so, the machine package would just integrate together all the APIs that we want to present to the machine agent.10:18
dimiterndanilos__: hey10:20
danilosdimitern, hey-hey10:20
danilosdimitern, welcome back, I hope it was nice two weeks off :)10:20
rogpeppedanilos: hiya10:20
dimiterndanilos__: have you started some work on the deployer api stuff?10:20
danilosrogpeppe, hey10:20
dimiterndanilos__: oh yeah it was :)10:20
danilosdimitern, yeah, barely10:20
danilosdimitern, basically, looking at that unification of watchUnits under Machine state object10:21
dimiterndanilos__: because i was thinking of picking that up10:21
dimiterndanilos__: if you don't mind10:21
danilosdimitern, hum, perhaps not a bad idea and I can focus on finishing the python-env stuff10:21
dimiterndanilos__: sgtm10:21
fwereaderogpeppe, that doesn't feel unreasonable to me -- it puts that responsibility in one place while we're firming up the stuff around it, and keeping it separate for now makes it easier to move it somewhere else if and when the need becomes apparent10:21
fwereaderogpeppe, I'd really prefer to keep things separated by default, and only combined when it's clear that doing so fits the rest of the model so well that it's actively *bad* to keep them separate10:23
rogpeppefwereade: do you want a separate package for each watcher type?10:23
danilosdimitern, some of unfinished code is up in lp:~danilo/juju-core/watch-units, though you might want to start over since it's not much (I've got some attempts at fixing tests uncommitted locally, but it doesn't solve them completely)10:23
danilosdimitern, want me to push that too or you don't care? :)10:23
rogpeppefwereade: i was planning on putting them all into apiserver/watchers10:24
dimiterndanilos__: i'll take a look at that and what we planned before i left to bring myself up to speed10:24
fwereaderogpeppe, depends where you mean... I think that what you describe may actually work pretty well for me -- IMO that's the one clear case we have of a really broadly shared capability10:25
danilosdimitern, some notes I gathered are up in http://paste.ubuntu.com/5741603/ (including a link to your pastebin)10:26
danilosdimitern, I meant https://docs.google.com/a/canonical.com/document/d/105xob7LVW63NoWoKoRhJNYN26_1GaAO-apiG8TKt_5s/edit :)10:26
dimiterndanilos__: cheers10:26
danilosdimitern, shared it with you10:26
jamfwereade, rogpeppe: by the same token, should it be called state/apiserver/upgrader/upgrader.go:UpgraderAPI  vs Upgrader ?10:26
jamgiven MachinerAPI10:26
fwereaderogpeppe, so you'd make the watcher API accessible separately to most of the other APIs, and those APIs' watch methods will return ids for use with the watcher service?10:27
fwereadejam, if it has it's own package it could just be upgrader.API, and if everything did it could be machiner.API, etc10:27
fwereaderogpeppe, jam, watcher.API :)10:28
jamfwereade: it does make using grep to find where this type is implemented pretty hard.10:28
rogpeppefwereade: i was thinking watchers.EntityWatcherAPI, watchers.EnvironWatcherAPI, etc10:29
rogpeppefwereade: because each one is actually really quite small10:29
fwereaderogpeppe, fwiw I think EnvironWatcher should just be an EntityWatcher really10:29
rogpeppefwereade: whatever10:29
rogpeppefwereade: it was the second watcher i could think of :-)10:29
fwereaderogpeppe, (bikeshed bikeshed: NotifyWatcher)10:30
rogpeppefwereade: if you wanna repaint that bikeshed, go for it10:31
rogpeppefwereade: (re-bikeshed: Watcher)10:31
fwereaderogpeppe, yeah, we can do all this later :)10:32
rogpeppefwereade: +110:32
rogpeppejam: don't use grep to find types :-)10:33
fwereaderogpeppe, I'm comfortable with the broad shape of what you propose10:33
rogpeppefwereade: cool10:33
rogpeppejam: also grep '^type SomeType' is a good way of finding types, if that's not how you do it already10:34
fwereaderogpeppe, so in *that* case can we make that machine package wither to almost nothing, and make it the individual workers' responsibility to pull the APIs they need off a single shared client object?10:35
rogpeppefwereade: yes10:36
fwereaderogpeppe, this sgtm10:36
jamrogpeppe: sure, but if everything is defined as type API ...10:36
jamfunc.*FuncName is reasonable to find functions too10:36
rogpeppejam: well, you'll only ever see it referred to a pkgname.API10:36
rogpeppejam: which is fairly unambiguous as to where that particular API type is defined10:36
jamrogpeppe: except in the package itself, and you don't know *which* file it is defined in10:37
jamrogpeppe: gives you a directory10:37
rogpeppejam: my favourite is ' Foo\(' for finding method definitions10:37
jamrogpeppe: func.*Foo finds both methods and free funcs10:37
rogpeppejam: but it also finds functions with Foo in the name10:38
rogpeppejam:  i like ' Foo\(' because it's pretty exact10:38
jamrogpeppe: except it finds calls of a free func Foo10:38
jam:)10:38
jamI agree that I put ( on when I need it10:39
rogpeppejam: yeah, it finds method and func defns10:39
rogpeppejam: in general though, i use godef :-)10:39
jamrogpeppe: which while it is a tool that works, it is also almost by definition a "tool that works for rogpeppe" :)10:39
jamas 99.999999% of humans don't have it installed on their machine10:40
rogpeppejam: other people use it too, honest :-)10:40
rogpeppejam: go get works10:40
jamrogpeppe: I would argue that the number of people comfortable using godef, and the number comfortable using grep10:40
jam...10:40
rogpeppejam: it's idiomatic in Go to have type names that work well when qualified by the package identifier10:41
rogpeppejam: i don't think it's necessary to make names that are unique across the code base because they're a bit awkward to find using grep.10:42
rogpeppejam: BTW i tried to write godef such that it would be almost trivial to integrate into another editor - all you need is the current file cursor position and the current file contents and it'll tell you where the definition is.10:43
rogpeppejam: perhaps i'll actually learn some vim programming and write some bindings for it at some point.10:44
wallyworld_jam: mgz_: danilos__: can we do standup now? it's half time in the State Of Origin football and I want to watch the 2nd half10:59
mgz_I could, not sure about de otros10:59
* wallyworld_ is hopeful10:59
dimiternwallyworld_: me too11:00
wallyworld_dimitern: hey, welcome back11:00
dimiternwallyworld_: thanks11:00
wallyworld_good holiday?11:00
jamwallyworld_, mgz_, danilos__, dimitern: I'm there https://plus.google.com/hangouts/_/8868e66b07fa02bdc903be4601200d470dae9ee311:01
dimiternwallyworld_: oh yeah, although it seemed shorter ;)11:01
jammgz_: de otros? los otros?11:01
mgz_something like that :)11:01
jamdanilos__: poke?11:03
dimiterndanilos_: https://plus.google.com/hangouts/_/8868e66b07fa02bdc903be4601200d470dae9ee311:10
danilos_dimitern, coming in a bit11:10
rogpeppejam: https://github.com/dgryski/vim-godef11:16
rogpeppejam: it's not fantastic (it doesn't know about the contents of the current buffer, so it will muck up if you're currently editing, but it might work ok otherwise)11:17
rogpeppejtv: reviewed11:23
TheMuefwereade: ping11:40
fwereadeTheMue, pong11:40
TheMuefwereade: wonna talk about autosync?11:41
fwereadeTheMue, sgtm, would you precis it here? would be worth the chance of other eyes passing over it I think11:41
TheMuefwereade: currently simply wanted to know how you see this feature. so far I only have this topic "auto sync-tools" and found one mail11:43
TheMuefwereade: as long as I understand it the need for an explicit sync-tools call shall be removed11:44
TheMuefwereade: instead it should be handled automatically during bootstrap if needed11:44
fwereadeTheMue, I had been hoping you had been analysing the problem in the meantime -- when should we do it, when not, what are the drawbacks of the approach you recommend for various users, why do we consider them to be a price wrth paying11:45
fwereadeTheMue, the problem is that sync-tools is an annoying step extra step for first-time users, and we want to make their experience better11:46
TheMuefwereade: so it's like I wrote. handle it automatically during bootstrap if needed (simplified)11:50
=== tasdomas is now known as tasdomas_afk
TheMuefwereade: maybe I'm underestimating it and simply don't see the troubles you expect11:55
TheMuefwereade: so it would be helpful what problems you see11:56
fwereadeTheMue, it's the "if needed" and the "simplified" I'd like to hear more about11:58
fwereadeTheMue, what's the trigger condition? what version(s) do we copy? what's the impact of these decisions?11:59
fwereadeTheMue, how do we make this a nice story for an isolated environment?11:59
TheMuefwereade: the latter is the largest problem, indeed12:01
fwereadeTheMue, don't underestimate the former12:01
TheMuefwereade: for the first parts I would use the same mechanisms like in sync-tools to check the existing tools in the environment12:01
TheMuefwereade: using the same decision logic of which tools are to sync12:02
fwereadeTheMue, sync-tools will sync tools even if you already have valid tools available12:04
TheMuefwereade: do we have known troubles with this logic (in sync-tools)?12:04
fwereadeTheMue, I don't think it's quite the same use case, is it?12:05
TheMuefwereade: it uses tools.ErrNoTools when inspecting the environs storage with tools.ReadList()12:06
TheMuefwereade: but public or private is explicitely set12:06
fwereadeTheMue, right -- there's a whole bunch of things to consider. what source we want now, whether we'll want alternative sources in the future, where we want to store the tools we autosync, etc12:07
fwereadeTheMue, at what point we check for tools, what sort of errors we could encounter, how we report those errors12:08
TheMuefwereade: thought the source topic is an extra one12:09
fwereadeTheMue, ok, but I'm asking you to solve a mid-scale problem, and I need you to think through the solutions12:09
TheMuefwereade: reasonable, ok12:10
fwereadeTheMue, the isolated environment case is one aspect of the problem; even if we don't solve that bit first (we won't) I'd like us to at least consider the problem's existence as we design it12:10
TheMuefwereade: but then I need more than simply the three words "auto sync tools". that's why I asked you, to get more information which problem has to be solved12:11
rogpeppejam, fwereade: i've changed things a little bit and added some more comments. you might want to take a look before i approve the branch: https://codereview.appspot.com/10259049/diff2/5001:16001/cmd/jujud/machine.go12:11
fwereadeTheMue, so one consequence of isolated environments is that we'll have to, at some stage, have a pluggable tools source12:11
fwereaderogpeppe, cheers, just a mo12:11
fwereadeTheMue, ok, I'm sorry, I thought I'd been clear that it's about streamlining the user's first experience12:12
TheMuefwereade: so let me simply create an issue for this topic (currently only a tiny kanban card exists) where we can collect everything that should be solved12:12
fwereadeTheMue, ec2 has a nice story from that POV, the others not so much12:12
TheMuefwereade: yeah, it's now more clear12:13
TheMuefwereade: and where I underestimated it *sigh*12:13
TheMuefwereade: hidden behind three nice words ...12:14
fwereadeTheMue, haha :)12:14
TheMuefwereade: ;)12:14
jamrogpeppe: you have some doc comments that are out of date (startStateWorker vs ensureStateWorker) otherwise LGTM12:15
fwereadeTheMue, the questions and answers are probably not actually that hard, it's just that I'd like to minimise surprising consequences and anything that involves copying several MB around is going to be a bit noticeable12:15
rogpeppejam: ah, i thought i'd fixed that, thanks12:15
TheMuefwereade: yep12:16
TheMuefwereade: what I got from the mail it sounded only like a "hey, we've got not tools. I want to bootstrap, so please call sync-tools"12:18
fwereaderogpeppe, LGTM,one thought12:24
rogpeppetrivial CL that fixes golang-tip govet against juju-core trunk: https://codereview.appspot.com/10607043/12:28
rogpeppeplease could someone have a look quickly, as the issue is stopping me from proposing anything currently.12:28
rogpeppefwereade: i believe you're on call :-)12:34
rogpeppefwereade: i invoke you12:34
fwereaderogpeppe, cheers12:34
fwereaderogpeppe, LGTM trivial12:35
rogpeppefwereade: ta12:36
jamfwereade, rogpeppe: pulling out some of the testing code in apiserver/machine into apiserver/testing so that I can reuse it for upgrader: https://codereview.appspot.com/10608043/12:45
jamrogpeppe: in the HA case, you'll still have the root machine that starts the HA off, right/12:47
jam?12:47
rogpeppejam: yes12:47
jamso the newly started will-be-root-nodes still have an API server to connect to to find out what they will be doing12:47
jamso there is still only 1 'machine/0' that bootstraps the whole process12:47
rogpeppejam: yes12:47
wallyworld_fwereade: no urgency, could you look at https://codereview.appspot.com/10534043/? i'm off to bed soon so will check any comments tomorrow12:47
rogpeppejam: yes12:48
rogpeppejam: but12:48
rogpeppejam: once it's bootstrapped the whole process, its jujud might restart12:48
jamwallyworld_: did you win?12:48
wallyworld_yes!12:48
fwereadewallyworld_, ack12:48
wallyworld_smashed them :-D12:48
jamwallyworld_: grats12:48
wallyworld_1-1 now. we need the 3rd game12:48
rogpeppejam: and then it's no longer appropriate to open state without opening the API first12:49
rogpeppejam: assuming there's at least one other API server out there12:49
fwereaderogpeppe, I'm wondering if there's some justification for always connecting to state if state info is available12:50
rogpeppefwereade: in the future, state info won't be passed in cloudinit12:51
fwereaderogpeppe, if a machine comes up without it, it can request it and write it, and the state workers that it'll start anyway can just fail repatedly until it's written out12:51
rogpeppefwereade: and it's better to connect with the latest info if we can12:51
fwereaderogpeppe, the bootstrap case remains special12:51
rogpeppefwereade: and the API holds the freshest info12:52
fwereaderogpeppe, sure, but I imagine the worker that keeps an eye on state info is going to have to be writing out updated versions every so often regardless12:52
jamrogpeppe: why do we believe the information about what API server to connect to is any fresher than what State to connect to?12:52
rogpeppefwereade: sure. but we might come up after being partitioned from the API for some time12:53
fwereaderogpeppe, in the bootstrap case only, write it in so the state workers can start and work immediately, and the api workers can sit and wait until there's an api available12:53
rogpeppejam: that's all we've got12:53
rogpeppefwereade: what happens when we can remove jobs from a machine?12:54
fwereaderogpeppe, delete the stateinfo and bounce the agent I guess?12:54
rogpeppefwereade: the machine is disconnected and comes back up, thinking that it needs to connect to the state, but actually it longer has that privilege, so it connects to the state repeatedly and fruitlessly12:55
jamrogpeppe: until it manages to connect to the API and finds out it is no longer able to do so.12:55
fwereaderogpeppe, what jam said12:55
rogpeppejam: yes, so either way the APIWorker code has to interfere with the StateWorker code.12:56
fwereaderogpeppe, at which point it gets deleted and everybody's happy12:56
rogpeppejam, fwereade: i'm not sure that we gain much by connecting to the state server regardless of the API.12:57
fwereaderogpeppe, that way the two tasks can be completely orthogonal, surely?12:57
rogpeppe[13:56:27] <fwereade> rogpeppe, at which point it gets deleted and everybody's happy12:58
fwereaderogpeppe, none of this conditional complexity and passing statey things into api methods12:58
rogpeppefwereade: the APIWorker has to do that deletion12:58
rogpeppefwereade: which makes the two tasks non-orthogonal12:58
fwereaderogpeppe, there's an api worker solely responsible for getting, updating, deleting state info -- but that's not something the api runner needs to take into consideration, is it?12:59
rogpeppefwereade: having the two tasks communicating via the shared stateinfo state seems wrong to me13:00
rogpeppefwereade: when we know exactly when a state worker is needed13:00
rogpeppefwereade: and can start it then13:00
fwereaderogpeppe, at the cost of forcing everyone who wants to know about the api tasks to also figure out wat the deal is with the state tasks13:01
rogpeppefwereade: ??13:01
rogpeppefwereade: the only shared knowledge is those few lines in machine.go13:01
rogpeppefwereade: noone writing an API-based worker needs to know about any of that stuff13:02
rogpeppefwereade: we're talking < 10 lines of code here.13:02
fwereaderogpeppe, agreed -- but to figure out what the agent does wrt the api, you need to derail into the state code13:02
rogpeppefwereade: i think that doing it directly is nicer than doing it indirectly by side-effect of changing the shared stateinfo13:03
rogpeppefwereade: which seems more like magic to me13:04
jtvrogpeppe: thanks for the review — would you also have time for another?  It's this one: https://codereview.appspot.com/10480045/13:05
fwereaderogpeppe, the actual state info is only "shared" between the thing that reads it and the thing that writes it13:05
fwereaderogpeppe, we need both those things already13:05
fwereaderogpeppe, why explicitly couple two distinct components when they'll work just fine independently anyway?13:06
fwereaderogpeppe, if the api runner needs to know the state info, that's a smell13:06
fwereaderogpeppe, if a component whose entire purpose is writing the state info knows about it, and *that* is run by an api worker, no problem13:07
fwereaderogpeppe, distinction seem meaningful?13:07
rogpeppefwereade: it still feels icky to me; that's perhaps because i haven't absorbed its possibilities. at the moment it feels like smearing responsibilities (changing the stateinfo not only changes the stateinfo, but also has the side effect, at some point in the future maybe, of triggering state-based tasks to connect and run.13:11
rogpeppe)13:11
rogpeppefwereade: it *may* be a better way of doing it, but i'd need to think hard about it for a while13:12
rogpeppefwereade: and for the time being, i'm reasonably happy with the current approach, which i think works ok13:12
rogpeppefwereade: and i really want to get this stuff in13:12
fwereaderogpeppe, yeah, I did LGTM it, I'm not trying to block you13:13
rogpeppefwereade: thanks13:13
fwereaderogpeppe, I'm also just wittering on about how I'd like to see it evolve13:13
ackkhi all, I have a question about juju-core API: after sending a WatchAll request, does the first AllWatchersNext response contain the whole current environment state, or can it be broken up in multiple responses?13:14
rogpeppefwereade: i'd rather that the component that was responsible for writing the stateinfo was also directly responsible for adding or deleting the state worker.13:14
rogpeppejtv: looking13:15
jtvthanks13:15
rogpeppejtv: reviewed13:18
jtvThanks again!13:20
jtvrogpeppe: not sure I understand your review comment... you want me to move the initialization of the environ's "name" attribute down by one line?13:28
rogpeppejtv: did you see what i did in the other providers to solve the same issue?13:28
jtvYes, I was the first to review the branch.  I did something as similar as I could here.13:28
rogpeppejtv: i made the setting of the name field entirely independent of SetConfig13:29
jtvYes, that's what I did too.13:29
rogpeppejtv: you set the name field within SetConfig, no?13:29
rogpeppejtv: i would set it in Open13:29
jtvThat's what I did.  I guess you just missed it then because it was so hassle-free.  :)13:29
jtvenv := azureEnviron{name: name}13:29
jtv←initializes "name" right from the start, for maximum protection as the ads say.  :)13:30
rogpeppejtv: ha ha!13:30
rogpeppejtv: i'd read the red code as green code13:30
rogpeppejtv: sorry for the bogus comment13:30
jtvRed code much better than Green squad code!13:30
rogpeppejtv: :-)13:30
mgz_now now, let's not have arguments :P13:30
rogpeppejtv: LGTM13:31
TheMuefwereade: thx for review13:31
dimiternfwereade: got a minute?13:31
jtvmgz_: Arguments much better than global state!13:32
jtvthanks again  rogpeppe13:32
mgz_death by joke!13:33
jtvIn Soviet Russia, we kill joke.13:33
jtvI'm sorry — stress-induced giddiness.13:33
fwereadedimitern, heyhey13:34
fwereadedimitern, sure13:34
dimiternfwereade: I'm running into some issues with the deployer tests after removing the units watcher arg13:35
fwereadedimitern, oh yes?13:35
dimiternfwereade: more specifically - take a look at TestDeployRecallRemovePrincipals and TestDeployRecallRemoveSubordinates13:36
dimiternfwereade: I changed the deployer to use the machineId instead of a tag to verify whether it's responsible13:36
dimiternfwereade: so now unassigned units cannot be deployed13:36
fwereadedimitern, cannot be removed you mean?13:37
dimiternfwereade: and the tests assuming that seem kinda crackful13:37
dimiternfwereade: that as well13:37
fwereadedimitern, hmm, don't forget this code *does* need to work with units started with an old deployer13:37
dimiternfwereade: but now I have this check: http://paste.ubuntu.com/5801495/13:38
dimiternfwereade: before it was deployerTag, ok := unit.DeployerTag(); ok { responsible == tag == d.tag }13:38
fwereadedimitern, ok, that looks reasonable, I thnk13:38
dimiternfwereade: the thing is - now the above 2 tests timeout right about there: http://paste.ubuntu.com/5801499/13:39
fwereadedimitern, then the only impact is making sure that we list units of old-style deployers as well in the manager13:39
dimiternfwereade: and I can't get the idea behind unassigning and then checking whether it's still deployed13:40
fwereadedimitern, well, unassigning is kinda madness and crack actually13:40
fwereadedimitern, because it's all fundamentally unknown13:41
dimiternfwereade: it's definitely confusing - what should it be instead?13:41
fwereadedimitern, well, it's not a feature we've designed properly at all, it's an 18-month-oldguess we still haven't found a usefor13:41
fwereadedimitern, so, to be clear , how do the tests fail? they don't remove the unit when it got unassigned, because..?13:42
=== wedgwood_away is now known as wedgwood
dimiternfwereade: http://paste.ubuntu.com/5801508/ - that's the test output13:43
fwereadedimitern, ah!13:43
rogpeppefwereade, anyone: i'd appreciate reviews of https://codereview.appspot.com/10494043/ and https://codereview.appspot.com/10554043/ if possible13:43
fwereadedimitern, so you saw a change13:44
fwereadedimitern, asked the machine for its id13:44
fwereadedimitern, it gave you that error and it filtered down13:44
fwereadedimitern, instead of trapping that error (in your first paste) and setting responsible = false13:44
dimiternfwereade: ah!13:44
dimiternfwereade: good catch13:44
dimiternfwereade: so this works: http://paste.ubuntu.com/5801529/ - but only for principals, the subordinates test still fails with the same timeout error13:49
dimiternfwereade: log http://paste.ubuntu.com/5801531/ - the only thing I changed is to add these lines in prepareSubordinates: http://paste.ubuntu.com/5801536/13:50
=== gary_poster is now known as gary_poster|away
fwereadedimitern, I think that one now needs to make into account that the principal will also be deployed?13:51
dimiternfwereade: you mean waitFor(c, isDeployed(ctx, u.Name()) first, then the sub?13:52
rogpeppejam: what does this mean: "No proposals found for merge of https://code.launchpad.net/~rogpeppe/juju-core/311-juju-bootstrap-state-change-password-1.6 into https://code.launchpad.net/~go-bot/juju-core/trunk."13:52
fwereadedimitern, I suspect so13:53
rogpeppejam: i just got it as an error from the Go Bot13:53
* rogpeppe wishes there was some way of knowing what tarmac is up to at any given moment13:55
fwereadedimitern, by the way, you know the separate branch I suggested that allows principal units to become dead without waiting for subs? I'm not quite so sure that's a good idea, don't waste any time removing that code :)13:56
dimiternfwereade: I'll propose it shortly as a first step13:57
fwereadedimitern, remember you can't land the change you're making without removing deployer from the unit agent13:58
dimiternfwereade: sure13:59
fwereadedimitern, and do please spin something up and test it live with an actual subordinate :)13:59
dimiternfwereade: ok, will do13:59
fwereadedimitern, and try taking things down in various ways13:59
fwereadedimitern, destroy principal unit/service, destroy subordinate service, destroy relation13:59
fwereadedimitern, at least you can --force machine and run all the test sin the same place ;p14:00
dimiternfwereade: i might need some help on that14:00
fwereadedimitern, sure -- I have a kanban meeting to get to now though14:00
mgz_rogpeppe: not sure, nothing seems obviously wrong, but tarmac seems to have not found the merge proposal14:00
rogpeppemgz_: hmm, weird14:01
mgz_ah, you reproposed after targetting the wrong branch?14:01
mgz_that seems likely to be the issue14:01
rogpeppemgz_: i don't *think* so, but it's possible14:01
rogpeppefwereade: kanban?14:02
=== gary_poster|away is now known as gary_poster
fssniemeyer: ping14:16
niemeyerfss: Yo14:31
rogpeppejam: any idea about this message: "No proposals found for merge of https://code.launchpad.net/~rogpeppe/juju-core/311-juju-bootstrap-state-change-password-1.6 into https://code.launchpad.net/~go-bot/juju-core/trunk."14:36
rogpeppejam: ?14:36
rogpeppejam: the go-bot has actually already merged that prereq14:36
rogpeppejam: despite it saying that it can't find it14:36
sidneirogpeppe: sometimes the lp api lags behind14:37
rogpeppejam: the weird thing is that the merge proposal has existed for days14:37
rogpeppesidnei: ^14:37
rogpeppesidnei: but maybe it's a misleading error message14:38
sidneiin one case i had to delete the cache that the lazr.restfulclient keeps locally to get it to notice14:39
* TheMue is stepping out for some time and will be back later14:43
fwereaderogpeppe, both LGTM14:47
rogpeppefwereade: thanks14:48
rogpeppethe API-connection branch has finally landed in trunk14:57
rogpeppeyay!14:57
dimiternrogpeppe: you've got 2 reviews14:57
rogpeppedimitern: thanks14:57
rogpeppedimitern: i'm just moving towards approval14:58
dimiternrogpeppe: only the panics are fishy14:58
rogpeppedimitern: the panics are there because i don't expect those methods to be called in the tests14:58
rogpeppedimitern: no point in writing code that's never used14:58
dimiternrogpeppe: hmm.. well, if it's there we should test it (at some point)14:59
rogpeppedimitern: i'm sure that they'll be fleshed out later when it becomes common test code14:59
rogpeppedimitern: all in good time :-)14:59
dimiternrogpeppe: ok then14:59
mgz_rogpeppe: what did you do to get your merge unstuck?15:00
rogpeppemgz_: nothing at all15:01
rogpeppemgz_: ah, no15:01
rogpeppemgz_: i re-approved it15:01
mgz_okay, good to know for future reference15:01
rogpeppe"There are additional revisions which have not been approved in review. Please seek review and approval of these new revisions."15:09
rogpeppeaarg15:09
rogpeppeh15:09
rogpeppedoes that mean i can't make changes that people suggest and then just land the branch?15:09
mgz_you just toggle the approval state for that15:09
mgz_you can't approve then push (as I found, I think)15:10
rogpeppemgz_: i don't think i did. but maybe i did.15:10
rogpeppemgz_: sigh15:10
mgz_there may also be a little lag15:10
rogpeppemgz_: i think tying the approval to the revno that was loaded in the lp page is bogus.15:11
rogpeppemgz_: too clever for its own good and ours15:11
dimiternfwereade: so once the deployer creation is removed from the unit agent, there's no need to test for deployment right?15:11
rogpeppemgz_: i'm spending far too much time shepherding the submission process15:11
dpb1fwereade: Thanks for the review.  So, in pyjuju setting "" even on the command line (juju set "foo=") would set an empty string in the config.  AFAIK, there was no way to get a nil/null/None into the charm config, since that state does not exist in Bash (lowest common denominator)15:16
fwereadedpb1, hmm, so what were we doing with "" values at config-get time? were we always stripping them out?15:23
dpb1"" -> remained as is.  something unset (no default in the config) was stripped out.  The only way to get a null was something unset in the config.yaml that you probed directly with json (config-get --format=json unset_key => null)15:28
dpb1fwereade: my brain hurts just typing that. :)15:29
fwereadedpb1, turned out pretty readable actually :)15:31
dimiternfwereade: https://codereview.appspot.com/10617043 - first attempt to see the general direction, will do live testing now15:31
fwereadedpb1, so, ok -- I guess there wasn't any way to explicitly clear a value in python and get back to the default then?15:31
fwereadedpb1, sorry, I've been doing go for too long15:32
fwereadedpb1, ah! right, I remember why we turned "" into nil on input -- because python couldn't distinguish between a default value and an explicit value that matched the default15:33
fwereadedpb1, and hence could not handle changing default settings on charm upgrade15:34
dpb1fwereade: correct.  once it is defined, no way to unset it, so "" was the defacto "unset"15:34
dpb1oh.15:34
fwereadedpb1, sorry, it was IIRC an overheard conversation at a sprint about a year ago15:34
* dpb1 parses15:34
fwereadedpb1, I'm frantically loading state myself15:35
fwereadedpb1, so, we took the ""-means-unset convention and formalized it a bit15:35
dpb1interesting15:35
fwereadedpb1, and reassured ourselves it couldn't possibly hurt anyone because the semantics were just the same15:35
fwereadedpb1, ofc you can see how well that turned out15:36
dpb1hehe15:36
dpb1well, it's a small thing, and easy to fix in the charms, its just... there are a lot of them. :)15:36
fwereadedpb1, but I think there's a germ of value in the idea somewhere15:36
fwereadedpb1, I firmly believe we need to fix this15:36
fwereadedpb1, we've broken people and we must unbreak them15:37
fwereadedpb1, to me the question is whether the fix is temporary, with a deprecation warning, or grandfathered in forever15:37
dpb1fwereade: agreed.  I was kind of surprised by it (as a charm author).  I'm sure others would be too15:37
dpb1fwereade: to me it seemed straightforward, "default: " in config.yaml maps to "" when you read it out.  But you threw that wrinkle in about distinguishing at charm upgrade time15:38
fwereadeTheMue, ping15:38
fwereadedpb1, yeah -- and the ability to reset to default is quite nice too15:39
dpb1fwereade: I guess yaml is just not expressive enough to consider a nil case15:39
fwereadedpb1, null is perfectly valid yaml but we don;t want people to have to type that ;p15:40
dpb1fwereade: ah...15:41
fwereadedpb1, yaml can probably express "please fire all the nuclear missiles" if you're not careful15:42
* dpb1 wonders what would happen if he typed juju set key=null :)15:42
fwereadedpb1, I don't *think* that goes through a yaml filter in python, and it certainly doesn't in go15:42
dpb1fwereade: iirc, I've tried that and you are right.  just strigifies to null15:43
fwereadedpb1, but if you used --config with a null value we'd interpret it as "please delete this setting" with the final impact being "replace with default"15:43
mgz_fwereade: that's what safe_load is for :P15:43
fwereadedpb1, except I'm suddenly not sure what would happen there in python15:43
fwereadedpb1, it might well start coming out as None15:43
dpb1fwereade: interesting.  I've never used this feature.  Since charms interpret "" as unset (for a variety of reasons we have discussed), that has always been how ive done it from the command line   juju set "key="15:44
dpb1fwereade: in any case, having some change in behavior when using juju-core in this manner is fine, better than having to change charms in subtle ways, IMO.15:46
fwereadedpb1, in the very narrow context of "" defaults being valid I completely agree15:48
fwereadedpb1, you've brought up a more disturbing question though15:48
dpb1I don't like the sound of that.15:49
fwereadedpb1, that `juju set option=` is now sometimes a *very* different operation across go and python15:49
fwereadedpb1, this only impacts string keys at least15:50
fwereadedpb1, for other values it's new and useful and I don't think it hurts15:51
fwereadedpb1, can you give a gut estimate for how often you used that for an option that had a non-empty default?15:52
dpb1fwereade: so you are saying in juju-core if I do "juju set option=" it will revert to the default value?15:53
fwereadedpb1, yeah15:53
fwereadedpb1, that was something that happened so long ago I'd completely forgotten the original behaviour :/15:53
dpb1well, that isn't entirely unexpected, I suppose.15:53
* fwereade wishes we hadn't developed in the dark for quite such a long time15:53
dpb1but it brings up how you would actually set the value to ""15:54
dpb1as you are getting at.15:54
fwereadedpb1, quite so15:54
mgz_`juju set option=\"\"` I guess15:54
fwereademgz_, except when `""` itself is valid15:55
* fwereade sighs15:55
dpb1For me, I use that idiom all the time, when I'm developing the charm.  unset, set back, see what the charm does15:55
fwereadedpb1, and this is when the default is not ""?15:55
dpb1but, as you can expect, that differs depending on how "dynamic" the charm is supposed to be.  ie., are options used in install or config-changed.15:55
dpb1fwereade: it's less important, since most options default to "" or have no default, especially as the charms grow.15:56
fwereadedpb1, I'm really worried about that whole issue in general actually -- I rather feel that a charm that *requires* options is somewhat naughty, and one that can't handle changes is very naughty indeed ;)15:57
dpb1fwereade: but, if I had found that before you mentioned it?  I probably wouldn't care an awfully lot.  it does something interesting (sets back to default).15:57
dpb1as long as there is a way to unset, I would just use that.  modifying that behavior slightly isn't hard.15:57
fwereadedpb1, where by "unset" you mean "set to empty", right?15:58
dpb1fwereade: yes.  I think there are cases where you need options, but in general, it's something you should and do learn in charm building 101.  Don't require anything, and respond to change dynamically.15:59
fwereadedpb1, ok, so, to summarize: `option=` is a nice way to reset, and you can live with that, but you need some way to express empty strings in general?16:00
dpb1fwereade: hehe, well, that is the crux of what we are talking about.  In pyjuju, the two were equivalent (except for the case of config-get --format=json unset_value)16:00
dpb1fwereade: ok, summarize... let me see16:01
dpb1fwereade: the bug specifically mentions "default:" and how that should go to empty string.16:02
fwereadedpb1, that's the easy bit -- if "" is valid, then we can keep that setting directly16:03
dpb1beyond that, I think most of what we have been talking about it theory.  I don't need "option=" on the command line to behave in any certain way.16:03
fwereadedpb1, so my issue is of global consistency16:03
fwereadedpb1, if we allow "" in charm defaults, which I think is good, we should be able to express "make this the empty string, not the default" for the cases where the default is *not* ""16:04
fwereadedpb1, as it is the range of possible values is constrained by input method and that kinda sucks16:04
dpb1fwereade: yes, that makes sense.16:04
dpb1fwereade: so something like , juju set --default option16:05
dpb1(just throwing that out there)16:05
dpb1afaik, pyjuju had no way of doing what you are saying, but I think it would be nice.  you can get the info out of "juju get" and just set it by hand.16:06
fwereadedpb1, the specific scenario is for, say, tuning changes -- we'd like to update the preferred setting for users who haven't expressed a preference, but not for those who explicitly did16:07
fwereadedpb1, the theory is that those who don't express a preference are those least likely to fix it manually and we should make it do the Right Thing by default16:07
dpb1fwereade: right, but in that case, they aren't going to be changing things at the command line in the first place, right?16:08
dpb1they will just deploy and let it work16:08
fwereadedpb1, but it should be easy for them to fiddle with the settings and reset the ones they didn't want to -- but still get the benefits from upgrades without having to look16:10
dpb1yes.16:10
dpb1in that case having an explicit setting like --set-default, makes even more sense, avoid relying on "option=" doing something perhaps unexpected.16:11
fwereadedpb1, I think I am becoming convinced, loath as I am to drop that cure little `float=` to reset to default in the cases where it's not clear16:13
dpb1fwereade: in any case, it's a separate issue.  want me to file a bug about it?16:14
fwereadedpb1, ok, that sounds good to me -- we fix charm defaults, with no deprecation warning, and file a bug that `juju set stringoption=` does the wrong thing when the default is not ""16:15
dpb1ok, great.  bug coming along shortly.16:16
fwereadeTheMue, were yuo following along there roughly?16:18
dpb1fwereade: https://bugs.launchpad.net/juju-core/+bug/119494516:19
_mup_Bug #1194945: juju set is overloaded <juju-core:New> <https://launchpad.net/bugs/1194945>16:19
fwereadedpb1, thanks16:19
dpb1fwereade: thx for chatting.  I'll be afk for a while now.16:20
fwereadedpb1, cheers, anytime16:20
rogpeppedpb1, fwereade: juju unset ?16:26
rogpeppefwereade: trivial, i think: https://codereview.appspot.com/1059504416:26
fwereaderogpeppe, I want to be able to do it in the same transaction as the sets, really16:26
fwereaderogpeppe, trivial16:27
fwereaderogpeppe, `juju set option-`? :)16:29
rogpeppefwereade: ?16:29
fwereaderogpeppe, "-" indicating removal ;p16:30
rogpeppefwereade: i think i'd prefer juju set !option, but i think people with dodgy shells might object :-)16:31
rogpeppejuju set myflag- foo=bar bar=tdfv16:33
rogpeppehmm, not entirely sure if that reads well16:33
fwereaderogpeppe, yeah, it's a better idea in my head than on the screen16:33
* fwereade bbiab16:34
dpb1I like unset from a readability point of view, for sure16:43
fwereadedpb1, I kinda feel it's good to be able to make all config changes via the same command (and, under the hood transactionally)16:44
dpb1fwereade: yes, I agree with you in that argument.16:47
rogpeppefwereade: another trivial? https://codereview.appspot.com/1062004316:54
arosaleshave folks seen that when a service unit goes down (outside of Juju) it is still made available to haproxy?17:00
mgz_AssertStrop... funny tyop17:02
dimiternrogpeppe: not sure increasing the ping timeout to 5m is a good thing17:13
rogpeppedimitern: why does it need to be faster?17:14
rogpeppedimitern: what are we guarding against?17:14
dimiternrogpeppe: we want dead connections to die quickly17:14
dimiternrogpeppe: i.e. to be detected and closed earlier17:15
rogpeppedimitern: if we're using them, they'll die quickly - if not, we don't care that much17:15
rogpeppedimitern: 5 seconds is way too fast17:15
dimiternrogpeppe: why so?17:15
rogpeppedimitern: because it's constant network traffic from every single node17:15
rogpeppedimitern: and a connection going down is a rare event17:16
dimiternrogpeppe: how about 1m then?17:16
rogpeppedimitern: that's probably ok17:16
TheMuefwereade: just returned to the screen and will now read the chat log17:16
dimiternrogpeppe: reviewed17:20
rogpeppedimitern: ta17:20
fwereaderogpeppe, LGTM also17:21
rogpeppefwereade: thanks17:22
rogpeppefwereade: do you have an opinion on the ping frequency?17:22
fwereaderogpeppe, a minute sounded reasonable17:22
fwereaderogpeppe, but probably only because it's the middle of the 3 values I was shown :)17:23
fwereaderogpeppe, that's the sort of number I'm perfectly happy tuning in response to observation though17:24
rogpeppefwereade: yeah. i changed it from 5 seconds because i was watching the request log and it seemed way too fast17:24
fwereaderogpeppe, fair enough, I feel like a minute is quite a nice resolution for now17:25
mgz_fwereade: https://codereview.appspot.com/1062304317:36
mgz_I can't find a nicer way of making those helpers shared17:37
TheMuefwereade: so, read it, good discussion17:41
TheMuefwereade: so when default: is '' the value is kept and only in this case (default: is a string) set foo= sets foo to an empty string17:46
TheMuefwereade: so foo default: 'bar' and set foo= sets foo not to bar (the default) but to ''17:47
TheMuefwereade: and the new issue will introduce unset foo or or set !foo to reset it to the default value (or delete it if no default is specified)17:48
TheMuefwereade: correct summarize?17:48
jammgz_: how was the Release meeting?17:52
mgz_jam: fine, I'll link you the notes17:54
jamrogpeppe: https://code.launchpad.net/~rogpeppe/juju-core/312-alt-api-jobs/+merge/17013518:00
jamhas a prerequisite of https://code.launchpad.net/~rogpeppe/juju-core/311-alt-juju-bootstrap-state-change-password-1.518:00
jam*not* 1.618:00
rogpeppejam: that's not the one that failed18:02
jamrogpeppe: which one failed?18:02
rogpeppejam: this one: https://code.launchpad.net/~rogpeppe/juju-core/305-alt-jujud-use-tasks-package/+merge/17085618:03
jamrogpeppe: so see if this fits what happened. You had a branch with a prereq, and they both got marked Approved at roughly the same time. Go-bot landed one of them, and then rejected the other with "No Proposals Found".18:06
rogpeppejam: sounds right18:07
rogpeppejam: i thought i could approve both branches at the same time and the prereq would get merged first18:07
jamrogpeppe: IIRC it doesn't sort the requests by prerequisite, so it tried to merge 305-alt-jujud first, which has an approved-but-not-merged-prereq, so it gets bounced, and then 311-alt-jujud- is triggered immediately after and lands.18:12
rogpeppejam: ha, that's not great18:12
rogpeppejam: so if there are prerequisites, you have to wait until the prereq is merged before approving the next in line18:13
jamrogpeppe: it probably doesn't help that the dependent branch comes alphabetically before the prereq branch.18:13
jam305 depends on 31118:13
rogpeppeha ha ha18:13
rogpeppeit sorts alphabetically?!18:13
rogpeppejam: it should really sort by approval time, if anything18:13
jamrogpeppe: I think it sorts by whatever order LP gives it, which could be alphabetically18:13
jamI don't think it does sort internally.18:13
rogpeppejam: can we change it?18:13
jamrogpeppe: https://launchpad.net/tarmac code is available18:14
jamrogpeppe: https://bugs.launchpad.net/tarmac/+bug/84570618:15
_mup_Bug #845706: Tarmac fails to resolve pre-req branches when they've already been merged <Tarmac:Triaged by rockstar> <https://launchpad.net/bugs/845706>18:15
jamrogpeppe: looking at the bug report18:15
rogpeppejam: but if we make changes, do they have to be back-ported to precise before we get the benefit?18:15
jamit appears launchpad actually does stop showing you the merge proposal for some reason.18:15
jamrogpeppe: I'm running from source, not a deb18:15
rogpeppejam: if you want to have a look, this branch relates somewhat to an earlier branch you proposed. https://codereview.appspot.com/10611046/18:16
jamI don't know the validaty of this bug comment: This is actually a bug in Launchpad, it seems. Sometimes it will fail to  give the branch in the requested list of proposals. This seems to  happen regardless of the prerequisite's merge status, but does seem to  occur more often when it has been merged.18:16
rogpeppejam: thanks for pointing me to that report18:25
rogpeppejam: i've reached eod. time to do some packing... see you tomorrow morning.18:26
rogpeppeg'night all18:26
jamrogpeppe: have a good evening. I'm about 6 hours past mine :)18:26
jamdimitern: you are attributed the code roger is removing in https://codereview.appspot.com/10611046/18:26
jamthere is an intermediate object that adds the Id and a pointer back to the resources map18:27
jamwhich doesn't seem to be used18:27
jamIf it isn't needed, we can remove it, I just wanted to check if you remember a need for it (that may just not have been written yet)18:27
jamIt is also possible you are marked with the code because you moved it in a refactoring, and not because you wrote it.18:27
dimiternjam: what?18:29
dimiternjam: I introduced the interface yes18:31
jamdimitern: so you have a Resource interface, and a registry which tracks them with secret srvResource type18:32
jamthe only thing srvResource provides over Resource18:32
jamis the 2 attributes Id, and a pointer back to the resource registry18:32
jamRoger's proposal removes those 2 attributes by just using a map of Resource directly.18:33
jamand I'm checking if you had a reason to have those attributes, which are currently unused.18:33
dimiternjam: well, originally I had to introduce the interface to decouple the srvRoot from the machiner facade18:34
dimiternjam: and I think that CL is just reorganizing things around, rather than removing functionality18:36
jamdimitern: well it specifically removes an intermediate object which has some attributes the new one doesn't have, but they don't seem to be used, and I didn't know if they had other plans to be used.18:36
dimiternjam: i don't think so - if we need these, we'll reintroduce them18:38
=== tasdomas_afk is now known as tasdomas
=== mramm2 is now known as mramm
=== tasdomas is now known as tasdomas_afk
=== BradCrittenden is now known as bac
rogpeppejam: those attributes were needed only because srvResource was originally embedded as an API object directly20:54
rogpeppejam: it implemented Stop20:54
rogpeppejam: so it needed to be able to find the resources struct to remove itself20:54
fssniemeyer: hi, took me all day to see your pong x)21:00
fssniemeyer: are you going to fisl next week?21:00
niemeyerfss: I'll try to21:13
thumpermorning21:22
thumperfwereade: around?21:33
fssniemeyer: I was wondering if we can chat about juju-core + vpc :)21:39
thumperstabby stabby22:28
thumperpretty happy that `bzr destroy-environment` doesn't do anything22:57
fwereadethumper, heyhey23:10
thumperfwereade: hey23:10
* thumper makes a sad face23:10
thumpermissing reviews23:10
fwereadethumper, haha, I've done that bzr/juju thing23:10
fwereadethumper, that's why I'm on :)23:10
thumperah23:10
thumperfwereade: I've just merged trunk23:11
thumperand made the lxc provisioner work with the new runner stuff23:11
thumperand tested on ec223:11
fwereadethumper, oh yes..?23:11
thumperhad to fix one thing23:11
fwereadethat sounds like the past tense which I infer is a good thing23:11
thumperwhere the api was assuming that for machine-0-lxc-0, that "0-lxc-0" was the amchine id23:11
thumperso used the MachineIdFromTag and it is all good23:12
thumperso we are back to where it works23:12
thumperdid you want me to repropose23:12
thumperactually, I'll do that anyway23:12
thumperI do think however that the pinger is doing something a little weird...23:13
thumperoh, perhaps not23:14
wallyworld_fwereade: hi, thanks for the extra comments on the instance metadata branch23:14
thumperit seems that the api writing code doesn't mention the source23:14
thumperand it is writted from the api serving side23:14
thumpernot the client side23:14
thumpermy dog is getting restless23:14
thumperI think she needs to go outside for a bit23:14
thumperfwereade, wallyworld_: Rietveld: https://codereview.appspot.com/1048904323:14
thumperthat is the branch that makes the lxc provisioner work out of the box23:15
wallyworld_fwereade: if i keep Nonce on machine and write InstanceId to both machine and instanceData for compatibility, i think that may address your concerns?23:15
thumperall previous pipes in that pipeline have landed23:15
* fwereade is working through it literally rght now23:15
thumperthis one branch makes us provision containers23:15
fwereadewallyworld_, yep, that sgtm23:15
wallyworld_fwereade: are you +1 on the sig change to machine.InstanceId() - returns (string, bool, error)23:15
thumperI have my daughter at home today finishing off her science fair project23:15
thumperbetter a day off school, than staying up until midnight and getting tired and stressed out23:16
thumperwallyworld_: what?23:16
thumperwallyworld_: why three results?23:16
fwereadewallyworld_, ah, hmm, not so sure there -- but if you keep instanceid on the machine for now we get to dodge that question for a bit, right?23:16
wallyworld_fwereade: sure, but i'd rather alter the sig now to match what is will be in the end23:17
fwereadewallyworld_, if it's an error that would conventionally replace the info encoded in the bool with a specific error23:17
wallyworld_thumper: results is string (the value), bool (is te machine provisioned), error (was there an error finding the id)23:17
thumperfwereade: I agree23:17
thumpererror NotProvisioned23:17
thumperand match against that23:17
thumpertwo, not three results23:17
fwereadewallyworld_, I wouldn't be comfortable predicting that change right now tbh23:17
wallyworld_ok, i can do that. i did like the bool23:17
wallyworld_i'll make it an error, that will cover both cases23:18
thumperwallyworld_: it just becomes one more step, error.IsNotProvisioned(err)23:18
fwereadewallyworld_, when we've got it behind the API I suspect we'll find out some interesting things about actual usage23:18
wallyworld_thumper: yes, agreed23:18
fwereadewallyworld_, but if you feel strongly I'm not too bothered23:18
* wallyworld_ doesn't feel strongly23:18
thumperfwereade: also, I have looked at a very interesting library23:19
thumperfwereade: that may fix watchers23:19
wallyworld_fwereade: with the constraints branch - i'm thinking i may need a deployment constraints struct that embeds the current constraints struct23:19
thumperfwereade: and give us distributed pub/sub23:19
thumperfwereade: http://code.google.com/p/go-router/23:19
thumperfwereade: worth looking into I think, but perhaps for later23:19
fwereadethumper, at the lowest level the watcher have to remain, but the distribution of the events is absolutely open for improvement imo23:20
thumperfwereade: it could mean we have one watcher on things, and just publishes the change23:20
wallyworld_fwereade: the deployment constraints stuct has the container constraint; that will avoid the need for rasing an error in places where the deployment constraint is not needed, like adding a machine23:20
fwereadethumper, yeah, definitely23:20
thumperlisteners instead are then just subscribers23:20
fwereadewallyworld_, yeah, I was wondering about that23:20
wallyworld_thumper: \o/ bring it on23:20
=== wedgwood is now known as wedgwood_away
thumperI want to poke around with it for a while to test efficiency and whether it does what we need23:21
wallyworld_fwereade: if you're not immediately -1 on the idea, i'll rework the brnach to see how it pans out23:21
thumperbut the docs look promising23:21
=== davechen1y is now known as davecheney
fwereadewallyworld_, but I'm a little reluctant too -- partly, I have a feeling that struct embedding may fuck unhelpfully with serialization23:21
wallyworld_really?23:21
fwereadewallyworld_, I think you should check before you go too far23:21
fwereadewallyworld_, and we're still wearing these schema compatibility chains ;p23:22
wallyworld_fwereade: you talking aout serialisation over the wire?23:22
fwereadewallyworld_, in mongo23:22
wallyworld_fwereade: there's a separate mongo doc already and conversion functions23:22
wallyworld_between the mongo doc and constraints entity23:22
fwereadewallyworld_, ah, that's nice, I knew I did that for a reason23:22
wallyworld_:-)23:22
wallyworld_so i think it will be all good23:23
wallyworld_tw, i would have done the same design if i were doing it, so you must have done it right :-)23:23
wallyworld_bte23:23
wallyworld_tw23:23
wallyworld_btw ffs23:23
fwereadewallyworld_, ok -- just one thought, though, that we're already expecting that different bits will pay attention to different constraints23:23
fwereadewallyworld_, and I'm not quite sure that there's quite such a clean separation, once you consider matching against existing instances as well23:24
fwereadewallyworld_, I might be wrong though23:24
fwereadewallyworld_, can't hurt to investigate23:24
wallyworld_i'm not sure right now, i'll look at how it pans out23:24
* fwereade goes back to thumper's branch23:25
* thumper appreciates that23:25
wallyworld_thanks for the input23:25
* thumper takes the dog otu23:25
wallyworld_sorry thumper for jumping the queue :-)23:25
davecheneyarosales: ping23:34
fwereadethumper, LGTM with waffling23:34
thumperfwereade: thanks I'll look at those shortly23:40
thumperneed to head into town now to get science fair photos printed :-)23:40
thumperwallyworld_: can I get you to look over that branch too?23:40
thumperperhaps I'll get it landed this afternoon, which would be great23:40
* thumper heads into town23:40
=== thumper is now known as thumper-afk
arosalesdavecheney, sorry we missed each other again23:47

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!