[00:00] fwereade_: The charm being upgraded to and the old charm are both in the same location [00:01] fwereade_: When we resolve, we read it to tell what's in there [00:01] fwereade_: If the user moved backwards, the metadata will have moved backwards with it [00:01] fwereade_: and juju will automatically put it back to work, and then upgrade it again if necessary [00:07] niemeyer, ok, yes, that makes sense... I'm going to have a lie down and a think about this for a bit, and if I'm lucky I won;t come back until tomorrow :) [00:08] fwereade_: I'll be wishing for the best :-) [00:08] fwereade_: Have a good night [01:00] davecheney: Heya [01:01] niemeyer: howdy! [01:01] davecheney: Sorry for being a bit behind on your reviews.. I know you have some good stuff in there [01:01] np, i know you have your hands full [01:02] i've got pleanty of bugs (most that I have registered :) to keep me busy [01:02] niemeyer: how is your transaction stuff going ? [01:03] davecheney: Pretty good.. working on it right now, finishing insert/remove [01:03] davecheney: The rest seems solid [01:03] awesome [02:04] davecheney: What are you up to? [02:05] niemeyer: working on the final part of https://bugs.launchpad.net/juju-core/+bug/1025656 [02:06] davecheney: Neat [02:06] davecheney: Do you have plans for what to do after that? [02:07] apart from fixing bugs, i'm looking to you and mark to provide direction [02:08] davecheney: Cool, I was wondering if you might appreciate working on being a bit of an integration champion for the next couple of months [02:08] niemeyer: i would love that [02:08] davecheney: I appreciated a lot the stuff you've done in the sprint and after that to make it work for real [02:08] aww shucks [02:09] davecheney: Having you actively looking for the missing pieces of the puzzle, and chasing down broken parts either directly or by poking people to fix stuff, will be invaluable [02:10] speaking of that, i tried to run juju on arm yesterday [02:10] with mixed results [02:11] niemeyer: i heard you have a juju intergratino testing tool [02:11] you were talking about that in the sprint [02:11] would you like to hand that over ? [02:11] davecheney: I have a half baked functional test runner that I think we should revive, but it's nothing to sweat at [02:12] davecheney: lp:~juju/juju/ftests [03:37] Internal compiler error.. boom [03:37] Silly one, though [07:10] mornin' all [07:30] morning [07:36] TheMue: hiya [07:36] rogpeppe: hi, how has your free day been? [07:37] TheMue: good thanks. it was our first wedding anniversary - we went for a nice walk and for a meal at a nice restaurant. [07:38] rogpeppe: oh, congratulations [07:38] rogpeppe: we'll have our silver anniversery in two years ;) [07:38] TheMue: i see the release meeting was postponed to today, so i didn't miss anything, right? [07:38] TheMue: wow! [07:39] rogpeppe: yep, yesterday no meeting, and mark felt ill [07:59] rogpeppe: he had been at PyCon [07:59] davecheney: ah, *that* sort of ill :-) [07:59] and you remember that story he told us at that Bar across the road from the Pirate Cafe in Lisbon ? [07:59] davecheney: mornin' BTW! [07:59] that might have been related [08:00] rogpeppe: yes hello [08:00] i'm just about to sign off for a bit, gotta get the dinner on [08:00] davecheney: k [08:00] davecheney: enjoy - see you for the meeting in a little bit [09:35] moin. [09:38] Aram: moin [09:39] so, merged trunk into my three open branches, now start a new one [09:46] Aram: regarding life, when calling State.RemoveMachine() you set the state of the machine to Dying. how do you remove a machine? IMHO one should Kill() and later Die() a machine before RemoveMachine() is checking that it's dead and then removes the machine from the state. [09:47] I don't understand the question, yes, you should call Kill + Die (or just die, alive -> die is valid now) before calling RemoveMachine. [09:47] is this not so? [09:47] Aram: not in trunk [09:48] Aram: here RemoveMachine() in mstate searches the machine with life = Alive and sets it to Dying [09:48] ?!? [09:48] let me take a look [09:49] aaah [09:49] yes [09:49] removemachine [09:49] state line 71ff [09:49] for some stupid reason I though you were talking about RemoveRelation. [09:49] yes, lifecycle for relation is not done yet. [09:49] erm [09:49] Aram: hehe, no [09:49] for machine [09:49] only relations have full lifecycle in mstate ATM. [09:50] Aram: ah, ok, so i'm still right with our Kill() -> Die() agreement [09:50] yes. [09:50] *pheeew* [09:50] +1 [09:50] in fact, lifecycle for machines is what I should do right now, after I push confignodes. [09:52] Aram: i only stumbled about it when doing a cross-check. sate has lifecycle now for relations as well as a life watcher and a service relation watcher (all in the review queue) and today i'm starting with units [09:52] why units before machines? [09:53] or are you talking about the unit watcher? [09:54] Aram: i'm going bottom up, as discussed with niemeyer. relations, units, services, machines [09:54] Aram: before you free a machine everything else has to be done [10:18] Aram: another question. Relations() of Service returns only alive relations in mstate. i thought we always return all, individual life state can then be retrieved with Relation.Life() [10:19] you're right, I haven't done that yet because I didn't want to modify the tests. [10:19] (yet). [10:24] Aram: ok [10:24] btw, confignodes were beautiful. [10:24] the implementation is really nice. [10:25] it takes advantage of the dynamic, schemaless nature of mongodb. [10:30] Aram: i'm pretty sure that it will be more elegant than the zk counterpart [10:56] meeting in 4 minutes? [10:56] yy [10:57] I'll be late ~10 minutes, start without me. [10:58] mate, we'll all be late [10:58] ain't no mramm yet [10:58] hhe. [10:58] nor niemeyer [11:01] davecheney: not yet awakened [11:03] Goooood morning! [11:03] heeeelo [11:04] niemeyer: heeeeeeya [11:04] Is it party time? [11:04] niemeyer: indeed, but mramm is not yet here [11:04] Okay, I'm sending invites then [11:04] niemeyer: seems to be. [11:05] rogpeppe: Hey! [11:05] rogpeppe: Welcome back [11:07] fwereade__: ping [11:07] rogpeppe, pong [11:08] fwereade__: meeting time... [11:08] rogpeppe, oh, so it is :) [11:08] mramm is in my time zone [11:08] should I call him ? [11:08] davecheney: good plan [11:08] imma skype him [11:08] davecheney: youa doa thata [11:09] Aram: ping? [11:16] sorry for being late guys [11:16] had an emergency [11:45] are we all also going to UDS ? [11:45] I only have an invite for the pre UDS kegger [11:49] and that was all anyone wrote [12:04] https://bugs.launchpad.net/juju-core/+bug/1028437 [12:12] https://launchpad.net/juju-core/+milestone/1.3 [12:43] thanks again everybody! [12:44] I know that putting those two meetings together made it very long [12:45] anyone fancy having a chat about the design of the upgrade command? [12:45] niemeyer, fwereade__, davecheney: ^ ? [12:46] rogpeppe, a little focused elsewhere atm but I'll try to keep half an eye on one if it happens :) [12:46] fwereade__: ta [12:49] niemeyer: does this LGTY, or had you just not finished the complete review yet? https://codereview.appspot.com/6463057/ [12:50] rogpeppe: I'm just grabbing some chimarrão.. [12:50] niemeyer: nop [12:50] np [12:51] night lads, pumpkin time [12:56] night all [12:57] rogpeppe: I haven't reviewed it after you fixed.. will check it out again now [12:57] niemeyer: thanks [12:58] rogpeppe: Well, and you fixed it 2h ago.. so that's not too surprising :-) [12:58] niemeyer: np. it just seemed like there might have been some more substantial comments waiting in the wings. i'm glad that there aren't! [12:59] rogpeppe: Ah, cool, that was mainly it really [13:00] rogpeppe: LGTM, thanks! [13:00] niemeyer: lovely, thanks a lot [13:00] niemeyer: any chance we could have a brief discussion about the juju upgrade command? [13:01] niemeyer: because that's what i'm doing next, i think. [13:03] rogpeppe: Of course [13:03] rogpeppe: Let's go [13:04] niemeyer: i've been wondering about two commands [13:04] niemeyer: juju upload-tools and juju upgrade [13:05] niemeyer: and i've been wondering how the flags to upgrade might work (if we decide to provide some capability to upgrade at finer granularity than "everything all at once") [13:05] rogpeppe: We already have an --upload-tools flag in bootstrap.. what about upgrade --upload-tools? [13:05] niemeyer: the thing is that upload-tools is useful independently of upgrade [13:06] niemeyer: (for instance if you want to make tools for several different platforms available) [13:06] rogpeppe: The only moment I can foresee upload-tools being used independently is to build the public bucket [13:07] rogpeppe: Otherwise, there's nothing wrong with doing upgrade --upload-tools on the different platform too [13:07] niemeyer: because nobody will ever do cross-platform stuff with a development version? [13:07] rogpeppe: I'm sure Dave will :-) [13:08] rogpeppe: upgrade --upload-tools should work [13:08] niemeyer: hmm, i suppose it doesn't really matter that we'll get half the upgrade done first, then half later. [13:08] rogpeppe: Right [13:08] niemeyer: but i was thinking about building the public bucket too [13:08] rogpeppe: Yeah, for that it may be useful, but the semantics may be different.. we don't need an environment running for that [13:09] niemeyer: true. [13:10] rogpeppe: So I suggest getting --upload-tools working first, since that gets us going development wise.. [13:10] rogpeppe: The command should probably be called "upgrade-juju", btw [13:10] niemeyer: ok. what about --version? do we allow the specification of a desired version or not? [13:10] niemeyer: +1 [13:10] rogpeppe: To avoid the ambiguity with upgrading charms [13:11] rogpeppe: Not for now, at least [13:11] niemeyer: and another thing: it would be nice to support this workflow: change source; juju-upgrade; test; change source; juju-upgrade; test [13:12] niemeyer: if i scanned the code right the ServiceUnitWatcher is not been used today. where will it be used later? [13:12] niemeyer: but we won't be able to do that unless the version number changes each time. [13:12] niemeyer: so i was wondering about --bump-version or something like that. [13:13] rogpeppe: Yeah, sounds nice to have something like this.. I wonder about the details, though [13:13] niemeyer: which would force to version to be one more than the current max version available in the private storage. [13:13] rogpeppe: Ah, ok, without touching the local version [13:13] niemeyer: yeah [13:13] rogpeppe: Sounds sensible [13:14] TheMue: There were a bunch of things that were going to use it.. watching upgrade flags, watching agent version, ... [13:14] TheMue: Funny enough, they both were moved in different directions [13:15] niemeyer: ic [13:16] niemeyer: i had some thoughts about major-version upgrades too, but i think they can wait [13:16] rogpeppe: +1 [13:16] niemeyer: just ask because of lifecycle [13:16] fwereade__: Do we have any need for the unit watcher nowadays in the uniter? [13:16] niemeyer: the watcher has to be changed to deliver alive, dying and dead units [13:16] niemeyer: so for now, here's the plan: juju upgrade-juju [--upload-tools] [--bump-version] [13:16] fwereade__: Or anything you can foresee? [13:17] rogpeppe: +1! [13:17] niemeyer: cool, will do [13:17] niemeyer, what TheMue said, I think [13:17] rogpeppe: So neat [13:17] niemeyer, it will want to know when it's Dying/Dead [13:18] fwereade__: Ah, of course [13:18] TheMue: ^ [13:18] fwereade__: yes [13:18] fwereade__: so if we'll have a user of the watcher once it's worse change it (like the service relation watcher) [13:19] fwereade__: but today it's unused ;) [13:19] TheMue, +1 [13:22] niemeyer: one other little twist occurs to me: the upgrade-juju command will need to know the series and architecture of the machine that each agent is running on. i'm *thinking* that it will find this out by looking at the agent's proposed tools (which should be set before the agent is assigned to a machine). does that sound reasonable? [13:22] TheMue: If we do the mstate migration fast enough we don't have to worry about that in state [13:23] rogpeppe: Yeah, sounds like a reasonable way forward [13:24] rogpeppe: We'll need constraint support in the near future, but we're not there yet, and it's not clear it'd be any better either way [13:24] niemeyer: ok, so i'll stop here now and continue with mstate after lunch ;) [13:25] niemeyer: yeah, i think that when we *do* have constraints, the proposed agent tools will still encapsulate the solved constraint in exactly the way that the tools require, so won't need changing. [13:25] lunchtime (today a bit later) [13:26] TheMue: Please synchronize carefully with Aram [13:27] TheMue: It's easy to parallelize, and it's also very easy to step on each other's toes [13:27] rogpeppe: Agreed [13:27] TheMue: Enjoy [13:27] niemeyer: great. [13:38] niemeyer, https://codereview.appspot.com/6441169 [13:44] fwereade__: looking [13:44] niemeyer, when you're done, I think I also need to talk about upgrades a bit more, I'll go make a sandwich and try to marshal my thoughts [13:45] fwereade__: Done [13:46] fwereade__: It looks good, but my first impression is that it could be a lot slimmer with a config node and no new ServiceCharm type [13:48] fwereade__: Pretty much: 1) read config node; 2) return value a value b [13:48] niemeyer, hm, maybe -- but the watcher needs a single type to return [13:48] niemeyer, and, honestly, the fiddling around with types seems to cost as many lines as just doing it with a serviceNode does [13:48] fwereade__: ServiceCharmChange.. [13:49] niemeyer, yeah, fair enough [13:49] fwereade__: I don't get your comment regarding fiddling with types [13:49] fwereade__: There's a lot of logic there that can go away entirely [13:49] niemeyer, casting stuff to strings/bools, dealing with what happens if they're not right [13:50] fwereade__: Well, just don't.. that's what you're doing now [13:50] niemeyer, you mean the retryChange on the serviceNode? that's only there because I thought you'd prefer it in case we added more fields [13:50] niemeyer, what, just panic if I get rubbish in the node? [13:50] fwereade__: Using such a type is no different than doing "v, _ := i.(bool)" [13:51] niemeyer, yeah, I guess, it just seemed like ConfigNode was overkill for two totally predictable fields [13:52] niemeyer, anyway, happy to do it as you suggest [13:52] fwereade__: I'd agree if we weren't doing non-fun stuff by hand as an alternative [13:53] niemeyer, heh, the confignode seemed to me like the unfun way there [13:53] niemeyer, anyway sgtm [13:53] * fwereade__ remembers why he'd put the toaster away in the cupboard in the first place, you need to watch the damn thing like a hawk :/ [13:54] fwereade__: Unfun as in significantly more logic, in exchange for ... ? [13:54] fwereade__: I may be on crack [13:54] fwereade__: Let's see how it looks like [13:59] niemeyer, it'll probably actually trun out better, your instincts are usually solid :0 [14:01] fwereade__: Cheers! :) [14:28] niemeyer, ok, should be ready to propose again in a mo, but I think I need another chat about charm upgrade errors [14:29] niemeyer, so, when we get an error, it's because there was a conflict trying to pull into the charm tree from the charms repo [14:30] fwereade__: Right [14:31] niemeyer, but I'm still not sure about how we handle it -- what does juju resolved imply here? [14:32] niemeyer, ISTM that it just means the user has done *something*, and they might themselves have updated the charm to the desired version and they might have reverted and made some more changes that should allow the upgrade to go ahead [14:32] fwereade__: We could make it do "bzr resolved --all + bzr commit -m ..." [14:33] niemeyer, and then we just try to pull again, which might do nothing or might upgrade successfully or might cause a new error [14:33] ? [14:33] niemeyer, and from that operation we determine what the new state of the charm really is? [14:34] fwereade__: I think we can just allow the charm to run again if the operation succeeds [14:34] fwereade__: Oh, wait.. we can't [14:35] fwereade__: Because we need to take a decision regarding whether to run the hook or not [14:35] niemeyer, ok, and the success is what determines what the version is? oh, bother [14:35] niemeyer, ah ok, yeah, that should be easy though [14:36] fwereade__: Yeah, I was just thinking that we could put the charm back in normal working mode and allow it to re-run the upgrade, but we can't.. we really have to take a decision right there if the upgrade is happening or not [14:36] niemeyer, yeah, I think that once we've started upgrading we shouldn't stop until we succeed [14:37] fwereade__: Allowing for a rollback also seems fine.. if diskVersion == oldVersion { abort upgrade and restart normally } [14:39] niemeyer, sorry, I'm not following [14:39] niemeyer, what causes us to do that? [14:39] fwereade__: We're saving the charm version within the bzr branch we use to prepare the upgrade, right? [14:40] niemeyer, perhaps; but where exactly? [14:41] niemeyer, and can we depend on that if it's something the user has access to? [14:41] fwereade__: I assumed it would be part of the charm directory itself.. we talked about unit.SetCharm(charm) when the unit is running, to report the version actually in use, right? [14:41] niemeyer, the second part, yes [14:41] fwereade__: Well, the user has access to the kernel of the machine too :-) [14:42] niemeyer, sure, but we aren't telling the user "hey, go and fix the kernel" [14:42] fwereade__: This is metadata that should be in a dot file of some sort within the charm directory.. if the user hacks that manually, good luck :) [14:43] fwereade__: Are we not? If the kernel breaks, we definitely say that [14:43] niemeyer, my instinct says that we should be storing it somewhere outside the charm directory and updating it based on our knowledge of the success of change operations, just as I expected we would the installing/upgrading/installed state of the charm [14:44] fwereade__: I don't get the contention in this bit.. you're saving the whole charm and snapshotting it.. how come it feels bad to save the version being snapshotted with it? [14:46] niemeyer, I'm not entirely sure it does, I just was expecting to leave the charm directory as the user's playground as much as possible [14:47] niemeyer, and to keep the state we depend upon separate [14:47] niemeyer, but I'm pretty comfortable either way [14:48] fwereade__: I think it's fine to have the state, as in, what's going on with the charm, somewhere else [14:49] fwereade__: But being able to tell what charm version is that thing we're looking at feels very convenient [14:49] fwereade__: It's much more likely that a version being snapshotted with the content will be right than something we keep ourselves independently [14:50] fwereade__: Otherwise, "bzr revert".. boom.. state is wrong [14:50] niemeyer, ok, that seems good then [14:51] niemeyer, so external state controls how we interpret internal state [14:51] fwereade__: Right.. external state tells what we've been doing about it.. internal state simply says what it is ATM [14:51] fwereade__: Well.. version really.. let's avoid one more "state" :-) [14:53] niemeyer, ok, I have some lurking uncertainty but I won't know what is is or whether it's real until I've implemented a bit more :) [14:53] fwereade__: I can sympathize [14:54] niemeyer, anyway, reproposed https://codereview.appspot.com/6441169 :) [14:55] fwereade__: How did it feel? [14:55] niemeyer, just as good -- not convinced it's really better but I don;t think it's any worse :) [14:56] niemeyer, it probably did get a little smaller though, I didn't count [14:57] fwereade__: Yeah, a few types/functions/closures less, thanks [15:07] fwereade__: Reviewed [15:08] fwereade__: Glad to see 100 lines going away [15:08] niemeyer, yeah, always nice :) [15:08] niemeyer, thanks [15:09] niemeyer, all looks sound [16:34] Lunch [18:23] fwereade_: ping [19:45] Somewhat of a quiet day.. [20:13] niemeyer: i hope to remedy that tomorrow, but i'm off now.... [20:13] niemeyer: see you tomorrow [20:13] rogpeppe: A calm day is good every once in a while.. [20:13] rogpeppe: Have a good time!