[00:57] jimbaker, you can do the encode/decode on the server [00:57] jimbaker, the transport is irrelevant to maintaining the format [00:58] ie transfer in yaml, encode/decode cycle in json [03:07] hazmat, i will see if i can make that work [03:07] hazmat, in principle this makes sense, it just is the approach i first tried. however, the code is much more refactored, so it's possible this is simpler to do [06:06] morning davecheney [06:15] fwereade: howdy [06:16] davecheney, how's it going? [06:18] fwereade: good; so we're back to lisbon in july i hear [06:18] davecheney, yeah, that's my understanding [06:18] davecheney, is that better or worse for you? [06:19] fwereade: probably better, although I dunno if mark will like the prices, 2700 euro [06:19] well, between 2000 and 2700 [06:19] davecheney, cool -- and, well, the cheapest I saw for me to POA was >2000 euros [06:19] that is what i decoded from the travel agent blather [06:19] fwereade: it was more the 45 hours travel that didn't do it for me [06:20] * fwereade winces *hard* [06:20] i think it went SYD -> the frikken moon -> POA [06:20] davecheney, haha [06:20] then a return trip via saturn [06:20] which is insane, because it's shorter to go SYD -> Perth -> POA, than SYD -> LAX [06:20] geographically speaking [06:21] davecheney, the longest I've had to deal with (without incorporating a bed, at least) is probably *just* over 24 hours door to door [06:21] davecheney, and that was bad enough [06:21] davecheney, if you have flights that long though it may well be worth checking with marianna if you can have an overnight break in an airport hotel [06:22] fwereade: yeah, save my back [06:22] davecheney, I wasn't expecting that, but my very first trip had what would otherwise have been 8 hours on the floor of an airport [06:22] 16 hours LAX -> SYD [06:22] davecheney, swapping that for 5 hours in an actualy bed was very much a good move [06:22] was pretty sore by the end of it [06:22] davecheney, I bet [06:23] oh, and a return trip to POA was > 5,000 AUD, which was the 'piss off, we don't want to do it price' [06:26] davecheney, crikey [06:31] heya TheMue [06:41] fwereade: Morning [07:32] rog, TheMue: I would welcome opinions on https://codereview.appspot.com/6317043 (which, in hindsight, reads somewhat passive-aggressively, but I don't really know what to do about that :/) [07:32] fwereade: Passive-aggressively? [07:33] TheMue, http://en.wikipedia.org/wiki/Passive%E2%80%93aggressive_behavior [07:34] TheMue, the problem is perhaps that I didn't express the issues I had with that approach in those terms during the discussion [07:35] TheMue, in my mind, the assumed context was "it's obvious that this is a lot more work", but that may in fact not have been obvious to all participants [07:36] fwereade: Understand === TheRealMue is now known as TheMue [07:42] fwereade: From a technological perspective LGTM. [07:43] fwereade: But I need more background to understand the problem you want to solve. [07:43] fwereade: Best in some simple words for a "newbie". ;) [07:44] fwereade: i see what you mean :-) [07:45] fwereade, TheMue: mornin' BTW [07:58] rog: Morning [07:58] hmm, my computer just crashed [07:58] if anyone said anything after my "mornin' BTW", i missed it [08:01] rog: Only my greeting after you re-entered the channel [08:01] TheMue: cool [08:05] TheMue, rog: rewritten; is it more helpful now? [08:07] TheMue, btw, wrt what you said earlier, that CL is IMO entirely unnecessary, and that's the reason I was whining so extravagantly in the description [08:08] rog, TheMue: btw, do you have a moment to discuss how I should test charm store access, from outside the charm package, without hitting the real charm store? [08:09] fwereade: sure [08:10] rog, ok... short of screwing up /etc/hosts, what *can* I do? :) [08:11] fwereade: is the charm store address hard-coded into the charm package? [08:11] rog, yeah [08:11] fwereade: well, one possibility would be to make it an exported variable that could be changed by testds [08:12] rog, sure; it just feels last-resorty [08:12] fwereade: of course then you've got to run some kind of your own server [08:12] fwereade: After reading the changed description it's more clear, thx. [08:12] rog, AIUI niemeyer is rather against exposing that at all [08:13] rog, as it is it's a const (because that was my understanding of niemeyer's perspective on the charm store) [08:14] fwereade: If the address is returned by a function we could introduce a package local variable switch by a function SetTestMode(bool). [08:14] fwereade: so what functionality are you needing to test exactly? [08:14] fwereade: When setting it to true the function returns localhost. [08:14] rog, that we can deploy a charm from the charm store [08:15] rog, I think there *may* be a case to be made that InferRepository is properly tested, and both local repos and the charm store are properly tested [08:16] rog, and that therefore hitting just one of the InferRepository code paths is actually adequate [08:16] rog, but my go-testing-fu is not adequate to determine whether this is defensible [08:16] fwereade: that sounds like it may be reasonable. [08:16] fwereade: there's no point in testing the same functionality in two different places [08:17] rog, yeah, that's what my heart says [08:18] rog, but it was hard work forcing myself to test everything a million different times in python when I joined the project [08:18] rog, and I haven't quite figured out how the prevailing rules have changed [08:19] rog, ok, I'll propose it as-is with an explanation [08:19] fwereade: seems reasonable. [08:46] rog, fwereade: Different topic, did you already booked you flights? [08:46] TheMue: yes, i've booked. arriving late sunday evening. [08:47] TheMue, ah yeah, I crash-stopped the booking the other day and haven't redone it yet [08:47] TheMue, ty for reminder [08:47] fwereade: np ;) [08:48] rog: Yes, I'll arrive at 21:45 and depart on Sat at 9:10. [08:50] * TheMue only hopes he's now right with location and date :D [09:21] fwereade: when you say "we discard the actual data and retain only the version." which code are you referring to? [09:21] fwereade: or is this stuff as yet to be written? [09:21] rog, the python, and the proposed code [09:22] fwereade: so, in your view, it would be better to have a ConfigNodeWatcher or something like that? [09:22] fwereade: that passes the settings along as they change [09:22] rog, well, I don't think we need anything other than a map[string]interface{} [09:23] rog, except hmm no [09:23] fwereade: no? [09:23] rog, blast, I haven't fully thought through the cross-relation settings-settings [09:24] fwereade: that, i think, might have been part of gustavo's point [09:24] So, added the flight data to the wiki page. [09:24] fwereade: but explain, please. [09:24] rog, I'm just mainly thinking that there is no reason whatsoever for settings of relations of *other* units to be writable at all [09:24] rog, offering the opportunity to do do so can only lead to bugs and confusion [09:24] fwereade: why does setting *writing* make any difference to what we're talking about, which is *reading* settings, no? [09:25] rog, maybe it's just a slight discomfort with ConfigNodes used in read-only contexts [09:26] rog, (and a ConfigNode can be refreshed, too, which doesn't fit my idea of what we should be doing) [09:26] fwereade: you could return a ConfigNode with a nil zk :-) [09:26] rog, heh, that could work :) [09:27] fwereade: but tbh, why does it matter? we're not going to use it to set values. [09:27] rog: Not nice, better an own type. [09:27] rog, but actually, it's not an issue anyway -- you'll never see changes of your own unit's relation settings [09:27] fwereade: a config node represents a node with settings. we can choose not to write to it. [09:27] rog, so nothing we are ever watching ever has any reason to be anything other than read [09:28] fwereade: i'd be happy if it was <-chan map[string]interface{} too. [09:29] fwereade: is there a problem with that? [09:29] rog, and IMO we shouldn't even offer the opportunity to write it or update it, for similar reasons to those people use to justify public/private distinctions in code [09:29] rog, I'd like that [09:29] rog, ok, people can still write to a map[string]interface{} but at least they have no way of screwing up Zk state by doing so [09:30] fwereade: Otherwise just do a little type struct { data map[string]interface{} } with a Get(string) interface{}. So there's no chance. [09:31] TheMue, indeed, that would work nicely [09:31] TheMue: nah, let them use the map directly. [09:31] TheMue: there's no harm in writing to it. [09:31] TheMue: and then it's easy to range over, etc [09:32] rog, in general my problem is simply with discarding well-sequenced information just so we can approximately reconstruct it later, with additional code to make sure the sequence breaks are less common than they might be [09:32] i'd say either use a map, or a ConfigNode, but not something in between [09:32] rog: No harm to ZK, yes. OK, and the range may be an argument. [09:33] TheMue: an argument? [09:33] rog, I'm not even saying this leads to perfect consistency (config-get is still out of sequence), but it's more consistent with less code and I just don't see the downside [09:33] rog: For using the map directly. [09:33] fwereade: i definitely don't see the point in a version watcher. [09:33] rog, ok, there is a downside: we will store more data locally on the nodes [09:34] fwereade: we've got to get that data anyway, right? [09:34] rog, that doesn't seem to me to be a strong argument for all the extra complexity :) [09:34] rog, indeed [09:35] rog: The ssh.go in state is by you, isn't it? [09:35] fwereade: and the version watcher is fetching all the content anyway. [09:35] rog, yeah [09:35] TheMue: no, that's me [09:35] rog, this is rather my point :) [09:36] rog: You *are* the ssh.go? ;) [09:36] * rog blushes [09:36] TheMue, ssh, we don't want reporters involved [09:36] ;) [09:37] TheMue: i think you meant to say "You are *the* ssh.go?" :-) [09:37] rog: There are several fmt.Printf() in it, are they still needed? [09:37] TheMue: erp [09:37] TheMue: i don't see any fmt.Prints [09:38] TheMue: i see log.Printfs [09:38] rog: Oh, my mistake, just seen log. Sh**! [09:38] rog: So here the prefix "state:" is also still ok. [09:38] TheMue: they're probably a little excessive tbh, but i find them useful for the time being. perhaps they should be log.Debugf's [09:39] TheMue: yeah. [09:40] rog: I'm adding the errorContextf() so I found them and thought it would be debugging code. But in the log it makes sense. [09:40] * rog goes downstairs to get a bowl of cereal [10:02] rog, I was just wondering some more about MachineUnitsWatcher [10:02] fwereade: uh huh? [10:02] rog, as you point out, we can't make a *Unit from a deleted unit [10:03] rog, but how are we keeping track of the things on the other end? might unit names be a better thing to pass around than units? [10:03] fwereade: i suggested that originally, but gustavo didn't like it [10:03] rog, isn't that how the client will be referring to the units in the first place? [10:03] fwereade: and actually, he's right [10:03] rog, bah, ok [10:03] rog, go on [10:04] fwereade: because if you look at the container API, it's phrased in terms of *Units [10:04] fwereade: which seems nice [10:04] rog, ah, ok, that does make sense [10:04] fwereade: although actually it might only use 'em for their names [10:05] rog, I kinda suspect it would ;p [10:05] rog, I have an analogous situation I would welcome input on [10:05] rog, ServiceRelationsWatcher [10:05] fwereade: yup. but... it's entirely possible that it might use more info from the unit for deployment in the future. [10:06] rog, yeah, could be [10:06] fwereade: although then we've got the out-of-date info problem, potentially. [10:07] fwereade: similar thing with ServiceRelationsWatcher, presumably? [10:08] rog, yeah; specifically I suspect we'll have a map[something]*RelatedUnitsWatcher that I'll want to remove watchers from (and stop them) as the service leaves relations [10:09] fwereade: so the ServiceRelationsWatcher is maintaining several watchers? [10:09] rog, I don't *think* it makes sense to use actual *Relations as keys (I should probably reread the spec and find out exactly what it takes to be usable as a map key) [10:10] fwereade: any pointer can be a map key [10:10] fwereade: (anything that defines equality can be a map key) [10:10] rog, no, the unit agent is maintaining several watchers which it starts/stops in response to the ServiceRelationsWatcher's changes [10:10] rog, ok, so actually I should be doing exactly what you're doing [10:11] rog, and define equality on Relations [10:11] fwereade: equality is defined on any pointer [10:11] fwereade: (the pointers are compared) [10:11] fwereade: you can't "define equality" on something [10:11] rog, ok, then that's surely not what I want [10:12] rog, so I had thought, my misinterpretation of your comments gave me hope [10:12] fwereade: what defines a relation in this context? [10:12] rog, the endpoints [10:12] fwereade: i.e. what information would you put in a key? [10:12] rog, hold on a sec [10:13] rog, what do you do in the machine agent when it comes up after an unexpected exit and wants to take down a container for a unit that no longer exists? [10:13] rog, how do you create the *Unit in that case? [10:14] rog, I have to solve the same problem in the unit agent with *Relations, I think [10:14] fwereade: there's no *Unit in that case [10:14] rog, how do you deal with it then? [10:14] fwereade: i have to look at running containers [10:14] fwereade: and see which of them do or don't correspond with current *Units [10:14] rog, and somehow find out what *Units... hmm, ok [10:15] fwereade: when we come up after an unexpected exit, we will never see units deleted. [10:15] rog, and how do you determine correspondence? by name, I presume? [10:15] fwereade: yeah, probably. [10:15] rog, yeah, I'm talking about purely at the agent level, when it has to diff last-known state against what the watcher says is current [10:16] rog, hold on [10:16] fwereade: depends how you store last-known-state [10:16] rog, can you give me an example of when we might want to know anything other than unit name to set up a container? [10:17] fwereade: perhaps the unit's service might have settings that influence how the LXC container is allocated [10:18] rog, isn't it a matter of setting up a named container, and poking in an upstart script to run `jujud unit foobar/17 ...`? [10:18] rog, the unit agent itself should surely be responsible for everything else [10:18] fwereade: i think things are a bit more involved when we're starting stuff within LXC containers [10:18] fwereade: but yeah. [10:19] rog, hmm, I fear I forsee another discussion-heavy afternoon :/ [10:20] fwereade: for your ServiceRelationsWatcher, what would you use that corresponds to the use of unit names in MachineUnitsWatcher? [10:20] rog, that's what I'm asking [10:20] fwereade: for the record, i'm off on holiday at the weekend for two weeks. so good to get the discussions in now... [10:20] rog, I'm inclined to tweak describeEndpoints to sort the endpoints before joining them with " " and using that as a key [10:21] rog, very good to, I'm off thu/fri [10:21] fwereade: ah, so this is last day we're online together before ages away... [10:21] rog, one of those irritating situations where, really, I love to spend time with my family but right now feels rather inconvenient [10:21] fwereade: yeah [10:22] rog, I suspect I'll be sneaking away to work at night occasionally, but I probably won't coincide with you [10:22] fwereade: we'll be in the camper van in cornwall. almost entirely offline. [10:23] rog, cool, have fun :) [10:23] fwereade: you can use a struct as a key BTW [10:23] fwereade: e.g. map[struct{e1, e2 string}] *Relation [10:23] fwereade: so no need to join with " " [10:24] rog, number of endpoints is abritrary [10:24] rog, and I'm pretty sure I can't use a slice as a key, right? [10:24] fwereade: that's right [10:25] fwereade: number of endpoints is arbitrary pending more relation kinds than provides/requires/peer [10:25] ? [10:25] fwereade: or is it not to do with that? [10:25] rog, ATM only possibilities are 1 and 2 [10:25] rog, that's exactly what it is [10:26] fwereade: why can't you use the relation key as a key? [10:27] rog, because I'm pretty certain maintaining the watchers is the unit agent's responsibility, and it doesn't feel like state is the right package [10:28] rog, ahhhh hmmm maybe I could use the relation ident [10:28] fwereade: we could add a Key method to Relation, i suppose. [10:28] rog, we expose that to users, even though it's just a thinly-disguised key [10:28] fwereade: relation ident? [10:28] rog, I'd prefer not to go down that path unless we have to [10:29] rog, for relation-0000000001, it's "relation-1" [10:29] fwereade: that seems like a fine thing to use for a key [10:29] fwereade: especially if it's exposed to users [10:30] rog, we export JUJU_RELATION_IDENT in foo-relation-* hooks so the charm can distinguish between the different runtime relations using the foo charm relation [10:30] fwereade: so maintaining a map of that to *Relation seems ideal to me [10:31] rog, I was thinking more a map of that to RelatedUnitsWatcher [10:31] fwereade: ok, sure. [10:31] rog, which might hold a *Relation, but I'm not even sure that's needed past construction time [10:32] * fwereade slopes off for cig/think time [10:56] hello everyone [11:02] Aram, heyhey [11:10] Aram: hiya [11:40] rog, fwiw, I was wrong: it's JUJU_RELATION_ID, not IDENT, and it's not relation-1: it's relation-name-according-to-service-1 [11:41] fwereade: well, maybe that's a reasonable form for the unit agent to key it on too. because it's not going to hold any relations for any other services, right? [11:42] rog, yeah, agreed, I just wanted to corrcet the misconceptions that might right now be burrowing through your brain :) [11:42] fwereade: rooted out! [12:08] rog, btw, should I do a quick CL for the MustErr watcher stuff we discussed yesterday? [12:08] fwereade: sounds good [12:08] rog, or are you on that? [12:08] rog, cool [12:08] fwereade: nope, i'm on unit agent tests currently [12:09] rog, ah cool, what sort of thing? [12:09] fwereade: oops, i meant machine agent [12:09] doh [12:09] rog, ah rigt :) [12:09] boring :-) [12:13] fwereade: i'm just looking at cmd/juju/deploy.go BTW [12:13] fwereade: i wondered if unpacking the charm in memory is a good idea [12:13] fwereade: perhaps it would be better to store in a temp file instead [12:13] rog, hmm, maybe so [12:14] fwereade: and then for *charm.Bundle you could use the raw file as is [12:14] fwereade: (no need to read twice, even) [12:15] fwereade: 'cos charms could potentially be arbitrarily big [12:15] rog, indeed so [12:18] fwereade: i wonder if the charm.Charm interface should support BundleTo. [12:19] rog, possibly, I'd have to think about it a bit :) [12:20] fwereade: yeah, there may well be a good reason it's not there [12:21] fwereade: at the very least, charm.Bundle could implement BundleTo :-) [12:22] fwereade: although that kinda loses the point [12:29] rog: a new inferno fan: http://debu.gs/entries/inferno-part-2-let-s-make-a-cluster [12:30] Aram: not new - this is the guy that did the post about my shell. [12:30] ah, yes. [12:30] Aram: but it's really nice to see him get excited about it all [12:30] Aram: it still excites me actually, but i've kinda left it behind [12:32] Aram: it's always great to see people play with toys you've made yourself. [12:32] yes, indeed. [12:32] hi niemeyer. [12:33] Hello! [12:33] niemeyer: yo! [12:36] niemeyer, heyhey [12:41] niemeyer, quick check: rick spencer is the head of my business unit, right? [12:41] fwereade: yep [12:42] niemeyer, cool, tyvm [12:45] fwereade: That reminds me of a guy back at Conectiva [12:45] niemeyer, oh yes? [12:45] fwereade: He ranted with someone online on an internal mailing list [12:46] fwereade: A couple of months after joining the company [12:46] fwereade: Then, right after sending his last email to the rant, he asks "Who's that silly guy!?" [12:46] fwereade: "It's your direct manager, dude.." [12:46] niemeyer, haha [12:47] niemeyer, in this case it's just because the travel agent wants to know [12:47] fwereade: Good thing you're not fighting with him yet! ;-D [12:47] niemeyer, but I hope this guy called "sabdfl" I've been insulting isn't anyone in particular ;p [12:47] LOL [12:49] niemeyer, btw, https://codereview.appspot.com/6298097 should be very quite, it's the mustErr stuff we discussed yesterday [12:49] s/quite/quick/ [12:49] fwereade: Looking [12:49] I intend to to a good pass on everything now, btw [12:50] Aram: Specially on yours long standing [12:50] niemeyer, cool, thanks :) [12:51] niemeyer: great, I have charms and units in the queue. === TheRealMue is now known as TheMue [12:59] fwereade: Done [13:00] Grrr, router had a problem. [13:00] niemeyer, cool, thanks [13:11] fwereade: did you run all tests before submitting https://codereview.appspot.com/6298097/ ? [13:12] fwereade: i suspect it may have broken other packages [13:12] rog, oh, crap, I may have just done state/... [13:12] fwereade: yeah, i think it breaks cmd/jujud [13:12] fwereade: though i haven't checked [13:12] rog, I saw a ... in history, and... [13:12] * fwereade hangs head in shame [13:13] rog, you're absolutely right :( [13:15] * rog goes for lunch in the blessed sunshine [13:17] niemeyer, observe screwup above; now I've fixed that and really truly properly verified it's unbroken, what would happen if I were to just lbox propose/submit again in the same branch? [13:18] meh, how can flight search technology be so bad. [13:18] I mean, it's a trivial problem to solve. [13:19] fwereade: Won't work [13:19] fwereade: Just branch it over with "bzr checkout -b -fixup" [13:19] fwereade: and then "lbox propose; lbox submit" [13:19] niemeyer, cheers, will do [13:20] hmm, lbox propose (without -cr) uses the previously created cr on rietveld if it exists? [13:31] rog, it appears to me that there's something up with the container tests as well... [13:31] Aram: Yep [13:31] greta [13:31] great [13:32] Aram: The previous CL/merge proposal/whatever.. it updates whatever was done [13:32] rog, http://paste.ubuntu.com/1050880/ [13:32] Aram: Well, almost.. it won't touch existing bugs [13:33] fwereade: When I now start with ServiceRelation.AddUnit(), would I conflict with you? [13:33] TheMue, yes, please don't touch ServiceRelation, it's gone [13:33] TheMue, what are you planning? [13:34] fwereade: I'm thinking of continuing with the relation methods, but I know you're doing some changes there. [13:34] fwereade: Relation is gone, but ServiceRelation not yet. [13:35] TheMue, I would very much prefer if it we didn't step on each other relation-wise [13:35] fwereade: That's why I asked before. ;) [13:35] TheMue, I have one CL out and a whole *bunch* of stuff waiting in the wings that wants only for agreement and go-ahead on the current bits [13:36] TheMue, there's a missing thing for state.Service though... we don't have any way to set service config [13:36] TheMue, and I think that then implies the whole format 2 business [13:36] fwereade: OK, so I'll concentrate on other parts. [13:36] Aram: Review sent [13:36] thanks [13:36] Aram: Please let me know if you'd like to talk [13:36] fwereade: OK, I'll take a look. [13:37] Aram: Most points should be straightforward, though [13:37] Aram: Glad to see this moving! [13:37] yes! [13:41] fwereade: Todays code only has a get_config. What would you exactly expect from a SetConfig()? Isn't it normally done by retrieving the config, even if it's empty and then do changes and write it again? [13:42] TheMue, huh, you're perfectly right [13:43] TheMue, maybe a look at the format 2 stuff then if you're not too much averse to it? [13:43] TheMue, I had it planned but I seem to have got rather caught up in these bits, and I think it'll take me a few days to burn it down again [13:45] fwereade: No problem, but where do I find infos about the format 2 stuff? [13:46] TheMue, just a mo; I'll collect what I can find for you [13:47] fwereade: Thx. [13:55] niemeyer: right now in ZK we use incrementing integer id's for some thing, this has the property of making nice names (/machine-0, /unit-1-0, etc), but in MongoDB incrementing integers are nasty: http://www.mongodb.org/display/DOCS/How+to+Make+an+Auto+Incrementing+Field [13:55] so so we still want to go with incrementing integers in mongodb, or are random numbers fine? [14:00] Aram: flight search is actually turing complete: http://research.swtch.com/ita [14:00] Aram: A machine id is an integer.. this isn't about MongoDB or ZooKeeper this is the API itself [14:01] niemeyer: I'm not debating whether it should be an integer or not, but whether should be an incrementing integer or a random one :)/ [14:01] Aram: It should remain an incrementing integer [14:01] Aram: 9879872198729831 won't be great :) [14:03] fwereade: ah! i see the problem. i've got jujud in my path, but you haven't. [14:03] fwereade: i think the test should probably create a fake jujud [14:03] fwereade: and set the path to point to it [14:03] rog, sounds sensible [14:04] rog: I know the problem is computationally hard, my issue is that nobody publicly exports the flight data so people could build better 3rd party software than the crap software flight agencies provide. [14:04] Aram: true too [14:04] Aram: Btw, how is it nasty to increment an integer in MongoDB? [14:05] Aram: It's a trivial inc operation [14:06] Aram: I suggest adding a "sequence" collection, where the _id is the collection name [14:06] Aram: and having a sequence(name string) (int, error) helper in State [14:06] niemeyer: so in that link I sent you, you prefer option #1 to option #2? [14:07] that's perfectly fine by me. [14:08] Aram: I prefer to have a scheme as suggested above [14:08] Aram: So we can reuse this next time we need a sequence number [14:09] yes, it's great. [14:09] ok. [14:18] niemeyer: is it possible to create a new bzr branch, but use the same CR as an existing branch? [14:18] Aram: nope [14:18] bummer [14:18] Aram: Well, maybe yes [14:18] Aram: But why? [14:19] niemeyer: I did the mistake of using my trunk for the changes you reviewed, I ported them to a different branch, but that also created a new CR... [14:20] Aram: I think it can be made to work: [14:20] Well, you already created a new CL, right? [14:20] yes [14:20] Aram: Edit the description in the *new* merge proposal, and change the very last link of codereview to point to the proper place [14:21] interesting [14:21] Aram: Then, mark the last branch as Rejected [14:21] Aram: and run "lbox propose" again [14:21] Aram: It's not great to be doing this because the merge proposal will lose part of the history [14:22] ok. [14:30] TheMue: Reviewed https://codereview.appspot.com/6300098/ again [14:30] niemeyer: Just seen, thx [14:30] TheMue: please ping me if you repush soon and I'll have an immediate look so we can integrate [14:31] Aram: Yeah, so we need to do something with this, since it's a copy of the last branch: https://codereview.appspot.com/6295103/ [14:31] niemeyer: Should be only a few moments. [14:31] Aram: Are you doing the trick mentioned? [14:31] niemeyer: yes [14:31] Aram: Super [14:33] niemeyer: how do I mark a branch as rejected? [14:33] Aram: Sorry, not the branch, the merge proposal [14:33] same question, where do I do that? :) [14:34] Aram: I've marked it as Work In Progress [14:34] Aram: In the merge proposal page itself [14:34] * niemeyer finds [14:35] "Sorry, there was a problem connecting to the Launchpad server." [14:35] heh [14:35] "Delete proposal to merge"? [14:35] this onw [14:35] one [14:35] ? [14:36] Aram: Done [14:36] Aram: No need to delete [14:36] Aram: It has some history there [14:36] Aram: I've marked it as Rejected [14:36] Aram: and added a note pointing out to the new MP [14:40] niemeyer: I think it worked, https://codereview.appspot.com/6304099 shows the correct branch. [14:40] niemeyer: but the issue is closed, is that expected? [14:46] Aram: No, this doesn't happen automatically.. have you closed it by hand? [14:46] Aram: lbox will only close on submit, which hasn't happened [14:46] niemeyer: ah, yes, sorry, my fault, forgot I closed it yesterday. [14:46] so I can open it again [14:46] done [14:46] great [14:46] Super [14:47] thanks again niemeyer [14:50] niemeyer: So, had a text conflict to resolve first, but now it's in again. [14:58] bcsaller, ping [15:15] Aram: np! [15:15] fwereade: you've got a review on https://codereview.appspot.com/6312048/ [15:15] fwereade: Let me know if you'd like to talk about it so we can get this in soonish [15:15] fwereade: I'll be stepping out for lunch in a moment [15:16] TheMue: Super, will look [15:17] niemeyer, I'd be happy to, reading now [15:18] TheMue: LGTM [15:18] niemeyer: Thx, will submit it now. [15:19] niemeyer, ok, I think I agree with what you have there [15:19] niemeyer, sorry about the Id, it seemed like a good idea at the time :/ [15:19] fwereade: Actually, LGTM with that in.. doesn't seem to be controversial in any way. [15:20] fwereade: No worries, I actually find the value of JUJU_RELATION_ID a good user-facing feature [15:20] fwereade: When sitting within the unit, that relation id makes perfect sense [15:20] fwereade: In state not so much [15:22] niemeyer, well, I'd love to get this in... but I guess I'm not sure what in particular differentiates the review from a LGTM-with-these-fixes one [15:23] niemeyer, most of your suggested changes seem sane and simple [15:23] niemeyer, which ones are subtler than I think at first glance? [15:23] fwereade: Actually, LGTM with that in.. doesn't seem to be controversial in any way. [15:23] fwereade: ? [15:23] niemeyer, ah, ok, I see [15:24] niemeyer, so, you're OK with the implementation of Id as it is? [15:24] fwereade: Nope, with the suggested change [15:24] niemeyer, just the int then, cool [15:24] fwereade: func (r *Relation) Id() int [15:24] niemeyer, great [15:24] fwereade: (no error even.. we have the key) [15:24] niemeyer, indeed [15:25] niemeyer, ok, I'll fix those and submit [15:25] niemeyer, tyvm [15:25] fwereade: Thanks a lot [15:25] niemeyer, a pleasure :) [15:38] fwereade: I'm afraid I don't have enough time to cover it before lunch, but maybe we should talk about the VersionWatcher stuff afterwards [15:39] fwereade: I'm missing the context [15:39] niemeyer, I would be very keen to [15:39] fwereade: You seem unhappy with it, and I don't even know why we need it, which means we should talk :) [15:39] niemeyer, definitely -- I was trying to talk about it yesterday, but in hindsight I was framing the discussion all wrong [15:40] niemeyer, I'm not sure precisely when I'll be around this evening [15:40] fwereade: No worries [15:40] fwereade: We can talk whenever [15:41] fwereade: We have enough overlap for this to not be an issue [15:41] niemeyer, and I'm off after that until the end of the week -- *but* this is a subject dear to my heart atm so I will definitely track you down at some point before the weekend [15:41] fwereade: Oh, I didn't know that, ok [15:41] niemeyer, (it is a pleasure to see my family... but curse the timing) [15:42] fwereade: Take a good rest and enjoy their presence [15:42] niemeyer, I *think* I mentioned it a while ago but I forgot to actually book it... [15:42] niemeyer, cheers :) [15:43] niemeyer, after that I think I'm saving my holiday for post 12.10 :) [15:43] fwereade: It will be quite a walk towards it :) [15:43] niemeyer, it feels like we're doing ok so far :) [15:44] fwereade: Indeed, very happy with how we've been pushing things forward [15:44] fwereade: Real momentum [15:46] niemeyer, absolutely, it makes even a short holiday feel quite the unwelcome intrusion ;) [15:49] fwereade: Hehe :) [15:49] Alright! [15:50] Time to find some nice lunch [16:07] fwereade: is it possible for a service to be subordinate to more than one other service? [16:10] rog, hmmm [16:10] rog, I wouldn't swar you couldn't write a charm that implied such a relationship :/ [16:10] fwereade: it certainly seems possible in the state API. [16:10] rog, but I don't think such an arrangement makes any sense [16:11] rog, how so? [16:12] rog, ah, wait, reparsed: I think it makes perfect sense [16:13] rog, deploy wordpress; deploy mysql; depoy logging; add-relation logging mysql; add-relation loggin wordpress [16:13] fwereade: you can d1 = AddService("wordpress"); d2 = AddService("wordpress"); sub1 = AddService(subord); d1.AddUnit(); sub1.AddUnitSubo .... you got it [16:13] rog, sorry, I have units on the brain at the moment, ahd totally wrong context [16:13] fwereade: ok, so that's fine. good [16:16] fwereade: thanks [16:52] niemeyer: did all the requested changes including monotonically increasing integer ids and state compatible errors. [16:55] niemeyer, fwereade: machine agent implementation: https://codereview.appspot.com/6294096/ [17:00] rog, cool [17:01] fwereade: if you manage to give some feedback today, that would be marvellous, as it would be nice to polish it tomorrow & friday. [17:01] rog, I'll do what I can :) [17:01] fwereade: ta! :-) [17:02] where should this be filed http://15.185.100.228/out.charmload [17:03] imbrandon: juju-core, i think [17:03] k [17:05] niemeyer: looks like your schema checking isn't quite sufficient... [17:09] https://bugs.launchpad.net/juju-core/+bug/1015700 :) [17:11] rog, initial thoughts posted [17:12] fwereade: thanks. BTW the fact that the watcher returns subord units too was decided after fairly extensive debate with niemeyer. [17:12] fwereade: that change was why i ended up adding the isPrincipal field to state.Unit [17:12] fwereade: which makes the current behaviour more or less cost-free [17:12] rog, cool, I'm always open to counterarguments/being overruled :) [17:13] rog, in that narrow context it seems like a slightly odd decision, but I'm sure there's context I'm not seeing [17:13] fwereade: and it's nice if status commands could potentially use Machine.Units to pull out stuff for a status, for example. [17:14] rog, I'm ok with that, but less sure that Units and WatchPrincipalUnits aren't both sensible methods for their particulat uses [17:14] fwereade: i don't see the extra four lines of code as particularly onerous [17:15] rog, it's the *indentation*! the horror, the horror! ;p [17:15] fwereade: oh well, if *that's* what you don't like, i'll do: if !u.IsPrincipal() {continue} :-) [17:15] rog, haha :) [17:16] what the hell does bzr want from me this time: http://paste.ubuntu.com/1051236/ [17:16] I only did a bzr pull [17:17] bzr revert fixed it [17:18] but... what's the procedure with these things? [17:18] Aram, was that a pull from a branch which included mate into a branch that had mstate but hadn't yet added it? [17:18] Aram, but I'm not sure it's any nicer when it has been added [17:19] fwereade: I pulled from lp:juju-core into my mstate branch. [17:19] Aram, why merge rather than pull? [17:19] Aram, er vice versa [17:19] hm? I should have done bzr merge instead of bzr pull? [17:20] Aram, I basically never pull, but I can't claim to have anything more than a vague intuitive understanding of bzr in the first place [17:21] hmm, this is strange. in hg/git you always pull, there's no other way. [17:21] so here pull does soemthing else... [17:21] Aram, my workflow is almost always `bzr branch `, followed by regular `bzr merge`s to keep the local branch up to date [17:21] ok, I'll try that. [17:23] fwereade: thanks, it worked. [17:23] Aram, excellent :) [17:23] basically I never have to use pull. [17:23] strange. [17:23] Aram, IME messing around with checkouts and pulls inevitably causes me pain [17:24] Aram, branch, merge, and push do basically everything I need [17:24] Aram, with ocasional judicious use of --remember on merge or push [17:34] fwereade: do you think all watchers should implement Err()? it seems to me like calling Stop is enough. [17:41] "TOO LONG (>50)" change summaries are limited to 50 chars? [17:44] fwereade: no more substantial comments? [17:44] fwereade: actually, publish them on the CL if you have some. i've gotta go! [17:44] fwereade: thanks a lot for the review BTW [17:45] fwereade: have a great long weekend, and i'll see you in a little over 2 weeks' time [17:45] * rog is off. see y'all (william excepted) tomorrow! [17:46] fwereade: i think i fixed that container test bug BTW. it'd be good if you could test it on your machine. [17:46] fwereade: https://codereview.appspot.com/6302103/ [17:49] rog: fwereade: TheMue: trivial one liner: https://codereview.appspot.com/6305121 [17:51] * niemeyer waves [17:51] Aram: LGTM [17:52] Aram: Just in time ;) [17:53] rog: LGTM too [17:53] niemeyer: what do I do now, lbox submit? why does it want to update the description again? [17:54] Aram: LGTM, I prefer this style of durations. [17:54] Aram: It's a chance for you to adjust it before committing, if necessary [17:54] niemeyer: ah, ok [17:55] niemeyer: I updated the mstate CR with everything you said [17:55] TheMue: yes, indeed. [17:55] Aram: Cool, will have a look now [18:02] Aram: Hmm.. your branch is not in the queue [18:03] Aram: Did you run lbox propose again? [18:03] yes [18:04] niemeyer, I'm around for a bit; are you free for a quick talk about VersionWatcher and related affairs? say in 10 mins perhaps, if you're in midflow with Aram? [18:04] Aram: So something isn't right [18:04] niemeyer: maybe I should click on "resubmit proposal" on the lpad website? [18:04] Aram: Nope.. just "lbox propose" on the right branch should do it [18:04] fwereade: Yeah, sounds good [18:04] Aram, the fantastic thing about lbox is you basically never need to visit launchpad ;) [18:05] niemeyer, cool, off for a ciggie then, bbs, ping me when you're free [18:05] Aram: Change *summaries* are limited to 50 chars.. change descriptions are not [18:05] niemeyer: yes, I figured. [18:05] Aram: These summaries end up in emails etc [18:05] As in, subjects [18:05] niemeyer: what about now, is it in th queue? [18:06] I did lbox propose again [18:06] Aram: It is [18:06] Aram: Thanks [18:06] niemeyer: ah, I did -wip before. [18:06] Aram: Ah, that'd explain it [18:07] I didn't want to spam you all with emails while I was doing experiments. [18:07] personally I get enough email already. [18:07] Aram: Did you change the branch name a third time? [18:08] niemeyer: I did change it three times, yes, but all the hacketry we did today was with name it has now... [18:08] Aram: Okay, please don't keep changing branch names as it spreads references around [18:09] Aram: The note I added to the previous MP is broken, for example [18:09] I'll try to. [18:10] Aram: There's something wrong in your setup still [18:10] Aram: "Aram Hăvărneanu (aramh) wrote 4 minutes ago:" [18:10] https://code.launchpad.net/~aramh/juju-core/juju/+merge/110920 [18:11] Aram: This means a local branch is being pushed onto the old location still [18:11] hmm? how can I check/debug? [18:13] Aram: try "bzr info" on the branch itself [18:13] Aram: Well, maybe lbox is actually picking the wrong one [18:14] Aram: Let's just delete the old MP [18:14] Aram: Done.. please delete this branch too: https://code.launchpad.net/~aramh/juju-core/juju [18:14] done [18:14] niemeyer: now what? [18:14] Aram: Okay, now run "bzr info" on bzr.. let's make sure it's not referring to the old name [18:15] on the branch, I mean [18:15] niemeyer: http://paste.ubuntu.com/1051331/ [18:15] Aram: Okay, I suppose you're not using cobzr [18:15] I am [18:16] white:mstate$ cobzr branch [18:16] master [18:16] * mstate-machine-basic [18:16] store-mgo-timeconsts [18:16] Aram: Oh, sorry, I misread [18:16] Aram: What "bzr nick" tells you? [18:16] white:mstate$ bzr nick [18:16] mstate-machine-basic [18:17] Aram: Looks good [18:17] Aram: Try lbox propose again then, with -v please [18:17] ok [18:19] niemeyer: http://paste.ubuntu.com/1051334/ [18:19] Aram: Yay, looks good [18:20] Aram: Aha, I know what's wrong [18:20] niemeyer: what is? is it on my side? [18:20] Aram: The email of the Launchpad MP on the CL is outdated [18:20] Aram: Can you please edit it as you're the only one with permission? [18:21] Aram: It should say 111235 rather than 110920 [18:21] ok, let me try it [18:22] done [18:22] Aram: Outstanding.. should all be in sync now [18:24] fwereade: Do you wanna talk before I dive into the review? [18:24] niemeyer, please [18:24] fwereade: Cool, so what's up.. tell me about a VersionWatcher :) [18:25] niemeyer, so... it seemed to me that we discussed the python style of related-unit watching [18:25] niemeyer, in which we track settings version changes [18:25] fwereade: Ok [18:26] niemeyer, and then, when we come to execute hooks, we grab the latest settings for a given unit relation and then cache them for the duration of the hook [18:26] niemeyer, this feels to me like unnecessary extra work -- we have the node data, we reduce it just to a version, and then later we grab the node data again and do extra work to make it appear constant [18:26] fwereade: Kind of.. I think you know it, but just to be super clear, it's cached at the time of first query [18:26] fwereade: Right? [18:27] niemeyer, yes, first query per unit relation per hook, AIUI [18:27] fwereade: Ok [18:27] niemeyer, it seems to me that this is an awful lot of work to go to when we could just grab the settings as they change [18:27] fwereade: Okay, can we please have a step back.. you seem to be assuming an implementation [18:28] fwereade: Where does that assumption comes from? [18:28] come [18:28] fwereade: Then, as a follow up question, what does "as they change" mean in the last sentence [18:28] niemeyer, well, I was trying to suggest that that implementation, despite the fact that it's what we do in python, is an awful lot of work for little benefit, when we have all the data available anyway [18:29] fwereade: What does grab the settings as they change mean? [18:30] niemeyer, it means we should, IMO, record the actual contents of the node (in one form or another... I kinda favour plain ol' map[string]interface{}, but that's by the by) [18:30] niemeyer, rather than just the version of the node [18:30] fwereade: Okay, so.. [18:30] niemeyer, I don;t think I'm assuming an implementation -- I'm comparing a proposed one with the existing python one [18:31] niemeyer, and trying to get buy-in on the idea that it is worth pursuing [18:31] fwereade: I meant it assumes an approach already, when it wasn't clear what this was about [18:31] fwereade: I'm not judging, just trying to understand the background [18:31] niemeyer, I understand, I clearly haven't communicated this especially well [18:31] fwereade: I think I have a better idea now [18:32] fwereade: As I understand it, there are two different problems, and they are being conflated in a single piece, which confuses the problem statement a bit more [18:32] niemeyer, I definitely took the wrong approach yesterday -- "this leads to a more consistent hook execution environment" is fundamentally not interesting, but "this saves us a bunch of work" hopefully leads to a more interesting discussion [18:33] fwereade: The first problem is caching [18:33] niemeyer, ok, yes; [18:33] fwereade: Caching is done so that we can implement more manageable data visibility for hooks [18:34] fwereade: So in effect, it's about *not showing changes* [18:34] fwereade: Slightly ironic, as the other problem is on the opposite end [18:34] fwereade: The second problem, is related to restarts [18:34] niemeyer, yes: during the execution of a single hook, we don't want the environment to visibly change [18:35] fwereade: (at least the settings data) [18:35] fwereade: So, with the second problem, we have the issue that we may lose events [18:35] fwereade: Because there's of course zero guarantee that we'll be permanently connected to whatever the backend turns out to be [18:36] niemeyer, indeed so [18:36] fwereade: So in that sense, we're interested in knowing whether things have changed since we last looked [18:36] fwereade: where "last looked" may be a process restart ago [18:37] fwereade: For that, we don't really care about what the content was [18:37] niemeyer, AIUI, though, the losing-events problem is always potentially in play... ZK doesn't guarantee I'll see 2 changes to a node that changes twice, does it? [18:37] fwereade: Because we just wanna know if we're supposed to tell the charm implementation that the data has changed, via a hook execution, or not [18:37] niemeyer, sure [18:37] fwereade: It doesn't, but that's fine. We offer at-least-once semantics [18:38] niemeyer, but the actual data tells us that just as well as the version does [18:38] niemeyer, in fact, it tells us better [18:38] fwereade: Without taking into consideration any implementation suggestions, [18:38] niemeyer, if there's a change followed by a revert while we're offline, diffing the settings reveals that there are no changes we care about [18:38] niemeyer, sorry, yes, go on [18:39] fwereade: it's a bit strange to cache a blob of, say, 64kb, when the text "42" would do [18:39] niemeyer, ok, so this is an optimization? [18:39] fwereade: It's not an optimization.. I'd call it non-silliness, from that specific perspective [18:40] niemeyer, ok, that is indeed a reasonable perspective [18:40] fwereade: But go on, you seem to have more detials [18:40] details [18:40] fwereade: This was just background [18:41] niemeyer, I'm not certain that perspective is worth the extra complexity involved in throwing away data and then reconstructing and caching it, when caching from the get-go leads, in my mind, to a much simpler model [18:42] fwereade: Sorry, but you're again jumping on a bit [18:42] fwereade: Why do we need to reconstruct and cache anything? [18:42] fwereade: This data is old.. we don't use it anymore. [18:42] niemeyer, well, distributed system, the data is always old [18:43] fwereade: No, that's not what I mean [18:43] niemeyer, ok; sorry to be dense; please restate [18:43] fwereade: In the scenario I just described, we track the version so that, in case of restarts, we know whether we must run a hook or not [18:43] fwereade: If we do detect an out-of-date situation, we run the hook [18:44] fwereade: We don't care about what the *data* used to be.. that 64kb fromt the old version is trash [18:44] niemeyer, ok, and that is a slightly eager but reasonable heuristic for rerunning hooks [18:45] fwereade: Eager? [18:45] niemeyer, a newer version does not necessarily imply that the data we care about has changed [18:45] fwereade: [18:45] niemeyer, but that's not a flaw worthy of the term flaw, let alone worth arguing over [18:45] fwereade: Does it generally change or not when we bump a version? [18:46] fwereade: We shouldn't be bumping the version of settings nodes without changes [18:46] fwereade: I'd consider that a bug [18:46] niemeyer, sure, understood, but versions don't revert when the data does -- but regardless, can we drop this? it's a derail [18:47] fwereade: Okay, yes, agreed regarding reverts, and agreed regarding derail. [18:48] fwereade: Okay, now, should we talk about VersionWatcher in that context, or is there some background that is missing in that brainstorm? [18:48] niemeyer, ok, that sounds interesting [18:49] niemeyer, if there's some that's relevant it will no doubt become apparent [18:49] niemeyer, my reading was that *if* we're tracking settings node versions, a VersionWatcher is just the ticket [18:49] fwereade: Well, I'm not sure, and that's what I'd appreciate gaining more insight into [18:50] niemeyer, I'm hoping that you have some insight I've missed here ;) [18:51] fwereade: When the agent *is* running, we're interested not only in the version but also in the data, since as we've discussed it may be queried and cached [18:51] niemeyer, ...ok, wait, I thought you were just arguing against caching the data outside of indivdual hook executions [18:52] fwereade: I'm arguing against saving it persistently [18:52] fwereade: and trying to understand whether we should be caching it in memory or not [18:53] fwereade: to solve problem *1* [18:53] niemeyer, ah-HA, this is a distinction that had entirely escaped me [18:53] niemeyer, ty for clarification [18:53] fwereade: Problem 2 doesn't require data.. problem 1 does [18:56] niemeyer, this is true; I maintain that the same data *could* be used to solve problem 2 as well, but I agree that it is not necessary when one considers problem 2 in isolation [18:57] fwereade: It could. It's just misoptimization. :) [18:57] niemeyer, that may well be so :) [18:59] niemeyer, although what I was trying to optimize when I thought of it was simplicity of implementation, which is IMO a pretty good candidate for first-thing-to-optimize [19:01] fwereade: Comparing two integers counts highly in my simplicity-meter [19:01] :) [19:01] fwereade: You may have further information that reveals why it is simpler to do something else, though [19:02] niemeyer, it's simply that we don't need to get and cache individual relation settings in a hook context if we already have them available [19:03] fwereade: I agree! [19:03] fwereade: That's about problem 1, though. [19:03] fwereade: And we get a version every time we get data out of ZooKeeper [19:04] fwereade: So there's a relationship between them [19:04] fwereade: We can solve both problems with a single watcher, cache the data for problem 1, and keep the version around for problem 2.' [19:05] niemeyer, ok, I did think of just extending ContentWatcher to include the version [19:05] fwereade: +1 [19:05] niemeyer, and let the clients discard what they will [19:06] niemeyer, I think I have to go now; I think I'll want to talk about this some more over the next couple of days though [19:07] fwereade: Sounds good. Hopefully that was useful. [19:07] niemeyer, thank you, I think we;re making progress :) [19:07] fwereade: Glad to hear it, and you're welcome [19:07] niemeyer, I'm not quite there yet though ;) [19:07] niemeyer, cheers [19:07] niemeyer, happy evening :) [19:15] fwereade: Have a good one [19:18] Aram: So, back to your branch [19:21] right === Aram2 is now known as Aram [19:32] Aram: Had another pass: https://codereview.appspot.com/6304099 [19:32] Aram: Probably the last one [19:32] Aram: It's looking great [23:08] niemeyer: thanks for the review. [23:08] Aram: My pleasure [23:14] howdy, anyone seen mark [23:19] https://twitter.com/campoy83/status/215255686592995331