| rog | davecheney, fwereade: hiya | 06:54 |
|---|---|---|
| fwereade | rog, heyhey | 06:54 |
| TheMue | Morning. | 07:15 |
| davecheney | morning all | 07:29 |
| TheMue | davecheney: Hi | 07:30 |
| === TheRealMue is now known as TheMue | ||
| twobottux | aujuju: How to SSH into local juju instance? <http://askubuntu.com/questions/152428/how-to-ssh-into-local-juju-instance> | 08:25 |
| TheMue | Hmm, t-storm above our home. | 08:31 |
| === mcclurmc_ is now known as mcclurmc | ||
| Aram | morning. | 10:07 |
| TheMue | Aram: Hi | 10:46 |
| voidspace | is this page still the "official" guide to writing charms? | 11:46 |
| voidspace | https://juju.ubuntu.com/docs/write-charm.html | 11:46 |
| voidspace | if so, then maybe it should start by referencing a directory that actually *exists* in juju trunk | 11:49 |
| voidspace | having to check out juju trunk when you're using an installed juju is not the friendliest way of starting a tutorial as well | 11:50 |
| voidspace | "Assuming the current directory is the juju trunk"... | 11:50 |
| voidspace | anyway, just some friendly feedback :-) | 11:51 |
| rog | niemeyer: yo! | 12:27 |
| fwereade | voidspace, thanks for pointing that out :) | 12:29 |
| fwereade | niemeyer, heyhey | 12:29 |
| Aram | hi niemeyer. | 12:30 |
| voidspace | fwereade: morning | 12:30 |
| fwereade | voidspace, how's it going? | 12:30 |
| voidspace | fwereade: mainly good :-) | 12:31 |
| fwereade | voidspace, mainly glad to hear it then :) | 12:31 |
| voidspace | fwereade: daughter has unexplained minor illness causing random temperature fluctuations | 12:31 |
| fwereade | voidspace, aww :( | 12:31 |
| niemeyer | Gooood morning all! | 12:31 |
| voidspace | fwereade: but she's happy and lively, so not *much* of a problem :-) | 12:31 |
| voidspace | niemeyer: morning | 12:31 |
| voidspace | fwereade: and yourself? | 12:31 |
| voidspace | our team is sprinting on getting us all up to scratch with juju this week... | 12:31 |
| fwereade | voidspace, yeah, very good thanks | 12:31 |
| fwereade | voidspace, ah, excellent | 12:32 |
| voidspace | fwereade: so expect more "observations" on your documentation :-) | 12:32 |
| fwereade | voidspace, they're greatly appreciated | 12:32 |
| voidspace | hehe | 12:32 |
| fwereade | voidspace, bug reports even more so ;) | 12:32 |
| Aram | niemeyer: I have some errands to attend to, I'm talking half a day off and will recuperate it later this evening/night. | 12:33 |
| voidspace | getting juju up and running with lxc inside a vm was astonishingly easy though - so kudos on that (using precise) | 12:33 |
| niemeyer | Aram: Sounds good, thanks for the note | 12:33 |
| voidspace | fwereade: want an issue for the charm writers guide? | 12:33 |
| fwereade | voidspace, it's very cool, isn't it -- sadly I can't take much credit for that bit :) | 12:33 |
| voidspace | fwereade: I wasn't 100% certain I was on the current-latest-correctest documentation page anyway | 12:33 |
| fwereade | voidspace, that would be fantastic | 12:33 |
| voidspace | heh | 12:33 |
| fwereade | voidspace, if it's what you find when you look for the docs then I think that's something we should address | 12:34 |
| voidspace | fwereade: yep | 12:34 |
| fwereade | niemeyer, when you have a mo please let me know if I still appear to be on crack re relation types | 12:35 |
| fwereade | niemeyer, I do think that RelatedService is applicable more widely than just in the unit agent | 12:36 |
| niemeyer | fwereade: Hmm | 12:37 |
| niemeyer | fwereade: Maybe.. that comment was made on a bad foundation. I still had in mind the old interface of AddRelation | 12:38 |
| fwereade | niemeyer, yeah, I was a bit surprised when I looked back at the python code and discovered that nobody ever paid any attention to the result of AddRelation | 12:38 |
| fwereade | niemeyer, add_relation_state, rather | 12:39 |
| niemeyer | fwereade: I still think eventually we'll need something that represents relations themselves, so we can more easily iterate. But that's a gut feeling rather than any kind of certainty, so I'm happy to see if we can tailor the model to something sensible for the use cases at hand | 12:40 |
| fwereade | niemeyer, I can well believe that, but I don;t think we're there atm -- have you looked at the Relation type in python recently? | 12:40 |
| fwereade | niemeyer, it has an internal_id property, and that's it :) | 12:41 |
| niemeyer | fwereade: Good point :) | 12:42 |
| fwereade | niemeyer, there's something else on my mind actually | 12:44 |
| fwereade | niemeyer, consistency of the history that unit agents see | 12:44 |
| fwereade | niemeyer, it really rather bugs me that we have a mix of zk-current and hook-queue-current state available when we're running hook | 12:45 |
| fwereade | niemeyer, and I think it exposes us to bug that we only don't hit through sheer luck, and coincidental properties of the rest of the implementation | 12:46 |
| niemeyer | fwereade: How so? | 12:46 |
| fwereade | niemeyer, for example, what should we do when we ask ZK for the current relations in a -joined hook and it turns out that the relation we're supposed to be joining was just deleted? | 12:47 |
| hazmat | fwereade, previously to relation-ids it all worked as expected | 12:48 |
| hazmat | fwereade, the hook could use apis see a state current at time of execution, and delete hook exec would be pending | 12:49 |
| niemeyer | fwereade: It's an excellent question, and we have several such issues all around the code base | 12:49 |
| fwereade | hazmat, ok; but we can also do things like try to look up settings for a relation that doesn't exist any more, and that's only safe because we don't GC anything under /relations | 12:49 |
| hazmat | fwereade, that wasn't a happy accident that was by design | 12:50 |
| hazmat | fwereade, we don't gc things that could be in current use | 12:50 |
| fwereade | hazmat, understood; but it feels to me like it'll be kinda tricky to ever GC relations given the current usage | 12:50 |
| niemeyer | fwereade: There are two options.. either we cope with the fact that the relation is being removed and offer a sensible answer | 12:50 |
| hazmat | fwereade, its quite simple actually.. no presence nodes, not in topology, its dead | 12:51 |
| niemeyer | fwereade: or we tag the relation (or whatever object) as dead and allow the code to handle it in a timely way | 12:51 |
| fwereade | hazmat, until some misfunctioning unit agent comes back up..? | 12:51 |
| hazmat | fwereade, and it would come back up and see the rel doesn't exist | 12:52 |
| hazmat | ie. it would check the topo | 12:52 |
| fwereade | niemeyer, or alternatively, we snapshot settings just as we do membership, and have unit agents execute hooks against the histroy we snapshotted | 12:52 |
| fwereade | hazmat, ok, GC isn't as tricky as I worried :) | 12:52 |
| hazmat | fwereade, if its disconnected for a while and hasn't notified yet | 12:52 |
| niemeyer | fwereade: Not sure I understand what that would mean | 12:53 |
| hazmat | then it will come back up and queue a broken hook in a separate context, and with queue reduction that wins | 12:53 |
| fwereade | niemeyer, hazmat: I may be missing something, but it seems to me that the unit agent gets notified of every relevant change to state, and can therefore keep a consistent local history through which it can run at its own pace (which isn't necessarily as slow as it might be, thanks to hook reduction) | 12:55 |
| niemeyer | fwereade: It can't.. assume the unit is down when a change happens | 12:55 |
| fwereade | niemeyer, it can; it has to know how to diff current state against its known latest state anyway | 12:56 |
| niemeyer | fwereade: Well, yes, and that goes against "every relevant change to state on consistent local history", right? | 12:56 |
| fwereade | niemeyer, I don;t see the tension there -- we always knew we might miss some changes, and we have queue reduction anyway... what's the difference? | 12:57 |
| hazmat | fwereade, terminology, its not seeing every change, it seeing deltas | 12:58 |
| hazmat | its | 12:58 |
| niemeyer | fwereade: I'm just pointing out that "it's fine to miss some changes" and "every relevant change to state on consistent local history" are not friends. | 12:58 |
| fwereade | hazmat, yeah, exactly | 12:58 |
| fwereade | niemeyer, I'm saying that within the constraints we're used to, we can run every hook against the state we knew to exist at the time we detected the need for a hook to run (collapsed as necessary), rather than running against a mixed state | 12:59 |
| fwereade | niemeyer, and I was wondering if there was an obvious downside to that strategy | 13:00 |
| niemeyer | fwereade: We can't snapshot the whole state at once | 13:01 |
| niemeyer | fwereade: Which means that by the time you discovered you had to run "joined", the state has already moved forward | 13:02 |
| niemeyer | fwereade: So there doesn't seem to be much of a point to me to be getting a state that is arbitrary in a time before it is needed | 13:02 |
| niemeyer | fwereade: But maybe I'm misunderstanding what you mean | 13:02 |
| fwereade | niemeyer, we're already snapshotting membership, and using that; we're snapshotting settings versions, and using those to figure out when to collapse; we could do the same with the actual settings, n'est-ce pas? | 13:03 |
| hazmat | its not really snapshotting | 13:03 |
| hazmat | its tracking | 13:03 |
| fwereade | niemeyer, when you do a relation-list, you're explicitly not getting the current related units; you're getting the units at the time the change was detected | 13:03 |
| hazmat | its still broken though | 13:04 |
| hazmat | er. not a snapshot | 13:05 |
| fwereade | hazmat, go on | 13:05 |
| hazmat | but conversely if you do a relation-get you do see a current value | 13:05 |
| hazmat | that's cached for the hook exec | 13:05 |
| niemeyer | fwereade: There's no way to get the data for multiple nodes at the same time.. it's a cache, rather than a consistent snapshot | 13:05 |
| niemeyer | fwereade: relation-list might work better in the case of relation-joined, specifically, but it won't work well in the case of relation-changed, for example | 13:05 |
| fwereade | hazmat, thatis rather my problem, that I'd be a lot more comfortable if we were grabbing the settings at the time they changed and using them | 13:06 |
| fwereade | hazmat, I don;t think we'd ever be too far out of date, because of queue reduction | 13:06 |
| hazmat | fwereade, agreed, these semantics are hard, without evolving a local state copy you can't be consistent throughout | 13:06 |
| niemeyer | fwereade: Because you detecting the change and listing the relations are two operations | 13:06 |
| niemeyer | fwereade: Done at different times | 13:06 |
| niemeyer | fwereade: Yes, not too far out of date, but that's not the point.. if it is out of date, what are we trying to solve then? | 13:06 |
| hazmat | fwereade, and even then its not clear that its truly consistent even then | 13:07 |
| fwereade | niemeyer, (we're always out of date a little) | 13:07 |
| niemeyer | fwereade: Exactly | 13:07 |
| fwereade | hazmat, this is to me the most interesting bit... how do you see us being inconsistent with local state? | 13:07 |
| niemeyer | fwereade: So doing a snapshot, say, 2 minutes ago so we can run a hook doesn't sound like solving a problem | 13:07 |
| niemeyer | fwereade: Reduction is evidence that we actually don't care about the intermediate state | 13:08 |
| hazmat | fwereade, well the question then is what is meant by consistent? that all values ever show to a hook are true as of the time of its related change event mvcc style? | 13:08 |
| niemeyer | fwereade: We just try to bring the unit aware of the latest state | 13:08 |
| fwereade | niemeyer, then why do we bother doing what we do with relation-list in the first place? | 13:08 |
| * hazmat needs some coffee to correct typos | 13:09 | |
| fwereade | niemeyer, I think I'd be comfortable with either "you always get current state" or "you always get consistent state" | 13:09 |
| fwereade | niemeyer, but it feels liek we're mixing the two and it makes my head hurt a little | 13:09 |
| rog | fwereade: +1 | 13:10 |
| niemeyer | fwereade: Current is impossible, as we both know.. | 13:11 |
| fwereade | niemeyer, and "current" has the problem that we could then be running relation-joined against some remote unit which relation-list tells us doesn't exist | 13:11 |
| rog | i wonder if we wouldn't be better having simple "something has changed"-style notifications | 13:11 |
| fwereade | niemeyer, "latest known" then | 13:11 |
| fwereade | niemeyer, but regardless, I agree that whatever you call it, it has problems | 13:11 |
| niemeyer | fwereade: Let's talk about relation-changed.. what do you suggest we do regarding relation-lsit? | 13:12 |
| hazmat | fwereade, so re not clear, there's also cross context manipulation now, you have to have a linear view across relations | 13:12 |
| hazmat | ie. relation-set from rel-a-hook to rel-b-hook | 13:13 |
| fwereade | niemeyer, AIUI what we currently do is keep track of which units have joined/departed according to local state, and always give that when asked to relation-list | 13:13 |
| hazmat | er.. rel-b | 13:13 |
| fwereade | hazmat, can you expand a little? I don't *think* that hurts us... but I'm probably missing something | 13:13 |
| fwereade | hazmat, a unit agent only sets its own settings, right? and that's in response to hooks running relation-set... and so that's automatically synchronised, isn't it? | 13:15 |
| niemeyer | fwereade: We could snapshot the state of the relation the specific hook is part of before we start the hook, if that's not done today | 13:15 |
| niemeyer | fwereade: Is that what you mean? | 13:15 |
| fwereade | hazmat, a unit's own settings will always reflect whatever it's decided to set given the hook that have so far been handled | 13:15 |
| niemeyer | fwereade: or rather, what you are suggesting? | 13:15 |
| fwereade | niemeyer, yes, I'm just suggesting that when we detect a settings change we store the settings as part of the history rather than just the fact of a change | 13:16 |
| fwereade | niemeyer, and that that is strictly more consistent than what we have now, and makes my head hurt less :) | 13:16 |
| fwereade | niemeyer, (and hopefully will make life less confusing for people when they're debugging tricky relations) | 13:17 |
| niemeyer | fwereade: How does that improve things? | 13:18 |
| niemeyer | fwereade: As we just discussed, when we detect that settings have changed, and go to get that, it's already out-of-date | 13:19 |
| fwereade | niemeyer, if we can divorce hook execution environments from zookeeper state (apart from the unit writing its own relation settings) we can eliminate having to think about an entire class of whoops-zk-changed-in-a-confusing-way situations | 13:19 |
| niemeyer | fwereade: Why is that out-of-date state any better than the out-of-date state we get when we actually run the hook? | 13:19 |
| hazmat | fwereade, right now the state tracking is mostly in independent relation contexts, it works cross relation through benefit of the db giving us a linear time view (albeit with per rel queue reduction), the point i was trying to raise is that we have cross relation activity now from hooks. i don't think its insurmountable if the linear view across relations is maintained, but it is something to keep in mind. ie any rel-a hook could change settings on a rel-b | 13:19 |
| hazmat | thing. | 13:19 |
| rog | niemeyer: FYI continuing weird behaviour with bzr (i've been stuck on it all morning). http://paste.ubuntu.com/1047256/ time to lose the branch, i think. if bzr is this broken, lbox has no hope. | 13:19 |
| fwereade | hazmat, ...whoa, ok, I missed that | 13:20 |
| fwereade | hazmat, how is that OK? | 13:20 |
| hazmat | fwereade, its not without some careful thought ;-) but generally it can work because we have a linear view of events changes across relations that the cache is kept up to date for, but it does require thought to the cache maintenance wrt to queue reduction | 13:21 |
| fwereade | hazmat, we're talking about a foo-relation hook on unit X relation A changeing settings on... unit X relation B? | 13:21 |
| niemeyer | fwereade: How is it not ok? | 13:21 |
| hazmat | fwereade, yes | 13:21 |
| fwereade | niemeyer, hazmat: it's not *necessarily* not ok, but it's a surprising extra channel | 13:22 |
| niemeyer | rog: Sorry, I can't really help since I can't reproduce the issue locally | 13:22 |
| niemeyer | fwereade: I find it pretty straightforward | 13:22 |
| rog | niemeyer: yeah, i have no idea. just thought you might be interested to see the bizarreness. | 13:22 |
| hazmat | rog, naked bzr works pretty well.. i kept running into issues with cobzr and gave it up | 13:24 |
| fwereade | niemeyer, ok, as long as it's all on the same unit, that's fine | 13:24 |
| niemeyer | hazmat: FWIW, I use it for every single bzr branch | 13:24 |
| niemeyer | fwereade: It is, the property "relation-set only changes myself" is sustained | 13:25 |
| rog | hazmat: i've never had a problem before. i think i mucked things up when i accidentally did a bzr commit as root. | 13:25 |
| hazmat | rog, interesting.. easy to fix with a chown. | 13:25 |
| fwereade | niemeyer, cool, I misunderstood on first reading, thought we were talking about something altogether more disturbing | 13:25 |
| rog | hazmat: that was the first thing i did | 13:25 |
| niemeyer | fwereade: So, still trying to understand what you mean | 13:25 |
| niemeyer | fwereade: Just so you get the background.. | 13:26 |
| niemeyer | fwereade: The idea of caching comes from giving people consistent answers across calls that read data from a remote unit, to make things a little more saner | 13:26 |
| niemeyer | fwereade: E.g. (relation-get username + relation-get password) will get data for the same person | 13:26 |
| rog | would someone be able to explain to me why what thing changes which relation settings is important here? | 13:27 |
| hazmat | niemeyer, also applies to the local unit reads via the writeback cache | 13:27 |
| niemeyer | hazmat: Rihgt | 13:27 |
| niemeyer | fwereade: So, you can see our model as "latest known state, snapshotted on first query" | 13:28 |
| fwereade | niemeyer, per queried unit, yeah | 13:28 |
| fwereade | niemeyer, and apart from the actual list of units, which is from longer ago | 13:29 |
| fwereade | niemeyer, none of these things *seriously* bother me | 13:29 |
| rog | i.e. why is cross-relation activity a difficulty? | 13:29 |
| niemeyer | rog: It's not.. was a red herring | 13:29 |
| rog | niemeyer: ok, thanks. | 13:30 |
| fwereade | niemeyer, but I'm equally not sure wat benefit we derive from having two different "now" states in play at a time | 13:30 |
| niemeyer | fwereade: We have N "now" states.. we have data from a distributed system, where each detail is arbitrarily old and invalid | 13:31 |
| hazmat | niemeyer, it can be | 13:31 |
| hazmat | niemeyer, with an attempt to have a consistent cache.. at least in the dictates of the py impl | 13:31 |
| fwereade | niemeyer, ok, but we have ordering guarantees from ZK, right? | 13:31 |
| niemeyer | fwereade: zk has ordering guarantees.. I don't know what you mean, though | 13:32 |
| hazmat | niemeyer, each rel has a separate queue subject to reduction rules, if a hook corresponding to event 3 in rel B executes at a time before a hook corresponding to event 2 in rel A, and they cross communicate it is an issue. | 13:32 |
| rog | fwereade: i'm with you, but i think for a slightly different reason. we *do* have potential consistency problems, but i think they should be solved as close to source as possible. so watchers should detect inconsistency if possible and act appropriately. by the time we get to the actual agent, all the information should be available in processed form. | 13:33 |
| niemeyer | hazmat: I don't know what you mean by "cross communicate".. hooks don't communicate | 13:33 |
| fwereade | niemeyer, I'm just not 100% convinced that it's never going to break anything if we get settings newer than the relation list | 13:33 |
| hazmat | niemeyer, cross communicate -> i mean hook in rel B modifies rel A setting | 13:33 |
| niemeyer | hazmat: I don't see any issue there | 13:33 |
| * fwereade cig/assimilate break, 3 mins | 13:34 | |
| niemeyer | hazmat: Yeah, don't see any issue there | 13:34 |
| niemeyer | fwereade: Cool, let's talk when you're back | 13:35 |
| niemeyer | fwereade: FWIW, I find your concern great.. it's nice to have that kind of problem in mind | 13:35 |
| niemeyer | fwereade: Let's figure the way we *want* relation-list and hooks to interact | 13:36 |
| hazmat | niemeyer, its only problematic for attempting to maintain a consistent view for each hook, because hook B modified settings in a separate rel using a different consistent view then the present time of the other rel. the linear view of time is subject to some jumps because of how we reduce the queue, it is consistent but only to a rel context, cross rel communication, requires a linear view that is scoped at a higher level to achieve consistency | 13:36 |
| rog | niemeyer: trying to get my head clear around this. this is only a problem because we're not getting all the info from a watcher, right? so the action that an agent is taking depends not only on the data received from the watch, but also some data that it reads from zk later? | 13:37 |
| niemeyer | hazmat: Sorry, I still don't see the problem. | 13:37 |
| hazmat | again it depends on what you looking for consistency, fwereade was looking for something akin to mvcc (ie. snapshot isolation) | 13:37 |
| fwereade | niemeyer, heh, I think what I just thought is what hazmat just said | 13:37 |
| niemeyer | hazmat: "B modified settings in a separate relation".. | 13:37 |
| niemeyer | hazmat: That's sequential | 13:37 |
| niemeyer | hazmat: Hooks aren't executing in parallel | 13:38 |
| rog | i don't think we can guarantee consistency across relations. | 13:38 |
| hazmat | niemeyer, but they are subject to time skips due to reduction | 13:38 |
| niemeyer | hazmat: Not time skips, no | 13:38 |
| niemeyer | hazmat: They are subject to ignoring events and acting on the latest known state, yes | 13:38 |
| hazmat | niemeyer, right their linear advancing, but yes the clock does skip event ticks | 13:38 |
| niemeyer | hazmat: I don't think that changes anything with regards to relations in different contexts, though | 13:39 |
| hazmat | it works now because we feed all of them (hook exec queues) into a linear scheduler | 13:39 |
| hazmat | yeah.. it might be a red herring | 13:39 |
| niemeyer | hazmat: A hook executing for relation A, B, C, or D, and doing changes for relation A, B, C or D, has the system as consistent as if the hook was doing changes on a single relation | 13:40 |
| rog | niemeyer: i don't think it does | 13:41 |
| fwereade | niemeyer, when relation B changes relation A's settings, it could plausibly affect how relation A acts... right? | 13:41 |
| rog | niemeyer: because the settings for each relation are in a different zk node | 13:41 |
| niemeyer | fwereade: Yes, hopefully :) | 13:42 |
| rog | niemeyer: and we can reorder zk watch firing across different nodes | 13:42 |
| fwereade | niemeyer, and if relation A doesn't have access to the same view of history that B used to make that change to A, I think we could end up in confusing states | 13:42 |
| fwereade | niemeyer, maybe it all shakes out, but I haven't convinced myself of that yet | 13:42 |
| hazmat | i'd need to white board to work it out, i haven't worked through it with the new cross rel communication work in, its something that needs thought though. | 13:42 |
| niemeyer | rog: DOesn't matter in that case we're discussing. We don't order those events. | 13:43 |
| hazmat | s/thought/verification | 13:43 |
| niemeyer | fwereade: I don't see how that's an issue | 13:43 |
| niemeyer | Quick break, biab | 13:43 |
| hazmat | rog, how's the upgrade work coming? | 13:45 |
| rog | hazmat: i finished that upgrader thingy. but it's judged premature for the time being, so i've put it on ice. currently working on parallel stuff towards the machine agent. | 13:46 |
| fwereade | niemeyer, ok, let's say B has set some setting per related unit on A, based on "current" state at the time a hook ran for B; and A then runs a hook, based on a new and incompatible "current" state for A; this feels liable to lead to upset and confusion | 13:47 |
| hazmat | rog, cool.. i got inspired to work on it as well | 13:47 |
| fwereade | niemeyer, whereas if B and A are both working with the same history, the "current" states don't matter -- both B and A will agree on (say) what units A is currently related to | 13:47 |
| hazmat | rog, although its turning out to be a lot more work than i had hoped. | 13:48 |
| rog | hazmat: yeah, i found that. but it wasn't too bad. if you want to copy the protocol, the code is here https://codereview.appspot.com/6307061 | 13:49 |
| rog | hazmat: i had a plan to automatically generate test permutations, but didn't get quite that far. | 13:50 |
| hazmat | rog, i eschewed the separate process for minor upgrades | 13:51 |
| rog | hazmat: ah, cool. | 13:51 |
| rog | hazmat: where are you finding the main difficulties? | 13:51 |
| hazmat | rog, i want to get to major upgrades though, which is where things get interesting.. else i'll never be able to merge any of the scaling work (json switch) | 13:51 |
| niemeyer | fwereade: That's a good direction for us to explore.. it'll be easier to get a point across with examples indeed | 13:52 |
| niemeyer | fwereade: I don't understand yours, though | 13:52 |
| niemeyer | fwereade: Let me try to translate | 13:52 |
| hazmat | rog, no part in particular is difficult.. i ended up going with some new abstractions (state protocols), its just a lot of pieces. | 13:52 |
| niemeyer | fwereade: We have two units, A and B, is that what you mean? | 13:52 |
| fwereade | niemeyer, two relations within the same unit | 13:52 |
| niemeyer | fwereade: Okay, so let's call two relations R1 and R2 | 13:53 |
| niemeyer | fwereade: and two units A and B | 13:53 |
| niemeyer | fwereade: Are R1 and R2 both between A and B? | 13:54 |
| fwereade | niemeyer, let's say yes | 13:54 |
| niemeyer | fwereade: Okay | 13:54 |
| fwereade | niemeyer, well, heh, R1 and R2 are each between the respective services of A and B | 13:55 |
| niemeyer | fwereade: So you mean that we have a hook R1-relation-changed executing within A? | 13:55 |
| fwereade | niemeyer, yes; and R1-relation-changed sets something on A.R2's settings | 13:55 |
| niemeyer | fwereade: Okay, I'm with you | 13:55 |
| * fwereade thinks a mo, maybe it doesn't matter... just a sec | 13:57 | |
| fwereade | niemeyer, ok, thinking aloud... R1 sets a key in A.R2 per unit related to A, which is currently [B] | 13:59 |
| niemeyer | fwereade: Okay | 13:59 |
| fwereade | niemeyer, B departs, and B' joins | 14:00 |
| niemeyer | fwereade: C joins, please :) | 14:01 |
| niemeyer | fwereade: Or you mean B joins again? | 14:01 |
| fwereade | niemeyer, ok, C joins | 14:01 |
| fwereade | niemeyer, now, I meant another unit of the service that B was a unit of | 14:01 |
| niemeyer | fwereade: Okay, let's call it C since the service relation doesn't really matter in this case.. it's a different unit in the relation | 14:02 |
| fwereade | niemeyer, yep, ok | 14:02 |
| fwereade | niemeyer, so, given the delays etc in play, we could well have a situation in which A.R1 still thinks it's related to B, but A.R2 thinks it is now related to C | 14:03 |
| niemeyer | fwereade: Oops, hold on | 14:03 |
| fwereade | niemeyer, A.R2 thinks A.R1 is now related to C | 14:03 |
| niemeyer | A.R1 doesn't think anything.. this is a settings node | 14:03 |
| niemeyer | fwereade: Neither does A.R2 | 14:04 |
| fwereade | niemeyer, yes: A.R1 is a node distinct from A.R2 though | 14:04 |
| niemeyer | fwereade: Yes, and neither of them "think" anything.. they are state | 14:04 |
| fwereade | niemeyer, s/thinks/has written state predicated on the assumption that/ | 14:05 |
| niemeyer | fwereade: Yes, and that's not a problem.. | 14:05 |
| niemeyer | fwereade: R1-relation-joined will be called for C | 14:05 |
| niemeyer | fwereade: and A will have a change to do whatever it pleases with it | 14:06 |
| niemeyer | fwereade: If the person maintaining the charm for A is doing crazy stuff that makes no sense, though, we can't avoid it | 14:06 |
| fwereade | niemeyer, I think I agree that we will eventually reach a steady state in which everything makes sense | 14:07 |
| niemeyer | fwereade: Yep, that was the main thinking | 14:07 |
| fwereade | niemeyer, but it seems wrong to let C see A.R1's settings in a state wildly divergent from A.R2's settings | 14:08 |
| fwereade | niemeyer, the eventual consistency may indeed trump all though | 14:09 |
| niemeyer | fwereade: You've lost me now | 14:11 |
| niemeyer | A.R1 and A.R2 were both set solely by A.. | 14:11 |
| niemeyer | fwereade: There's no way for them to be "divergent" | 14:11 |
| fwereade | niemeyer, the "wildly divergent" is hyperbolic, but is meant to indicate that A.R1 could respond to 30 events before A.R2 responds to any | 14:12 |
| fwereade | niemeyer, unless we're thinking a little about the cross-relation stuff alluded to above | 14:12 |
| fwereade | niemeyer, I think that would take a pathological scheduling situation and it might be impossible but I can't comfortable state that I'm certain it is impossibvle | 14:13 |
| niemeyer | fwereade: We can improve the scheduling algorithm over time to make events across relations "fair", in case they don't naturally get fair | 14:14 |
| niemeyer | fwereade: We've lost the use of practical examples, though | 14:14 |
| fwereade | niemeyer, well observed | 14:14 |
| fwereade | niemeyer, I might have to try to turn this into an email | 14:14 |
| niemeyer | fwereade: R1-relation-changed can *really* have 30 events happening before R2-relation-changed happens | 14:14 |
| rog | fwereade, niemeyer: is it this kind of scenario we're potentially concerned about? http://paste.ubuntu.com/1047364/ | 14:16 |
| niemeyer | rog: I'm not aware of any scenario we're concerned about, after the above conversation | 14:17 |
| fwereade | rog, I don't follow that I'm afraid | 14:17 |
| niemeyer | rog: The password case I mentioned works today | 14:17 |
| rog | niemeyer: yes, but it might not work if the username and password were stored in different relations, right? | 14:18 |
| niemeyer | rog: That's an artificial example that we're not concerned about | 14:18 |
| rog | niemeyer: ok. i was trying to understand fwereade's concerns, sorry. | 14:18 |
| niemeyer | rog: No worries, I'm just explaining what I've been discussing with fwereade | 14:19 |
| rog | niemeyer: i was trying to come up with a concrete example of | 14:19 |
| rog | [15:08:36] <fwereade> niemeyer, but it seems wrong to let C see A.R1's settings in a state wildly divergent from A.R2's settings | 14:19 |
| rog | niemeyer: of course, it *is* an artificial example | 14:20 |
| niemeyer | rog: See my comments that follow that :-) | 14:20 |
| rog | niemeyer: and i'm happy for us to say that settings are only consistent within a relation. | 14:20 |
| niemeyer | rog: Cool, that's a given today, and I agree it doesn't feel useful to go over the trouble | 14:21 |
| niemeyer | fwereade: Okay, can we go back to that branch we were talking about when the conversation diverged? | 14:22 |
| rog | niemeyer: i do think that it might make sense to make the agents use info read by the watchers though, rather than going into the zk state again. it would mean the watchers can detect inconsistencies early on and produce a consistent-looking stream of events which will lead to determinstic actions. | 14:23 |
| niemeyer | rog: The agents use info read by the watchers today. | 14:23 |
| rog | niemeyer: if they do, that's no problem. i seem to remember that wasn't the case though. | 14:24 |
| niemeyer | rog: They do. Of course, there are a lot of details about this, but those details should be taken into account when saying "we should do foo". | 14:25 |
| niemeyer | rog: The whole conversation above was exploring those details | 14:25 |
| rog | niemeyer: looking at this before, it seemed to me that more stuff needed to be cached in the state types for this to be true. but perhaps things have changed, or the agents don't need as much info as i thought. | 14:27 |
| niemeyer | rog: Sorry, but I really don't know what you're suggesting. | 14:27 |
| rog | niemeyer: one example: when the machine agent is watching units for a machine, it sees the topology change and sends a *Unit down a channel. by the time the agent comes to ask the unit for its CharmURL, the unit might be gone. | 14:33 |
| rog | niemeyer: on a completely different subject: how do you feel about the possibility of using build tags for conditional tests instead of flags? (i'm thinking of -amazon here, to start with) | 14:35 |
| niemeyer | rog: I've seen the thread.. sounds fine, but we shouldn't be worrying about this right now | 14:35 |
| rog | niemeyer: i'm just about to add another flag, and it seems a bit wrong. and it's awkward to test everything with the flags. | 14:36 |
| rog | niemeyer: it would be quite and easy to change | 14:36 |
| rog | s/quite/quick/ | 14:36 |
| niemeyer | rog: What flag are you adding? | 14:36 |
| rog | niemeyer: -root | 14:36 |
| niemeyer | rog: Hmm.. why do we need it? | 14:36 |
| rog | niemeyer: because the only way of testing the container package with any usefulness is to run the tests as root | 14:37 |
| rog | AFAICS | 14:37 |
| niemeyer | rog: I can't see why that would be the case right now | 14:37 |
| niemeyer | rog: We're not adding support for LXC today | 14:37 |
| rog | niemeyer: not possible to add upstart scripts without being root | 14:38 |
| rog | niemeyer: i could just leave out testing entirely | 14:38 |
| niemeyer | rog: You can add upstart scripts to a path that you define | 14:38 |
| rog | niemeyer: putting yet another shim second-guessing what commands the upstart package will be invoking seems wrong | 14:38 |
| rog | niemeyer: really? running upstart myself, then? | 14:39 |
| rog | niemeyer: i hadn't tried that. | 14:39 |
| niemeyer | rog: Yes, that actually works | 14:39 |
| niemeyer | rog: Although I'd probably just verify that the script was put in place | 14:39 |
| niemeyer | rog: Well, depending on what else is being tested in that path | 14:40 |
| rog | niemeyer: i quite like having a test that really verify that it actually works with upstart for real. | 14:40 |
| niemeyer | rog: Either way, definitely not require root for tests | 14:40 |
| rog | niemeyer: indeed. | 14:40 |
| rog | niemeyer: hence the flag. | 14:40 |
| niemeyer | rog: Those will be run close to never | 14:40 |
| rog | niemeyer: same as the amazon tests :-) | 14:40 |
| niemeyer | rog: Not really.. I run those fairly frequently | 14:41 |
| niemeyer | rog: I don't run *any* test suite as root, though | 14:41 |
| rog | niemeyer: really? when was the last time you did? last few times they've failed for me. | 14:41 |
| niemeyer | rog: We may need that for LXC, unfortunately | 14:41 |
| niemeyer | rog: For EC2/S3, every single time I change the tests | 14:41 |
| rog | niemeyer: that's why i thought it was reasonable to do it now. | 14:41 |
| niemeyer | rog: We're not implementing LXC now | 14:42 |
| niemeyer | rog: Are you working on LXC? | 14:42 |
| rog | niemeyer: nope | 14:42 |
| niemeyer | rog: So I'm lost | 14:42 |
| niemeyer | rog: Only need for LXC => That's why I'm doing it now => Doing LXC? => No => Vhat geev? | 14:43 |
| rog | niemeyer: i thought that since the container package was going to need to have root tests eventually, and that the current tests are more effective running as root, that it would be reasonable to have root tests there now. | 14:43 |
| niemeyer | rog: Testing as root is bad practice, and you know that | 14:44 |
| rog | niemeyer: i agree. but how do you suggest i provide a meaningful test of this code: https://codereview.appspot.com/6312044/diff/1/container/container.go | 14:45 |
| rog | ? | 14:45 |
| rog | niemeyer: second-guessing what the upstart package is going to do seems wrong to me | 14:45 |
| niemeyer | rog: Please have a look at how fwereade tested the upstart package | 14:45 |
| rog | [15:45:50] <rog> niemeyer: second-guessing what the upstart package is going to do seems wrong to me | 14:46 |
| rog | i can't test this without knowing exactly what commands upstart is going to call, where it's storing the files, what they look like, etc | 14:46 |
| niemeyer | rog: Sorry, but I don't know what that means.. I'm suggesting having a look at the technique used by the upstart package to test things | 14:46 |
| niemeyer | rog: The actions from these package look trivial | 14:46 |
| rog | niemeyer: they are | 14:47 |
| niemeyer | rog: I hope we can find a trivial way to test them | 14:47 |
| rog | niemeyer: i haven't succeeded yet! | 14:47 |
| niemeyer | rog: that doesn't involve running them as root | 14:47 |
| rog | niemeyer: suggestions very welcome. | 14:48 |
| rog | niemeyer: i've looked at the upstart tests BTW | 14:48 |
| rog | niemeyer: anyway, gotta go to the dentist. back in a bit. | 14:49 |
| niemeyer | rog: Isn't this merely writing a file on disk right now? | 14:49 |
| niemeyer | rog: I hope we don't need root to test that! :-/ | 14:49 |
| niemeyer | fwereade: ping | 14:50 |
| fwereade | niemeyer, pong | 14:50 |
| rog | niemeyer: yes. but to verify that file, i've got to know what file is written and what it looks like. given that the upstart package itself doesn't have a single test that actually verifies that it really works with upstart, we should do that *somewhere* i think. | 14:50 |
| niemeyer | <niemeyer> fwereade: Okay, can we go back to that branch we were talking about when the conversation diverged? | 14:50 |
| niemeyer | rog: Okay.. if you can't figure how to test that file, please move on to something else and I'll figure it. | 14:51 |
| niemeyer | rog: It's a trivial text file | 14:51 |
| fwereade | niemeyer, probably, if you tell me where you consider it to have diverged... | 14:51 |
| niemeyer | rog: That somewhere is the upstart package, by the way.. | 14:51 |
| niemeyer | fwereade: I just mean following up the conversation | 14:52 |
| niemeyer | fwereade: We were talking about the CL and then moved on to something else.. I'd just like to see what we do with it | 14:53 |
| fwereade | niemeyer, ah, cool, the RelatedService one | 14:53 |
| niemeyer | fwereade: Yeah | 14:53 |
| fwereade | niemeyer, well... not sure where to go really: have I allayed your fears somewhat? | 14:54 |
| niemeyer | fwereade: Yeah, I'm happy with the overall concept | 14:54 |
| niemeyer | fwereade: I'm not entirely happy with the type and method itself, though | 14:54 |
| fwereade | niemeyer, ah, ok, go on | 14:54 |
| niemeyer | fwereade: The data in the RelatedService method feels very weird to me | 14:54 |
| niemeyer | fwereade: It alludes a bit to that conversation we had the other day | 14:55 |
| niemeyer | fwereade: About how we store data | 14:55 |
| niemeyer | fwereade: I'm looking at https://codereview.appspot.com/6307099/diff/1/state/service.go | 14:56 |
| fwereade | niemeyer, I'm more than happy to stick "relation" on the front of everything | 14:56 |
| niemeyer | fwereade: Line 239 | 14:56 |
| niemeyer | fwereade: Not about naming.. about the actual data | 14:56 |
| fwereade | niemeyer, ah, ok, the RelatedRole? | 14:56 |
| fwereade | niemeyer, or the totality of bits of data that make up RelatedService? | 14:57 |
| niemeyer | fwereade: If s is service A, with a requirer relation R1 named "cache".. this is storing {A, R1, provider(!)} | 14:58 |
| niemeyer | fwereade: If s is service A, with a requirer relation R1 named "cache".. this is storing {A, R1, "cache", provider(!)} | 14:58 |
| niemeyer | (added the name) | 14:58 |
| fwereade | niemeyer, whoa, I would appear to be totally on crack there | 14:59 |
| niemeyer | fwereade: This RelatedService is *not* the remote service.. this is the *local* idea of a relation for service s.. look at the service key in the data. | 14:59 |
| fwereade | niemeyer, I switched that back and forth several times, must have desynchronised myself | 14:59 |
| niemeyer | fwereade: The previous doc for the type, and the data stored in the relation, was apparently right | 14:59 |
| fwereade | niemeyer, my intent there was to store the *other* service, and its role | 15:00 |
| niemeyer | fwereade: Yeah, I'm not sure it's right, though | 15:00 |
| fwereade | niemeyer, well, the relationKey and scope are the same on either side | 15:01 |
| niemeyer | fwereade: The relation key and scope are.. the relation role and relation name are not | 15:01 |
| niemeyer | fwereade: We're losing those | 15:01 |
| fwereade | niemeyer, the relation name should IMO come from the local side, and the role and service key from the remote side | 15:02 |
| niemeyer | fwereade: Exactly.. that sounds a bit messy to be honest | 15:02 |
| fwereade | niemeyer, but I'm pretty sure that that is the actual data we need | 15:03 |
| fwereade | niemeyer, it's everything about the remote service, plus the local name for that service | 15:03 |
| fwereade | niemeyer, ok not for the service | 15:03 |
| niemeyer | fwereade: ;-) | 15:04 |
| fwereade | niemeyer, for the relation through which we are connected to that service | 15:04 |
| fwereade | niemeyer, hell, just "for the relation" | 15:04 |
| niemeyer | fwereade: You're getting into the confusion that I'd like to avoid :-) | 15:04 |
| niemeyer | fwereade: At the moment s.Relations() returns the participation for s in all its relations | 15:05 |
| niemeyer | fwereade: {s.key, relation key, relation name, relation role} | 15:06 |
| niemeyer | fwereade: All of that data is for *s* | 15:06 |
| fwereade | niemeyer, yes, that is true; and it looks nice and simple there, and then we have to collect all the remote data later | 15:07 |
| fwereade | niemeyer, which is not so nice and simple, and happens more than once | 15:07 |
| niemeyer | fwereade: Yep, we can add another call which is also nice and simple that returns the other services participating in the same relation | 15:08 |
| niemeyer | fwereade: and in fact we should cache that data so we don't have to query it again | 15:08 |
| fwereade | niemeyer, what would you call that other method? | 15:08 |
| niemeyer | fwereade: Relation.Services? | 15:09 |
| niemeyer | fwereade: Alternatively, | 15:10 |
| fwereade | niemeyer, ah, so you're suggesting a top-level Relation type? | 15:10 |
| fwereade | niemeyer, it really feels like just one more step to go through | 15:10 |
| niemeyer | fwereade: I'm trying to imagine how to solve it rather | 15:11 |
| niemeyer | fwereade: Another suggestion: | 15:11 |
| niemeyer | func (s *Service) Relations() ([][]*ServiceRelation, err error) | 15:11 |
| niemeyer | fwereade: One relation per entry | 15:12 |
| fwereade | niemeyer, ok, and then we still have to go through the leaves discarding or not-discarding the various Servicerelations that refer to the original service | 15:12 |
| niemeyer | fwereade: First ServiceRelation in the relation is guaranteed to be s | 15:12 |
| fwereade | niemeyer, picking what to do based on the length of the []ServiceRelation is kinda nasty though | 15:13 |
| fwereade | niemeyer, IMO the advantage of RelatedService is it's something that all the clients I'm aware of can treat the same regardless of role or scope | 15:14 |
| niemeyer | fwereade: Doesn't have to be based on the length.. l[0].RelationRole() | 15:14 |
| niemeyer | fwereade: The disadvantage is that it's mixing up roles and services in a way not seen anywhere else | 15:14 |
| niemeyer | fwereade: We AddRelation() a list of endpoints.. we add-relation a list of relations with service:rel on the same side.. etc | 15:15 |
| niemeyer | fwereade: We store them in the topology in the same way.. | 15:16 |
| fwereade | niemeyer, it doesn't seem to me to be enormously controversial to suggest that user-facing representations of data do not necessarily map perfectly to internal representations of same | 15:17 |
| niemeyer | fwereade: The internal representation is *also* the same.. | 15:17 |
| niemeyer | fwereade: topology stores data exactly like that | 15:17 |
| niemeyer | fwereade: This is an edge case not seen anywhere else | 15:17 |
| fwereade | niemeyer, ...and in different contexts internally, the same data sometimes wants to be stored in different ways | 15:17 |
| niemeyer | fwereade: and one of the reasons why it is an edge case is that it's being customized for a representation that so far we don't enforece | 15:18 |
| niemeyer | fwereade: enforce | 15:18 |
| niemeyer | fwereade: Relations can have three services today, if we choose to | 15:18 |
| niemeyer | fwereade: and as we debated today, this isn't so far fetched (e.g. peer relations) | 15:18 |
| niemeyer | fwereade: I'm not suggesting we start implementing that now, but it sounds silly to start designing things in a way that is unlike everything else and make that impossible | 15:19 |
| niemeyer | fwereade: (when we can trivially not) | 15:20 |
| fwereade | niemeyer, it strikes me as a lot of speculative generality... AFAICT multi-peer relations should remain easy whatever I do here, and pro/req remain brain-bendingly hard | 15:21 |
| fwereade | niemeyer, I feel that you're less happy with the general concept than perhaps you thought... either the RelatedService concept is sane, in which case by its nature it ends up combining useful bits of data from potentially more than one service... or it's not, and we have amore fundamental disagreement in play :) | 15:24 |
| niemeyer | fwereade: I would agree on the speculative generality if we were not in a situation where everything else works the same way. | 15:24 |
| niemeyer | fwereade: and if we didn't have an option at hand that was trivial to implement. | 15:25 |
| niemeyer | fwereade: What you're suggesting is a departure from what we have now, | 15:25 |
| niemeyer | fwereade: And you just mixed up both sides while *suggesting it* above | 15:25 |
| niemeyer | fwereade: I'd really appreciate if we could find a model that was not like that | 15:26 |
| niemeyer | fwereade: Doesn't have to be my suggestion, though. You know the constraints and concerns we have. Anything else that solves these issues is also fine. | 15:26 |
| fwereade | niemeyer, I mixed it up in the aftermath of a long conversation, involving many of us, which would seem to imply that the whole subject is not quite as simple as it might at first appear | 15:26 |
| niemeyer | fwereade: In my humble opinion, you've mixed up in the precise place I'm concerned about, because it is confusing. | 15:27 |
| niemeyer | fwereade: Having {service key, relation key, relation name} where relation name is not for {service key, relation key} is *extremely* confusing and unnecessary. | 15:28 |
| fwereade | niemeyer, ok.... I'll think some more | 15:29 |
| fwereade | niemeyer, I'm just very reluctant to remake the mistakes we made in the python version and then duplicated in the go version | 15:30 |
| fwereade | niemeyer, (it's just a doc issue... the docs in both places claim a perspective that is not accurate) | 15:31 |
| fwereade | niemeyer, maybe fixing the types to match the docs is not the right way though | 15:31 |
| * fwereade cig, think | 15:31 | |
| niemeyer | fwereade: I've been explaining why it's not just a doc issue.. we have a model used across the board | 15:32 |
| niemeyer | fwereade: and this is not it | 15:32 |
| niemeyer | fwereade: I'm not suggesting we follow blindly what we do with Python, though | 15:32 |
| niemeyer | fwereade: You've been making changes to the relation stuff that have been accepted | 15:32 |
| fwereade | niemeyer, that wasn't what I was trying to imply, sorry | 15:36 |
| fwereade | niemeyer, how would you feel about something with the same core idea, but hopefully a little more explicit and less confusing: {remoteServiceKey string, relationKey string, localEndpoint RelationEndpoint}? | 15:37 |
| fwereade | niemeyer, combining "what we're connected to" and "how we're connected to it" without leading to the confusion you rightly disapprove of | 15:38 |
| fwereade | niemeyer, (I would still say that having all the existing types documented to embody a perspective that they actually don;t *is* a doc issue) | 15:39 |
| niemeyer | fwereade: Where are we doing this today? | 15:41 |
| fwereade | niemeyer, // ServiceRelation represents an established relation from | 15:42 |
| fwereade | / the viewpoint of a participant service. | 15:42 |
| niemeyer | fwereade: As far as I understand, that is true with the current implementation | 15:43 |
| niemeyer | fwereade: It's not great, though, I agree | 15:43 |
| fwereade | niemeyer, ha, yes, I see that... :( | 15:44 |
| niemeyer | fwereade: What about [][]RelationEndpoint? | 15:45 |
| niemeyer | fwereade: We wouldn't even need the RelatedService/ServiceRelation red herring | 15:45 |
| niemeyer | fwereade: This is pushing in the direction you were already doing too, with AddRelation(endpoints...) and RemoveRelation(endpoints...) | 15:46 |
| fwereade | niemeyer, hmm, there's still a certain amount of tomfoolery necessary to extract the data we care about in both the unit agent and in status | 15:46 |
| niemeyer | (no docs!) | 15:46 |
| niemeyer | fwereade: I don't see how it's any worse than the RelatedService interface, to be honest | 15:47 |
| niemeyer | fwereade: rels := s.Relations(); for i, endpoints := rels { ... } | 15:47 |
| niemeyer | fwereade: rels := s.Relations(); for i, endpoints := range rels { ... } | 15:47 |
| niemeyer | fwereade: That's really it | 15:47 |
| niemeyer | fwereade: With the additional guarantee that endpoints[0] is always for s | 15:47 |
| niemeyer | fwereade: and len(endpoints) > 0 | 15:48 |
| fwereade | niemeyer, plus fooling around to determine which endpoint we care about, which the s[0] thing doesn't address | 15:48 |
| niemeyer | fwereade: This sounds really easy to deal with | 15:48 |
| niemeyer | fwereade: That went over my head | 15:48 |
| fwereade | I guess s[len(s)-1] is not too hard | 15:48 |
| niemeyer | fwereade: INdeed, and I bet that we'll need to special case the peer case anyway | 15:50 |
| fwereade | niemeyer, the point of this is to avoid special-casing peers | 15:50 |
| niemeyer | fwereade: Yeah, I understand, I'm saying that I suspect we'll need to be careful about it anyway | 15:51 |
| niemeyer | fwereade: Peer relations are established by hand, etc | 15:51 |
| niemeyer | fwereade: Erm, manually by juju | 15:51 |
| fwereade | niemeyer, yeah, that is also something that bugs me | 15:51 |
| fwereade | niemeyer, but not all that much :) | 15:51 |
| niemeyer | fwereade: I quite like the above interface, actually.. it seems quite appropriate that we can, for example, iterate over the results of s.Relations() and use the result to call state.RemoveRelation(endpoints...) directly | 15:52 |
| fwereade | niemeyer, yeah, I don't have a strong negative reaction to that one | 15:53 |
| fwereade | niemeyer, I'll see where I can take it | 15:53 |
| niemeyer | fwereade: I also love the fact we don't have to document ServiceRelation :-) | 15:53 |
| fwereade | niemeyer, ha, yeah :) | 15:53 |
| fwereade | niemeyer, offhand it seems this *does* imply a Relation type... {key string endpoints []RelationEndpoint} | 15:55 |
| niemeyer | fwereade: Indeed.. but is it any better than [][]RelationEndpoint? | 15:55 |
| niemeyer | fwereade: For that specific use case? | 15:55 |
| niemeyer | fwereade: I see this as []Relation | 15:56 |
| niemeyer | fwereade: We just happened to have leaning towards representing Relation as []RelationEndpoint, apparently | 15:56 |
| fwereade | niemeyer, ...hmm, I guess I can keep the relation key out of it for a while | 15:56 |
| niemeyer | have been | 15:56 |
| fwereade | niemeyer, we can grab it from the topology when we need it | 15:57 |
| niemeyer | fwereade: I'm not suggesting that's a bad idea, by the way.. sounds like a perfectly good type | 15:57 |
| niemeyer | fwereade: We might just hold off a bit to see if we need it as such in practice | 15:58 |
| niemeyer | fwereade: If you feel the logic would be more sensible with it, I'd be happy too | 15:58 |
| niemeyer | fwereade: (e.g. maybe we do need the relation key materialized for the algorithms) | 15:59 |
| fwereade | niemeyer, I'm sure enough we'll want it soon that I'd like to wrap the []RelationEndpoint in a Relation type, even if I leave the key out until it's needed | 15:59 |
| niemeyer | fwereade: +1 then | 16:00 |
| fwereade | niemeyer, just to reduce the churn from []RelationEndpoint to *Relation | 16:00 |
| fwereade | niemeyer, cool, I'll see what I can do with that | 16:00 |
| niemeyer | fwereade: We can have the key from the get go.. sounds sensible enough even if unused | 16:00 |
| fwereade | niemeyer, even better :) | 16:00 |
| niemeyer | fwereade: Woohay agreement | 16:01 |
| niemeyer | :) | 16:01 |
| fwereade | niemeyer, thank you... as usual, I feel wiser :) | 16:01 |
| niemeyer | robbiew: Do we have the cloud consistency call today? | 16:01 |
| robbiew | niemeyer: let me check with zaid | 16:01 |
| niemeyer | fwereade: My pleasure, love those conversations too | 16:01 |
| niemeyer | robbiew: Cool, leaving for lunch now.. will be back on time no matter what | 16:03 |
| robbiew | niemeyer: bet on it being cancelled | 16:03 |
| hazmat | robbiew, ? why | 16:29 |
| robbiew | hazmat: b/c sabdfl isn't here ;) | 16:30 |
| hazmat | cool | 16:30 |
| * hazmat lunches | 17:03 | |
| * rog is in need of a good name. The provisioning agent has ProvisioningAgent (the command) and Provisioner (the implementing type, renewed when the state connection fails). I'm looking for an equivalent to Provisioner for the machine agent. | 17:09 | |
| rog | i'm wondering about machineAgent, but that might be confusing. | 17:10 |
| niemeyer | rog: MachineAgent + Machiner? :-) | 17:11 |
| rog | niemeyer: Machiner seems weird though. i did think of that. | 17:12 |
| * rog didn't see the smiley | 17:12 | |
| niemeyer | rog: It does sound weird, but I was only half-joking ;) | 17:12 |
| rog | niemeyer: ok, Machiner it is for the time being, pending better thoughts | 17:18 |
| niemeyer | rog: Sounds good | 17:18 |
| niemeyer | mramm: ping | 17:20 |
| fwereade | rog, hmm, Uniter is actually a surprisingly good name for what the unit agent does, too :) | 17:32 |
| rog | fwereade: unite all the rest of the juju functionality in one command, you mean? | 17:33 |
| fwereade | rog, it also unites various services, if you squint | 17:33 |
| * rog can't squint that hard :-) | 17:34 | |
| * fwereade has -6.5, -7.0 eyesight and can squint with the best | 17:34 | |
| niemeyer | fwereade: RelatedFoo tends to be a bad name to refer to something that is part of a relation | 17:35 |
| niemeyer | fwereade: Everything there is related | 17:35 |
| fwereade | niemeyer, crap, you're absolutely right, but Counterpart didn't seem quite right there | 17:35 |
| fwereade | niemeyer, still, if I can teach myself to default to that every time I want to type Related it'd probably be better, wouldn't it | 17:36 |
| niemeyer | fwereade: Right.. and this seems related to a more fundamental issue | 17:36 |
| niemeyer | fwereade: *Relation* shouldn't have these methods | 17:36 |
| niemeyer | fwereade: For the reasons we discussed earlier.. a Relation has no side | 17:36 |
| fwereade | niemeyer, hence the unordered endpoints | 17:37 |
| fwereade | niemeyer, my feeling was that the serviceName specified a side quite well | 17:37 |
| niemeyer | fwereade: Hm.. how so? | 17:38 |
| * fwereade is really glad he's talking about this now and not just seeing the review tomorrow am ;) | 17:38 | |
| fwereade | niemeyer, back up a sec: do you have a problem with (*Relation)Endpoint(serviceName string) (RelationEndpoint, error)? | 17:39 |
| fwereade | niemeyer, or just with RelatedEndpoint(serviceName) | 17:39 |
| niemeyer | fwereade: Hmm | 17:40 |
| niemeyer | fwereade: The former sounds totally fine | 17:40 |
| fwereade | niemeyer, I freely admit the name of the latter is awkward | 17:40 |
| fwereade | niemeyer, I *think* it's a valid thing to ask for though | 17:40 |
| niemeyer | fwereade: I'm starting to think we should go back on CounterpartRole, or at least make it private.. | 17:44 |
| niemeyer | fwereade: This seems to have been guiding your thinking in those branches | 17:44 |
| rog | fwereade: OtherEndpoint ? | 17:44 |
| fwereade | niemeyer, not consciously... | 17:44 |
| niemeyer | fwereade: and I would prefer to not create APIs that sparkle these assumptions on the whole code base | 17:44 |
| niemeyer | fwereade: There's no RelatedEndpoint | 17:45 |
| fwereade | niemeyer, it just seems that there were a couple of places where we needed to match up counterparts, and I anticipate another, so... | 17:45 |
| niemeyer | fwereade: There are participants in a relation | 17:45 |
| niemeyer | fwereade: The only assumption a given endpoint should make is that it is part of a relation with other endpoints | 17:45 |
| niemeyer | fwereade: The number of other endpoints depends on role | 17:45 |
| fwereade | niemeyer, or without any other endpoints whatsoever ;) | 17:46 |
| niemeyer | fwereade: There's nothing in our current model that binds the number of endpoints to one, two, three, etc | 17:46 |
| niemeyer | fwereade: Over the last few days, I've been trying to maintain that model, and we have been arguing back and forth around issues that deep inside are related to that | 17:46 |
| fwereade | niemeyer, yeah, this sounds like it's the heart of it | 17:47 |
| niemeyer | fwereade: RelatedEndpoint makes no sense if there are three endpoints | 17:47 |
| fwereade | niemeyer, agreed | 17:47 |
| niemeyer | fwereade: and it makes little sense if there is a single one | 17:47 |
| niemeyer | fwereade: (the related endpoint of A is A? huh?) | 17:47 |
| fwereade | niemeyer, well, it makes more sense than you might think | 17:47 |
| fwereade | niemeyer, if you read the docs for RelatedEndpoint | 17:47 |
| niemeyer | fwereade: "In the case of a peer relation," | 17:48 |
| fwereade | niemeyer, I *think* I was careful to write it in such a way that it did make sense for peer relations | 17:48 |
| niemeyer | fwereade: That's in the docs | 17:48 |
| niemeyer | fwereade: It means "Yo reader! Special case ahead!" | 17:48 |
| fwereade | niemeyer, on the contrary, it means "yo reader! you probably didn;t read the last sentence carefully enough!" ...which I concede is not *much* better | 17:49 |
| fwereade | niemeyer, ok, in both significant cases I am aware of, I want to be able to easily get the service which has units that units of a given service might want to relate to | 17:50 |
| niemeyer | fwereade: Hmm.. is that implementation right? | 17:50 |
| niemeyer | fwereade: Oh, ok, nevermind | 17:50 |
| fwereade | niemeyer, would it make you happier if I made that service*s* which have units which units of a given service might want to relate to? | 17:50 |
| niemeyer | fwereade: It is a special case indeed.. | 17:50 |
| niemeyer | fwereade: Look at the implementation | 17:50 |
| niemeyer | fwereade: "If it's not my service, than pick the other" | 17:51 |
| niemeyer | Erm | 17:51 |
| fwereade | niemeyer, yes | 17:51 |
| niemeyer | fwereade: "If it's not my name, than pick the other" | 17:51 |
| fwereade | niemeyer, indeed | 17:51 |
| fwereade | niemeyer, the trouble is that we have this task I described above | 17:51 |
| fwereade | niemeyer, it doesn't seem to me to be a good move to insist on sprinkling special cases per relation type through the code right now, every time we want to do this, on the basis that we might want to have more special cases later | 17:53 |
| niemeyer | fwereade: Agreed, but that seems to have different meanings for us | 17:54 |
| fwereade | niemeyer, apparently so :) | 17:54 |
| niemeyer | fwereade: RelatedEndpoint in my mind is *huge* special casing | 17:55 |
| niemeyer | fwereade: It special cases the concept of provider and requirer, of peer, and of the assumption that there are only 1 or two relation endpoints | 17:55 |
| niemeyer | fwereade: All in that small block of code | 17:55 |
| fwereade | niemeyer, all in that small block of code *tacked onto the Relation type* | 17:55 |
| fwereade | niemeyer, I think we agree that there are special cases regardless | 17:56 |
| niemeyer | fwereade: No, not at this level | 17:56 |
| fwereade | niemeyer, I want my relation-related special cases packaged up neatly in the place I expect | 17:56 |
| niemeyer | fwereade: The existence of a Relation is generic | 17:56 |
| niemeyer | fwereade: The fact multiple endpoints participate in a relation is generic | 17:56 |
| niemeyer | fwereade: The fact they may have arbitrary types is generic | 17:56 |
| niemeyer | fwereade: The fact they are arbitrarily numbered is generic | 17:56 |
| niemeyer | fwereade: All of those were properly generic in the previous API | 17:57 |
| niemeyer | fwereade: CounterpartRole changes that | 17:57 |
| niemeyer | fwereade: RelatedEndpoint changes that | 17:57 |
| fwereade | niemeyer, weeeeeeelll. | 17:57 |
| fwereade | niemeyer, I invite you to take a look at just what would be involved in adding multi-endpoint relations to the python | 17:57 |
| niemeyer | fwereade: and I'm feeling bad about it because it feels like we're going down that path for no good reason | 17:57 |
| fwereade | niemeyer, but I *am* trying to figure out a middle ground :) | 17:58 |
| fwereade | <fwereade> niemeyer, would it make you happier if I made that service*s* which have units which units of a given service might want to relate to? | 17:58 |
| niemeyer | fwereade: I didn't parse that sentence.. that's mainly why I didnt answer | 17:58 |
| niemeyer | fwereade: "services which have units which units of given service"? | 17:59 |
| fwereade | niemeyer, so, (*Relation) OtherEndpoints(serviceName string) ([]RelationEndpoint, error) | 17:59 |
| fwereade | niemeyer, ok, not Other | 17:59 |
| niemeyer | fwereade: No, that actually sounds fine | 17:59 |
| niemeyer | fwereade: With Other too | 17:59 |
| fwereade | niemeyer, because I would still want to return the peer endpoint there | 17:59 |
| niemeyer | fwereade: Uh oh | 18:00 |
| niemeyer | fwereade: Then no | 18:00 |
| fwereade | niemeyer, it's a matter of "[units of which services] might my units want to communicate with?" | 18:00 |
| niemeyer | fwereade: You have a very specific use case in mind, and is tailoring a generic interface to that very specific use case | 18:00 |
| niemeyer | fwereade: Just special case it in your use case.. I bet it's a three or four lines function | 18:00 |
| niemeyer | fwereade: Exactly.. there no "my units" concept outside of "your unit" | 18:01 |
| fwereade | niemeyer, yes, I am primarily focused on getting an interface that allows me to express what I need in the same way in the 2 cases we currently have | 18:01 |
| fwereade | niemeyer, so you wouldn't be opposed to a function that basically did that, but you don't like having it on Relation? | 18:02 |
| niemeyer | fwereade: Silly example: it's a well known model to have Primary, Secondary, and Voter | 18:03 |
| niemeyer | fwereade: If I have a Secondary, what's the other Unit? | 18:03 |
| fwereade | niemeyer, I don't really see how that fits our scenario... can't secondaries become primaries, etc? | 18:04 |
| niemeyer | fwereade: That's not the point.. | 18:05 |
| niemeyer | fwereade: What's the CounterpartRole? | 18:05 |
| rog | niemeyer: a couple of small reviews for you, if you like: https://codereview.appspot.com/6302089/ and https://codereview.appspot.com/6312044/ | 18:06 |
| niemeyer | rog: Thanks, I'll include them in the set of reviews | 18:06 |
| rog | i've gotta go, see y'all tomorrow | 18:07 |
| niemeyer | rog: Have a good one | 18:07 |
| fwereade | niemeyer, look, I tried to start this conversation several days ago, and you brushed it off with words to the effect of "I don't know, maybe we'll need more, maybe we're fine with what we have" | 18:07 |
| fwereade | niemeyer, I'm happy to drop CounterpartRole | 18:07 |
| niemeyer | fwereade: Thanks, I'm starting to feel like this is more important than I thought, because it will affect our thinking elsewhere | 18:07 |
| fwereade | niemeyer, I understand the high-level concept of "there may be more relation types" | 18:07 |
| fwereade | niemeyer, I don;t follow how primary/secondary/voter fits in with what we have, because roles can change | 18:08 |
| niemeyer | fwereade: But i was actually trying to explain why I see issues in the OtherEndpoint stuff | 18:08 |
| niemeyer | fwereade: It was an example.. I do think we'll have other relations in the future, and I don't know how they look like right now | 18:08 |
| niemeyer | fwereade: We'll need smart brains, etc | 18:08 |
| niemeyer | fwereade: We'll have maintenance of filesystems and whatnot | 18:09 |
| fwereade | niemeyer, it seems to me that the roles we have in play are used in really quite a lot of places | 18:09 |
| niemeyer | fwereade: I don't know how those will look like | 18:09 |
| niemeyer | fwereade: But I'm trying to point out that there's no reason why relations are singletons of pairs only | 18:09 |
| niemeyer | fwereade: Primary/Secondary/Voter is a model that I thought was easier to get across | 18:09 |
| niemeyer | But maybe not.. | 18:09 |
| fwereade | niemeyer, I'm trying to handle one single problem: given a relation R, and a service S in that relation, independent of the number of services | 18:10 |
| fwereade | niemeyer, *what* services in that relation might units of S need to respond to changes in? | 18:10 |
| fwereade | niemeyer, I think that this is a generic question | 18:11 |
| niemeyer | fwereade: Processing your (clean) statement.. hmm | 18:11 |
| fwereade | niemeyer, I understand your reservations re CounterpartRole, but I'm not sure it's as simple as dropping it... what about topology, where we had the counterpart dict beforehand? | 18:13 |
| niemeyer | fwereade: This is facilitating an algorithm that takes into account the specific roles.. that algorithm knows the roles, and deals with them appropriately | 18:13 |
| fwereade | niemeyer, CanRelateTo takes one argument, and that encodes the 2-service asusmption as well | 18:14 |
| niemeyer | fwereade: Not really | 18:14 |
| fwereade | niemeyer, at least, surely, we need a CanMakeRelationWith(roles ...RelationRole) ? | 18:14 |
| niemeyer | fwereade: CanRelateTo(A, B).. CanRelateTo(B, C).. | 18:14 |
| niemeyer | fwereade: Not necessarily, no | 18:15 |
| niemeyer | fwereade: Just like you don't need a GreaterThan(numbers ...) to see if a number is larger than a set of numbers | 18:15 |
| niemeyer | fwereade: FWIW, that API was conceptualized in that form with this in mind | 18:16 |
| fwereade | niemeyer, I think relations between services are perhaps a little more involved than comparisons between reals, but I worry that I'm straying off the point | 18:16 |
| fwereade | niemeyer, does it make sense to have an incomplete multi-endpoint relation though? | 18:17 |
| niemeyer | fwereade: CanRelateTo(Voter, Primary)... CanRelateTo(Primary, Secondary)... | 18:17 |
| niemeyer | fwereade: These are examples.. I hope they make some sense to you | 18:17 |
| fwereade | niemeyer, *kinda*, but... does it really make sense to have, say, *just* voters and primaries? | 18:18 |
| niemeyer | fwereade: In a number of cases, yes | 18:19 |
| fwereade | niemeyer, I'm not clear what impact the votes could have then | 18:20 |
| fwereade | niemeyer, but can we take it as read that there are more kinds of relation than are dreamed of in my philosophy | 18:20 |
| niemeyer | fwereade: For a typical database, maybe it doesn't do much, but it actually works, and may be a fine step towards establishing the full topology wanted | 18:20 |
| niemeyer | fwereade: The real point is that CanRelateTo(Primary, Primary) would fail.. and CanRelateTo(Peer, Requirer) would also fail | 18:21 |
| fwereade | niemeyer, ok, that makes sense | 18:21 |
| fwereade | niemeyer, but regardless... can you think of a generic way to expose an answer to the problem I'm trying to solve? | 18:23 |
| fwereade | niemeyer, OtherEndpoints would almost work, except for peer relations it's not really an "other" endpoint | 18:23 |
| fwereade | niemeyer, RelatedEndpoints feels to me like it's the closest I've heard yet | 18:24 |
| fwereade | niemeyer, but I suspect you will dislike that word for perfectly rational reasons | 18:24 |
| niemeyer | fwereade: I'm still processing your previous (clean) statement.. that was a great way to put it | 18:24 |
| fwereade | niemeyer, thanks :) | 18:25 |
| fwereade | niemeyer, I'm also fretting that, really, we've got a *lot* of generic code to put in place to support this | 18:25 |
| fwereade | niemeyer, for example, I can't in good conscience implement unit agents with only one presence node per relation | 18:26 |
| fwereade | niemeyer, I can't watch related units without watching a range of possible locations for the presence nodes | 18:26 |
| fwereade | niemeyer, it all makes me very nervous | 18:26 |
| fwereade | sorry, brb | 18:27 |
| niemeyer | fwereade: Okay, your thinking seems to be reaching my head | 18:30 |
| fwereade | b | 18:32 |
| niemeyer | fwereade: :) | 18:32 |
| fwereade | niemeyer, indeed, I'm not actually blind to your perspective, but it has been rather hard to incorporate it at the point where the bits hit the disk, as it were ;) | 18:33 |
| niemeyer | fwereade: Your idea of RelatedEndpoint sounds fine.. I can't think of a better name either | 18:33 |
| niemeyer | fwereade: Can we do one tweak: | 18:34 |
| fwereade | niemeyer, surely :) | 18:34 |
| niemeyer | fwereade: Considering all the conversation about regarding multiple endpoints and the unknown coming future, | 18:34 |
| niemeyer | s/about/above | 18:34 |
| niemeyer | fwereade: I'd appreciate if we could make counterpartRole private, and make RelatedEndpointS (plural) return a slice | 18:35 |
| fwereade | niemeyer, SGTM | 18:35 |
| niemeyer | fwereade: and attempt to keep in mind the fact we don't have a clearly defined set of relation roles when doing the lower level algorithms | 18:35 |
| fwereade | niemeyer, if I'd been thinking I would have made that private in the first place | 18:35 |
| fwereade | niemeyer, I'll do my best :) | 18:36 |
| niemeyer | fwereade: I don't want to go crazy with that, though | 18:36 |
| niemeyer | fwereade: It's just a way to force us to *try* to go in that direction if it's not too much pain | 18:36 |
| niemeyer | fwereade: If it turn out to make no sense, we can always talk again and see if we should reevaluate our position | 18:36 |
| fwereade | niemeyer, ok, I'll see how we do :) | 18:37 |
| niemeyer | fwereade: Thanks a lot, and sorry for making such a minor step difficult somehow | 18:37 |
| fwereade | niemeyer, on the upside, multi-peer relations really shouldn't be too hard even as we are now | 18:37 |
| fwereade | niemeyer, np, a pleasure, we all end up the richer for these discussions ;) | 18:38 |
| niemeyer | fwereade: That's an easy case to keep in mind.. if we can handle that with minor trouble, we're probably in a good direction | 18:38 |
| fwereade | niemeyer, cool | 18:38 |
| fwereade | niemeyer, I should probably go and see some physical human beings now, gn :) | 18:38 |
| niemeyer | fwereade: Haha | 18:39 |
| niemeyer | fwereade: Sounds like a good plan, thanks :-) | 18:39 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!