[06:54] davecheney, fwereade: hiya [06:54] rog, heyhey [07:15] Morning. [07:29] morning all [07:30] davecheney: Hi === TheRealMue is now known as TheMue [08:25] aujuju: How to SSH into local juju instance? [08:31] Hmm, t-storm above our home. === mcclurmc_ is now known as mcclurmc [10:07] morning. [10:46] Aram: Hi [11:46] is this page still the "official" guide to writing charms? [11:46] https://juju.ubuntu.com/docs/write-charm.html [11:49] if so, then maybe it should start by referencing a directory that actually *exists* in juju trunk [11:50] having to check out juju trunk when you're using an installed juju is not the friendliest way of starting a tutorial as well [11:50] "Assuming the current directory is the juju trunk"... [11:51] anyway, just some friendly feedback :-) [12:27] niemeyer: yo! [12:29] voidspace, thanks for pointing that out :) [12:29] niemeyer, heyhey [12:30] hi niemeyer. [12:30] fwereade: morning [12:30] voidspace, how's it going? [12:31] fwereade: mainly good :-) [12:31] voidspace, mainly glad to hear it then :) [12:31] fwereade: daughter has unexplained minor illness causing random temperature fluctuations [12:31] voidspace, aww :( [12:31] Gooood morning all! [12:31] fwereade: but she's happy and lively, so not *much* of a problem :-) [12:31] niemeyer: morning [12:31] fwereade: and yourself? [12:31] our team is sprinting on getting us all up to scratch with juju this week... [12:31] voidspace, yeah, very good thanks [12:32] voidspace, ah, excellent [12:32] fwereade: so expect more "observations" on your documentation :-) [12:32] voidspace, they're greatly appreciated [12:32] hehe [12:32] voidspace, bug reports even more so ;) [12:33] niemeyer: I have some errands to attend to, I'm talking half a day off and will recuperate it later this evening/night. [12:33] getting juju up and running with lxc inside a vm was astonishingly easy though - so kudos on that (using precise) [12:33] Aram: Sounds good, thanks for the note [12:33] fwereade: want an issue for the charm writers guide? [12:33] voidspace, it's very cool, isn't it -- sadly I can't take much credit for that bit :) [12:33] fwereade: I wasn't 100% certain I was on the current-latest-correctest documentation page anyway [12:33] voidspace, that would be fantastic [12:33] heh [12:34] voidspace, if it's what you find when you look for the docs then I think that's something we should address [12:34] fwereade: yep [12:35] niemeyer, when you have a mo please let me know if I still appear to be on crack re relation types [12:36] niemeyer, I do think that RelatedService is applicable more widely than just in the unit agent [12:37] fwereade: Hmm [12:38] fwereade: Maybe.. that comment was made on a bad foundation. I still had in mind the old interface of AddRelation [12:38] niemeyer, yeah, I was a bit surprised when I looked back at the python code and discovered that nobody ever paid any attention to the result of AddRelation [12:39] niemeyer, add_relation_state, rather [12:40] fwereade: I still think eventually we'll need something that represents relations themselves, so we can more easily iterate. But that's a gut feeling rather than any kind of certainty, so I'm happy to see if we can tailor the model to something sensible for the use cases at hand [12:40] niemeyer, I can well believe that, but I don;t think we're there atm -- have you looked at the Relation type in python recently? [12:41] niemeyer, it has an internal_id property, and that's it :) [12:42] fwereade: Good point :) [12:44] niemeyer, there's something else on my mind actually [12:44] niemeyer, consistency of the history that unit agents see [12:45] niemeyer, it really rather bugs me that we have a mix of zk-current and hook-queue-current state available when we're running hook [12:46] niemeyer, and I think it exposes us to bug that we only don't hit through sheer luck, and coincidental properties of the rest of the implementation [12:46] fwereade: How so? [12:47] niemeyer, for example, what should we do when we ask ZK for the current relations in a -joined hook and it turns out that the relation we're supposed to be joining was just deleted? [12:48] fwereade, previously to relation-ids it all worked as expected [12:49] fwereade, the hook could use apis see a state current at time of execution, and delete hook exec would be pending [12:49] fwereade: It's an excellent question, and we have several such issues all around the code base [12:49] hazmat, ok; but we can also do things like try to look up settings for a relation that doesn't exist any more, and that's only safe because we don't GC anything under /relations [12:50] fwereade, that wasn't a happy accident that was by design [12:50] fwereade, we don't gc things that could be in current use [12:50] hazmat, understood; but it feels to me like it'll be kinda tricky to ever GC relations given the current usage [12:50] fwereade: There are two options.. either we cope with the fact that the relation is being removed and offer a sensible answer [12:51] fwereade, its quite simple actually.. no presence nodes, not in topology, its dead [12:51] fwereade: or we tag the relation (or whatever object) as dead and allow the code to handle it in a timely way [12:51] hazmat, until some misfunctioning unit agent comes back up..? [12:52] fwereade, and it would come back up and see the rel doesn't exist [12:52] ie. it would check the topo [12:52] niemeyer, or alternatively, we snapshot settings just as we do membership, and have unit agents execute hooks against the histroy we snapshotted [12:52] hazmat, ok, GC isn't as tricky as I worried :) [12:52] fwereade, if its disconnected for a while and hasn't notified yet [12:53] fwereade: Not sure I understand what that would mean [12:53] then it will come back up and queue a broken hook in a separate context, and with queue reduction that wins [12:55] niemeyer, hazmat: I may be missing something, but it seems to me that the unit agent gets notified of every relevant change to state, and can therefore keep a consistent local history through which it can run at its own pace (which isn't necessarily as slow as it might be, thanks to hook reduction) [12:55] fwereade: It can't.. assume the unit is down when a change happens [12:56] niemeyer, it can; it has to know how to diff current state against its known latest state anyway [12:56] fwereade: Well, yes, and that goes against "every relevant change to state on consistent local history", right? [12:57] niemeyer, I don;t see the tension there -- we always knew we might miss some changes, and we have queue reduction anyway... what's the difference? [12:58] fwereade, terminology, its not seeing every change, it seeing deltas [12:58] its [12:58] fwereade: I'm just pointing out that "it's fine to miss some changes" and "every relevant change to state on consistent local history" are not friends. [12:58] hazmat, yeah, exactly [12:59] niemeyer, I'm saying that within the constraints we're used to, we can run every hook against the state we knew to exist at the time we detected the need for a hook to run (collapsed as necessary), rather than running against a mixed state [13:00] niemeyer, and I was wondering if there was an obvious downside to that strategy [13:01] fwereade: We can't snapshot the whole state at once [13:02] fwereade: Which means that by the time you discovered you had to run "joined", the state has already moved forward [13:02] fwereade: So there doesn't seem to be much of a point to me to be getting a state that is arbitrary in a time before it is needed [13:02] fwereade: But maybe I'm misunderstanding what you mean [13:03] niemeyer, we're already snapshotting membership, and using that; we're snapshotting settings versions, and using those to figure out when to collapse; we could do the same with the actual settings, n'est-ce pas? [13:03] its not really snapshotting [13:03] its tracking [13:03] niemeyer, when you do a relation-list, you're explicitly not getting the current related units; you're getting the units at the time the change was detected [13:04] its still broken though [13:05] er. not a snapshot [13:05] hazmat, go on [13:05] but conversely if you do a relation-get you do see a current value [13:05] that's cached for the hook exec [13:05] fwereade: There's no way to get the data for multiple nodes at the same time.. it's a cache, rather than a consistent snapshot [13:05] fwereade: relation-list might work better in the case of relation-joined, specifically, but it won't work well in the case of relation-changed, for example [13:06] hazmat, thatis rather my problem, that I'd be a lot more comfortable if we were grabbing the settings at the time they changed and using them [13:06] hazmat, I don;t think we'd ever be too far out of date, because of queue reduction [13:06] fwereade, agreed, these semantics are hard, without evolving a local state copy you can't be consistent throughout [13:06] fwereade: Because you detecting the change and listing the relations are two operations [13:06] fwereade: Done at different times [13:06] fwereade: Yes, not too far out of date, but that's not the point.. if it is out of date, what are we trying to solve then? [13:07] fwereade, and even then its not clear that its truly consistent even then [13:07] niemeyer, (we're always out of date a little) [13:07] fwereade: Exactly [13:07] hazmat, this is to me the most interesting bit... how do you see us being inconsistent with local state? [13:07] fwereade: So doing a snapshot, say, 2 minutes ago so we can run a hook doesn't sound like solving a problem [13:08] fwereade: Reduction is evidence that we actually don't care about the intermediate state [13:08] fwereade, well the question then is what is meant by consistent? that all values ever show to a hook are true as of the time of its related change event mvcc style? [13:08] fwereade: We just try to bring the unit aware of the latest state [13:08] niemeyer, then why do we bother doing what we do with relation-list in the first place? [13:09] * hazmat needs some coffee to correct typos [13:09] niemeyer, I think I'd be comfortable with either "you always get current state" or "you always get consistent state" [13:09] niemeyer, but it feels liek we're mixing the two and it makes my head hurt a little [13:10] fwereade: +1 [13:11] fwereade: Current is impossible, as we both know.. [13:11] niemeyer, and "current" has the problem that we could then be running relation-joined against some remote unit which relation-list tells us doesn't exist [13:11] i wonder if we wouldn't be better having simple "something has changed"-style notifications [13:11] niemeyer, "latest known" then [13:11] niemeyer, but regardless, I agree that whatever you call it, it has problems [13:12] fwereade: Let's talk about relation-changed.. what do you suggest we do regarding relation-lsit? [13:12] fwereade, so re not clear, there's also cross context manipulation now, you have to have a linear view across relations [13:13] ie. relation-set from rel-a-hook to rel-b-hook [13:13] niemeyer, AIUI what we currently do is keep track of which units have joined/departed according to local state, and always give that when asked to relation-list [13:13] er.. rel-b [13:13] hazmat, can you expand a little? I don't *think* that hurts us... but I'm probably missing something [13:15] hazmat, a unit agent only sets its own settings, right? and that's in response to hooks running relation-set... and so that's automatically synchronised, isn't it? [13:15] fwereade: We could snapshot the state of the relation the specific hook is part of before we start the hook, if that's not done today [13:15] fwereade: Is that what you mean? [13:15] hazmat, a unit's own settings will always reflect whatever it's decided to set given the hook that have so far been handled [13:15] fwereade: or rather, what you are suggesting? [13:16] niemeyer, yes, I'm just suggesting that when we detect a settings change we store the settings as part of the history rather than just the fact of a change [13:16] niemeyer, and that that is strictly more consistent than what we have now, and makes my head hurt less :) [13:17] niemeyer, (and hopefully will make life less confusing for people when they're debugging tricky relations) [13:18] fwereade: How does that improve things? [13:19] fwereade: As we just discussed, when we detect that settings have changed, and go to get that, it's already out-of-date [13:19] niemeyer, if we can divorce hook execution environments from zookeeper state (apart from the unit writing its own relation settings) we can eliminate having to think about an entire class of whoops-zk-changed-in-a-confusing-way situations [13:19] fwereade: Why is that out-of-date state any better than the out-of-date state we get when we actually run the hook? [13:19] fwereade, right now the state tracking is mostly in independent relation contexts, it works cross relation through benefit of the db giving us a linear time view (albeit with per rel queue reduction), the point i was trying to raise is that we have cross relation activity now from hooks. i don't think its insurmountable if the linear view across relations is maintained, but it is something to keep in mind. ie any rel-a hook could change settings on a rel-b [13:19] thing. [13:19] niemeyer: FYI continuing weird behaviour with bzr (i've been stuck on it all morning). http://paste.ubuntu.com/1047256/ time to lose the branch, i think. if bzr is this broken, lbox has no hope. [13:20] hazmat, ...whoa, ok, I missed that [13:20] hazmat, how is that OK? [13:21] fwereade, its not without some careful thought ;-) but generally it can work because we have a linear view of events changes across relations that the cache is kept up to date for, but it does require thought to the cache maintenance wrt to queue reduction [13:21] hazmat, we're talking about a foo-relation hook on unit X relation A changeing settings on... unit X relation B? [13:21] fwereade: How is it not ok? [13:21] fwereade, yes [13:22] niemeyer, hazmat: it's not *necessarily* not ok, but it's a surprising extra channel [13:22] rog: Sorry, I can't really help since I can't reproduce the issue locally [13:22] fwereade: I find it pretty straightforward [13:22] niemeyer: yeah, i have no idea. just thought you might be interested to see the bizarreness. [13:24] rog, naked bzr works pretty well.. i kept running into issues with cobzr and gave it up [13:24] niemeyer, ok, as long as it's all on the same unit, that's fine [13:24] hazmat: FWIW, I use it for every single bzr branch [13:25] fwereade: It is, the property "relation-set only changes myself" is sustained [13:25] hazmat: i've never had a problem before. i think i mucked things up when i accidentally did a bzr commit as root. [13:25] rog, interesting.. easy to fix with a chown. [13:25] niemeyer, cool, I misunderstood on first reading, thought we were talking about something altogether more disturbing [13:25] hazmat: that was the first thing i did [13:25] fwereade: So, still trying to understand what you mean [13:26] fwereade: Just so you get the background.. [13:26] fwereade: The idea of caching comes from giving people consistent answers across calls that read data from a remote unit, to make things a little more saner [13:26] fwereade: E.g. (relation-get username + relation-get password) will get data for the same person [13:27] would someone be able to explain to me why what thing changes which relation settings is important here? [13:27] niemeyer, also applies to the local unit reads via the writeback cache [13:27] hazmat: Rihgt [13:28] fwereade: So, you can see our model as "latest known state, snapshotted on first query" [13:28] niemeyer, per queried unit, yeah [13:29] niemeyer, and apart from the actual list of units, which is from longer ago [13:29] niemeyer, none of these things *seriously* bother me [13:29] i.e. why is cross-relation activity a difficulty? [13:29] rog: It's not.. was a red herring [13:30] niemeyer: ok, thanks. [13:30] niemeyer, but I'm equally not sure wat benefit we derive from having two different "now" states in play at a time [13:31] fwereade: We have N "now" states.. we have data from a distributed system, where each detail is arbitrarily old and invalid [13:31] niemeyer, it can be [13:31] niemeyer, with an attempt to have a consistent cache.. at least in the dictates of the py impl [13:31] niemeyer, ok, but we have ordering guarantees from ZK, right? [13:32] fwereade: zk has ordering guarantees.. I don't know what you mean, though [13:32] niemeyer, each rel has a separate queue subject to reduction rules, if a hook corresponding to event 3 in rel B executes at a time before a hook corresponding to event 2 in rel A, and they cross communicate it is an issue. [13:33] fwereade: i'm with you, but i think for a slightly different reason. we *do* have potential consistency problems, but i think they should be solved as close to source as possible. so watchers should detect inconsistency if possible and act appropriately. by the time we get to the actual agent, all the information should be available in processed form. [13:33] hazmat: I don't know what you mean by "cross communicate".. hooks don't communicate [13:33] niemeyer, I'm just not 100% convinced that it's never going to break anything if we get settings newer than the relation list [13:33] niemeyer, cross communicate -> i mean hook in rel B modifies rel A setting [13:33] hazmat: I don't see any issue there [13:34] * fwereade cig/assimilate break, 3 mins [13:34] hazmat: Yeah, don't see any issue there [13:35] fwereade: Cool, let's talk when you're back [13:35] fwereade: FWIW, I find your concern great.. it's nice to have that kind of problem in mind [13:36] fwereade: Let's figure the way we *want* relation-list and hooks to interact [13:36] niemeyer, its only problematic for attempting to maintain a consistent view for each hook, because hook B modified settings in a separate rel using a different consistent view then the present time of the other rel. the linear view of time is subject to some jumps because of how we reduce the queue, it is consistent but only to a rel context, cross rel communication, requires a linear view that is scoped at a higher level to achieve consistency [13:37] niemeyer: trying to get my head clear around this. this is only a problem because we're not getting all the info from a watcher, right? so the action that an agent is taking depends not only on the data received from the watch, but also some data that it reads from zk later? [13:37] hazmat: Sorry, I still don't see the problem. [13:37] again it depends on what you looking for consistency, fwereade was looking for something akin to mvcc (ie. snapshot isolation) [13:37] niemeyer, heh, I think what I just thought is what hazmat just said [13:37] hazmat: "B modified settings in a separate relation".. [13:37] hazmat: That's sequential [13:38] hazmat: Hooks aren't executing in parallel [13:38] i don't think we can guarantee consistency across relations. [13:38] niemeyer, but they are subject to time skips due to reduction [13:38] hazmat: Not time skips, no [13:38] hazmat: They are subject to ignoring events and acting on the latest known state, yes [13:38] niemeyer, right their linear advancing, but yes the clock does skip event ticks [13:39] hazmat: I don't think that changes anything with regards to relations in different contexts, though [13:39] it works now because we feed all of them (hook exec queues) into a linear scheduler [13:39] yeah.. it might be a red herring [13:40] hazmat: A hook executing for relation A, B, C, or D, and doing changes for relation A, B, C or D, has the system as consistent as if the hook was doing changes on a single relation [13:41] niemeyer: i don't think it does [13:41] niemeyer, when relation B changes relation A's settings, it could plausibly affect how relation A acts... right? [13:41] niemeyer: because the settings for each relation are in a different zk node [13:42] fwereade: Yes, hopefully :) [13:42] niemeyer: and we can reorder zk watch firing across different nodes [13:42] niemeyer, and if relation A doesn't have access to the same view of history that B used to make that change to A, I think we could end up in confusing states [13:42] niemeyer, maybe it all shakes out, but I haven't convinced myself of that yet [13:42] i'd need to white board to work it out, i haven't worked through it with the new cross rel communication work in, its something that needs thought though. [13:43] rog: DOesn't matter in that case we're discussing. We don't order those events. [13:43] s/thought/verification [13:43] fwereade: I don't see how that's an issue [13:43] Quick break, biab [13:45] rog, how's the upgrade work coming? [13:46] hazmat: i finished that upgrader thingy. but it's judged premature for the time being, so i've put it on ice. currently working on parallel stuff towards the machine agent. [13:47] niemeyer, ok, let's say B has set some setting per related unit on A, based on "current" state at the time a hook ran for B; and A then runs a hook, based on a new and incompatible "current" state for A; this feels liable to lead to upset and confusion [13:47] rog, cool.. i got inspired to work on it as well [13:47] niemeyer, whereas if B and A are both working with the same history, the "current" states don't matter -- both B and A will agree on (say) what units A is currently related to [13:48] rog, although its turning out to be a lot more work than i had hoped. [13:49] hazmat: yeah, i found that. but it wasn't too bad. if you want to copy the protocol, the code is here https://codereview.appspot.com/6307061 [13:50] hazmat: i had a plan to automatically generate test permutations, but didn't get quite that far. [13:51] rog, i eschewed the separate process for minor upgrades [13:51] hazmat: ah, cool. [13:51] hazmat: where are you finding the main difficulties? [13:51] rog, i want to get to major upgrades though, which is where things get interesting.. else i'll never be able to merge any of the scaling work (json switch) [13:52] fwereade: That's a good direction for us to explore.. it'll be easier to get a point across with examples indeed [13:52] fwereade: I don't understand yours, though [13:52] fwereade: Let me try to translate [13:52] rog, no part in particular is difficult.. i ended up going with some new abstractions (state protocols), its just a lot of pieces. [13:52] fwereade: We have two units, A and B, is that what you mean? [13:52] niemeyer, two relations within the same unit [13:53] fwereade: Okay, so let's call two relations R1 and R2 [13:53] fwereade: and two units A and B [13:54] fwereade: Are R1 and R2 both between A and B? [13:54] niemeyer, let's say yes [13:54] fwereade: Okay [13:55] niemeyer, well, heh, R1 and R2 are each between the respective services of A and B [13:55] fwereade: So you mean that we have a hook R1-relation-changed executing within A? [13:55] niemeyer, yes; and R1-relation-changed sets something on A.R2's settings [13:55] fwereade: Okay, I'm with you [13:57] * fwereade thinks a mo, maybe it doesn't matter... just a sec [13:59] niemeyer, ok, thinking aloud... R1 sets a key in A.R2 per unit related to A, which is currently [B] [13:59] fwereade: Okay [14:00] niemeyer, B departs, and B' joins [14:01] fwereade: C joins, please :) [14:01] fwereade: Or you mean B joins again? [14:01] niemeyer, ok, C joins [14:01] niemeyer, now, I meant another unit of the service that B was a unit of [14:02] fwereade: Okay, let's call it C since the service relation doesn't really matter in this case.. it's a different unit in the relation [14:02] niemeyer, yep, ok [14:03] niemeyer, so, given the delays etc in play, we could well have a situation in which A.R1 still thinks it's related to B, but A.R2 thinks it is now related to C [14:03] fwereade: Oops, hold on [14:03] niemeyer, A.R2 thinks A.R1 is now related to C [14:03] A.R1 doesn't think anything.. this is a settings node [14:04] fwereade: Neither does A.R2 [14:04] niemeyer, yes: A.R1 is a node distinct from A.R2 though [14:04] fwereade: Yes, and neither of them "think" anything.. they are state [14:05] niemeyer, s/thinks/has written state predicated on the assumption that/ [14:05] fwereade: Yes, and that's not a problem.. [14:05] fwereade: R1-relation-joined will be called for C [14:06] fwereade: and A will have a change to do whatever it pleases with it [14:06] fwereade: If the person maintaining the charm for A is doing crazy stuff that makes no sense, though, we can't avoid it [14:07] niemeyer, I think I agree that we will eventually reach a steady state in which everything makes sense [14:07] fwereade: Yep, that was the main thinking [14:08] niemeyer, but it seems wrong to let C see A.R1's settings in a state wildly divergent from A.R2's settings [14:09] niemeyer, the eventual consistency may indeed trump all though [14:11] fwereade: You've lost me now [14:11] A.R1 and A.R2 were both set solely by A.. [14:11] fwereade: There's no way for them to be "divergent" [14:12] niemeyer, the "wildly divergent" is hyperbolic, but is meant to indicate that A.R1 could respond to 30 events before A.R2 responds to any [14:12] niemeyer, unless we're thinking a little about the cross-relation stuff alluded to above [14:13] niemeyer, I think that would take a pathological scheduling situation and it might be impossible but I can't comfortable state that I'm certain it is impossibvle [14:14] fwereade: We can improve the scheduling algorithm over time to make events across relations "fair", in case they don't naturally get fair [14:14] fwereade: We've lost the use of practical examples, though [14:14] niemeyer, well observed [14:14] niemeyer, I might have to try to turn this into an email [14:14] fwereade: R1-relation-changed can *really* have 30 events happening before R2-relation-changed happens [14:16] fwereade, niemeyer: is it this kind of scenario we're potentially concerned about? http://paste.ubuntu.com/1047364/ [14:17] rog: I'm not aware of any scenario we're concerned about, after the above conversation [14:17] rog, I don't follow that I'm afraid [14:17] rog: The password case I mentioned works today [14:18] niemeyer: yes, but it might not work if the username and password were stored in different relations, right? [14:18] rog: That's an artificial example that we're not concerned about [14:18] niemeyer: ok. i was trying to understand fwereade's concerns, sorry. [14:19] rog: No worries, I'm just explaining what I've been discussing with fwereade [14:19] niemeyer: i was trying to come up with a concrete example of [14:19] [15:08:36] niemeyer, but it seems wrong to let C see A.R1's settings in a state wildly divergent from A.R2's settings [14:20] niemeyer: of course, it *is* an artificial example [14:20] rog: See my comments that follow that :-) [14:20] niemeyer: and i'm happy for us to say that settings are only consistent within a relation. [14:21] rog: Cool, that's a given today, and I agree it doesn't feel useful to go over the trouble [14:22] fwereade: Okay, can we go back to that branch we were talking about when the conversation diverged? [14:23] niemeyer: i do think that it might make sense to make the agents use info read by the watchers though, rather than going into the zk state again. it would mean the watchers can detect inconsistencies early on and produce a consistent-looking stream of events which will lead to determinstic actions. [14:23] rog: The agents use info read by the watchers today. [14:24] niemeyer: if they do, that's no problem. i seem to remember that wasn't the case though. [14:25] rog: They do. Of course, there are a lot of details about this, but those details should be taken into account when saying "we should do foo". [14:25] rog: The whole conversation above was exploring those details [14:27] niemeyer: looking at this before, it seemed to me that more stuff needed to be cached in the state types for this to be true. but perhaps things have changed, or the agents don't need as much info as i thought. [14:27] rog: Sorry, but I really don't know what you're suggesting. [14:33] niemeyer: one example: when the machine agent is watching units for a machine, it sees the topology change and sends a *Unit down a channel. by the time the agent comes to ask the unit for its CharmURL, the unit might be gone. [14:35] niemeyer: on a completely different subject: how do you feel about the possibility of using build tags for conditional tests instead of flags? (i'm thinking of -amazon here, to start with) [14:35] rog: I've seen the thread.. sounds fine, but we shouldn't be worrying about this right now [14:36] niemeyer: i'm just about to add another flag, and it seems a bit wrong. and it's awkward to test everything with the flags. [14:36] niemeyer: it would be quite and easy to change [14:36] s/quite/quick/ [14:36] rog: What flag are you adding? [14:36] niemeyer: -root [14:36] rog: Hmm.. why do we need it? [14:37] niemeyer: because the only way of testing the container package with any usefulness is to run the tests as root [14:37] AFAICS [14:37] rog: I can't see why that would be the case right now [14:37] rog: We're not adding support for LXC today [14:38] niemeyer: not possible to add upstart scripts without being root [14:38] niemeyer: i could just leave out testing entirely [14:38] rog: You can add upstart scripts to a path that you define [14:38] niemeyer: putting yet another shim second-guessing what commands the upstart package will be invoking seems wrong [14:39] niemeyer: really? running upstart myself, then? [14:39] niemeyer: i hadn't tried that. [14:39] rog: Yes, that actually works [14:39] rog: Although I'd probably just verify that the script was put in place [14:40] rog: Well, depending on what else is being tested in that path [14:40] niemeyer: i quite like having a test that really verify that it actually works with upstart for real. [14:40] rog: Either way, definitely not require root for tests [14:40] niemeyer: indeed. [14:40] niemeyer: hence the flag. [14:40] rog: Those will be run close to never [14:40] niemeyer: same as the amazon tests :-) [14:41] rog: Not really.. I run those fairly frequently [14:41] rog: I don't run *any* test suite as root, though [14:41] niemeyer: really? when was the last time you did? last few times they've failed for me. [14:41] rog: We may need that for LXC, unfortunately [14:41] rog: For EC2/S3, every single time I change the tests [14:41] niemeyer: that's why i thought it was reasonable to do it now. [14:42] rog: We're not implementing LXC now [14:42] rog: Are you working on LXC? [14:42] niemeyer: nope [14:42] rog: So I'm lost [14:43] rog: Only need for LXC => That's why I'm doing it now => Doing LXC? => No => Vhat geev? [14:43] niemeyer: i thought that since the container package was going to need to have root tests eventually, and that the current tests are more effective running as root, that it would be reasonable to have root tests there now. [14:44] rog: Testing as root is bad practice, and you know that [14:45] niemeyer: i agree. but how do you suggest i provide a meaningful test of this code: https://codereview.appspot.com/6312044/diff/1/container/container.go [14:45] ? [14:45] niemeyer: second-guessing what the upstart package is going to do seems wrong to me [14:45] rog: Please have a look at how fwereade tested the upstart package [14:46] [15:45:50] niemeyer: second-guessing what the upstart package is going to do seems wrong to me [14:46] i can't test this without knowing exactly what commands upstart is going to call, where it's storing the files, what they look like, etc [14:46] rog: Sorry, but I don't know what that means.. I'm suggesting having a look at the technique used by the upstart package to test things [14:46] rog: The actions from these package look trivial [14:47] niemeyer: they are [14:47] rog: I hope we can find a trivial way to test them [14:47] niemeyer: i haven't succeeded yet! [14:47] rog: that doesn't involve running them as root [14:48] niemeyer: suggestions very welcome. [14:48] niemeyer: i've looked at the upstart tests BTW [14:49] niemeyer: anyway, gotta go to the dentist. back in a bit. [14:49] rog: Isn't this merely writing a file on disk right now? [14:49] rog: I hope we don't need root to test that! :-/ [14:50] fwereade: ping [14:50] niemeyer, pong [14:50] niemeyer: yes. but to verify that file, i've got to know what file is written and what it looks like. given that the upstart package itself doesn't have a single test that actually verifies that it really works with upstart, we should do that *somewhere* i think. [14:50] <niemeyer> fwereade: Okay, can we go back to that branch we were talking about when the conversation diverged? [14:51] rog: Okay.. if you can't figure how to test that file, please move on to something else and I'll figure it. [14:51] rog: It's a trivial text file [14:51] niemeyer, probably, if you tell me where you consider it to have diverged... [14:51] rog: That somewhere is the upstart package, by the way.. [14:52] fwereade: I just mean following up the conversation [14:53] fwereade: We were talking about the CL and then moved on to something else.. I'd just like to see what we do with it [14:53] niemeyer, ah, cool, the RelatedService one [14:53] fwereade: Yeah [14:54] niemeyer, well... not sure where to go really: have I allayed your fears somewhat? [14:54] fwereade: Yeah, I'm happy with the overall concept [14:54] fwereade: I'm not entirely happy with the type and method itself, though [14:54] niemeyer, ah, ok, go on [14:54] fwereade: The data in the RelatedService method feels very weird to me [14:55] fwereade: It alludes a bit to that conversation we had the other day [14:55] fwereade: About how we store data [14:56] fwereade: I'm looking at https://codereview.appspot.com/6307099/diff/1/state/service.go [14:56] niemeyer, I'm more than happy to stick "relation" on the front of everything [14:56] fwereade: Line 239 [14:56] fwereade: Not about naming.. about the actual data [14:56] niemeyer, ah, ok, the RelatedRole? [14:57] niemeyer, or the totality of bits of data that make up RelatedService? [14:58] fwereade: If s is service A, with a requirer relation R1 named "cache".. this is storing {A, R1, provider(!)} [14:58] fwereade: If s is service A, with a requirer relation R1 named "cache".. this is storing {A, R1, "cache", provider(!)} [14:58]