[06:33] morning [06:57] TheMue, heyhey [06:58] fwereade: heya [08:52] ooops, disconnected *hmmm* [09:44] moin. [09:55] Aram: moin [10:42] lunchtime [10:45] fwereade: fairly trivial: https://codereview.appspot.com/6576053 [10:46] rogpeppe, cheers [10:47] rogpeppe, LGTM [10:47] fwereade: ta [10:52] fwereade: and another, even more trivial: https://codereview.appspot.com/6566060 [10:53] rogpeppe, LGTM [10:53] fwereade: tyvm [10:56] fwereade: i think i'll submit both, given your remark [10:56] rogpeppe, +1 [11:33] have one too: https://codereview.appspot.com/6567060 [11:43] TheMue: i would look, but i can't seem to pull from lp currently [11:43] TheMue: rather, i was looking, but i wanted to look more closely but failed to talk to launchpad [11:43] TheMue: i can't even pull from trunk currently [11:43] rogpeppe: oh, which error message? [11:43] Aram, fwereade: are you having any problems accessing launchpad through bzr? [11:44] rogpeppe: i just pulled trunk and it worked [11:44] same [11:44] workes [11:44] TheMue: http://paste.ubuntu.com/1230255/ [11:44] works [11:45] rogpeppe: iiirks, never seen that [11:45] TheMue: yeah, i don't know what's going on - it worked fine earlier [11:46] i wonder whether my password has expired or something [11:50] i suspect it may be something to do with using a long branch name [11:50] lbox just managed to push fine. weird [12:00] TheMue: all working now. who knows what was going on... [12:00] rogpeppe: maybe service-side [12:01] lol, i was trying to find the definition for the "delete" function, totally forgetting it was built in! [12:02] :) [12:16] Aram: seem i found a bug in the machines watcher [12:17] Aram: typically a machine can be removed after EnsureDead() [12:18] TheMue: you've got a review [12:18] Aram: when testing the firewaller and remove a machine i get nor error returned but the machine watcher panics with "machine removed before being dead" [12:18] rogpeppe: just seen, cheers [12:19] TheMue: I know, the implementation was correct and niemeyer broke it. [12:19] rogpeppe: good hints, thx [12:19] * Aram is back in 15 minutes. [12:21] fwereade: another one (this one makes live tests pass, yay!): https://codereview.appspot.com/6566058/ [12:21] * fwereade looks [12:24] rogpeppe, LGTM :) [12:25] fwereade: thanks! [12:39] oh why can i never remember how to print a full revision id in bzr?! [12:39] nor is it easy to google for [12:59] fwereade: meh, how I wish we had actual container entities. [12:59] Aram, heh, me too, I remain totally convinced it's the Right Thing to do [13:00] the machine units watcher is ridiculously complex becase we don't, I have to watch 1) the machine, 2) n principal units, 3) all the units and take great care to integrate and use all this knowledge. [13:04] fwereade: the uniter tests seem to spend ages and ages doing: want resolved mode '\x00', got '\x02'; still waiting [13:04] fwereade: and similar [13:04] rogpeppe, hell, niemeyer had that yesterday; I was unable to repro :/ [13:04] fwereade: the uniter tests passed for me just now but took 124s to run [13:05] rogpeppe, they do indeed take a long time: some of it may be the wanton fsyncing but probably not all [13:06] rogpeppe, I'm working on a suggestion of niemeyer's at the moment which changes the uniter quite interestingly, though so I was kinda hoping that would induce magical improvement somewhere ;) [13:06] fwereade: i'll paste you a copy of the output of the test, with timestamps so you can see where the time is going [13:06] fwereade: lol [13:06] fwereade: for the machine units watcher, we only want to deliver events if there's a change in lifecycle, not a change in any other attribute, right? [13:07] fwereade: interesting, when i ran it that time, it failed with "never reached desired status" [13:07] rogpeppe, yeah, indeed, there is clearly something funky wrt Resolved -- that is one of the bits that is changing, actually [13:08] Aram, I think so, yes [13:08] Aram, tedious, innit [13:09] fwereade: http://paste.ubuntu.com/1230376/ [13:09] rogpeppe, cheers [13:10] fwereade: that ouput is actually from a branch other than trunk [13:10] fwereade: i'll try in trunk and see if the same thing happens - i might have broken something [13:10] there's a huge impedance mismatch between state/watcher and what we actually want. I don't find any purpose to state/watcher, we could have implemented watchers just the way we wanted withoout an aditional 1kLOC layer, and they would have been simpler. [13:11] Aram: how would you have implemented watchers? [13:11] rogpeppe: I actually did like 4 or 5 times. [13:11] let me find a branch. [13:11] rogpeppe, I wouldn't worry too much about it, I will be looking into it [13:12] Aram: i think the point of state/watcher is that it's efficient enough to cope with many thousands of documents [13:12] niemeyer: yo! [13:13] Good mornign! [13:14] morning [13:15] rogpeppe: Just sent some comments on the branch [13:16] rogpeppe: Thanks for fixing the tests [13:16] niemeyer: cheers [13:16] rogpeppe: The tools waiter looks like a nice idea, but it's sprawling too far IMO [13:16] rogpeppe: A tool waiter should wait for the tools, and nothing else. It's messing up the already messy situation even further. [13:18] niemeyer: ok. i thought it was quite neat to bundle up the watcher with the thing it's watching, but evidently not. [13:18] fwereade: ping [13:18] niemeyer, pong [13:18] rogpeppe: It's a tool waiter [13:18] rogpeppe: It's not nice to bundle things with arbitrary things so we can use the underlying things' interface [13:18] fwereade: yo [13:18] fwereade: Couple of questions [13:19] fwereade: Did you figure what was wrong with the uniter tests? I think I can still reproduce [13:19] niemeyer, listening (btw thank you for excellent advice re uniter, it seems to be coming along pretty nicely) [13:19] niemeyer, afraid not; rog was just hitting it today [13:19] fwereade: My pleasure, glad it was useful [13:19] fwereade: Okay, I'll dive in right away then [13:19] niemeyer: the reason it made things nicer is that otherwise we end up always passing around the object along with the watcher channel. [13:19] fwereade: Please keep going in whatever you were doing before.. :-) [13:19] niemeyer, my suspicion is that my ClearResolved() is funky, it looked off when I was changing it today [13:20] niemeyer: the waiter object made that more straightforward by bundling them together [13:20] rogpeppe: It's a tools waiter.. it's not reasonable to have a tool waiter sprawling the live tests.. makes no sense whatsoever to be saying ToolsWaiter.EnsureDead (!!) [13:21] niemeyer: i admit that was a bit of a short cut :-) [13:21] rogpeppe: Heh [13:21] rogpeppe: These tests should be cleaned up, reduced in size and scope.. that's the opposite of it [13:22] niemeyer: i'd like to have a chat about that [13:22] rogpeppe: Cool, let's just extinguish the fire first, and then I'm happy to discuss it [13:22] niemeyer: i'd like to do that, but i don't want the time taken to run the tests to double. [13:22] rogpeppe: They won't double.. we already have BootstrapOnce [13:23] niemeyer: tests like this one start a unit. [13:23] niemeyer: that takes almost as long as bootstrapping [13:23] niemeyer: so it makes sense to test some stuff with the unit once we've started it, i think. [13:24] rogpeppe: BootstrapOnce can create a default unit, with a well known testing charm [13:24] rogpeppe: Which follow the same rules of the first machine (do not mess with it, and if you do, put it back in place) [13:24] niemeyer: i think that's wrong, for the same reason it was wrong in the other tests we changed recently. [13:24] rogpeppe: Curiously, you won't be able to do that for upgrades, so I don't think it's too much to upgrade it [13:24] niemeyer: we don't want loads of arbitrary context in the suite [13:25] rogpeppe: Erm, to deploy a new unit [13:25] rogpeppe: Yep, so let's pay the price of time [13:25] rogpeppe: Pick your fight :) [13:25] niemeyer: live tests already take 10 minutes. i don't want them to take 20. [13:25] rogpeppe: I don't care [13:26] rogpeppe: I won't be sitting and looking at the screen in either case [13:26] rogpeppe: I do care, though, that we have incomprehensible and long-winded tests [13:26] rogpeppe: Which took a week to fix when we changed something fundamental [13:26] rogpeppe: *one* test [13:28] niemeyer: the difficulty i'm having is that all the tests i'd factor out are subsets of the current Bootstrap and Deploy test. that is, to test upgrading, i think we need to do almost exactly what we already do in that test. [13:29] rogpeppe: That's why we have a BootstrapOnce test [13:29] rogpeppe: s/test/helper/ [13:29] rogpeppe: Because it's fundamental to live tests [13:29] rogpeppe: Everything else is not [13:30] rogpeppe: Of course you need bootstrap to test deploy, of course we need bootstrap to test upgrade, of course etc etc [13:30] niemeyer: how can i test upgrade without doing a deploy? [13:30] rogpeppe: But that's certainly not reasoning to put *all of those tests* in the same place [13:30] rogpeppe: Otherwise all we know is "Huh.. the big blob of code is broken again!" [13:31] rogpeppe: You can do a deploy without testing upgrade [13:32] niemeyer: sure. then we have a set of tests, each of which includes the other as a prefix. i'm with you, but i don't want our live tests to take 2 hours [13:32] rogpeppe: What upgrade is a prefix of? [13:32] niemeyer: deploy is a prefix of upgrade. maybe upgrade is as far as it gets, yes. [13:32] rogpeppe: bingo [13:33] rogpeppe: Bootstrap and deploying things are fundamental.. everything else is not [13:33] niemeyer: so... does it make sense to have deploy as a separate test, when upgrade will test *exactly* the same path? [13:33] rogpeppe: Does it make sense to know that deploy works, despite the fact that upgrade doesn't? [13:34] niemeyer: sure. but our test failure will tell us that by where it failed. i don't think it justifies an extra three minutes testing time. [13:35] rogpeppe: So let's erase all the tests.. 0 minutes [13:35] niemeyer: now yer being silly :-) [13:35] rogpeppe: Where's the value? [13:35] rogpeppe: I am [13:35] rogpeppe: But that's the point.. you're worried about timing and ignoring everything else [13:35] rogpeppe: THere are other ways to optimize time [13:35] rogpeppe: Running tests in parallel for instance [13:36] rogpeppe: But if you take away the value of the test, there's little reason to have them [13:36] rogpeppe: All we have is a black/white works/doesn't work [13:36] niemeyer: does that mean table-driven tests are of no value because they're all bundled into a single test? [13:36] rogpeppe: Can't see that leap [13:37] rogpeppe: "table tests" are generally very contained in scope [13:38] rogpeppe: Bootstrapping, deploying, upgrading, deploying more units, remove units, test firewaller, open ports, close ports, check expose, blah blah blah [13:39] niemeyer: ok, i'll change it and see how it looks [13:40] rogpeppe: hold on, this is all future work.. let's please make things stable first [13:40] niemeyer: ok. so what do you want me to do about the current branch? [13:40] rogpeppe: Look at the review, and move forward? [13:41] niemeyer: ok. i thought you weren't keen on most of what i was doing there. [13:41] rogpeppe: I've sent a review that details exactly my feeling about what's in that branch [13:42] rogpeppe: And I've started the conversation saying that I appreciate the refactoring you did to wait for tools [13:42] rogpeppe: And I also said I didn't want to get into this discussion before the fire was extinguished [13:42] niemeyer: fair enough [13:44] niemeyer: how about if toolsWaiter was named differently? i think the bundling together of object and channel works well - it made the code simpler. [13:45] rogpeppe: :-( [13:46] rogpeppe: Can we please make the tools waiter *wait for tools*!? [13:46] niemeyer: ok, i think i'm there [13:46] rogpeppe: Thanks [14:03] fwereade: Curiously, the tests are passing now :-( [14:03] * niemeyer looks at the paste [14:04] niemeyer, yeah, I did see it once myself -- have a small amount of patience, because your suggestions led to a certain amount of rearrangement that happened to somewhat affect error resolution [14:04] niemeyer, but I'm not 100% there yet [14:04] fwereade: Cool [14:08] niemeyer: i think this might be better: https://codereview.appspot.com/6566058 [14:09] * rogpeppe goes for a bite of lunch [14:15] rogpeppe: Thanks, LGTM. Just sent a couple of questions for your consideration [14:18] niemeyer: the advantage of having watcher as an embedded field is just so we don't need to define another Stop method. but i guess it's probably best to be explicit and do that anyway. [14:19] rogpeppe: Okay, I personally don't mind much either way in that case.. whatever you're happy with [14:19] niemeyer: we don't need a refresh because waitAgentTools has already done a Refresh, but again, maybe it's best to do it so it's obvious [14:20] niemeyer: i'm happy to do that if you prefer [14:21] rogpeppe: Yeah, sounds slightly more resilient.. we could change the tools waiter without breaking the test [14:21] niemeyer: ok, will do. thanks for the review BTW [14:21] rogpeppe: My pleasure, and sorry for the confusion [14:21] niemeyer: np [14:25] I think I was the go fmt offender.. [14:26] I stumble upon it every other commit [14:29] niemeyer: i do it quite a bit too :-) [14:30] fwereade: You might appreciate this one: https://codereview.appspot.com/6565057 [14:30] fwereade: Figure it while having a look at uniter tests runs.. :) [14:30] rogpeppe: ;) [14:30] niemeyer, yay! [14:31] niemeyer, LGTM [14:31] niemeyer, that was on my copious-free-time list too :) [14:32] fwereade: The uniter test that broke in the paste was relative to resolution [14:32] fwereade: I'm wondering if it's just a sync race [14:32] fwereade: I'll see if I can reproduce this afternoon [14:32] niemeyer: ping [14:33] TheMue: Yo [14:33] niemeyer, hmm, I feel like that bit *should* be pretty clean, but I'm not certain [14:33] niemeyer: the MachinesWatcher seems to have a problem [14:33] TheMue: Oh? [14:33] niemeyer: while the test of RemoveMachine() in state works I now tried it for the firewaller [14:34] niemeyer: and there I get no error in EnsureDead() and RemoveMachine() but a panic in the watcher telling me that the machine is not dead [14:35] TheMue: Ah, interesting.. I'd like to have a closer look at that, as I'm about to refactor that watcher to use a slice of changes too [14:35] TheMue: Do you have a reproducer? [14:36] niemeyer: currently in an own little branch of the firewaller test. but i'll set up an extra branch for you [14:36] TheMue: ? [14:37] TheMue: Thanks, please push something, or just put the test in a paste and send a link [14:37] TheMue: I'll have a look after lunch [14:37] niemeyer: ok, will do [14:39] I'll step out for an earlier lunch today to join Ale.. see you all soon [14:39] niemeyer, fwereade: i still get sporadic failures of worker/uniter [14:40] niemeyer: but that's the only failure [14:40] niemeyer: enjoy! [14:40] rogpeppe: It's known.. I'll try to reproduce in the afternoon, and fwereade is also refactoring things somewhat [14:40] niemeyer: a slightly reddish shade of green then :-) [14:41] rogpeppe: The message is the same: tests should pass before you merge. If something broke, don't commit [14:41] * niemeyer steps out [15:42] uniter upgrade tests pass, yay! [15:42] (live) [15:57] * fwereade cheers ar rogpeppe [15:58] well holy shit [15:59] after that *brutal* refactoring, the uniter tests pass in about half the time they did before [15:59] fwereade: trivial: https://codereview.appspot.com/6566063 [15:59] * rogpeppe cheers at fwereade [16:00] rogpeppe, (and worked second time after I got it to build, too, the only bug (detected thus far...) was errorContextf-related) [16:00] fwereade: nice one [16:00] rogpeppe, LGTM [16:01] rogpeppe, bbs [16:01] fwereade: so how long do the uniter tests take now? [16:01] k [16:01] * niemeyer waves [16:02] rogpeppe, bit more than a minute [16:02] TheMue: ping [16:02] niemeyer, that suggestion was *awesome* [16:02] fwereade: Woohay! [16:02] niemeyer: pong [16:02] niemeyer, I think I remained true to its spirit, just got it building, and OMG at least twice as fast [16:03] niemeyer, but I have to go... you'll get a CL tonight for sure, unless I discover some subtle screwup [16:03] niemeyer: it is not reproducable standalone, only in the firewaller [16:03] niemeyer: shall i push that branch so that you can test it there? [16:04] TheMue: Yeah, or even just a tiny test for the firewaller that exposes the bug [16:04] niemeyer: looked into the watcher. the revno = -1 seems to come to early, because the machine is still in w.alive [16:04] TheMue: I've sent a review too [16:05] niemeyer: cheers [16:05] TheMue: Yeah, I want to look at that.. I've put the panic mainly so I'd see those cases [16:05] niemeyer: I paste the test, one moment please [16:05] TheMue: Thanks! [16:05] fwereade: Superb, that sounds very exciting! [16:06] niemeyer: uniter upgrade, finally: https://codereview.appspot.com/6561063 [16:06] rogpeppe: Looking [16:08] niemeyer: http://paste.ubuntu.com/1230645/ , the panic is in merge() of the MachinesWatcher [16:08] TheMue: Thanks [16:09] TheMue: please let me know if you have any questions on the review sent [16:13] niemeyer: the fixed bug had no failing effect, I only discoverd that my removing of the service not worked. and then i've seen that the map contained the unit with two ids: the service name and the unit name. but we never looked for the service name, so no error. but my length test just didn't became 0. [16:13] niemeyer: the other comments a pretty clear, thanks [16:14] TheMue: That's how you discovered the bug [16:14] TheMue: But does it really have "no failing effect"? Why do we use this map at all then? [16:15] niemeyer: yes, but before it showed no effect [16:15] TheMue: This sounds like a missing test? [16:15] TheMue: Why is the map used? [16:15] niemeyer: the map only contains one entry with the service name and the last added unitd [16:16] niemeyer: now it is used for me to stop the serviced, but i have to look why we have added it once [16:16] TheMue: Sorry, that doesn't sound right [16:16] TheMue: Feels very hand-wavy [16:16] TheMue: Why have you added that map in the first place? [16:16] TheMue: Where is it used? [16:17] niemeyer: that's what i wanted to say. i right now don't remember and have to look for the reason [16:17] TheMue: Bingo. [16:18] niemeyer: now it is used for me to stop the serviced, but i have to look why we have added it once <= last sentence, i have to look [16:19] TheMue: niemeyer: the map only contains one entry with the service name and the last added unitd [16:19] TheMue: That's what I was talking about [16:19] niemeyer: if you don't want me to describe what i've seen just let me know [16:19] TheMue: Heh [16:21] TheMue: I don't need you to tell me that you've seen a map with a service name and a unit [16:21] TheMue: unitd.serviced.unitds[unit.ServiceName()] = unitd [16:21] TheMue: I can read the code [16:22] niemeyer: ok, so i don't describe stuff like that in future [16:22] TheMue: I also don't need you to tell me that tests were passing before.. as that's obvious. [16:22] TheMue: What I do need your help with, is making sure we have a test covering the fix [16:22] TheMue: I do need you for that, a lot. [16:23] niemeyer: I'll do my best. [16:23] TheMue: And that's the only thing I've suggested in the review [16:26] TheMue: Do we have any test covering two services? [16:28] niemeyer: The map has been used after an exosed change. So the one (last added) unit has been passed twice to flushUnits() for opening/closing of ports [16:28] niemeyer: so far the map has been used no where else [16:28] TheMue: Have you seen the line right below the one you changed/fixed? [16:29] niemeyer: yes [16:30] TheMue: and? [16:30] niemeyer: here a changed unit is added directly to the slice, so only once [16:30] TheMue: Slice? [16:31] niemeyer: but after the exposed we iterate of the serviced.unitds and the one unit is twice in the map and added to the slice [16:31] niemeyer: yes, the slice that is passed to flushUnits() [16:31] TheMue: I mean line 130 and 131 [16:32] TheMue: The line you fixed, and the line right below it [16:33] niemeyer: ah, ic what you mean [16:33] niemeyer: the referenced serviced is the same [16:34] TheMue: Yes, we have the unit twice in the same map with different keys [16:34] niemeyer: so one line isn't needed [16:34] TheMue: Right, the bottom one [16:34] TheMue: Do we have any tests with multiple services? [16:35] niemeyer: have to look, afaik not yet [16:35] TheMue: Can we have one in a follow up branch? [16:35] niemeyer: will add it, sure [16:35] TheMue: Happy to have this in just with the review comments sorted, plus the duplicated line removed [16:35] TheMue: Thank you [16:36] niemeyer: yw [16:42] niemeyer: just a heads up: i'm rolling the provisioner into the machine agent, and i don't particularly see any reason to treat the firewaller as inherently bundled with the provisioner, so i'm creating two bools on the Machine - Provisioner and Firewaller. does that seem reasonable to you? [16:42] niemeyer: i think it makes the implication of the flags more clear, as well as being more flexible. [16:45] rogpeppe: Sounds like it could be a list.. machine.AgentWorkers() []WorkerKind [16:45] niemeyer: yeah, i was wondering about that kind of thing too [16:45] niemeyer: or a map[WorkerKind]bool [16:46] rogpeppe: Doesn't sound right.. [16:47] niemeyer: we can have more than one instance of a given worker? [16:47] niemeyer: well, i guess so [16:47] rogpeppe: Probably not, but that's unrelate [16:47] d [16:48] niemeyer: oh - that was the reason i suggested it. [16:48] rogpeppe: Yep.. that's unrelated [16:48] niemeyer: so why don't you think it's good like that? [16:48] rogpeppe: Lists without duplications are a fine concept [16:49] niemeyer: ok. where would you put the check? or would we just trust? [16:49] rogpeppe: Where would you put the check? [16:50] niemeyer: probably in the agent itself. but it depends whether we want to allow multiple instances of a given worker in principle [16:50] niemeyer: BTW would the agent worker kinds include the machiner too? [16:51] rogpeppe: Good question.. I think it should, for consistency [16:51] niemeyer: it kinda doesn't seem right to leave it out, but then again i can't think of a situation where we don't want to run it [16:52] rogpeppe: Let's have it in.. it reflects reality and reduces implicit assumptions [16:52] rogpeppe: machine.SetAgentWorkers([]WorkerKind{MachineWorker, ProvisionerWorker, FirewallerWorker}) [16:52] niemeyer: ok, so if the machine agent starts and no workers are set, it... does what? [16:52] rogpeppe: It explodes and laughs [16:53] niemeyer: sounds good [16:53] niemeyer: to relive some typing, i think perhaps SetAgentWorkers(... state.WorkerKind) [16:53] rogpeppe: +1 [16:53] rogpeppe: SetWorkers sounds good too [16:53] relieve [16:53] niemeyer: +1 [16:54] niemeyer: and type AgentWorker string ? [16:54] rogpeppe: Uh oh.. we have a problem [16:54] rogpeppe: A race, more specifically [16:54] rogpeppe: type sounds good [16:55] rogpeppe: We have to create the machine with its workers set [16:55] niemeyer: ah yes [16:56] niemeyer: we could put the workers into AddMachine [16:56] rogpeppe: +1 [16:56] niemeyer: cool. then i won't provide a SetWorkers method, so there's no pretence [16:56] rogpeppe: Should fail if MachineWorker isn't set, or if there are duplicates [16:57] niemeyer: sounds good [16:57] niemeyer: although we could just add the MachineWorker implicitly [16:58] niemeyer: and return it from machine.Workers [16:58] rogpeppe: Let's please make it explicit. Very soon it may well make sense to have a machine that has no units. That's a trivial step from now. [16:58] niemeyer: ok, that seems good [16:59] niemeyer: actually we could make it not return an error, but do as requested (assuming at least one worker). we'd just need to tweak AssignToUnusedMachine to choose a machine with a MachineWorker [17:00] niemeyer: still i'll start by returning an error if no MachineWorker [17:08] rogpeppe: One thing at a time, please [17:08] OK: 124 passed, 1 skipped [17:08] how can one skip a test? [17:09] is this an old or new feature? [17:09] Aram: Hah, I think that's actually a bug [17:09] Aram: I mean, in the sense that we want the test running [17:09] Aram: c.Skip(reason) [17:09] I see [17:09] c.Skip("Marshalling of agent tools is currently broken") [17:09] I'm probably on an older branch [17:09] Aram: you are, and they aren't [17:10] I'll merge trunk at some point [17:12] Aram: https://codereview.appspot.com/6576056 [17:13] niemeyer: i thought i'd already done that... [17:14] niemeyer: probably just in some other abandoned branch [17:14] rogpeppe: Maybe.. one of the reasons why small/in-focus branches rokc [17:14] niemeyer: point taken :-) [17:14] niemeyer: CL is in again [17:15] niemeyer: I have to leave, familiy needs me, but I'll look into mail via phone later [17:15] niemeyer: was just talking on a call and we were discussing the possibility of adding some things into metadata.yaml... [17:16] niemeyer: How would you feel about a field at the same level as 'interface' that is 'recommends' and lists charms that are the recommended other side of that relationship? [17:16] niemeyer: and on a related note, how about a text field called 'description' for the relation itself, that is displayed whenever there is confusion on which relation to establish? [17:19] SpamapS: Unless we define behavior for such a field, I'm -1 on it [17:20] SpamapS: This sounds like getting into stacks, which I very much hope we start diving into soon [17:20] SpamapS: On the description one, sounds more interesting [17:22] niemeyer: The thought was to help people build stacks. The only behavior for it is to make use in the charm store and such. [17:22] niemeyer: its for the case where something is optional, like memcached, but there may be several implementations that provide: memcache protocol. [17:23] niemeyer: prefer may be a better term than recommend [17:24] SpamapS: Understood.. I'm still -1 on adding such a field before we define how stacks work and what behavior we want from juju itself when it looks at it [17:25] niemeyer: ok, mind if I open up a bug in juju-core? [17:26] SpamapS: Not at all, thanks for that.. If you don't mind, please try to phrase the bug in terms of the problem you're trying to solve, rather than the solution [17:26] hmm, seems like i've used up all my flocks: [17:26] bzr: ERROR: Could not acquire lock "/home/rog/src/go/src/launchpad.net/juju-core/.bzr/checkout/dirstate": [Errno 11] Resource temporarily unavailable [17:27] niemeyer: indeed [17:34] I wish Rietveld's copy & pasting worked as nicely as pastebin.ubuntu.com [17:41] niemeyer, https://codereview.appspot.com/6568060/ is still WIP but I think it hews closer to your vision [17:42] niemeyer, the tests worked first time I got it building (ok, ok, second time) and the tests runs about twice as fast as they did before :) [17:43] fwereade: Oh, nice! [17:43] fwereade: Why do we need the wantFoo stuff? [17:44] Oh, maybe I misunderstand [17:44] * niemeyer reads more [17:45] Yeah, I think I did.. nice [17:51] fwereade: if you have a moment, you might want to have a glance at https://codereview.appspot.com/6561063 [17:54] i'm now off for the evening [17:54] night all [17:58] niemeyer: how do I turn off debug output from state/watcher? [18:21] fwereade: Sent comments.. great stuff [18:22] niemeyer, sweet :D [18:22] niemeyer, I'll have time to look at them properly later [18:26] fwereade: Oh, I think there are a few races in the filter logic, but they're fixable.. I'll send a note [19:03] Aram: Just started hacking on the MachinesWatcher [19:04] Aram: Curious to see the effect of the model we discussed yesterday [19:04] niemeyer: cool, so how can I disable debug output from state/watcher wile still keeping it from state? [19:06] Aram: Have you had a look at the package? [19:18] fwereade, Aram: I wonder about the right moment to show Dead things [19:20] If we show entities that first show up as Dead after the initial event, we'll have to cache them to avoid showing them repeatedly [19:20] Hmm.. or perhaps not, actually [19:21] We can clean up when we find the entity removed [19:21] That sounds sane... nevermind [19:23] fwereade, Aram: http://paste.ubuntu.com/1231019/ [19:23] Sane? [19:24] whenever one or more machines is added or changes its lifecycle. [19:24] Aram: Sounds good [19:25] niemeyer: Machine.Workers: https://codereview.appspot.com/6564063 [19:25] rogpeppe: Looking [19:27] niemeyer: thanks. i really am gone now :-) [19:34] rogpeppe: have a good evening! [19:35] rogpeppe: And a laugh, in case you're still around: [19:35] 128 » ProvisionerWorker WorkerKind = "firewaller" [19:35] 129 » FirewallerWorker WorkerKind = "provisioner" [19:52] machine units test pass [19:52] hell yeah [19:52] now to add more tests to test for more stuff [19:55] Aram: Woot! [20:05] // test -gocheck.f TestMachineWaitAgentAlive [20:05] comment in the source [20:05] I did NOT do that :). [20:05] that was roger [20:15] proposed [20:15] * Aram has dinner [20:15] bye [21:41] niemeyer, ping [21:41] fwereade: yo [21:42] niemeyer, heyhey [21:42] niemeyer, clarification re: MachinesWatcher comment you pasted above [21:43] fwereade: Cool [21:43] niemeyer, but I'm having trouble writing what I want clearly :/ [21:44] fwereade: It changed slightly after Aram's suggestion [21:45] niemeyer, ok, is it possible that the watcher will ever miss a Dead, and only observe the removal? and; if that is possible, and it occurs, will it send a Dead anyway? [21:46] fwereade: It is possible, and it does report anyway [21:46] niemeyer, excellent, such was my supposition; I think there might be a nicer way to express that aspect of it [21:46] niemeyer, or maybe it's not worth it :) [21:47] fwereade: I think the documentation actually guarantees it [21:47] niemeyer, it may just be my faulty reading then [21:47] fwereade: You know too much :-D [21:47] niemeyer, what I really want to talk about is the uniter change I suggested earlier [21:48] niemeyer, I agree that resolved() is an ugly little knot of code [21:48] fwereade: 'k [21:48] niemeyer, and I also have some sympathy for the notion that ModeHookError is really 2 modes, one Alive and one Dying, although I am somewhat resistant here [21:49] niemeyer, but the tension between the two is what really bothers me [21:49] fwereade: I'm not sure if it was clear from my comment, but I wasn't suggesting getting rid of the current mode [21:49] niemeyer, in that unpacking the resolved method leads to duplication of slightly fiddly logic in 2 places [21:50] fwereade: I was suggesting that it sounds fine to have it hand off parts of its work to two independent modes [21:50] s/parts/part [21:50] niemeyer, or 4 if ModeHookErr and ModeConflicted are both split [21:50] niemeyer, and in either case, the resolved-handling and the upgrade-handling are duplicated [21:51] fwereade: I'm hoping we can remove the fiddleness enough for it to not be an issue. [21:51] niemeyer, I don;t think the <-u.charmUpgrades(false) suggestion will fly -- getUpgrade only works right when it's run on the main goroutine [21:51] fwereade: Upgrade handling, if the suggestions are made are possible, is simply return ModeUpgrade(foo) [21:52] fwereade: Why? [21:52] fwereade: What I see is actually a race caused by the fact it runs elsewhere [21:53] niemeyer, filter goroutine reads uniter state: ok, cool, we're not upgrading | main goroutine starts an upgrade | filter goroutine reads the "current" state from the charm dir, instead of using the upgrade state [21:54] niemeyer, it's rather deliberate that the filter doesn't have access to the uniter itself, anyway [21:54] fwereade: Hmm [21:55] fwereade: Yeah, the fact it doesn't access the uniter sounds good [21:56] fwereade: You see the race, though? [21:56] niemeyer, ah, sorry, just saw the new review, need to read that [21:58] niemeyer, yeah, you're absolutely right, and I would be more than happy to send both ResolvedModes and CharmThingys down the channels [22:00] fwereade: Cool, we're in sync then.. I also see why you want getUpgrade in the uniter [22:00] fwereade: It'll just loose some of its inner workings since we'll get that info from the channel [22:00] lose [22:01] niemeyer, yeah -- and I suspect resolve will look a little different in this light, too [22:01] niemeyer, might give me a way out of the ickiness [22:01] fwereade: Sweet [22:02] niemeyer, thinking about the Alive/Dying (sub)Modes a bit more [22:04] niemeyer, switching channels on and off in response to events on other channels is IMO not all that unpleasant a technique; it should surely not be used arbitrarily, but in each of those cases we have a situation in which... [22:05] niemeyer, the stimulus/response set in Dying is a strict subset of that in Alive; and the only valid transition is from Alive to Dying [22:05] niemeyer, it seems a shame to duplicate the identical bits any more than they have to be [22:05] niemeyer, but it will look different once I fix filter [22:09] fwereade: The amount of logic you have added to enable the shifting is worth about exactly the same of the logic that you actually need in the dying case by itself. [22:09] fwereade: One of them is entirely straightforward.. the other is not [22:09] fwereade: I suggest, once you're done with everything else, at least trying to see how it looks [22:09] fwereade: If you still feel the same way, I won't argue much :) [22:10] niemeyer, I'll see how it goes :) [22:10] fwereade: and by worth I mean the number of lines [22:10] niemeyer, I'm really not sure about that... we lose the upgrade stuff, but the resolved stuff is still needed when dying [22:10] niemeyer, in both cases [22:11] niemeyer, it all hinges on how terse I can make the resolved handling ;) [22:18] fwereade: Right, the resolved stuff is the only duplication in both cases [22:18] fwereade: You only need tombDying next to it [22:18] fwereade: In one of them [22:19] niemeyer, hmm, I think I should probably always have tombDying :) [22:19] fwereade: or default.. [22:19] niemeyer, in that one specific case in the main uniter loop, ok, yeah :) [22:20] niemeyer, actuall that has tombDying *and* default [22:20] niemeyer, I must be missing something [22:20] fwereade: Yeah, that's slightly surprising [22:20] niemeyer, when in a Mode is a select default useful? [22:21] niemeyer, (that bit is actually totally unnecessary -- it really just allows us to indirectly check for unit-Dead in between modes [22:21] ) [22:22] fwereade: When in a mode is an if statement useful? :-) [22:22] niemeyer, d'oh, ok, yes, I use them [22:23] niemeyer, but in a for { select loop, surely all they will cause is spinning? [22:24] fwereade: Ah, definitely, that'd be wrong [23:13] davechen1y: Morning! [23:15] morning [23:15] niemeyer: thank you for your review, just responding to your comments now [23:15] davechen1y: Cheers [23:16] wrt. splitting hostFromTarget into two methods, one for machines, the other for units [23:16] i'd still like to have a single method that dispatches to both of those [23:16] so I can reuse it in scp [23:16] what do you think ? [23:18] davechen1y: Sounds like a good idea [23:20] niemeyer: also, Unit.PublicAddress only works when called from the unit itself [23:20] :( [23:21] davechen1y: Isn't that EnvironProvider.PublicAddress? [23:21] niemeyer: let me check before making mroe incorrect statements [23:22] sorry, you are correct [23:23] right ... so that comes out of the document, which means the UA sets that field, i'm guessing on Alive [23:25] davechen1y: Yeah, it should set it at some point === davechen1y is now known as davecheney [23:38] niemeyer: what are your thoghts on changing environs.Environ.Instances([]string) to be ...string ? [23:42] davecheney: I personally don't mind too much in either direction, but I wonder if environ.Instance(instId string) might be more useful [23:42] niemeyer: in the majority of cases where i've used that method, i only have a single id I want to resolve [23:43] so wrapping it in a []string is overkill [23:43] making the param varadic means we don't have to have two methods on the interface [23:43] davecheney: It also means we'll call InstanceS when we want one, and have to unwrap a slice [23:43] davecheney: WE also are using a real slice less than we should [23:44] davecheney: The provisioner, for example, has a TODO for avoiding the overkill of per instance query [23:44] davecheney: Slightly off topic, though, I know [23:44] the providers all expect a slice [23:44] as in the method takes a []string now [23:44] so changing it to be variadic would have no impact on them [23:45] davecheney: I mean that in "foo, err := environ.Instances(whatever)", foo is a slice too [23:45] yeah, that also doth suck [23:45] just as common is foo, err := ... ; foo[0].something() [23:45] davecheney: So if we're missing a handier "give me one", I'd suggest a real one [23:45] niemeyer: fair call [23:45] i will propose something [23:46] davecheney: Cheers [23:46] davecheney: I'll step out to hack a home appliance here.. wish me luck :) [23:47] roger [23:47] send more paramedics