| davecheney | wallyworld: is there a page on the wiki describing the days the company is closed this year ? | 00:52 |
|---|---|---|
| wallyworld | davecheney: not sure, i know there was something about the 28th. let me see if i have anything in my email | 00:53 |
| wallyworld | davecheney: this is all i know - it says company closed between xmas and new year https://wiki.canonical.com/PeopleAndCulture/Policies/Leave | 00:55 |
| davecheney | i love how out of date this information is | 00:55 |
| wallyworld | but there's a separate page somewhere with reference to the 28th being a free day | 00:55 |
| davecheney | it makes it really useful | 00:55 |
| wallyworld | yeah | 00:55 |
| davecheney | there is an up to date pdf on the page | 00:56 |
| davecheney | thanks | 00:56 |
| wallyworld | davecheney: i think that pdf is the one with the free day etc | 00:58 |
| TheMue | Morning | 06:48 |
| === TheMue_ is now known as TheMue | ||
| fwereade | TheMue, rogpeppe: heyhey | 07:19 |
| TheMue | fwereade: Hi | 07:19 |
| TheMue | fwereade: The improved firewaller is in at https://codereview.appspot.com/6875053/ | 07:21 |
| fwereade | TheMue, cool, looking | 07:21 |
| TheMue | fwereade: Cheers | 07:21 |
| fwereade | TheMue, what happens when a machine has a unit removed after initGlobalPorts and createMachineData? | 07:42 |
| fwereade | s/and/and before/ | 07:42 |
| TheMue | fwereade: This is catched by the error check for an unassigned machine. | 07:43 |
| fwereade | TheMue, and what happens in response? | 07:44 |
| fwereade | TheMue, I don't get how you can distinguish between "this unit is not assigned to the machine I'm looking at: keep it around until we find the machine that does" | 07:45 |
| TheMue | fwereade: Today the unit is skipped, but you're right, it should be dropped from those temporary stored units. | 07:45 |
| fwereade | TheMue, but then the first machine you process will trash all the others, won't it? | 07:46 |
| TheMue | fwereade: Why? | 07:46 |
| fwereade | TheMue, because it will go through all the valid data saying "nope, not mine, trash it" -- won't it? | 07:47 |
| TheMue | fwereade: Each machine data, when created, is scanning the remaining units from the initial scan and checks, if that unit is assigned to the machine the data is representing. | 07:47 |
| TheMue | fwereade: No, I only should trash those without any assigned machine id. Today I just skip it. | 07:47 |
| fwereade | TheMue, yes | 07:47 |
| fwereade | TheMue, and you *have* to | 07:48 |
| TheMue | fwereade: Line 228. | 07:48 |
| fwereade | TheMue, because that unit might be assigned to another machine | 07:48 |
| TheMue | fwereade: So far it's by accident as good as it is. ;) | 07:48 |
| fwereade | TheMue, ok, what I'm afraid I'm saying is that you have I think rearranged the problems but not fixed them | 07:49 |
| TheMue | fwereade: I had this when during testing units have been created while init runs in background and those units are not yet assigned. | 07:49 |
| fwereade | TheMue, the fundamental problem is that you cannot effectively take a delta between the initial-scan state and the state at the point where you've started all the watches you need | 07:50 |
| TheMue | fwereade: machineData and unitData are proper initialized before returning to the loop. I had masses of runs with debugging statements. They shown that, depending on who is faster, init, adding or mixed, the state afterwards is always correct. | 07:51 |
| TheMue | fwereade: Yes, you never can, because scanning it initially takes time, especially when state is large, and during this time state can be changed. | 07:52 |
| TheMue | fwereade: So this CL fetches its initial state as good as it can and at the time machineData and unitData are created it looks if those initial units match to them. | 07:54 |
| fwereade | TheMue, so how are you in a position to assume that the initial-changes state that you build up corresponds to the globalUnitPorts state you built up before? | 07:54 |
| fwereade | TheMue, ISTM that a removed unit will stay in globalUnitPorts forever | 07:55 |
| TheMue | fwereade: Yes, that's the only problem I still have. I have no kind of safe signal that says "Now you can clean it up, guy." | 07:56 |
| fwereade | TheMue, yeah | 07:56 |
| fwereade | TheMue, and this is an inescapable consequence of the decision to separate the initial scan from the creation of the watchers, isn't it? | 07:57 |
| TheMue | fwereade: One way could be a kind of timer checking those entries from time to time if they are removed. | 07:57 |
| fwereade | TheMue, it's just another layer of paper over the problem | 07:57 |
| TheMue | fwereade: Sure. | 07:57 |
| TheMue | fwereade: We need the initial scan to look if the current open ports are those we need when the Firewaller starts. | 07:58 |
| TheMue | fwereade: And the initial scan needs time. | 07:58 |
| TheMue | fwereade: And during this time the state can (and will) change. | 07:59 |
| fwereade | TheMue, understood -- but why don't you also create the *Datas at that point? | 07:59 |
| fwereade | TheMue, they hold the watchers which have properties that allow us to avoid this problem | 07:59 |
| TheMue | fwereade: Thought about it, niemeyer disliked it. | 07:59 |
| TheMue | fwereade: But here it's the same problem. | 07:59 |
| TheMue | fwereade: Starting those watchers still produces a gap while scanning. | 08:00 |
| fwereade | TheMue, the initial events from those watchers *give* you the initial state, don't they? | 08:04 |
| TheMue | fwereade: At least if I start the right ones which may possibly not happen because the state changes while I'm scanning to know which watchers I have to start. | 08:05 |
| fwereade | TheMue, er, don't you just need to start the machines watch and handle the initial event? | 08:06 |
| TheMue | fwereade: That's what the firewaller has done in his first "release", just that. | 08:07 |
| TheMue | fwereade: :/ | 08:07 |
| fwereade | TheMue, I've never seen the FW setting up initial UD state by consuming the first event from the unit watcher before starting the watchLoop | 08:08 |
| fwereade | TheMue, did it really do that> | 08:08 |
| fwereade | TheMue, AFAICS the only way to do this reliably is to follow that model | 08:09 |
| rogpeppe | fwereade, TheMue: morning! | 08:09 |
| TheMue | fwereade: Please rephrase the UD sentence. | 08:09 |
| TheMue | rogpeppe: Morning. | 08:09 |
| fwereade | TheMue, if everything in the data tree makes sure it's initialized via initial event before starting its watch loop | 08:09 |
| fwereade | TheMue, then all you need to do the global port open/close (just once) at the end of the first machinesw.Changes event | 08:10 |
| fwereade | s/need to/need is to/ | 08:10 |
| fwereade | no that s/ is crack | 08:11 |
| fwereade | whoops, no, it's what I meant | 08:11 |
| fwereade | TheMue, er, am I making any sense? | 08:11 |
| TheMue | fwereade: I have to think about it more. | 08:13 |
| TheMue | fwereade: It would avoid possibly remaining removed units in that one map, yes. | 08:13 |
| TheMue | fwereade: But it would totally change the way of initialization and have to check how this effects the instance mode, because we here talk about the global mode. | 08:14 |
| TheMue | fwereade: *sigh* | 08:14 |
| fwereade | TheMue, yeah, I'm sorry :( | 08:19 |
| TheMue | fwereade: Fridays discussion ended with this proposal, but seems it falls too short. | 08:19 |
| fwereade | TheMue, I was always trying to communicate that separating watcher setup from initial scanning was intrinsically broken -- sory I haven't been very effective at that | 08:22 |
| TheMue | fwereade: Funnily it works really fine, it only has the I-dont-know-if-those-left-units-are-removed-during-startup-in-global-mode-problem. | 08:22 |
| fwereade | TheMue, which is exactly the bug we're concerned about | 08:22 |
| fwereade | TheMue, fwiw, you've also introduced a goroutine-unsafety bug in unitData.watchLoop | 08:23 |
| fwereade | TheMue, ud.ports is used on the main goroutine | 08:23 |
| TheMue | fwereade: Could you point me to the line? | 08:24 |
| fwereade | TheMue, old: line 542 removed; new: lines 612, 615 changed | 08:24 |
| TheMue | fwereade: IMHO the goroutine is started after initialization (btw opposit to the former solution). | 08:25 |
| fwereade | TheMue, sorry, rephrase please? | 08:25 |
| TheMue | fwereade: Seems to be independent, sorry. I have a deeper look. Thought you meant something different. | 08:26 |
| dimitern | jam, wallyworld, rogpeppe: PTAL https://codereview.appspot.com/6877054/ | 08:27 |
| TheMue | fwereade: But I see the problem, yes. I stumbled about the same naming ports as field (where it's just used by the Firewaller itself) and the local variable. Shit, has been intended, but not lucilly. | 08:28 |
| TheMue | luckilly | 08:28 |
| TheMue | Hehe, this word doesn't exist. | 08:28 |
| TheMue | Just lucky. | 08:28 |
| fwereade | TheMue, sorry, I'm still a bit confused :) | 08:28 |
| TheMue | fwereade: In the old version the unitData has a field named ports and in watchLoop() a variable named ports. I thought that this has been a mistake, but that has been intended. | 08:30 |
| TheMue | fwereade: So my "correction" has been wrong. | 08:30 |
| fwereade | TheMue, yeah, it's definitely required, but it demands close reading -- probably worth a comment for future reference | 08:30 |
| TheMue | fwereade: Yes. | 08:30 |
| fwereade | TheMue, otherwise people will definitely try the same "correction" in future :) | 08:30 |
| TheMue | fwereade: Maybe also a better naming. Dunno, will think about it. First I'll mark it again as WIP. | 08:31 |
| fwereade | TheMue, cool, thanks | 08:32 |
| TheMue | fwereade: Done | 08:32 |
| TheMue | fwereade: So the problem now is, that following your/our approach is a larger change. | 08:33 |
| TheMue | fwereade: And starting the datas based on the initial event is independent of instance or global mode. | 08:34 |
| TheMue | fwereade: The later ony is interesting to mark ports as opened and to manage the ref count. | 08:34 |
| TheMue | fwereade: Which can be handled during this startup phase. | 08:35 |
| TheMue | fwereade: So, before entereing the firewaller loop, init is called waiting for the first machine watcher event. | 08:36 |
| TheMue | fwereade: It starts the machines, but then? Hmm, how do I know that all unit watchers and service watchers are set up based on it? | 08:37 |
| TheMue | fwereade: All machines have to notify the init via a channel when they are done and the same with units and services, bottom up. | 08:38 |
| TheMue | fwereade: Does that makes sense? | 08:38 |
| fwereade | TheMue, sorry, I'm still processing | 08:39 |
| * TheMue thinks of a concurrently initializing tree which at least has to signal the firewallers init that it's done so that it can continue and enter the loop. | 08:40 | |
| fwereade | TheMue, the trouble is partly that I haven't ever analysed the non-global mode | 08:40 |
| TheMue | fwereade: Hehe | 08:41 |
| fwereade | TheMue, but I *think* that the "initial scan" can be perfectly synchronous if the MD creation goes something like... | 08:42 |
| fwereade | w := m.WatchUnits() | 08:42 |
| fwereade | units := <-w.Changes() | 08:42 |
| fwereade | for _, u := range units { | 08:42 |
| fwereade | // create unit data similarly | 08:42 |
| fwereade | } | 08:42 |
| fwereade | go md.loop(w) | 08:42 |
| fwereade | TheMue, (very poorly sketched ofc) | 08:43 |
| TheMue | fwereade: When are the ud loops started? | 08:44 |
| fwereade | TheMue, exactly the same model as the MD ones | 08:44 |
| TheMue | fwereade: Then it's very similar to my thought, only expressed in code. ;) | 08:45 |
| fwereade | TheMue, so, after you've created all your MDs (synchronously) in the first event | 08:45 |
| fwereade | TheMue, sorry -- it's the thought I've been trying to communicate for a week or 2 ;p | 08:46 |
| TheMue | fwereade: *rofl* | 08:46 |
| TheMue | fwereade: To come back to the mode question. | 08:46 |
| TheMue | fwereade: IMHO independent of the mode the startup of the watcher is always the same. What differs is, how the firewaller itself (the type Firewaller) handles those informations based on the mode. | 08:47 |
| TheMue | fwereade: And here instance and global mode have to be handled differently, yes. | 08:48 |
| fwereade | TheMue, yes, indeed -- in global mode you can do a flush-everything at the end of the first machine event, and in instance mode just flush each machine as it's handled | 08:48 |
| TheMue | fwereade: +1 | 08:49 |
| fwereade | TheMue, I *think* this also means you can drop everything global* except globalPortRef | 08:49 |
| TheMue | fwereade: A ref != 0 means true, yes | 08:50 |
| fwereade | TheMue, you just need to be a little bit careful about updating it for initial events in create methods *only* before the first init run has completed | 08:50 |
| fwereade | TheMue, and subsequently doing so only as a consequence of flushes | 08:50 |
| fwereade | TheMue, ...or maybe I'm overcomplicating it | 08:51 |
| TheMue | fwereade: Could you please explain your thoughts more. | 08:51 |
| fwereade | TheMue, I think that I misspoke -- you never need to touch the refcounts while you're doing the "initial scan", they can be handled purely on the main goroutine as they currently are | 08:53 |
| TheMue | fwereade: Am I right you're thinking of a kind of "init" argument set to true during startup and later being false and internally work as described above only when it's true? | 08:54 |
| fwereade | TheMue, I *was*, but I don't think it's needed by anything except the main FW loop, so it can just be a var therein | 08:54 |
| fwereade | TheMue, sorry, bbiab | 08:55 |
| TheMue | fwereade: Hmm, I need the refcounts at the end of the startup to compare the needed ports to the currently open ports. | 08:55 |
| fwereade | TheMue, you can build the refcounts during the initial global flush, can't you? | 08:55 |
| TheMue | fwereade: Pardon? | 08:56 |
| fwereade | TheMue, sorry, back | 09:02 |
| TheMue | fwereade: NP | 09:02 |
| fwereade | TheMue, when you first do the initial global mode flush, won't you have a complete set of MDs and UDs available? | 09:03 |
| fwereade | TheMue, so you can just build the refcounts from that data | 09:03 |
| fwereade | TheMue, I think? | 09:04 |
| TheMue | fwereade: So you think of an explicit flush at the end of the init? Otherwise I have somewhere to remember that it's the first time. | 09:05 |
| fwereade | TheMue, yeah, exactly -- I'm saying that I think the only place you need that information is within the main FW loop | 09:05 |
| fwereade | TheMue, I could well be wrong though -- I'm hardly conversant with all the details of the FW | 09:07 |
| TheMue | fwereade: In the loop? Opposit to a firewaller field? | 09:07 |
| fwereade | TheMue, yeah -- I can't see when else you'd need it -- but you know the FW better than I do | 09:07 |
| TheMue | fwereade: We're still talking about the ref count? Just to make sure. | 09:08 |
| fwereade | TheMue, maybe "initial global mode flush" is the wrong term? | 09:08 |
| fwereade | TheMue, yeah, we're talking about the refcounts | 09:08 |
| fwereade | TheMue, they can be build up in initGlobalMode | 09:08 |
| TheMue | fwereade: I need access to it in the method flushGlobalPorts(). | 09:09 |
| fwereade | TheMue, but *that* only needs to be called, once, at the end of the first machines change | 09:09 |
| fwereade | TheMue, doesn't initGlobalMode call flushGlobalPorts? I may be confused | 09:09 |
| TheMue | fwereade: Not today, but I would do it at the end of the init after the change we talked about and when global mode is active. | 09:10 |
| fwereade | TheMue, ah, no, I'm on crack, sorry | 09:10 |
| fwereade | TheMue, so "flush" was the wrong term | 09:10 |
| TheMue | fwereade: But it's also needed later, it is the alternative to flushInstancePorts() | 09:10 |
| TheMue | fwereade: You smoke too much. *lol* And maybe the wrong stuff. | 09:10 |
| TheMue | fwereade: *scnr* | 09:11 |
| fwereade | TheMue, what I'm trying to say is that if you call initGlobalMode after handling the first machines change | 09:11 |
| fwereade | TheMue, you can use all the MD/UDs that have been built up -- and never have to hit state at all | 09:11 |
| fwereade | TheMue, it's a simple loop through all the units to count up the port refs | 09:11 |
| fwereade | TheMue, and then do the usual open/close as you currently do at the end of initGlobalMode | 09:12 |
| TheMue | fwereade: Yes, that's right, here I would update the refcounts based on the datas, compare it to the open ports and handle the needed opens and closes. | 09:12 |
| fwereade | TheMue, ...and then you're done, because you *know* that any changes from that state are already being watched for | 09:12 |
| fwereade | TheMue, and will be handled sanely once you hit the main loop select again | 09:13 |
| TheMue | fwereade: Bingo! | 09:13 |
| fwereade | TheMue, the only wrinkle about which I am still uncertain is service exposure | 09:14 |
| TheMue | fwereade: If that changes during init? | 09:15 |
| fwereade | TheMue, just in general -- I know there's code to handle it but I've never needed to tease out the details | 09:15 |
| fwereade | TheMue, but, yes, the same general strategy needs to be used, I think | 09:15 |
| TheMue | fwereade: We're watching the expose flag to see if a service is exposed. That's covered today in init (what will change) and during runtime (by the watches). | 09:16 |
| fwereade | TheMue, yeah -- my point is that you need to unify init and watcher setup in the same way we discussed for MD/UD | 09:18 |
| TheMue | fwereade: Yep, the whole chain MD/UD/SD. | 09:19 |
| fwereade | TheMue, I think it's pretty easy so long as you're careful to make sure all your *Data creation is done on the main goroutine | 09:20 |
| fwereade | TheMue, including for subsequent events | 09:20 |
| TheMue | fwereade: Yes, sounds reasonable. | 09:21 |
| fwereade | TheMue, cool :) | 09:21 |
| TheMue | fwereade: So I think I'll exactly do this now. | 09:22 |
| fwereade | TheMue, great | 09:22 |
| fwereade | TheMue, tyvm for your time on this, I'm sorry it's such a hassle | 09:22 |
| TheMue | fwereade: No, it's absolutely great to do this kind of design review. | 09:23 |
| TheMue | fwereade: I have to thank you. | 09:23 |
| fwereade | TheMue, always a pleasure :) | 09:25 |
| fwereade | btw TheMue, or rogpeppe, I need another review on https://codereview.appspot.com/6864050/ if either of you has a moment | 09:41 |
| rogpeppe | fwereade: looking | 09:43 |
| rogpeppe | fwereade: you've got a review | 10:06 |
| rogpeppe | fwereade: BTW i don't *think* it's possible for the transaction mechanism to determine which op failed. | 10:07 |
| rogpeppe | fwereade: because the asserts are executed in a distributed way | 10:07 |
| fwereade | rogpeppe, I'm still not sure it's a good idea -- but it should at least be able to figure out the doc, right? | 10:07 |
| rogpeppe | fwereade: i'm not sure | 10:07 |
| rogpeppe | fwereade: because it can't do the check and write the doc in the same moment | 10:08 |
| rogpeppe | fwereade: if someone else happens to execute the transaction, how do we get that information back to the entity that started that transaction? | 10:09 |
| fwereade | rogpeppe, you could be right | 10:12 |
| fwereade | rogpeppe, I suspect it would still be possible, but almost certainly not worth the effort :) | 10:12 |
| rogpeppe | fwereade: i'm not sure actually. seeing when a transaction fails and writing some information about it is non-atomic, and it needs to be atomic to work. | 10:13 |
| fwereade | rogpeppe, maybe so, I certainly don't claim to understand all the nuances of txn's implementation | 10:16 |
| rogpeppe | fwereade: neither me :-) | 10:17 |
| fwereade | rogpeppe, TheMue: I have a couple of responses to the reviews, would appreciate your thoughts | 10:31 |
| fwereade | rogpeppe, (I'm not disagreeing that a comment is called for, btw, I'm explaining the logic in case I'm on crack) | 10:31 |
| TheMue | fwereade: Already writing. | 10:31 |
| fwereade | morning Aram | 10:33 |
| Aram | yo | 10:33 |
| rogpeppe | fwereade: replied | 10:40 |
| rogpeppe | Aram: hiya | 10:41 |
| Aram | hey there | 10:41 |
| fwereade | rogpeppe, TheMue, cheers guys | 10:41 |
| TheMue | fwereade: YW | 10:41 |
| TheMue | Aram: Hiay | 10:41 |
| TheMue | Hiya | 10:41 |
| Aram | hello. | 10:41 |
| TheMue | Grmpf, fingers too fast. | 10:42 |
| fwereade | oh, bugger, doc appointment, bb after lunch | 10:44 |
| wallyworld | dimitern: hi, i just commented on your mp | 11:33 |
| dimitern | wallyworld: hey, 10x | 11:33 |
| wallyworld | sorry about the delay, i was eating dinner and having a few tasty drinks :-( | 11:33 |
| wallyworld | :-) i mean | 11:33 |
| dimitern | wallyworld: no worries :) | 11:34 |
| dimitern | wallyworld: the FlavorDetail used internally is enough to return lists of either Entity or FlavorDetail with the HTTP API | 11:34 |
| wallyworld | dimitern: ok, so long as the current goose nova client can unmarshall the data as returned by the double, then i;m happy :-) | 11:36 |
| dimitern | wallyworld: certainly | 11:36 |
| wallyworld | dimitern: i'll plug in the live/local tests once this lands and we can see how it all pans out | 11:37 |
| dimitern | wallyworld: yep, can't wait already | 11:38 |
| dimitern | rogpeppe: when you can, could you look at my CL please? | 11:39 |
| rogpeppe | wallyworld: i've just sent another review of your errors CL | 11:39 |
| wallyworld | dimitern: yeah, when you have worked on a branch for a little while, you just want it to land! | 11:39 |
| rogpeppe | wallyworld: perhaps we could have a chat about it some time | 11:39 |
| rogpeppe | dimitern: which one? | 11:39 |
| wallyworld | rogpeppe: ok, let me take a look | 11:40 |
| dimitern | rogpeppe: https://codereview.appspot.com/6877054/ | 11:40 |
| rogpeppe | dimitern: looking | 11:40 |
| wallyworld | rogpeppe: happy to chat when you have a moment, after you finish reviewing. just ping me | 11:43 |
| rogpeppe | dimitern: sorry, "d" is shorthand for "delete this line" (ed(1) syntax :-]) | 11:46 |
| dimitern | rogpeppe: I see :) | 11:47 |
| niemeyer | Hello all | 11:47 |
| rogpeppe | niemeyer: mornin' | 11:47 |
| dimitern | niemeyer: hiya | 11:47 |
| rogpeppe | dimitern: you've got another review | 12:15 |
| rogpeppe | wallyworld: ping | 12:15 |
| dimitern | rogpeppe: thanks! | 12:16 |
| wallyworld | rogpeppe: mumble? | 12:16 |
| rogpeppe | wallyworld: what *is* mumble?? | 12:16 |
| wallyworld | oh, maybe google hangout? | 12:16 |
| wallyworld | rogpeppe: mumble is a chat program, like skype | 12:16 |
| rogpeppe | wallyworld: i'd be happy to use mumble if i knew what it was :-) | 12:16 |
| wallyworld | except free, open source etc | 12:16 |
| rogpeppe | wallyworld: ah | 12:16 |
| rogpeppe | wallyworld: i'll give it a go. presumably i can apt-get it? | 12:17 |
| wallyworld | yeah, believe so. but you need a passwrd etc for the server - it's on y=the wiki | 12:17 |
| wallyworld | i'll see if i can dig up a link | 12:17 |
| wallyworld | rogpeppe: https://wiki.canonical.com/StayingInTouch/Voice/Mumble | 12:18 |
| rogpeppe | wallyworld: ok, i'm on mumble | 12:27 |
| rogpeppe | wallyworld: which space do you use? | 12:28 |
| wallyworld | rogpeppe: goto cloud engineering kitchen? | 12:28 |
| dimitern | rogpeppe: settings links to []nova.Link{} prevents AddFlavor/Server from generating them (it does that when links == nil). Not setting them will cause DeepEquals to fail and will force the unnecessary IMHO creating the full list of links every time I need to compare what's in the state vs. what I set there. | 12:33 |
| dimitern | s/settings/setting | 12:34 |
| * niemeyer => reboot | 12:50 | |
| niemeyer | fwereade: ping | 13:12 |
| fwereade | niemeyer, pon | 13:12 |
| fwereade | er, pong | 13:12 |
| niemeyer | fwereade: Heya :) | 13:12 |
| niemeyer | fwereade: pon is great, though | 13:12 |
| fwereade | niemeyer, how's it going? | 13:12 |
| niemeyer | fwereade: We should totally switch over | 13:12 |
| niemeyer | pin.. pon | 13:13 |
| fwereade | niemeyer, yeah, think how much typing we'd save :) | 13:13 |
| fwereade | niemeyer, thanks for your reviews last night, very helpful | 13:13 |
| niemeyer | fwereade: No worries.. that's actually what I'm pinging about.. is there anything you'd like to chat on to unblock? | 13:14 |
| fwereade | niemeyer, I think I'm good, thanks | 13:15 |
| fwereade | niemeyer, I'm still marshalling my thoughts re testing methods | 13:16 |
| niemeyer | fwereade: Coolio | 13:17 |
| fwereade | niemeyer, I guess the differing perspective may be that I see the testing methods as implying "do all required busywork to get to the point where I can call, say, RemoveService" | 13:17 |
| fwereade | niemeyer, but the current uses are not very compelling | 13:18 |
| fwereade | niemeyer, I'm fine dropping AddAnonymousService, but I reserve the right to get it out and wave it around if and when I spot a situation that deserves it, even if it probably never deserves to go on a shared test case | 13:19 |
| fwereade | s/case/suite/ | 13:19 |
| fwereade | ;) | 13:20 |
| niemeyer | fwereade: Hehe :) | 13:22 |
| niemeyer | fwereade: Sounds good :) | 13:22 |
| TheMue | lunchtime, biab | 13:38 |
| niemeyer | rogpeppe: Regarding this comment: | 14:00 |
| niemeyer | """ | 14:00 |
| niemeyer | one possibility would be to allow forcing the version number only, and have | 14:00 |
| niemeyer | version.Current always report the current series. | 14:00 |
| niemeyer | """ | 14:00 |
| niemeyer | rogpeppe: I don't get the idea | 14:00 |
| niemeyer | rogpeppe: version.Current.Series already does report the current series | 14:00 |
| rogpeppe | on a call, back in mo | 14:01 |
| niemeyer | rogpeppe: Suppa | 14:02 |
| rogpeppe | niemeyer: version.Current.Series doesn't report the current series if the version is forced, i think | 14:06 |
| rogpeppe | niemeyer: i may be wrong, let me check | 14:07 |
| rogpeppe | niemeyer: no, that's right | 14:07 |
| niemeyer | rogpeppe: Ah, I see what you meant | 14:07 |
| niemeyer | rogpeppe: Hacking the hack sounds a bit hackish :-) | 14:08 |
| rogpeppe | niemeyer: actually we'd be making the hack slightly smaller | 14:08 |
| rogpeppe | niemeyer: by making it change the version number only | 14:08 |
| niemeyer | rogpeppe: I guess so | 14:09 |
| niemeyer | rogpeppe: I'll have a look at this | 14:09 |
| fwereade | brb cleaner cleaning | 14:12 |
| niemeyer | rogpeppe: Actually, on a second thought, I don't understand why that matters for that particular CL | 14:18 |
| rogpeppe | niemeyer: aren't you trying to verify that an instance with the given series is actually started? | 14:18 |
| niemeyer | Yes, and where's current series picked from? | 14:19 |
| niemeyer | Ah, from the system | 14:19 |
| niemeyer | That's slightly unexpected (by me) | 14:19 |
| rogpeppe | niemeyer: where else could it come from? | 14:20 |
| niemeyer | It basically means what we get out of the agent tools is reporting the machine, not the actual running tools | 14:20 |
| niemeyer | Not sure which way is more useful, though | 14:21 |
| rogpeppe | niemeyer: they're both potentially useful | 14:22 |
| niemeyer | Yeah | 14:22 |
| rogpeppe | niemeyer: but it's hard to bake the current series into the compiled executable | 14:22 |
| niemeyer | rogpeppe: Well.. I'm coding a change right now to *avoid* doing it, so it shouldn't be so hard.. 8) | 14:22 |
| niemeyer | rogpeppe: Either way, we'd be guessing in both directions.. so we don't have to make any changes to that right now | 14:23 |
| niemeyer | I'll just do that one side-change as an independent CL and see how it looks | 14:23 |
| niemeyer | After lunch, though | 14:24 |
| niemeyer | biab | 14:24 |
| fwereade | dimitern, wallyworld, jam: is http://paste.ubuntu.com/1416964/ expected? I just updated goose... | 14:59 |
| dimitern | fwereade: well, that seems you're missing some openstack env vars | 15:00 |
| fwereade | dimitern, surely the tests should pass on a clean system? | 15:01 |
| dimitern | fwereade: if you don't pass -live to the tests it should not matter | 15:01 |
| fwereade | dimitern, I'm pretty certain my env vars should not be able to cause tests to either fail or succeed :) | 15:01 |
| fwereade | dimitern, I didn't, though | 15:01 |
| fwereade | dimitern, I concede the point re -live though :) | 15:02 |
| dimitern | fwereade: hmm.. well, I don't know if wallyworld is around - that are his last changes | 15:02 |
| fwereade | dimitern, bah, I never know whether to dive in and fix myself, or revert, or flag and move on | 15:03 |
| dimitern | fwereade: as a workaround until fixed, you'll need to set these to non-empty strings: OS_USERNAME, OS_PASSWORD, OS_TENANT_NAME, OS_REGION_NAME | 15:04 |
| dimitern | fwereade: if tests are failing I think revert is the best bet | 15:05 |
| fwereade | dimitern, yeah, probably sensible :) | 15:06 |
| fwereade | rogpeppe, TheMue: https://codereview.appspot.com/6864050 comments addressed I think | 15:06 |
| fwereade | (fwiw, does anyone know best practice for reversing a merge into trunk? absent better ideas I'm just going to copy the old version over the top and ask for a trivial LGTM on trust) | 15:08 |
| fwereade | bah, it's not the latest | 15:09 |
| rogpeppe | fwereade: last time i did it, i used patch with a revert diff | 15:09 |
| rogpeppe | reverse diff | 15:09 |
| rogpeppe | fwereade: LGTM BTW | 15:10 |
| fwereade | rogpeppe, yeah, I'm *sure* bzr would do it for me if I knew what I were doing :) | 15:10 |
| TheMue | fwereade: Review is in. | 15:13 |
| rogpeppe | niemeyer: next stage in bottom-up API implementation: https://codereview.appspot.com/6913043 | 15:27 |
| niemeyer | rogpeppe: Cheers | 15:28 |
| niemeyer | dimitern: Can you give a hand there and propose a fix to avoid the issue for the moment? | 15:39 |
| dimitern | niemeyer: what fix? | 15:40 |
| niemeyer | dimitern: One that unbreaks tests | 15:40 |
| dimitern | niemeyer: ok, I'll take a look | 15:40 |
| niemeyer | dimitern: Thanks a lot | 15:40 |
| fwereade | niemeyer, dimitern: I just proposed a branch with that commit straight-up reverted -- https://codereview.appspot.com/6907049 | 15:40 |
| fwereade | niemeyer, dimitern: I'll go ahead and reject it if a fix is incoming | 15:41 |
| niemeyer | fwereade: I'm assuming that a trivial temporary fix for this would be under 10 lines of code.. if it seems any controversial, +1 on reverting | 15:41 |
| dimitern | fwereade: I have no idea what the fix should be yet, looking.. | 15:41 |
| fwereade | rogpeppe, fwiw, `bzr diff -r-2..-3` gave me a patch that I could just apply | 15:41 |
| rogpeppe | fwereade: yeah, that's what i was suggesting | 15:42 |
| fwereade | niemeyer, gut says it's a bit of trivial test setup | 15:42 |
| rogpeppe | fwereade: sorry if i wasn't clear enough | 15:42 |
| fwereade | rogpeppe, not to worry | 15:42 |
| dimitern | fwereade: i cannot even compile the tests from trunk - go test -i in environs/openstack says: ../../state/open.go:72: undefined: mgo.DialWithInfo; mgo.DialInfo | 15:43 |
| fwereade | dimitern, if it looks like it'll be complex then maybe just give my CL a quick check and let me know if it's sane | 15:43 |
| fwereade | dimitern, go get -u labix.org/v2/mgo/... | 15:44 |
| niemeyer | dimitern: Update your mgo | 15:44 |
| niemeyer | What fwereade says | 15:44 |
| niemeyer | dimitern: How long since you last run tests, btw? :-) | 15:44 |
| dimitern | fwereade: ok I'll check out the CL in the mean time | 15:44 |
| fwereade | niemeyer, I think he's been on goose :) | 15:44 |
| niemeyer | Ah, that'd explain it | 15:44 |
| dimitern | niemeyer: a week probably - been working on goose mostly, not juju-core | 15:44 |
| niemeyer | fwereade: Can we do that on our side maybe, if we know what to do? Is it just a matter of putting those vars in the env? | 15:45 |
| fwereade | niemeyer, no idea, it probably is | 15:46 |
| fwereade | niemeyer, I was trying to avoid distracting myself... didn't actually manage that, though :) | 15:46 |
| niemeyer | fwereade: If it is, +1 on adding them next to a TODO: This is wrong as the test shouldn't have to hack the environment to work out-of-the-box. Please fix at next chance. | 15:46 |
| niemeyer | fwereade: LOL | 15:46 |
| dimitern | fwereade: the CL looks fine to me | 15:47 |
| fwereade | dimitern, I'll see if there's an easy fix -- we can race :) | 15:47 |
| dimitern | fwereade: I don't think the revert was premature - ListServers() for example in the test I have has some arguments in the provider, but not in goose, so I think it's half-baked anyway | 15:49 |
| fwereade | dimitern, yeah, looking at the tests I haz a bit of a confuse, it looks like it *ought* to work | 15:50 |
| rogpeppe | it's going to be interesting keeping goose and juju-core in sync | 15:52 |
| niemeyer | Slightly unfortunate but if the APIs are diverging, there's no other way | 15:52 |
| niemeyer | rogpeppe: Feels pretty easy | 15:52 |
| niemeyer | rogpeppe: We've been doing that with every other package for quite awhile | 15:53 |
| rogpeppe | niemeyer: true, but the external packages weren't perhaps changing quite as fast | 15:53 |
| niemeyer | rogpeppe: Sure, but the idea is still the same.. merge only when both as ready to go in | 15:53 |
| niemeyer | s/as/are | 15:53 |
| niemeyer | rogpeppe: https://codereview.appspot.com/6907050 | 15:58 |
| niemeyer | dimitern: It doesn't look like the API is broken, though? | 16:04 |
| niemeyer | dimitern: It seems to compile fine here | 16:04 |
| dimitern | niemeyer: maybe I'm not getting something right then | 16:04 |
| fwereade | dimitern, ah, no, it's just a broken string (and tests which appear to run twice), plus whatever the deal is with local_test which I don't really get at all | 16:06 |
| dimitern | fwereade: I think the local tests are supposed to use the test doubles | 16:06 |
| fwereade | dimitern, yeah, but none of those tests actually hit anything let alone the test double | 16:08 |
| dimitern | fwereade: no idea, wallyworld would know I haven't looked at them yet | 16:09 |
| rogpeppe | niemeyer: looking | 16:09 |
| fwereade | dimitern, ISTM that they're failing to construct an env and trying to get the provider from that | 16:09 |
| niemeyer | fwereade: I have a fix.. pushing | 16:11 |
| fwereade | dimitern, it looks like that's what we do in ec2, though, although I can't figure out *why* | 16:11 |
| dimitern | fwereade: probably it needs both tenant and region, which is not needed for ec2? | 16:13 |
| fwereade | dimitern, it doesn't need an env at all :/ | 16:13 |
| fwereade | dimitern, it's creating one for absolutely no reason | 16:14 |
| dimitern | fwereade: I see, so it shouldn't? | 16:14 |
| rogpeppe | niemeyer: LGTM | 16:14 |
| fwereade | dimitern, AFAICT it should just be doing Provider("openstack").PrivateAddress() | 16:15 |
| fwereade | dimitern, PrivateAddress has to work with no credentials or it's worthless | 16:15 |
| niemeyer | fwereade, dimitern: https://codereview.appspot.com/6912047 | 16:15 |
| fwereade | niemeyer, LGTM as far as it goes, but it might be nice to drop the redundant Test() func in config_test.go | 16:16 |
| niemeyer | fwereade: Yep, there's no point in having credentials there | 16:16 |
| fwereade | niemeyer, and frankly local_test.go should just be deleted | 16:16 |
| niemeyer | fwereade: DOing it | 16:17 |
| dimitern | niemeyer: LGTM | 16:18 |
| fwereade | niemeyer, LGTM | 16:19 |
| niemeyer | dimitern, fwereade: Thanks. I've sent an improvement to the comment as well, to point out that what was done there is a temporary hack to get things building, but definitely not the right thing to do. | 16:21 |
| dimitern | ok | 16:21 |
| fwereade | niemeyer, lovely, cheers | 16:21 |
| fwereade | niemeyer, ah, sorry, I thought you were submitting - LGTM again, I guess, but really almost all that file is useless and confusing | 16:26 |
| fwereade | niemeyer, hard to fault the implementer, though, it looks exactly like the ec2 version -- I think I'll propose my own on top when you've submitted | 16:28 |
| niemeyer | fwereade: Yes, my attempt is not to fix anything other than doing the smallest possible to help unrelated work to continue | 16:31 |
| fwereade | niemeyer, actually it's not quite as simple as it looks | 16:31 |
| fwereade | niemeyer, yeah, sensible | 16:31 |
| niemeyer | fwereade: What's that? | 16:31 |
| fwereade | niemeyer, I think the *Address tests should have their own metadataHost patching | 16:32 |
| fwereade | niemeyer, and I don;t think any of the other tests have any justification for doing so | 16:33 |
| niemeyer | fwereade: Agreed on both counts | 16:33 |
| fwereade | niemeyer, ok, it is pretty easy, I'll dash it off a bit later | 16:35 |
| rogpeppe | niemeyer: i was in the process of implementing jujud server (to serve the API) then realised that perhaps it might be good as another machine worker. what do you think? | 16:39 |
| rogpeppe | fwereade: ^ | 16:39 |
| fwereade | rogpeppe, I don't *think* so | 16:46 |
| fwereade | rogpeppe, but would you expand? | 16:47 |
| rogpeppe | fwereade: i started doing it separately and then realised that it had almost everything in common | 16:47 |
| fwereade | rogpeppe, hum, interesting -- if you can pull it out cleanly then that would probably be a Good Thing then | 16:47 |
| rogpeppe | fwereade: it looks at state and does stuff to it. almost the same as any other worker. | 16:48 |
| rogpeppe | fwereade: the only thing i'd want to do (to make things consistent) is to remove the --ca-cert argument from jujud | 16:48 |
| rogpeppe | fwereade: and have it pull the file from a known file within data-dir | 16:48 |
| fwereade | rogpeppe, +1 to that for sure | 16:48 |
| rogpeppe | s/the file/the cert/ | 16:48 |
| rogpeppe | fwereade: then the API worker can look in data-dir too to get the server cert and key | 16:49 |
| rogpeppe | fwereade: hmm, maybe it's crack though actually | 16:50 |
| rogpeppe | fwereade: they are almost identical *now* | 16:50 |
| fwereade | rogpeppe, I'm still a bit confused by it, it is true, but laura is shouting persistently in the other room so I am finding it tricky to concentrate | 16:50 |
| rogpeppe | fwereade: but in the future, the agents will need to talk to the API server, not the db | 16:51 |
| fwereade | rogpeppe, ah, yes, good point | 16:51 |
| rogpeppe | fwereade: yeah, so the upgrader loop must be different, because they're talking to two different kinds of state server. | 16:52 |
| rogpeppe | fwereade: i mean the state-reconnect loop | 16:53 |
| fwereade | rogpeppe, maybe best to keep them separate for now then | 16:53 |
| rogpeppe | fwereade: definitely | 16:53 |
| rogpeppe | fwereade: thanks for the feedback | 16:53 |
| fwereade | rogpeppe, it's always nice to be thanked for saying "er... dunno" :) | 16:54 |
| rogpeppe | fwereade: it could be worse, you might be a teddy bear | 16:54 |
| rogpeppe | fwereade: it will be a little weird though, 'cos i think the API server agent will need to listen to itself... | 16:56 |
| fwereade | rogpeppe, haha | 16:56 |
| rogpeppe | fwereade: the worker does need an entity name though, as it needs to store its password in state and in a directory. how about just "server" for the entity name, and having State.SetAPIServerPassword ? | 17:00 |
| rogpeppe | fwereade: oh bugger, that doesn't work | 17:05 |
| rogpeppe | anyone wanna have a look at this CL? https://codereview.appspot.com/6913043 | 17:10 |
| fwereade | rogpeppe, niemeyer: can I get a trivial LGTM on https://codereview.appspot.com/6907051 please? I ran all the tests :) | 17:23 |
| niemeyer | fwereade: Looking | 17:23 |
| niemeyer | fwereade: That seems rather controversial | 17:23 |
| fwereade | niemeyer, :p | 17:23 |
| niemeyer | fwereade: LGTM :-) | 17:23 |
| fwereade | niemeyer, cheers | 17:24 |
| rogpeppe | fwereade: how did that compile before? | 17:24 |
| fwereade | rogpeppe, it didn't, niemeyer owes us cookies :p | 17:24 |
| niemeyer | rogpeppe: Probably didn't.. the guy was incompetent | 17:24 |
| niemeyer | :-) | 17:24 |
| rogpeppe | ha ha | 17:24 |
| fwereade | rogpeppe, niemeyer: the trouble is I didn't spot it when I did the full test run for my previous submit | 17:25 |
| rogpeppe | niemeyer: that probably cancels out a few cookies i owe you :-) | 17:25 |
| niemeyer | rogpeppe: I'm not sure.. I should write down a footnote about the effects of the rule on build fixes | 17:25 |
| * niemeyer laughs in an evil way | 17:25 | |
| niemeyer | Reminds of that game where we make up rules as we go | 17:26 |
| niemeyer | We should play that at some point | 17:26 |
| fwereade | rogpeppe, niemeyer, I don't suppose there's a way to get go test to put the bad output, that I care about, at the end where I see it more obviously? | 17:26 |
| niemeyer | fwereade: Can I get a second review on this easy one: https://codereview.appspot.com/6907050 | 17:26 |
| fwereade | niemeyer, looking | 17:27 |
| niemeyer | fwereade: Hmm.. not that I know of | 17:27 |
| fwereade | niemeyer, just a NO YUO or something, that's all I ask :) | 17:27 |
| niemeyer | That's the game, btw: http://en.wikipedia.org/wiki/Mao_(card_game) | 17:29 |
| fwereade | niemeyer, can you briefly explain the context? code looks good but I don't quite understand what it helps us do | 17:29 |
| niemeyer | fwereade: It's a pre-req of.. | 17:29 |
| niemeyer | fwereade: https://codereview.appspot.com/6868070/ | 17:29 |
| fwereade | niemeyer, ah, ok | 17:30 |
| fwereade | niemeyer, LGTM | 17:31 |
| niemeyer | fwereade: Cheers | 17:33 |
| fwereade | I think I'm done for the day | 17:34 |
| fwereade | dimitern, I think we're all going down to get ribs at la rive, join us if you fancy it, they're pretty good :) | 17:34 |
| dimitern | fwereade: cool, when? | 17:35 |
| fwereade | dimitern, imminently :) | 17:35 |
| dimitern | fwereade: :) be there in 20m | 17:35 |
| fwereade | sweet, see you shortly then :) | 17:35 |
| fwereade | happy weekends everyone | 17:35 |
| dimitern | from me too :) happy weekend! | 17:36 |
| fwereade | niemeyer, (don't quite know how I failed to get to the deployer rework today :( -- it'll be there soon though :)) | 17:38 |
| niemeyer | fwereade: All good | 17:39 |
| niemeyer | fwereade, rogpeppe: I'll be off on Monday, btw, in exchange for the 28th that I missed.. Back Tuesday and on | 17:52 |
| rogpeppe | fwereade: have a good one! | 17:53 |
| rogpeppe | niemeyer: ok, have a great weekend | 17:53 |
| niemeyer | fwereade, rogpeppe: A great weekend to all as well | 17:54 |
| niemeyer | rogpeppe: api-server LGTM | 17:57 |
| rogpeppe | niemeyer: cool, thanks | 17:57 |
| rogpeppe | niemeyer: next up is adding a conventional Stop method, then adding a jujud server command. | 17:57 |
| niemeyer | rogpeppe: Sweet, sounds sensible | 17:58 |
| rogpeppe | niemeyer: i'm wondering how we should manage the api server's state password | 17:59 |
| rogpeppe | niemeyer: it probably needs its own entity management stuff in state, which i'm not too keen on doing, as it seems a lot of work for not much gain | 18:00 |
| rogpeppe | niemeyer: i've been toying with the idea of making the state server part of the machine agent | 18:01 |
| niemeyer | rogpeppe: How do you mean | 18:01 |
| niemeyer | ? | 18:01 |
| rogpeppe | niemeyer: it can't *quite* be "just another worker" but maybe not far off | 18:01 |
| niemeyer | rogpeppe: Soryr, let me be more specific | 18:01 |
| niemeyer | rogpeppe: Why state password would the API server need? | 18:02 |
| niemeyer | s/Why/what | 18:02 |
| rogpeppe | niemeyer: i think we probably want to keep mongodb password access | 18:02 |
| rogpeppe | niemeyer: even after the API server is done | 18:03 |
| niemeyer | rogpeppe: I see | 18:03 |
| niemeyer | rogpeppe: The API server is a bit of an interesting case | 18:03 |
| rogpeppe | niemeyer: and so we still need to go through the same initial password dance that we do now | 18:03 |
| rogpeppe | niemeyer: yeah, it is. but it's perhaps not as special as we might initially think. | 18:04 |
| niemeyer | rogpeppe: It can be, depending which way we go | 18:04 |
| rogpeppe | niemeyer: yea | 18:04 |
| niemeyer | rogpeppe: We could opt to use a unix socket, for example | 18:04 |
| rogpeppe | h | 18:04 |
| niemeyer | Which isn't supported by mgo right now | 18:04 |
| niemeyer | rogpeppe: I'm not sure about how much that'd make things easier/nicer, though | 18:05 |
| rogpeppe | niemeyer: i don't think that works so well, as i think we want to be able to have a state server fan out to several mongos, perhaps | 18:05 |
| rogpeppe | niemeyer: and the state server might implement some caching, so it might make sense to have more state servers than mongods | 18:05 |
| niemeyer | rogpeppe: Yeah, that's an interesting schema to keep in the sleeve, even if we don't do it right away | 18:06 |
| rogpeppe | niemeyer: i'm thinking that in fact, if we choose it to be, the state server is actually very similar to a normal worker. | 18:07 |
| rogpeppe | niemeyer: except that, eventually, it will act on the mongo state rather than the api state | 18:07 |
| niemeyer | rogpeppe: Agreed | 18:07 |
| niemeyer | rogpeppe: Sounds like a good way to put it | 18:08 |
| rogpeppe | niemeyer: and thinking that way, perhaps it makes sense to make it just another worker in the machine agent's arsenal. | 18:08 |
| niemeyer | rogpeppe: Yep | 18:08 |
| rogpeppe | niemeyer: there will be an interesting little dance for machine agents that also happen to be api servers, but i *think* it's possible | 18:09 |
| rogpeppe | niemeyer: the upgrade logic will want to talk to the api that's being served by the same machine agent, for example, but that will probably work ok. | 18:10 |
| niemeyer | rogpeppe: Yeah, it might work alright | 18:15 |
| rogpeppe | right, time for me to go too | 18:25 |
| rogpeppe | night all! have great weekending. | 18:25 |
| === carif_ is now known as carif | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!