[00:07] niemeyer: ahh, `yaml:"flow"`, might be the savation [00:08] davecheney: Ah, hold on [00:08] davecheney: You want to compare output? [00:08] davecheney: for testing purposes? [00:09] niemeyer: yes [00:09] davecheney: That's probably not a good direction [00:09] davecheney: We've had a lot of pain comparing yaml like that [00:09] ok [00:09] davecheney: It's easier to unmarshal the yaml back [00:10] davecheney: and compare the value [00:10] davecheney: This also helps because you can compare bits, rather than full docs [00:10] using deep equals ? [00:10] davecheney: Yeah [00:10] davecheney: Indentation is doable, etc [00:11] ok, i'll keep working on it [01:41] niemeyer: is this a bug ? http://play.golang.org/p/P1dQvMC9T4 [01:43] niemeyer: nm, the channel corrected my mistake [01:44] davecheney: Yeah, no commas [06:22] davecheney, fwereade__, rog: morning btw [06:22] davecheney, fwereade__, TheMue: yo! [06:38] rog: themue, hello! [06:39] rog: https://codereview.appspot.com/6432045 if you have any comments on the test harness [06:39] i would be glad to hear them [06:40] testing json and yaml is a pain the butt [06:40] but at least this way, it isn't a pain in the butt when we're pressed for time and someone points out json output is manditory [06:43] * rog looks [06:46] davecheney: i wonder if we should use MarshalIndent to produce nice looking json. just a thought. [06:47] rog: sure [06:47] davecheney: also, i wonder if the status tests could just specify the value of the data structure required; then we could unmarshal the output (in whatever format we want to check) and make sure it deep equals the [06:48] required value [06:48] rog: tried that, didn't wokr [06:48] davecheney: ah, because the unmarshal differently? [06:48] theyt [06:48] they [06:48] :-) [06:48] yes, json produces map[string]interface{} [06:48] yaml produces map[interface{}]interface{} [06:49] davecheney: hmm, but can't you unmarshal into a data structure of the expected kind? [06:50] rog: yes, but then the next level down becomes m[i]i [06:51] m[i]i ? [06:51] oh yeah [06:51] hmm [06:51] i don't think it's solveable [06:52] anyway, that is enough to start mergeing in my working branches [06:52] and if the harness is uttery wrong [06:52] then i'll tackle it then [06:54] davecheney: one last thought: could you start with the expected data structure, then marshal it and unmarshal it again, *then* check against the original unmarshalled value you got? [06:55] or marshal it, and compare it to the data in cxt.Stdout [06:55] davecheney: no, that won't work [06:55] why not [06:55] davecheney: because marshalling isn't deterministic [06:56] for json it is, but i take your point [06:56] davecheney: oh really, json sorts map keys? cool, i didn't know that. [06:56] gustavo argued that goayml can do the same thing [06:56] i'm going to hvae to change goyaml slightly anyway [06:56] davecheney: it would make life easier [06:57] davecheney: and json is a good precedent [06:57] * davecheney thinks that json is about as crack headed as javascript [06:57] for example, struct { Machines map[int]interface{} } [06:57] cannot be json marshalled [06:57] yet _must_ be marshalled by yaml [06:58] so, because of that, i have to conver machine.Id to a string, then change goyaml to accept a formatting flag to convert the string keys back to ints [06:58] davecheney: oh that's a pity [06:59] anyway, dinner time [06:59] then i'll ponder your idea [06:59] if that means we describe the output once, that is a big win [07:00] davecheney: given what you've said above, i'm not sure it can work. [07:00] davecheney: pity machine ids aren't strings [07:04] rog: Let the Firewaller only use a passed state has been a nice simplification, but how do I get the environment now. Somehow lost the path. [07:05] TheMue: ha ha. you'll need to pass the environment in as an argument too. [07:05] rog: sh** [07:06] rog: Already expected it *sigh* [07:11] rog: Btw, on Sunday, do we try to get seats next to each other or do we just meet at the baggage claim? [07:11] TheMue: it would be nice to sit together, but i can't quite work out how :-) [07:12] morning all [07:12] rog: That's the problem. [07:12] fwereade_: Morining [07:12] TheMue: except, i guess, we can try to arrive at the checkin desk together... but most seats are gone by then. [07:12] TheMue: i have an idea [07:13] * TheMue listens [07:13] TheMue: i won't have access to a computer, but perhaps i could give you my booking details and you could try and select a seat for both of us on the airline website [07:13] TheMue: in fact, perhaps they allow us to choose a seat now. i'll have a look. [07:13] rog: That could work [07:20] rog: Your Environ.OpenPorts() wants all ports that have to be open or just a delta that should be opened with this action? [07:21] TheMue: just a delta [07:21] rog: Good [07:21] TheMue: otherwise there wouldn't be a ClosePorts method :) [07:21] rog: Makes it more simple [07:21] rog: Sounds logical :D [07:21] TheMue: yeah, i toyed with both approaches, but this seemed better [07:22] rog: Yes, otherwise I would have to maintain a list of all open ports on a machine inside the firewaller. Now I just tell you to open it. [07:22] TheMue: yeah [07:25] rog: Hmm, if a port can't be opened (so you return an error), how hard is this error? I could do a panic, but that wouldn't be helpful. Better would be a kind of retry. [07:26] TheMue: yeah, it's a good question. [07:26] TheMue: not very hard, i'd say [07:26] TheMue: perhaps you could put the retry in later, to avoid cluttering the initial logic [07:27] rog: But have to leave now until noon, some shopping with Carmen. We can talk later again. [07:27] rog: So far I'll ad a TODO, yes [07:54] Hello all [08:12] asachs_, hi [08:12] everyone, sorry, I have to pop out for 30 mins [08:12] bbs [09:05] anyone seen a MaaS based bootstrap fail to install zookeeper ? [09:13] asachs_, not offhand; can you ssh into the instance at all? [09:13] fwereade_: yeah i could - but no zookeeper installed :( [09:14] * fwereade_ desperately tries to remember where cloudinit logs to [09:15] asachs_, can you take a look at /var/log/cloud-init-output.log on the instance? [09:15] fwereade_, asachs_: /var/log/cloud-init.log [09:15] rog, ah ok [09:15] fwereade_, asachs_: also, perhaps: cloud-init-output.log [09:17] fwereade_: here's a script i use occasionally for debugging, which automatically copies the logs from an instance to my local machine: http://paste.ubuntu.com/1101520/ [09:17] rog, nice :) [09:37] fwereade_: wanna swap a book or two again? [09:39] rog, I'll see if I can find something worth reading :) [09:49] rog: can you paste that link again please ? [09:51] davecheney: which link? [09:51] the one you emailed me [09:51] play.g.o [09:52] davecheney: http://play.golang.org/p/AMq8qURfBW [09:53] ta [10:39] rog: thanks - its for a maas setup so the ec2 stuff will not work so nice :) [10:41] asachs_: np. it should be easy enough to port the concept though, if you want. i doubt you have the rc shell installed anyway :-) [10:42] rog: it looks fairly straightforward, will have a go at it once my new MAAS server is up [10:42] rog: and thx [10:42] asachs_: tbh, i usually ended up doing ssh and tail -f for a more interactive view. [11:30] fwereade_: i wonder if you might take a look at this and see whether it seems reasonable. i haven't yet done the extra tests in the juju package, but the cmd/juju test suite passes: https://codereview.appspot.com/6428061/ [11:32] rog, very shortly, have much state loaded atm [11:32] fwereade_: np [11:33] fwereade_: it's fairly trivial refactoring, nothing substantial [11:33] rog, cool === TheMue_ is now known as TheMue [12:12] just off for lunch, then travelling this afternoon - i'll probably leave here in a couple of hours [12:12] rog: enjoy [13:12] rog, ok, I am at last on your review, lunch intruded [13:12] rog, sorry [13:20] fwereade: it's still WIP BTW [13:21] fwereade: 3 possible books: Crystal Rain (Tobias Buckell), Harmony (Project Itoh) and Wolf Hall (Hilary Mantel). read any of 'em? [13:24] rog, none of them :) [13:24] fwereade: last one's historical rather than sf, but great anyway [13:27] rog, excellent [13:27] rog, what's your general view of bleak sprawling messed-up fantasy? [13:28] rog, reviewed btw [13:30] rog, and I did the linked list thing with HookQueue and I think it worked out really nicely [13:30] rog, so many thanks for that [13:30] rog, https://codereview.appspot.com/6422049 [13:31] rog, nothing is O(1) though [13:31] rog, Add is O(len(changed_units)) [13:31] rog: I have some troubles with http://paste.ubuntu.com/1087489/, line 41 ff. [13:31] rog, Next is O(len(members)) [13:32] rog, but those are both better than before :) [13:33] rog: It seems like your outline iterates twice over the change (which is the list of open ports). [13:37] fwereade: last night i couldn't resist seeing what the linked list idea might look like; i was going to throw it away, but... http://paste.ubuntu.com/1101809/ [13:38] * rog is trying desperately to pack [13:40] rog, I'm afraid I really can't judge it without tests ;) [13:40] fwereade: indeed [13:51] Good mornings! [13:52] niemeyer, heyhey [13:53] niemeyer: Moin [13:53] Heyas! [13:55] niemeyer: yo! i'm heading to the airport in 25 mins... [13:56] rog: Oh, going to Lisbon already? [14:07] niemeyer: first leg only. laying over for the weekend with some university pals in amsterdam (planned ages and ages ago, and cut short a bit by the sprint). [14:07] rog: Ah, nice [14:08] rog: Have some good fun there [14:08] niemeyer: should be fun. at least one of them i haven't seen for 20 years [14:08] rog: WOah [14:08] niemeyer: i'll try not to arrive too hungover! [14:08] rog: Have you graduated 20 years ago!? 8) [14:08] niemeyer: old bugger, me [14:09] rog: LOL [14:10] niemeyer, fwereade, TheMue: right, i'm off. might be online for a bit from the airport though. [14:10] otherwise see y'all in lisbon! [14:10] i'm looking forward to it! [14:10] rog, have fun! [14:10] rog: Enjoy, we'll see at least at Lisbon airport [14:10] rog: Indeed! [14:10] rog: Have fun! [14:10] toodle pip [14:19] fwereade: So, the validation stuff is up for review [14:19] niemeyer, I'm reading it now, it looks *awesome* so far [14:21] fwereade: Very glad (and relieved :-) to hear it [14:25] niemeyer, LGTM, but I think control-bucket should also be immutable [14:26] fwereade: Is it not? [14:26] niemeyer, huh, couldn't see it [14:26] * fwereade reads again [14:26] niemeyer, it's not mentioned in Validate [14:27] niemeyer, did you do something clever? [14:27] fwereade: Isn't that awesome? :-) [14:27] fwereade: Oh, wait [14:27] niemeyer, but the structure of it all is perfect [14:27] fwereade: You mean immutable as in cannot be changed from old to cfg [14:27] Okay [14:27] niemeyer, yeah, sorry [14:28] fwereade: I read that as in *config.Config is immutable [14:28] fwereade: WE don't ever change, or even reference, control-bucket within Validate [14:28] niemeyer, nah, Config seems fine to me :) [14:28] fwereade: Which is what I claimed is awesome [14:28] niemeyer, ohhhhh [14:28] niemeyer, hmmmmm [14:29] niemeyer, can't quite decide whether that's awesome or evil [14:29] fwereade: But, you're right I think.. we shouldn't allow the control-bucket to change, at least for now [14:29] fwereade: Really? [14:29] fwereade: Why would it be evil? The schema is doing the whole validation for us [14:30] niemeyer, sorry, I misunderstood what you said [14:30] niemeyer, not to worry :) [14:30] fwereade: Cool :-) [14:34] niemeyer, a suggestion: we should store relation resolved nodes under the relation (like settings and presence), not under the unit [14:35] niemeyer, (and we should do the same for the workflow nodes, if they're not already there, which I don't immediately recall) [14:35] niemeyer, oh, hmm, maybe that's problematic [14:35] * fwereade goes off to read code [14:36] fwereade: I do think there's an awesometastic potential for simplifying things in that state machine, but indeed we have to think through the error management [14:38] fwereade: "If it's worth checking for a missing endpoint (as we do in Open) I maintain it's worth checking for here rather than there :)." [14:38] niemeyer, ...but it turns out you dropped that check entirely, and I'm fine with that [14:38] fwereade: Part of the beauty is that this is now done in both locations, and in SetConfig [14:38] niemeyer, oh? sorry, where? [14:38] fwereade: I haven't dropped it.. this branch actually *increases* validation significantly, perhaps to my surprise also :-) [14:39] fwereade: There's a single code path to grab an ecfg [14:39] fwereade: and it validate [14:39] s [14:40] niemeyer, there's also a funny little bit in Open, that you deleted, that checks the EC2Endpoint on the chosen region [14:40] niemeyer, I don;t see EC2Endpoint anywhere else in the diff [14:41] fwereade: Oh? [14:41] niemeyer, but I don't really see why we should need to check it in the first place [14:41] fwereade: Where is that? [14:41] niemeyer, https://codereview.appspot.com/6416055/diff/4001/environs/ec2/ec2.go#oldcode117 [14:41] fwereade: Oh, sorry [14:41] fwereade: I did misread that code [14:42] fwereade: I've read the error message, and inferred the test [14:42] fwereade: If the region exists in aws.Region, I claim we can use it [14:43] fwereade: We're not checking S3Endpoint, for example [14:43] niemeyer, agreed [14:43] fwereade: Thanks, though, I did miss the real test [14:43] niemeyer, I don't think that bit ever had an explicit test anyway [14:44] fwereade: Indeed [14:44] fwereade: I'll add the control-bucket check [14:44] niemeyer, cool [14:46] niemeyer, btw, I added those methods to Relation, in https://codereview.appspot.com/6430055/ [14:46] niemeyer, and in the CL I suggest that maybe they should be on Unit [14:47] niemeyer, but in *fact* I now think a RelationUnit type, which just holds the stuff we repeatedly calculate in Relation.unitScope, is really the right place for them [14:48] niemeyer, (*Relation)Unit(*Unit) (*RelationUnit, error) [14:48] niemeyer, (*RelationUnit) Watch() (*presence.Pinger, error) [14:48] niemeyer, (*RelationUnit) Settings() (*ConfigNode, error) [14:48] niemeyer, (*RelationUnit) Watch() (*RelationUnitsWatcher, error) [14:49] niemeyer, and then... [14:49] niemeyer, something a bit like (*RelationUnit) WaitResolved() (<-chan bool, error) [14:50] niemeyer, and similar methods for Workflow etc etc [14:50] niemeyer, would you be OK with that? [14:51] niemeyer, sorry, way up there, s/unitScope/unitInfo/ [14:53] (what the hell, how is it 5pm? time flies :)) [14:53] fwereade: I'm on the fence about it, mostly because I don't see the actual benefit and do see increased API surface, and even usage burden (e..g you now must check error twice to reach a Relation's Settings). [14:54] fwereade: Do you see simplification on the implementation? And if so, can you describe a bit of it? [14:55] niemeyer, the benefit IMO is that we don't need to pass a Relation and a Unit around everywhere we're dealing with the relation lifecycle [14:55] niemeyer, I think it'll also have methods like Workflow and SetResolved and so forth [14:56] niemeyer, which would be icky on Unit (because it will want stuff like that itself) [14:56] fwereade: Sounds sensible [14:56] fwereade: +1 [14:56] niemeyer, and the big issue with the current form is (*Relation)Watch(*Unit), which *really* looks like we're watching a unit [14:57] fwereade: Agreed.. I dislike that too [14:57] fwereade: It's also nice that we can have an *actual* Relation.Watch method, taking no parameters [14:57] fwereade: That doesn't ignore any units [14:57] niemeyer, ooh, nice [14:57] fwereade: I'm looking forward to use that in a monitoring tool :-) [14:58] niemeyer, ok, I'll wip that and make the change [14:58] fwereade: Thanks a lot [14:58] niemeyer, but https://codereview.appspot.com/6422049/ is also ready(?) and I think it's pretty cool actually [14:59] niemeyer, it lacks the occasional pathological performance characteristics of the first one [14:59] niemeyer, I think :) [15:01] fwereade: Sweet, will have a look [15:02] Actually, I'll review that first thing in the afternoon if that's ok [15:02] niemeyer, no rush :) [15:03] niemeyer, I have two nicely complementary streams of work at the moment, both UA related, and easy to switch between [15:03] niemeyer, when they collide I'll be screaming for reviews, so take it easy while you can ;) [15:04] fwereade: LOL [15:04] fwereade: Sweet [15:36] fwereade: is there any particular reason that StartUnit should be StartUnits? it's trivial for the caller to write a for loop, and it means that the caller can decide what to do if one fails after several have succeeded. [15:37] rog, just seems more appropriate for a high-level interface somehow [15:38] rog, matter of taste, not really bothered, we'll find out what we really need later [15:39] fwereade: if we were going to potentially optimise AddUnits(n) so that it wasn't just a simple for loop around AddUnit, i think i'd agree. but otherwise i can't see the point. [15:39] fwereade: it's not *that* high level - we assume the caller can program :-) [15:39] rog, that's what I kinda think we should be doing :) [15:40] fwereade: yeah, ok, will go for AddUnits then [15:41] fwereade: then if the state API changes to allow a more efficient approach, the juju API can remain the same [15:41] rog, yeah, exactly [15:42] rog, niemeyer: hey, why does AddUnitSubordinateTo exist? [15:42] fwereade: and one approach to that would be to add AddUnits to the state API and not download the charm for every unit! probably a better approach than caching [15:43] fwereade: Hm? [15:43] rog, niemeyer: should this not be handled at AddUnit time, with something like Service.Subordinates()? [15:43] rog, niemeyer: anyway, total derail really, but I can't see any benefit to exposing the functionality [15:43] fwereade: i guess it depends what level you see the state API [15:44] fwereade: i see it as potentially allowing things outside the scope of what is allowed in the high level juju view [15:44] rog, regardless of level, I'm not sure it should let us do things that aren't internally consistent [15:45] rog, without the appropriate relations in place, isn't it straight-up nonsensical to try to do that? [15:45] fwereade: Yeah, could be internalized I guess [15:45] rog, niemeyer: one for our Copious Free Time, I think, anyway ;) [15:45] fwereade: i think i agree [15:45] fwereade: It doesn't let us do things that aren't internally consistent already, but if we can simplify it that's a bonus [15:59] fwereade: I've added the control-bucket stuff and re-proposed.. if all looks good will submit once I'm back from lunch [16:00] niemeyer, cool, I'll take a look [16:01] niemeyer, just unwipped https://codereview.appspot.com/6430055 [16:01] niemeyer, according to how the tests feel, it was a good move :) [16:01] fwereade: Sweet! [16:02] niemeyer, dammit, I'm meant to be going out and having fun this evening ;p [16:03] fwereade: no rest for the wicked! [16:03] mramm2, haha === imbrandon is now known as Guest41572 [16:04] fwereade: You really should :-) [16:04] niemeyer, oh, I will, but it's really inconvenient right now :) [16:04] fwereade: Next week we'll have tons of time together to push awesomeness forward :) [16:05] fwereade: LOL [16:05] fwereade: I know how that feels [16:05] fwereade: I was hacking until post midnight yesterday [16:05] fwereade: Hard to stop when in a roll :) [16:05] niemeyer, yeah :) [16:14] TheMue, ping === Guest41572 is now known as imbrandon [16:15] fwereade: mom === imbrandon is now known as imbrandon_ [16:16] TheMue, it always disturbs me when you call me mom :) [16:16] fwereade: You're to me like a mom ;) [16:16] TheMue, aww :) [16:16] TheMue, well, all right darling === imbrandon_ is now known as imbrandon [16:20] TheMue, (I'll ask now; but no rush, please just respond when you're ready, I might have to go in a sec, I'll see your response later) [16:20] TheMue, I'm getting a slight urge to do violence to the ResolvedMode stuff [16:21] TheMue, (1) I don't think we have any need for the 1000/1001 magic numbers [16:22] TheMue, (2) I don't think we have any need for anything other than SetResolved, ClearResolved, and WaitResolved() (retry bool, err error) [16:23] TheMue, ie I can't see a use case for WatchResolved; am I missing something? [16:24] TheMue, (hmm, actually, what I'd *really* like is a one-shot <-chan bool watch, I think it will mesh very nicely with my plans for the hook executor) [16:24] TheMue, anyway, let me know your thoughts [16:24] niemeyer, I would also appreciate your perspective on the above [16:26] I'm thinking that I want, on both Unit and RelationUnit: [16:26] SetResolved(retry bool) error [16:26] ClearResolved() error [16:27] WaitResolved() (retry <-chan bool, err error) [16:27] but ofc I'm open to mockery and dismissal :) [16:27] anyway, later all [16:29] niemeyer, (also, am I right in thinking that we can't separately resolve errors in relations of the same name? is this deliberate, or am I misreading juju/control/resolved.py?) [16:36] fwereade: So, back again. Our daughter had problems today. [16:37] TheMue: sorry to hear about that [16:38] oops [16:39] TheMue, no worries, I might be able to pop back and read in 5-10 mins [16:39] TheMue, hope all is ok now [16:40] mramm2: Thankfully it's not her direct problem. It a harder problem by her best girl friend. But now our daughter has problems with her employer because she came later to the afternoon stint [16:41] fwereade: Yes, it's ok. Thx. [16:42] fwereade: So you're looking if the ResolvedMode is needed? Or what is your question? [16:43] fwereade: I've translated it from the python code, but if we don't need it anymore we should remove it. [16:50] TheMue, sorry, I'm really just asking if you're aware of any uses for it that I may have missed [16:50] TheMue, that can't be covered by what I proposed [16:50] TheMue, or if there's anything otherwise obviously ludicrous about what I suggest [16:52] fwereade: Right now I don't know. I'll take a look. [16:55] fwereade: So far it's not used, yes. I only ported it without looking who later will use it. [16:57] fwereade: In Py it's used by the agent. [17:26] fwereade: Hah.. so I screwed up the copy & paste.. [17:26] That's what I get for not self-reviewing [18:27] So, next proposal for the firewaller is in and I'm out. If we don't see tomorrow evening here we'll see on Sunday in Lisbon. [18:27] Have a nice evening. [18:37] TheMue: Thanks, have a great trip! [18:38] niemeyer: Thx, yes, and you have a great trip too. Will be a long flight. [18:42] TheMue: Thanks, this one is going to be surprisingly good, actually [18:42] TheMue: A single leg.. first time ever [18:42] niemeyer: Yeah, so it's a good location. Only Dave will always ever have problems. [18:43] niemeyer: So we once should visit him. [18:43] :D [18:43] TheMue: True :-) [18:44] niemeyer: Next UDS is very near. I'm already ching if not train will be better. [18:44] TheMue: Nice, that should be a breeze indeed [18:45] niemeyer: Yes, but that's still some month in front. Let's face our sprint next week. [18:45] TheMue: Yeah, and I have nightmareish trip before that, in early August [18:45] 4 legs each way.. :-( [18:46] niemeyer: Ouch