[09:07] <fwereade> heya TheMue
[09:24] <TheMue> fwereade: Hi
[09:24] <TheMue> fwereade: Phew, it's raining cats and dogs here.
[09:25] <fwereade> TheMue, it's pretty hot here
[09:25] <fwereade> TheMue, we're just approaching the too-damn-hot point of the year
[09:25] <fwereade> TheMue, give me a few days and I'll be begging for a decent rainstorm :)
[09:26]  * TheMue dcc's fwereade some rain.
[09:26] <fwereade> TheMue, :)
[09:27] <fwereade> TheMue, so how's it going? I hope the format 2 stuff isn't too much of a hassle -- I feel like I maybe should have done it myself, but I got caught up in the relations and I felt it was getting pretty important
[09:29] <TheMue> fwereade: I'll do firewall first, then format 2.
[09:29] <fwereade> TheMue, ah, excellent
[09:30] <fwereade> TheMue, are we going with the security groups style or are we doing it properly this time?
[09:31] <TheMue> fwereade: I'm talking about todays firewall.py in state. security and auth will be handled later.
[09:31] <fwereade> TheMue, ah cool
[09:31] <TheMue> fwereade: firewall is used by the PA.
[09:32] <TheMue> fwereade: So I'm moving it out of state to cmd.
[09:32]  * fwereade suddenly gets suspicious
[09:32]  * fwereade goes to read code a mo
[09:33] <fwereade> TheMue, doesn't implementing that presuppose the security groups approach?
[09:33] <TheMue> fwereade: As far as I've seen yet not.
[09:34] <TheMue> fwereade: But I've just started.
[09:34] <fwereade> TheMue, it seems to me that if the PA is going to use it, then we're assuming that the PA will remain responsible for opening/closing ports
[09:35] <fwereade> TheMue, a proper solution using firewalls on the units surely shouldn't involve the PA at all?
[09:36] <TheMue> fwereade: Sorry, don't know.
[09:36] <fwereade> TheMue, blast, wish niemeyer was on
[09:37] <TheMue> fwereade: So how would your solution look like?
[09:38] <fwereade> TheMue, unit agent messing with iptables, rather than PA messing with the provider
[09:38] <fwereade> TheMue, we've certainly talked about our use of security groups being a serious problem, and about the need for a cross-provider firewall solution
[09:39] <TheMue> fwereade: Pls go on ...
[09:40] <fwereade> TheMue, but it would not necessarily be *irrational* for us to go with the tried, tested, known-working-at-small-scale solution (given the time constraints that are starting to wear at me slightly)
[09:41] <fwereade> TheMue, the problems with security groups are (1) aws is really not designed to handle what we're doing with them and (2) the solution only works for aws
[09:41] <fwereade> TheMue, (2) is not important wrt our critical short-term goals
[09:42] <fwereade> TheMue, but disregarding (1) feels like the sort of decision that we should get some sort of consensus on before writing code that presupposes it
[09:42] <fwereade> TheMue, s/presupposes it/presupposes that approach/
[09:43] <TheMue> fwereade: That are worries, ok, but how would a proper solution look like?
[09:44] <fwereade> TheMue, I'm afraid I don't have a clear idea of the *precise* problem with our use of security groups... just that we're not meant to use any, and an apocryphal amazon engineer was said to look somewhat horrified by the prospect :)
[09:44] <fwereade> TheMue, I think it comes down to the *unit* agents watching the ports that should be open in their container and taking charge of it themselves
[09:45] <fwereade> TheMue, we'd still need *some* security groups, but probably just 2: one for PA machines and one for everything else
[09:45] <fwereade> TheMue, make sense?
[09:46] <TheMue> fwereade: Yep, so far understandable.
[09:46] <TheMue> rogpeppe: Hey, you are not here. ;)
[09:46] <rogpeppe> TheMue: that's right. i'm an invisible ghost.
[09:47] <rogpeppe> TheMue: i've been given special dispensation :-)
[09:47] <TheMue> rogpeppe: Ah, ok, then it's ok.
[09:47] <fwereade> TheMue, that's pretty much it...
[09:47] <fwereade> rogpeppe, heyhey
[09:47] <rogpeppe> fwereade: yo!
[09:47] <fwereade> rogpeppe, are you aware of any official preference as to how we implement firewalling this time round?
[09:47] <rogpeppe> i seem to remember we've got a meeting scheduled in 13 minutes, so i thought i'd try and turn up for it...
[09:47] <rogpeppe> (maybe i've got it wrong though!)
[09:48] <fwereade> rogpeppe, btw, finished To Hold Infinity, very enjoyable
[09:48] <rogpeppe> fwereade: cool, glad you enjoyed it. am enjoying wwz, in a slightly grim kinda way
[09:48] <rogpeppe> hmm, firewalling
[09:49] <rogpeppe> until we containerise everything, i think the current approach is probably the only one
[09:49] <fwereade> rogpeppe, enjoying Axiomatic too, fun to have a more thinky, less experiencey read once in a while
[09:49] <fwereade> rogpeppe, ah, expand please? I don't see the issue
[09:50] <fwereade> rogpeppe, after all everything is containerised already... in a sense... which feels like the appropriate sense for this context
[09:50] <rogpeppe> fwereade: how do we firewall without making use of ec2's facilities?
[09:50] <fwereade> rogpeppe, iptables?
[09:51] <rogpeppe> fwereade: can't anything get around that?
[09:51] <fwereade> rogpeppe, I have always presumed that it works as advertised, but I can't point to anything proving that
[09:52] <fwereade> rogpeppe, and I'm not saying we don't use security groups at all -- we have to -- but we know that using one per machine is a problem
[09:52] <fwereade> rogpeppe, I just don't know whether it's the sort of problem we want to fix now, or the sort of problem we leave for 13.04
[09:53] <rogpeppe> fwereade: am i right about the meeting, BTW?
[09:53] <fwereade> rogpeppe, er, I have no idea... I had a vague feeling it was weds, but maybe I missed another change
[09:54] <fwereade> rogpeppe, but davecheney is on, and that may lend support to your theory ;p
[09:55] <rogpeppe> dammit, it's an hour later
[09:55] <rogpeppe> bugger, my dispensation is invalid
[09:55] <rogpeppe> fwereade: iptables are manipulatable by root, and the charms run as root.
[09:55] <rogpeppe> fwereade: we need to talk to niemeyer about this
[09:56] <rogpeppe> fwereade, TheMue: well, gotta go. will miss the meeting, i think. have fun, and post any interesting/relevant conversations to juju-dev, where i will see 'em and sneakily read 'em...
[09:57] <fwereade> rogpeppe, yeah, indeed -- I'm not even sure I have a strong position on this, I just feel it's something we should get niemeyer's input on before we implement code that supposes either way
[09:57] <fwereade> rogpeppe, enjoy the holiday :)
[09:57] <TheMue> rogpeppe: OK, have fun.
[09:59] <fwereade> TheMue, I think that either way you can certainly implement something that keeps an eye on both sets of conditions, and emits events when ports should actually open or close
[09:59] <rogpeppe> fwereade: we could cache groups, because we're unlikely to have too many configurations of ports.
[09:59] <rogpeppe> fwereade: which might mitigate the issue
[09:59] <TheMue> fwereade: That's what firewall does today.
[10:00] <rogpeppe> TheMue: ah, it must've changed since i last looked
[10:00] <rogpeppe> TheMue: i thought there was one group for each machine
[10:00] <rogpeppe> anyway, gotta go
[10:00] <TheMue> rogpeppe: The firewall.py does not very much. It's only used by the PA.
[10:01] <fwereade> TheMue, where does it do that?
[10:01] <fwereade> TheMue, I don't see anything that shares groups in there
[10:02] <TheMue> fwereade: I didn't say anything about groups. I meant watcing the ports.
[10:02] <fwereade> TheMue, if anything does that, it's in the individual provider's open_port/close_port methods
[10:02] <fwereade> TheMue, ah got you
[10:03] <fwereade> TheMue, all I'd suggest then is to make sure that the thing that watches an individual machine remains distinct from the thing that watches all machines
[10:05] <fwereade> TheMue, do I appear to be approximately sane there?
[10:05] <TheMue> fwereade: I'll keep it in mind. I'm not yet deep enough in it. Just started the porting and as a prerequisite the watcher for the exposed flag.
[10:06] <fwereade> TheMue, cool
[10:06]  * fwereade starts to wonder whether he's right about it being up to the UA... maybe the MA would be better...
[10:07] <TheMue> fwereade: You've got more insight than me. I sometimes miss an architecture graphics where the components, their responsibilities and roles and how they communicate are visible.
[10:07]  * TheMue is a very visual being.
[10:07] <fwereade> TheMue, I think the issue there is that the responsibilities in python are not necessarily as they should be
[10:08] <fwereade> TheMue, eg theMA being responsible for the first download of the charm, and the UA being responsible for subsequent ones
[10:08] <TheMue> fwereade: OK, then two graphics: todays implementation and wanted implementation
[10:09] <fwereade> TheMue, the first one is of limited value and the second one is subject to change as we figure out *how* we should be doing things...
[10:09] <fwereade> *should*
[10:09] <fwereade> TheMue, hopefully without succumbing to second-system effect
[10:11] <TheMue> fwereade: That's a problem of working remote. I've used whiteboards a lot for a discussion of how something is and how it should change.
[10:11] <TheMue> fwereade: My intention is now first class diagram
[10:11] <TheMue> s/now/no/
[10:34] <Aram> moin.
[10:54] <fwereade> Aram, heyhey
[11:03] <TheMue> Aram: Moin
[11:05] <Aram> fwereade: TheMue: had a little bit of fun yestarday: http://play.golang.org/p/D-qPq8uIw3
[11:14] <fwereade> Aram, haha, nice
[11:18] <TheMue> Aram: *lol*
[13:08] <TheMue> Hmm, seems it's time for a topology watcher.
[14:05] <TheMue> fwereade: Any experiences with the size of topologies in large installations?
[14:06] <fwereade> TheMue, all I know is that yaml was too big for the 2k deployment, json makes it small enough for that with room to spare
[14:06] <TheMue> fwereade: I'm asking because topology watchers keep an old one in memory and pass it and a new one to the using callbacks/watcher users.
[14:07] <fwereade> TheMue, IIRC max ZK node size is 1MB, so order of that, I guess
[14:07] <TheMue> fwereade: I would store it already parsed, so there should be not whitespace problem.
[14:08] <fwereade> TheMue, it shouldn't be an overwhelming load though
[14:08] <TheMue> fwereade: ok
[14:08] <fwereade> TheMue, however you way want to look at recent topology watchers in go, which don't keep a whole topology around
[14:08] <fwereade> TheMue, they just keep the bits they're interested in
[14:09] <TheMue> fwereade: WHich ones you're talking about? Most I've seen so far watch simple nodes.
[14:09] <fwereade> TheMue, MachinesWatcher and MachineUnitsWatcher
[14:10] <TheMue> fwereade: Also the event of change always forces me to at least read one complete node.
[14:10] <fwereade> TheMue, also ServiceRelationsWatcher, new in review today
[14:10] <fwereade> TheMue, yeah, you always read the whole new topology
[14:10] <fwereade> TheMue, no reason to keep unit info around when all you care about is relations for one service
[14:10] <TheMue> fwereade: Thx, will take a look. I need it for the ServiceUnitsWatcher.
[14:11] <fwereade> TheMue, cool
[14:11] <fwereade> TheMue, a suggestion, don't know if it applies:
[14:12]  * TheMue listens
[14:12] <fwereade> TheMue, when doing the ServiceRelationsWatcher, it was very convenient to add (*Service)relationsFromTopology(t *topology) and use it both in Relations and the watcher
[14:13] <fwereade> TheMue, haven't looked at MW or MUW to see whether they'd benefit from similar
[14:13] <TheMue> fwereade: OK, will look, it sounds good.
[14:14] <fwereade> TheMue, it may be that the code to extract the stuff we care about is small enough not to bother in those cases and maybe in yours
[14:15] <TheMue> fwereade: Huh, the last sentence is difficult for me to understand.
[14:15] <fwereade> TheMue, sorry
[14:16] <fwereade> TheMue, I'm saying that getting a []*Relation from a service and a topology is enough work to make it worth factoring out
[14:16] <fwereade> TheMue, but getting a []*Unit from a service and a topology may be trivial enough that it's better to duplicate the code
[14:17] <fwereade> TheMue, similar may apply to MW and MUW
[14:17] <TheMue> fwereade: OK, understand, I will see how much it is.
[15:19] <niemeyer> Hellos!
[15:31] <TheMue> niemeyer: Hello to the far west.
[15:35] <niemeyer> TheMue: Hi :)
[15:35] <niemeyer> TheMue: How's been the weekend?
[15:37] <TheMue> niemeyer: Fine, a but support for my brother in law, he is building a house, and sitting on the couch on Sunday, it rained cats and dogs.
[15:37] <TheMue> niemeyer: And your travel to SFO?
[15:37] <niemeyer> TheMue: Hah :)
[15:38] <niemeyer> TheMue: The trip was quite fine
[16:17] <niemeyer> Hmm.. so it seems that Go's behavior on redirections has changed somehow.. lpad seems broken :(
[16:17]  * niemeyer investigates
[16:27] <fwereade> niemeyer, heyhey
[16:28] <fwereade> niemeyer, TheMue: please confirm that it is not safe to select on a send to a channel that might be closed
[16:28] <niemeyer> fwereade: It is actually safe
[16:29] <fwereade> niemeyer, really? oh, cool
[16:29] <niemeyer> fwereade: It depends a bit on what you mean by that, though
[16:29] <niemeyer> fwereade: Oh, wait.. *send*.. hmm
[16:29] <fwereade> niemeyer, select {dodgy <- event: blah; <-t.Dying()}
[16:29] <fwereade> niemeyer, select {dodgy <- event: blah; <-t.Dying():}
[16:30] <niemeyer> fwereade: No, that's not ok, sorry for the misinfo
[16:30] <fwereade> niemeyer, no worries :)
[16:32] <niemeyer> fwereade: It's considered a bad practice (hence why it blows up) because it's a clear statement that the life time of the channel is messed up.
[16:32] <fwereade> niemeyer, that was what I thought
[16:32] <Aram> hi niemeyer, how's SF?
[16:32] <fwereade> niemeyer, and I'm pretty sure I'm in a situation where I can just leave the channel alone without ever closing it anyway :)
[16:33] <niemeyer> Aram: Pretty nice, sunny.. had a good time with Andrew yesterday as well
[16:33] <Aram> niemeyer: nice.
[16:33] <niemeyer> fwereade: That's a possible answer
[16:33] <Aram> niemeyer: did you see my silly paste entry? http://play.golang.org/p/D-qPq8uIw3
[16:34] <niemeyer> Aram: Yeah, that was awesome :)
[16:35] <niemeyer> robbiew: ping
[16:35] <Aram> niemeyer: I could have made it an actual animated PNG, but animated PNGs don't work in webkit browsers yet.
[16:35] <robbiew> niemeyer: pong
[16:35] <niemeyer> robbiew: Heya
[16:35] <niemeyer> robbiew: Do we have a meeting today?
[16:36] <robbiew> niemeyer: heh...as usual, I have no idea...checking
[16:36] <niemeyer> Aram: Surprisingly short
[16:36] <niemeyer> robbiew: Cool.. I better find out a good way to call out of the hotel if so
[16:36] <robbiew> niemeyer: no meeting
[16:37] <niemeyer> robbiew: Super, thanks for checking
[17:28] <fwereade> gn all
[18:08] <fwereade> niemeyer, btw, I have to go again in a sec, but I meant to ask:
[18:08] <fwereade> niemeyer, are we planning to replicate the security-group firewalling for 12.10?
[18:09] <niemeyer> fwereade: Yeah
[18:09] <niemeyer> fwereade: Should be easy, and gets us parity
[18:09] <niemeyer> fwereade: We can then fix it another way later
[18:09] <niemeyer> fwereade: But,
[18:09] <niemeyer> fwereade: We should try to make the implementation sensible, so that we can reuse bits
[18:09] <fwereade> niemeyer, yep, I approve (despite emotionally wanting to Do It Right ;))
[18:09] <niemeyer> fwereade: I've been talking to Frank about that
[18:10] <niemeyer> fwereade: He's working on the firewall port watcher stuff
[18:10] <fwereade> niemeyer, excellent, I realised I didn't know what plan we were following when he mentioned it this morning
[18:10] <niemeyer> fwereade: That we have under state/firewall.py in Python
[18:10] <niemeyer> fwereade: But with some twists.. the Python version assumes it knows about a provider and what not
[18:10] <niemeyer> fwereade: The Go version will be a normal watcher
[18:10] <fwereade> niemeyer, yeah, I presume we'll just be outputting changes
[18:10] <fwereade> niemeyer, perfect
[18:11] <niemeyer> fwereade: Exactly
[18:11] <fwereade> niemeyer, I would guess two levels of watchers so we can reuse the inner one when it becomes the MA (UA???)'s responsibility?
[18:12] <niemeyer> fwereade: Yeah, we actually already have one in the unit
[18:12] <niemeyer> fwereade: So this is adding the second one, on Machine
[18:12] <fwereade> niemeyer, ah, nice
[18:12] <niemeyer> fwereade: WatchPorts
[18:12] <niemeyer> fwereade: I think we'll use the exact same thing when we move
[18:12] <niemeyer> fwereade: The difference is that the machine agent will call Machine.WatchPorts, rather than the provisioning
[18:13] <fwereade> niemeyer, perfect :)
[19:27] <robbiew> mramm: looking for me?
[19:33] <Aram> niemeyer: somethins intriguing is happening... compare this: http://bazaar.launchpad.net/~gophers/juju-core/trunk/view/head:/mstate/state.go#L56 with this: https://codereview.appspot.com/6304099/diff2/9002:18002/mstate/state.go
[19:33] <Aram> the machine function
[19:33] <Aram> is different :)
[19:33] <Aram> how can this be?
[19:33] <Aram> the AllMachines function is the same though, and both have been altered in the same commit.
[19:34] <niemeyer> Aram: Why should they be the same, just so I get the context?
[19:35] <Aram> niemeyer: because I submitted what's on codereview, and what's in launchpad seems an earlier version.
[19:36] <niemeyer> Aram: Ah, it's actually not
[19:37] <niemeyer> Aram: https://codereview.appspot.com/6330045/
[19:37] <Aram> interesting.
[19:38] <Aram> why the removal of that branch?
[19:38] <niemeyer> Aram: The new error will look like "can't get machine 42: not found", which is fine
[19:38] <niemeyer> Aram: I had to touch that logic due to the NotFound renaming
[19:38] <niemeyer> Aram: (ErrNotFound now)
[19:38] <Aram> yes, yes.
[19:39] <niemeyer> Aram: But rather than replacing it, I just dropped and allowed the underlying error to go through as per the message above
[19:39] <Aram> well yes, that was my initial version as well.
[19:39] <niemeyer> Aram: Not really
[19:39] <niemeyer> Aram: your initial version was the opposite.. any error would lead to "not found"
[19:40] <Aram> right.
[19:41] <Aram> niemeyer: anyway, thanks for clearing the confusion.
[19:41] <niemeyer> Aram: np, and sorry for the trouble.. I wanted to ask for your review on it too, but at the same time didn't want to leave trunk broken
[19:41] <Aram> of course
[20:01] <Aram> niemeyer: first piece of the puzzle: https://codereview.appspot.com/6341050
[20:01] <niemeyer> Aram: Awesome, thanks!
[20:12] <Aram> niemeyer: the diff on codereview is always done against lp:juju-core? can't I do it against some other branch I have?
[20:13] <niemeyer> Aram: You can, with -req
[20:13] <niemeyer> Aram: It only allows trees rather than graphs, but it works
[20:14] <Aram> strange, that's what I did, lbox propose -cr -wip -req="lp:~aramh/juju-core/mstate-charm-basic"
[20:14] <niemeyer> Aram: -req has to be used at propose time
[20:14] <Aram> but it generated this: https://codereview.appspot.com/6325057 which is wrong because it should only be two lines
[20:14] <niemeyer> Aram: After the merge proposal is created, it doesn't work anymore
[20:14] <niemeyer> Aram: (because Launchpad doesn't allow changing it)
[20:14] <Aram> can I delete a merge proposal and do it again from the same branch?
[20:14] <niemeyer> Aram: Yeah
[20:14] <niemeyer> Aram: That works fine
[20:15] <Aram> ok, thanks
[20:17] <niemeyer> np
[20:33] <niemeyer> Okay, lpad works again.. I'll go out for finding some food, and will be back to work on reviews
[22:50] <Aram> morning davecheney
[22:51] <davecheney> morning Aram
[22:51] <davecheney> hows it going ?
[22:51] <Aram> great
[22:56] <Aram> niemeyer: I believe three pieces of the puzzle should be in the queue now
[22:56] <niemeyer> Aram: Super, thanks!
[22:56] <niemeyer> davecheney: Heya
[22:57] <davecheney> howdy lads