/srv/irclogs.ubuntu.com/2012/10/15/#juju-dev.txt

davecheneyfwereade: thank you for your review06:09
fwereadedavecheney, np, the tweaks should be straightforward :)06:10
davecheneyfwereade: already done06:10
fwereadedavecheney, cool06:11
davecheneyfwereade: could I press you to expand on this statement06:19
davecheneyFWIW, I maintain that user-config is not the same thing as internal-config, and06:19
davecheneytrying to pretend that they are causes us nothing but pain. Not sure that topic06:19
davecheneyis really open for discussion, though...06:19
davecheneyi too feel the sting of too many battles over configuration06:19
fwereadedavecheney, I was trying to argue for it when we were originally having the config, and lost... but I'm still not quite sure *why* I lost, and so I can't figure out whether this situation qualifies as a reason to reopen the discussion (and indeed I'm not really sure whether I even want to...)06:21
davecheneyfwereade: so this is a diffrence between the environments configuration (passwords and shit that the agents need)06:21
davecheneyvs the pure minimum of config as a connection string06:22
fwereadedavecheney, I hadn't looked at it exactly that way -- in my eyes the problem is simply that we're trying to make a single parser do different things in different contexts06:25
davecheneyfwereade: well, IMO the wrinkle is not in the parser06:27
davecheneybut the way BootstrapConfig fucks with the config, then tries to parse it again06:28
TheMuemorning06:31
davecheneyhello06:31
fwereadedavecheney, ISTM that the only reason we're messing around with the config is so that we can pretend that the two situations are the same06:31
* davecheney nods06:31
fwereadedavecheney, and that that is the cause of the ugliness you reference06:31
fwereadedavecheney, so yeah, not exactly the parser itself, but a decision made in the parser's intellectual scope, if you like06:32
fwereadeTheMue, heyhey06:32
TheMuefwereade: heya, just reading your review06:32
fwereadeTheMue, probably all these things were covered in a prereq, but I didn't see that I'm afraid06:32
TheMuefwereade: no, they are not06:33
TheMuefwereade: one simply is a typo (c'n'p error) - the "dummy" in the ec2 config. i'll change it06:34
TheMuefwereade: the "default" is still in discussion and may be changed later. but i still prefer the word "default" so that one explicitely state in the config, that he wants the default behavior instead by leaving it empty06:36
TheMuefwereade: could you please describe your question regarding the change of the mode more? do you mean a change at runtime?06:37
fwereadeTheMue, feels wrong to me to have 2 names for the same state, but mileages may reasonably vary I guess06:37
fwereadeTheMue, yeah06:37
fwereadeTheMue, there's handling somewhere for things like attempts to change the env type, we can probably just do something similar there06:39
TheMuefwereade: we don't have two names, we only have, if somebody writes nothing, a default mode. but how would you otherwise explicit define that you want the default mode?06:39
fwereadeTheMue, honestly I don't think this should be user-configurable at all06:39
fwereadeTheMue, it feels like we're repeating the placement mistake06:39
TheMuefwereade: environments like ec2 have different modes, later maybe also a third one (one group per service). how to control that?06:40
fwereadeTheMue, ask the environ what it prefers?06:41
TheMuefwereade: the environment prefers nothing, it depends on your cloud size and services06:41
fwereadeTheMue, in that case, surely, it shouldn't have a default value: it's is an important decision that we should require an explicit value for06:43
fwereadeTheMue, but... well, look, we'll be dealing with a whole bunch of environments soon enough06:43
fwereadeTheMue, some of them have no concept of security groups in the first place, eg MAAS (as far as I know, not very up to date with progress)06:44
fwereadeTheMue, for those envs we will actually have to implement a local firewaller in every machine agent anyway, I think06:45
TheMuefwereade: yes, that's why niemeyer wanted a default, because you today can't tell if instance and global are ok for each environment. default is start, more modes to come.06:45
TheMuefwereade: exactly06:45
TheMuefwereade: but more important is your thought about changing it at runtime06:46
fwereadeTheMue, ok, if we implement local per-machine firewalling with iptables or something06:46
fwereadeTheMue, why would we need "global" or "instance" modes at all?06:47
fwereade<fwereade> TheMue, ok, if we implement local per-machine firewalling with iptables or something06:47
fwereade TheMue, why would we need "global" or "instance" modes at all?06:47
fwereadeTheMue, envs like MAAS will require that we add local firewalling, I think06:48
fwereadeTheMue_ ^^06:48
TheMue_fwereade: yes, but other envs maybe not06:48
fwereadeTheMue_, if we have local firewalling implemented for just one env we can just do that everywhere06:49
TheMue_fwereade: niemeyer wanted to keep it more abstract, so we later can support multiple envs with the same configuration machanism06:49
fwereadeTheMue_, and just have a thin layer of the global handling in the environments that need it on top of that06:49
TheMue_fwereade: you now change the whole firewalling concept06:49
TheMue_fwereade: please let's do it step by step06:49
fwereadeTheMue_, in that situation global-firewalling is just Wrong06:50
=== TheMue_ is now known as TheMue
fwereadeTheMue_, how so? AIUI I merely bring up what everyone has always known had to be done from the beginning06:50
fwereadeTheMue, I'm not demanding specific action :)06:50
fwereadeTheMue, but I am worried that we're adding a piece of totally transitory dev-relevant-only config06:51
fwereadeTheMue, that we will end up releasing and have to go through all the tediousness of deprecating06:51
TheMuefwereade: what's wrong with security groups for the ec2 env?06:51
TheMuefwereade: especially, if the user configures abstract modes like instace or global, the change from sec groups to iptables is transparent for the user06:53
fwereadeTheMue, they scale like shit, and ISTM that the global firewaller provides so flimsy a veil of kinda-security that , while it may allow us to scale, will not be used in any serious deployments06:53
fwereadeTheMue, what is the purpose of the global firewaller if not to paper over the security group issues?06:53
TheMuefwereade: using them today lets us sart while a change to a different implementation later is transparent for the user. so we ca release early with today demands but still have a comfortable solution for the future06:54
fwereadeTheMue, today's demands are "match python"; I though python had no global firewaller06:55
fwereadeTheMue,  the most transaprent thing is to do the best we can for the user given our capabilities06:55
fwereadeTheMue, if we want to target, right, now, users who want either scalability or security, but not both, then I guess we have no choice06:56
Guest2629TheMue, fwereade, davecheney: mornin'!06:56
TheMuefwereade: afaik it's just a simple temporary solution which later is easy to change. how fast could you set up iptables inclusive testing?06:56
TheMueGuest2629: morning06:56
=== Guest2629 is now known as rogpeppe
fwereadeTheMue, but if we're asking them to make a choice like *that* I think we should be upfront about it and have no default06:57
fwereaderogpeppe, heyhey06:57
TheMuefwereade: could you add this with your reasons to the review so that we could discuss it with niemeyer later. i'm not good as a man in the middle.06:58
fwereadeTheMue, clearly the decision to write the global firewaller has already been made, but when I review the code I lack the context behind the decisions and need to ask questions06:58
TheMuefwereade: yes, for sure06:59
fwereadeTheMue, I will expend my review, thanks06:59
TheMuefwereade: but i'm not very good in repeating your questions later to niemeyer. it's better when you ask him too06:59
fwereadeer extend06:59
fwereadeTheMue, absolutely, np :)07:00
TheMuefwereade: as long as this topic is in move changes are more simple than later07:00
rogpeppedavecheney: ping07:01
davecheneyrogpeppe: ack07:01
rogpeppedavecheney: i see your issue with my admin-secret fix, but i think that moving the admin-secret check earlier in the Bootstrap function would fix that.07:02
davecheneyrogpeppe: fair call07:02
davecheneywhat you go, ie the blowup at cloud init level07:02
rogpeppedavecheney: admin-secret can't be a required field in the config, unfortunately AFAICS07:02
davecheneywas where i started with07:02
davecheneyi got most of the way with making admin-secret rquired07:02
davecheneybut BoostrapConfig screwed me07:02
rogpeppedavecheney: the fundamental issue is that we don't want to push admin-secret into the state.07:03
davecheneyrogpeppe: what about just overwriting the secret with 'your secret isn't here'07:03
davecheneyconfig.New would be happy07:03
davecheneyand we _know_ we're never putting the real value into the state07:04
rogpeppedavecheney: i don't see how that's better than just not requiring the admin secret.07:04
davecheneyrogpeppe: i would argue the opposite07:04
davecheneybut not streniously07:04
rogpeppedavecheney: it's like we're saying, "yeah it's required, but only means something in some cases"07:06
davecheneyrogpeppe: can't we leave it out of config/config07:06
davecheneyand apply it at environs/ec2/config.go07:06
davecheneyit is already in there07:06
davecheneybut specfied as schema.Omit07:06
rogpeppedavecheney: no07:06
rogpeppedavecheney: because we need to be able to create an environment without admin-secret, i think07:07
rogpeppedavecheney: or... maybe that is a possibility, let me think07:08
davecheneyrogpeppe: yeah, can't we do that just by changing environs/ec2/config.go ?07:08
davecheneyrogpeppe: it would make admin-secret an environ specific value07:08
davecheneyunknownattrs, or whatever07:08
rogpeppedavecheney: no, we do need to be able to create an environment without admin-secret07:08
davecheneyrogpeppe: i don't understand that sentance07:09
davecheneyan environment == ec2 ?07:09
rogpeppedavecheney: environs == environs.Environ07:09
rogpeppes/environs/environment/07:09
davecheneyyeah, that is easy07:09
davecheneyremove the admin-secret atom from environs/config/config.go07:10
rogpeppedavecheney: but can we then open an ec2 environment with that config?07:11
davecheneyrogpeppe: depends if it is a valid ec2 config07:11
rogpeppedavecheney: i thought you were suggesting that the ec2 config required admin-secret07:12
davecheneyrogpeppe: indeed i am07:12
rogpeppedavecheney: so when an agent opens the environment, it must make up an admin secret?07:12
davecheneyrogpeppe: NFI07:13
davecheneyi dont know how it works ont he agents07:13
rogpeppedavecheney: it works ok. agents don't need admin-secret07:13
rogpeppedavecheney: only the client has the admin secret07:14
davecheneyyeah, they would use environs/config/config.go07:14
davecheneywhich only requires a generic config07:14
davecheneyso removing admin-secret from e/c/config.go is a valid suggestion07:14
rogpeppedavecheney: they've still got to get it through environs/ec2/config.go,m no?07:14
davecheneyrogpeppe: don't thye do that by getting the providor, then the provider can validate the config ?07:15
rogpeppedavecheney: i'm not sure i understand. where are you suggesting we make the check for admin-secret's presence?07:15
davecheneyenvirons/ec2/config.go07:16
davecheneyit is already there07:16
davecheneybut is optional atm07:16
rogpeppedavecheney: so what happens when we don't have an admin secret?07:16
davecheneynot a valid config07:16
rogpeppedavecheney: then how can an agent create an Environ?07:17
davecheneywell, you got me there07:17
davecheneyi guess they are screwed07:17
davecheneyand we are back to sq one07:17
rogpeppedavecheney: that's the reason i decided that the best place to put the check was where admin-secret is actually needed - in Bootstrap07:17
davecheneyrogpeppe: fair enough07:18
davecheneybest of a bad situation07:18
davecheneyoh07:18
rogpeppedavecheney: yeah. i'm very happy to entertain other possibilities, but i haven't seen a better one yet07:18
davecheneywhat about if admin secret in ec2 worked the same way as authorised-keys in e/c/config.go ?07:19
davecheneyie, if it were blank, they values are sourced from somewhere else07:19
davecheney(else == whereever the agents stash their secret value)07:19
rogpeppedavecheney: so you can't stash your secret in environments.yaml?07:19
davecheneyrogpeppe: i believe authorised-keys has a fallback mech07:20
davecheneyyaml first, disk second07:20
rogpeppedavecheney: yeah it does.07:20
rogpeppedavecheney: but... how does this help?07:20
davecheneydoes it solve the problem of the e/ec2/config.go being able to require admin-secret, but allow it to come from several locations07:20
rogpeppedavecheney: i think we've got a fundamental issue here - in some places in the system, we don't *have* an admin secret07:20
davecheneyagreed07:21
rogpeppedavecheney: ... where it's coming from another location or not07:21
davecheneyi think this it the point fwereade says he told us so07:21
rogpeppedavecheney: i agree with fwereade's point, but i don't mind this solution *too* much07:21
davecheneyrogpeppe: althoght, it does bail out after writing the data to the s3 store07:22
davecheneywhich means you have to use juju destroy-environment before trying again07:22
rogpeppedavecheney: yeah, that's a bug07:23
rogpeppedavecheney: as it suggested earlier, the check should come further up07:23
rogpeppes/it/i07:23
rogpeppedavecheney: i'll fix that07:24
davecheneysweet07:24
davecheneyafk for a while07:24
rogpeppedavecheney: afk?07:24
fwereaderogpeppe, away from keyboard07:28
rogpeppefwereade: thanks07:28
TheMuefwereade: thx for your notes. both sound very good to me.07:59
fwereadeTheMue, cool, cheers :)07:59
Arammoin.08:32
TheMueAram: moin moin08:33
TheMue*: Always the same question. *sigh* When having a chain of CLs which shall bei updated to trunk, then after merge only commit and push or also lbox propose of already submitted pre-requesites?08:37
TheMuerogpeppe, fwereade: any hint ^^?09:13
fwereadeTheMue, I generally don't repropose unless there were conflicts09:13
TheMuefwereade: so only commit and push? i twice made it wrong in the beginning and had too much code in the reviews09:15
fwereadeTheMue, it seems to work ok for me09:15
fwereadeTheMue, I only get too much code when I forget a -req09:16
TheMuefwereade: thx, i'll make a note to keep it in mind next time. ;)09:16
AramTheMue: no need to repropose before submitting.10:10
Aramjust make sure you merge trunk.10:10
Aramso you merge the prereq10:10
niemeyerMorning jujuers!12:08
fwereadeniemeyer, heyhey12:13
niemeyerfwereade: Heya12:13
niemeyerfwereade: Good weekend?12:13
fwereadeniemeyer, very nice thanks -- saw cath's cousins and their new baby :)12:13
niemeyerfwereade: Oh, nice12:14
fwereadeniemeyer, and also stacked up a few reviews which will, I hope, lead to interesting conversations if not necessarily immediate progress12:14
niemeyerfwereade: /me is all sensitive about "new babies" :-)12:14
fwereadeniemeyer, he's about 4 months old and smiley and dribbly and cute12:15
niemeyerfwereade: LOL12:15
TheMueniemeyer: morning and thx for your comment12:22
niemeyerTheMue: Heya, np12:23
* TheMue 's small baby - she is now 16 yrs old ;) - has her first archery tournament this afternoon. we'll all watch her.12:23
niemeyerTheMue: Wow, very nice. Good luck there!12:27
fssniemeyer: morning :-)12:28
niemeyerfss: Heya12:35
TheMueniemeyer: thank you, I will tell her when she's back from school. just had to watch our older one leaving home with her driving teacher. one of the last lessons before driving test.12:36
fwereadelate lunch, bbiab12:48
niemeyerfwereade: Enjoy12:49
niemeyerfwereade: https://codereview.appspot.com/6651060/ is waiting for you13:15
fwereadeniemeyer, cheers13:38
niemeyerAram: ping14:11
fssniemeyer: ping14:22
niemeyerfss: hi14:22
fssdid you see that iamtest cl?14:23
niemeyerfss: Still churning CLs pushed over the holiday and/or weekend14:24
fssniemeyer: oh, right :-) please let me know if I can help with anything14:27
niemeyerfss: Cheers!14:31
niemeyerfwereade_: ping14:33
fwereade_niemeyer, pong14:34
niemeyerfwereade_: Yo14:34
niemeyerfwereade_: Just started looking at relation-lifecycles14:34
fwereade_niemeyer, oh yes14:35
niemeyerfwereade_: The N=100k indeed feels troublesome there14:35
fwereade_niemeyer, does my wolly justification for an end-run around txn hold any water?14:36
fwereade_s/wolly/wooly/14:36
niemeyerfwereade_: I'm thinking too.. in principle it sounds doable14:36
niemeyerfwereade_: But even better would be to have these settings collected in a more distributed way14:37
niemeyerfwereade_: Even if we do RemoveAll, 100k is still a lot14:37
fwereade_niemeyer, not really sure where it makes sense to distribute it to -- a CA that just does a bit of work every few seconds might do the trick I guess :)14:38
niemeyerfwereade_: Yeah, but that sounds worryingly hand-wavy14:39
niemeyerfwereade_: We've decided to leave settings around, and that seems to work well, but is there any real point where we can say settings for a unit leaving aren't relevant anymore?14:40
fwereade_niemeyer, I don't really think we can at the unit level, can we? We could make subordinate relations be cleaned up incrementally by their own units14:41
fwereade_niemeyer, but that doesn't help with the big ugly case14:41
niemeyerfwereade_: Right14:42
fwereade_niemeyer, the CA case doesn't seem too bad to me though -- it's just one extra op to create a document with the relation's ID (or I guess just directly its scope key prefix)14:45
fwereade_niemeyer, the CA then just removes a few matching docs at a time until it runs out, and then deletes the "droppings" doc14:45
niemeyerfwereade_: The extra doc for?14:46
fwereade_niemeyer, or adds it into an existing doc, sorry, haven't considered tradeoffs here14:46
niemeyerfwereade_: Ah, I see14:47
fwereade_niemeyer, brb phone14:47
niemeyerfwereade_: Cool, thinking meanwhile14:47
niemeyerfwereade_: Yeah, you're right.. the CA may be a good idea after all14:54
fwereade_niemeyer, however I'd prefer not to switch focus to that just now14:55
fwereade_niemeyer, do you think that an interim solution of either monster-transaction, or remove-all-settings-before-transaction, would be plausible?14:56
niemeyerfwereade_: Not greatly comfortably with either15:01
* fwereade_ is crestfallen15:02
* niemeyer looks in the dictionary to see what fwereade_ is15:02
fwereade_niemeyer, how about deleting no settings, but creating a droppings doc for future use?15:03
* niemeyer laughs at the "-- said of a horse." part of the explanation15:03
* fwereade_ doesn't know which dictionary niemeyer has15:04
niemeyerfwereade_: That was from gcide15:09
fwereade_niemeyer, ah nice :)15:10
niemeyerfwereade_: Sounds good regarding injecting doc into cleanups collection15:10
fwereade_niemeyer, great15:11
fwereade_niemeyer, and that's a nicer name than droppings15:11
niemeyerfwereade_: {Id: bson.ObjectId, Type: "settings", Prefix: "r#..."}15:12
fwereade_niemeyer, LGTM15:12
niemeyerfwereade_: /Type/Kind/ perhaps15:12
fwereade_niemeyer, +115:12
niemeyerfwereade_: We'll need the CA soon enough anyway15:12
niemeyerfwereade_: To recover lost transactions15:12
fwereade_niemeyer, indeed15:13
niemeyerI'll step out for lunch.. back in a bit to continue on reviews15:14
fwereade_wrtp, Aram, niemeyer: is http://paste.ubuntu.com/1281389/ expected/familiar? just merged...16:24
wrtpfwereade_: i haven't seen that before16:25
wrtpfwereade_: ISTM that's probably a consequence of recent fw changes, renaming "default" to ""16:25
fwereade_wrtp, yeah, I guess we will see a fix tomorrow16:27
niemeyerI guess no cloud consistency call today..16:31
niemeyerfwereade_: Can you paste the link again please?16:31
fwereade_niemeyer, http://paste.ubuntu.com/1281389/16:31
niemeyerfwereade_: Cheers16:31
niemeyerfwereade_: I have a try at fixing it quickly16:32
niemeyerSo we don't stick with a broken trunk16:32
niemeyerrobbiew: Cloud consistency call today?16:35
robbiewniemeyer: no idea...I know I won't be attending (at ODS)16:35
robbiewniemeyer: I would say "no"16:36
robbiew...as I don't know who will be around to attend16:36
niemeyerrobbiew: Cheers16:36
robbiew;)16:36
wrtpi'm off for the day. see y'all tomorrow.17:21
niemeyerwrtp: Enjoy the evening17:28
niemeyerfwereade: https://codereview.appspot.com/6699044 .. nothing exciting, luckily17:28
=== andrewsmedina is now known as almanegra
fwereade_niemeyer, ping20:15
fwereade_niemeyer, re service-lifecycles EnterScope txns -- I suspect the logic *is* faulty, but the mistake is not, I think, in the inability of a unit to re-enter a scope -- I think that is a feature20:17
fwereade_niemeyer, thoughts?20:17
niemeyerfwereade_: Yo20:21
fwereade_niemeyer, heyhey20:21
niemeyerfwereade_: In principle I don't see a great reason to allow it, but at the same time it feels bad to have faulty logic that doesn't really bring any benefit and misbehaves in such cases20:22
fwereade_niemeyer, to expand: when a unit leaves the scope, it has run the relation-broken hook and represented to the charm that its participation in the relation is entirely done20:22
niemeyerfwereade_: Sure, but in theory it could happily re-enter the scope by reporting to the charm that it's entering again20:23
fwereadeniemeyer, I am 100% with you on the faulty logic, I will rework and explain nicely, unless you feel there is something deeply wrong with the approach -- I think that it is possible to determine the various situations cleanly20:24
fwereadeniemeyer, which feeds into ErrScopeDying20:24
fwereadeniemeyer, which should maybe be ErrScopeClosed -- indicating that either the unit has already left and can't come back, or just that the scope is not open to new members...20:25
fwereadeniemeyer, ...but in either case, the correct response for the uniter is to ignore the error and all further references to that relation20:26
niemeyerfwereade: Is there anything that misbehaves today if we enter the scope again, as far as the the state/watchers/etc are concerned?20:26
fwereadeniemeyer, I don't think so, no20:27
niemeyerfwereade: So I don't see yet the motivation to be actively preventing the unit from entering the scope. It feels like a property we can easily explore for good reasons in the future, and don't have to think much right now20:28
fwereadeniemeyer, the intent is that a unit "re-entering" a scope should basically do nothing except possibly update the settings if the private address has changed20:29
fwereadeniemeyer, that's from the API perspective20:30
niemeyerfwereade: Yep, sounds fine20:30
niemeyerfwereade: I'm talking about leaving and entering, though20:30
niemeyerfwereade: I don't see a reason to actively disallow it20:31
fwereade_niemeyer, gaah, sorry20:31
fwereade_niemeyer, I saw/said:20:31
fwereade_<niemeyer> fwereade: Yep, sounds fine20:31
fwereade_<fwereade> niemeyer, from the model perspective I think it is sensible to make this part of the model agree with the part which thinks that relation-broken is the dividing line between "in the relation" and "left, never ever coming back"20:31
fwereade_niemeyer, I would favour consistency for the time being and consider experimentation later20:33
niemeyerfwereade> niemeyer, that's from the API perspective20:33
niemeyer<niemeyer> fwereade: Yep, sounds fine20:33
niemeyer<niemeyer> fwereade: I'm talking about leaving and entering, though20:33
niemeyer<niemeyer> fwereade: I don't see a reason to actively disallow it20:33
niemeyerfwereade_: I don't see the reason to building onto the model that "left, never ever coming back"20:33
fwereade_niemeyer, I understand that to be the meaning of the relation-broken hook20:34
fwereade_niemeyer, I had a feeling we did state somewhere that it would be called once per relation and it would be the last thing called20:34
fwereade_niemeyer, if that is the case (er, I'll see if I can find anywhere...) then I think it's best to be consistent20:35
niemeyerfwereade_: We're getting into shady areas20:36
niemeyerfwereade_: We never discussed what happens when you start a unit in a different machine, for example20:36
niemeyerfwereade_: Do we Leave and then Join from the other machine? Etc20:36
niemeyerfwereade_: I'm not suggesting we should specify any of that right now, though20:36
niemeyerfwereade_: I'm just suggesting we don't start hardcoding unnecessary behavior that closes doors we haven't talked about yet20:37
fwereade_niemeyer, I shall meditate upon this :)20:38
niemeyerfwereade_: Hehe :)20:38
fwereade_niemeyer, hm, actually, on https://juju.ubuntu.com/docs/charm.html it says "Runs when a relation which had at least one other relation hook run for it (successfully or not) is now unavailable"20:39
fwereade_niemeyer, that is not really a good description IMO but it does if anythig carry connotations of potentially being rerunnable20:39
niemeyerfwereade_: Overall, the lifecycle branches look like going in a pretty good direction.. the leave scope one has a few things we'll have to discuss too to make sure we understand what's up, but it's all getting together nicely20:39
fwereade_niemeyer, yeah, I thought there'd probably be a bit of discussion for each of them20:40
fwereade_niemeyer, the one that sits on top of them all does make me happy, though20:40
niemeyerfwereade_: It looks lke surprisingly small amount of logic, thankfully20:40
fwereade_niemeyer, yeah, it's not too bad20:41
fwereade_niemeyer, there may be a simplification waiting to get out but I haven't got round to implementing it against that tree yet so it could yet surprise me20:42
niemeyerfwereade_: Cool20:43
niemeyerfwereade_: I'll step out to do some end-of-afternoon errands.. back later20:44

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!