[08:51] <rog> morning!
[09:23] <rog> fwereade_: i wonder if we should think about logging a little deeper
[09:23] <rog> fwereade_: when i played with juju, it seemed to me that it didn't do logging very well, and perhaps we can do better.
[09:24] <fwereade_> rog, heh, I won't disagree, but I think I've surgically removed my credibility on all logging-related issues for a little while :p
[09:24] <rog> fwereade_: :-)
[09:25] <fwereade_> rog, what were you thinking of though?
[09:26] <rog> fwereade_: i was wondering about a way to amalgamate many (distributed) logs into a unified view, so one can see what's happening throughout the system in a moderately timely way
[09:26] <fwereade_> rog, how would that differ from `juju debug-log`?
[09:27] <rog> fwereade_: well, current debug-log goes through zk, right? which isn't great.
[09:27] <rog> fwereade_: and debug-log only includes juju log messages
[09:28] <rog> fwereade_: but it would be good to be able to see other logs too (preferably in a unified view), e.g. stuff in /var/log
[09:28] <fwereade_> rog, hmm, interesting -- I think we kinda expect to hand that level of detail off to a subordinate charm if it's wanted
[09:28] <fwereade_> rog, I agree that zk is not really the ideal log server :p
[09:29] <fwereade_> rog, I do think that it would actually be pretty cool if we *did* basically have rsyslog integrated somehow
[09:29] <rog> fwereade_: i'm not sure that i want to go to the trouble of deploying a subordinate charm in order to see log messages without ssh'ing to a node. but maybe it's ok.
[09:30] <rog> fwereade_: part of the problem is that you don't want to shove log messages into the system unless you want them - there's a lot of potential wasted bandwidth if you do
[09:30] <fwereade_> rog, this use case immediately makes me think that it might make sense to have "env-level" subordinate charms, but I haven't really kept up on that so maybe someone already thought of it
[09:31] <rog> fwereade_: maybe could think of the whole env as one "service"...
[09:31] <fwereade_> rog, (that is, that it might be nice to be able to auto-add a particular subordinate (say, a logging one) to *all* services)
[09:32] <rog> yeah
[09:32] <fwereade_> rog, yeah, something like that
[09:32] <rog> except that means that services can overlap so maybe not great
[09:32] <fwereade_> rog, yeah, we'd have to be careful
[09:32] <fwereade_> rog, and I guess it's a bit speculative at this stage
[09:32] <fwereade_> rog, worth keeping in mind though
[09:36] <rog> fwereade_: yeah. i think it's really really important to have good logging support in a distributed system.
[09:40] <fwereade_> rog, no argument here :0
[12:33] <niemeyer> Greetings!
[12:41] <rog> niemeyer: hiya
[12:46] <niemeyer> Oops.. will need to give some attention downstairs.. biab
[13:27] <niemeyer> Woohay
[13:29] <TheMue> niemeyer: ??? What happened?
[13:30] <niemeyer> TheMue: https://plus.google.com/u/0/107994348420168435683/posts/PZmwXqzbyGi
[13:32] <TheMue> niemeyer: ah, great. our older daughter has an e-piano. she's learning it autodidactic.
[13:33] <niemeyer> TheMue: Very nice!
[13:33] <niemeyer> TheMue: I'm half and half.. I have some great friends which are amazing piano players/teachers.. that's what motivated me to get into it
[13:34] <niemeyer> TheMue: So I've been having some classes in addition to studying by myself
[13:35] <TheMue> niemeyer: sounds good. i wanted to learn it too. but i never started with it. insted i'm drifted into writing (and a bit into drawing again).
[13:38] <niemeyer> TheMue: I've started to learn music itself, rather than the instrument, but I slowly got to appreciate the instrument itself
[13:38] <TheMue> niemeyer: fantastic
[14:42] <niemeyer> fwereade_: ping
[14:42] <fwereade_> niemeyer, pong, how are you?
[14:42] <niemeyer> fwereade_: Heya
[14:42] <niemeyer> fwereade_: Excellent.. great Monday so far :)
[14:42] <niemeyer> fwereade_: How about yourself?
[14:43] <fwereade_> niemeyer, pretty good thanks
[14:44] <fwereade_> niemeyer, so, what can I do for you?
[14:44] <niemeyer> fwereade_: Just wanted to see if there's anything you'd like to catch up on regarding the pipeline/branch/conversation/whatever
[14:44] <fwereade_> niemeyer, ah, yeah, let me think a mo
[14:44] <fwereade_> niemeyer, just an am-i-on-crack check really
[14:45] <niemeyer> fwereade_: I don't think you are, but go on
[14:45] <fwereade_> niemeyer, the motivation is that the python hook commands are kinda nasty, because there are far too many layers of plumbing
[14:46] <fwereade_> niemeyer, and that we'll get a much simpler situation if we make the client as thin as possible
[14:46] <niemeyer> fwereade_: Motivation for ... (just to make sure I get what you're really saying)
[14:46] <niemeyer> fwereade_: Ah, no need, I see already
[14:47] <fwereade_> niemeyer, basically, implementing a single jujup command which we specialise purely by symlink
[14:47] <niemeyer> fwereade_: That sounds fine for sure
[14:47] <niemeyer> fwereade_: I'd just prefer to name it as something else of your choice
[14:47] <fwereade_> niemeyer, not jujup?
[14:47] <niemeyer> fwereade_: jujud and jujup are graphically extremely close
[14:48] <fwereade_> niemeyer, well, I wouldn't expect anyone to ever call jujup directly, but good point nonetheless
[14:48] <niemeyer> fwereade_: And also in the way we pronounce it
[14:48] <niemeyer> fwereade_: Naming it something else could avoid it
[14:48] <fwereade_> niemeyer, yeah, another good point
[14:48] <niemeyer> fwereade_: jujut (tools?)
[14:49] <niemeyer> fwereade_: ?
[14:49] <fwereade_> niemeyer, jujut sounds good to me
[14:49] <fwereade_> niemeyer, although, hah, pronunciation vs jujud
[14:49] <niemeyer> Hmm.. even though I guess my pronounciation argument falls short
[14:49] <niemeyer> Yeah :)
[14:49] <rog> i wouldn't mind a longer name
[14:50] <niemeyer> fwereade_: jujuh?
[14:50] <rog> noone's ever going to type it...
[14:50] <niemeyer> hooks
[14:50] <niemeyer> rog: That goes both ways..
[14:50] <fwereade_> niemeyer, pronunciation again :p
[14:50] <niemeyer> fwereade_: jujuc?
[14:50] <rog> jujud is good because there's an existing convention
[14:50] <niemeyer> fwereade_: (from client)
[14:50] <fwereade_> niemeyer, I was thinking that and liking it just-about-enough
[14:50] <fwereade_> niemeyer, yeah, same reasoning
[14:51] <rog> jujucallback :-)
[14:51] <niemeyer> fwereade_: Cool, +1..
[14:51] <fwereade_> rog, we're just extending the noble and hallowed 1-char extension
[14:51] <fwereade_> ...tradition
[14:51] <rog> hmm
[14:51] <niemeyer> rog: 1 char is fine, IMO, precisely because no one has to remember the command name.. we can deal with a short name for ourselves
[14:51] <rog> fwereade_: for me, jujud means juju daemon. jujuc means... nothing.
[14:51] <niemeyer> rog: client.. that's what it means
[14:51] <rog> jujuice
[14:51] <TheMue> juice *scnr*
[14:52] <niemeyer> rog: Oh man.. shhhh
[14:52] <niemeyer> rog: Let's reserve that one for something nice :)
[14:52] <TheMue> rog: h5
[14:52] <rog> yeah, jujuc's ok.
[14:52] <fwereade_> motion carried :)
[14:52]  * TheMue likes jujuc too
[14:53] <rog> only thing is, it's not really the juju client. it's part of the juju infrastructure. the client is really the one doing juju deploy etc.
[14:54] <TheMue> niemeyer: I'm now doing the Agent stuff. So I need an idea on how you wonne have watches implemented.
[14:54] <fwereade_> rog, but nobody will ever know c is short for client unless we tell them, and it can be out little secret :p
[14:54] <rog> jujuhook might be a little better, but it's not the hook itself, it's something inside the hook
[14:55] <rog> TheMue: i'll paste you what we wrote down in the lobby that time
[14:55] <TheMue> rog: great, thx
[14:56] <niemeyer> rog: Cheers
[14:57] <rog> http://paste.ubuntu.com/869964/
[14:58] <rog> it's just the client code, but i think that was the important bit
[14:59] <rog> i'm wondering why Stop isn't just a close of the event channel, but there's probably a good reason
[14:59] <rog> ah yes
[14:59] <TheMue> rog: yep, most stuff is clear. Changed delivers just a bool?
[15:00] <rog> TheMue: changed delivers anything you like
[15:00] <rog> TheMue: the idea is it can deliver deltas
[15:00] <rog> TheMue: unlike the usual zk channels
[15:00] <rog> TheMue: each watcher might have a different event type
[15:01] <TheMue> rog: ok, so the watch isn't one type, they are individual types depending on what the watch
[15:01] <TheMue> rog: yep
[15:01] <rog> TheMue: yeah, i think so
[15:03] <TheMue> rog: so i need to start a goroutine reading the zk-watcher events, transform then, and once it closes i'll send the Stop and end the goroutine
[15:03] <TheMue> rog: pretty simple
[15:04] <rog> TheMue: i don't think Close should send on Stop.
[15:04] <rog> TheMue: i *think* that Stop will be global to the state.
[15:05] <rog> TheMue: so when the state itself is closed, it closes the Stop channel, which then propagates through to all watchers
[15:06] <TheMue> rog: ok, so far the state doesn't knows a Stop(). and I've got to think about dependencies.
[15:06] <rog> TheMue: so far State doesn't have Close AFAIK
[15:07] <TheMue> rog: yep, that's what I meant
[15:07] <rog> ah yes
[15:19] <niemeyer> TheMue: Note that the watch method will generally be on *State or some more specific type
[15:20] <TheMue> niemeyer: yep
[15:21] <niemeyer> rog: Why do we need two channels again?
[15:21] <rog> niemeyer: i'm just trying to remember why
[15:22] <niemeyer> rog: We need a way to notify the goroutine itself that it's being stopped
[15:22] <rog> niemeyer: it depends how we want to propagate the state close
[15:22] <TheMue> so, have to leave until this evening, have to drive my older daughter to a workshop. thankfully she's making her driver license soon.
[15:22] <niemeyer> rog: But I can't see why we'd make that public
[15:23] <TheMue> niemeyer, rog : please keep on discussing, will read it later
[15:23] <rog> TheMue: ok, see you tomorrow
[15:23] <niemeyer> TheMue: LOL
[15:34] <niemeyer> TheMue, rog, fwereade_: http://paste.ubuntu.com/870023/
[15:34] <niemeyer> Erm.. type is misnamed
[15:35] <niemeyer> Fixed: http://paste.ubuntu.com/870026/
[15:36] <fwereade_> niemeyer, looks good to me
[15:36] <rog> niemeyer: does FooWatcher.Change get closed?
[15:36] <niemeyer> rog: Ah, it should.. at the bottom of break.. fixing it
[15:36] <niemeyer> s/break/for
[15:37] <niemeyer> TheMue: With rog's point: http://paste.ubuntu.com/870029/
[15:37] <rog> niemeyer: ah, that break should break two levels too
[15:38] <niemeyer> rog: Hmm.. could have to indeed
[15:38] <rog> niemeyer: might as well defer(close(w.Change)) and return
[15:38] <niemeyer> rog: I'll fix in a better way
[15:38] <niemeyer> rog: http://paste.ubuntu.com/870030/
[15:39] <niemeyer> Hard to get it wrong like that
[15:40] <niemeyer> Argh.. Unity is driving me nuts with the latest ALT-<anything> sequence bringing up the hud at all times
[15:40] <rog> niemeyer: i'd like to see it filled out. for instance, what if  the zk watch case is sending on FooWatcher.Change when FooWatcher is stopped?
[15:40] <niemeyer> rog: What's the problem with that?
[15:41] <niemeyer> rog: In fact, how can it possibly happen? Nothing sends on Change if FooWatcher is stopped
[15:41] <rog> niemeyer: there's a potential deadlock if the thing reading on FooWatcher.Change is the one calling Stop
[15:42] <niemeyer> rog: Sorry, still don't get it.. a) Only one of these may be done at a time.. b) Change is closed on Stop
[15:42] <rog> niemeyer: ok, here's the scenario:
[15:43] <rog> niemeyer: watcher is started. it quickly generates a few events and blocks trying to send on FooWatcher.Change
[15:43] <rog> niemeyer: the thing that started the watcher decides it doesn't need it any more and calls Stop on it
[15:43] <rog> niemeyer: the watcher will block forever
[15:43] <niemeyer> rog: Definitely not
[15:43] <niemeyer> 		case <-w.tomb.Dying:
[15:43] <niemeyer> 			return
[15:43] <niemeyer> 		case <zk watch>:
[15:43] <niemeyer> 			...
[15:43] <niemeyer> 		}
[15:43] <rog> niemeyer: because the thing that started the watcher never received the events it's blocking on
[15:44] <rog> niemeyer: what's inside the <zk watch> case?
[15:44] <rog> niemeyer: you'll have to guard the send to FooWatcher.Change too, i think
[15:45] <niemeyer> rog: <-w.tomb.Dying is all that's necessry
[15:45] <niemeyer> necessary
[15:45] <niemeyer> rog: If there are other selects, the same must be done of course
[15:45] <niemeyer> rog: Check out the blog post for a detailed explanation: http://blog.labix.org/2011/10/09/death-of-goroutines-under-control
[15:46] <rog> niemeyer: it's not just in the case of selects, it's any channel operation.
[15:46] <niemeyer> rog: Which are done inside selects? :-)
[15:46] <rog> niemeyer: but that's why i wanted you to fill out the example
[15:46] <rog> niemeyer: not necessarily...
[15:46] <rog> niemeyer: but in this case, they'll have to be.
[15:46] <niemeyer> rog: LOL
[15:47] <niemeyer> rog: Yeah.. they have to be if we don't want to consciously introduce bugs
[15:47] <rog> niemeyer: one thing this example doesn't address is what we do when State itself is closed
[15:48] <niemeyer> rog: Indeed.. it must Stop too
[15:48] <niemeyer> rog: Actually, not really
[15:48] <niemeyer> rog: That was the idea I had, but you correctly pointed out that this is a job for the loop
[15:49] <niemeyer> If the state is closed, zk is closed, and the loop will error out..
[15:49] <niemeyer> If we use tomb.Fatal, as per the post, it will eventually reach the guy calling Stop() too
[15:50] <niemeyer> Hmm.. I'll have to break out to pick up Ale for lunch..
[15:50] <niemeyer> rog: biab, in case you want to continue on that, but I think we have a reasonable plan
[15:50] <rog> niemeyer: any chance you might be able to review some of my merge requests this week BTW?
[15:50] <niemeyer> rog: Yep, good chances ;)
[15:50] <rog> niemeyer: that would be nice
[15:51] <rog> niemeyer: (i've been trying not to badger you :-])
[15:51] <niemeyer> rog: That's appreciated, as I've been reviewing tons of branches pretty much every day
[15:51] <niemeyer> I'll give some priority to the store early this week still, though
[15:51] <niemeyer> biab
[15:51] <rog> ok
[16:57] <rog> niemeyer: are you back yet?
[17:03] <robbiew> niemeyer: around for juju call?
[17:03] <niemeyer> robbiew: Here
[17:03] <niemeyer> robbiew: Is it happening?
[17:04] <robbiew> niemeyer: yes
[17:04] <robbiew> on with sabdfl now :)
[17:04] <robbiew> just us
[17:04] <niemeyer> robbiew: Sorry, will be on in a sec
[17:06] <niemeyer> rog: I am, kind of :)
[17:06] <robbiew> hazmat: of course you're welcome too ;)
[17:06] <rog> niemeyer: ok, when you have a moment, wouldn't mind a chat about tomb
[17:14] <niemeyer> rog: Sounds good
[17:25] <hazmat> sigh
[17:25] <hazmat> think i missed it
[17:25] <hazmat> lunch'd out
[18:08] <rog> i'm off for the evening. see you tomorrow.
[18:13] <niemeyer> rog: I'm off the call, but I guess you're gone
[21:13] <niemeyer> Alright.. skeleton in place, HTTP server works.. need to implement the actual actions.. maybe today still!
[21:13] <niemeyer> But now it's time for some outsiding
[21:13] <niemeyer> Laters
[21:38] <rog> niemeyer: yes, sorry. tomorrow...