[01:24] davecheney: Yo! [01:24] davecheney: Are you staying around for a bit? [01:25] niemeyer: yy [01:29] davecheney: I guess the answer is no.. :) [01:29] yeah, i'm arround [01:29] davecheney: Ah, hey :) [01:30] davecheney: I'm about to get live tests working, and have a set of changes in the pipeline.. if you'll be around for a bit, we can quickly interact on those [01:30] niemeyer: excellent [01:30] davecheney: It's all pretty agreeable stuff [01:30] let me know what you need me too do [01:30] davecheney: Btw, I have a connected RPi turning a led on and off here :-) [01:31] nice [01:31] niemeyer: http://dave.cheney.net/2012/09/25/installing-go-on-the-raspberry-pi [01:31] Boards with GPIO <3 [01:31] finally wrote it up last night [01:31] davecheney: Oh, sweet [01:33] davecheney: Very cool [01:33] davecheney: I got Go working yesterday too [01:33] odessa(~/devel/src/launchpad.net/juju-core/testing) % go test ../cmd/juju [01:33] --- FAIL: TestPackage (0.00 seconds) [01:33] mgo.go:65: exec: "mongod": executable file not found in $PATH FAIL [01:33] FAIL launchpad.net/juju-core/cmd/juju 0.116s [01:33] ^ with your review comments applied [01:33] ? [01:34] davecheney: I don't understand.. I didn't suggest any changes to the path or whatever? [01:34] niemeyer: nah, this is just responding to your comments in ,https://codereview.appspot.com/6560045 [01:34] davecheney: Ah, ok, just showing how it looks like.. super, thanks [01:34] davecheney: Phew.. I thought it was all broken :) [01:35] nah, just testing that it does work as expected if you don't have mgo installed [01:36] davecheney: Looks good [01:38] ty [01:55] Amazon is taking ages to allocate a machine apparently.. and S3 is failing.. not a good night to test it live [01:55] Anyway, I'll break the CLs down [02:00] https://codereview.appspot.com/6572050 [02:02] niemeyer: Proposal: https://code.launchpad.net/~dave-cheney/goetveld/002-add-darwin-termios-support/+merge/126362 [02:02] for some reason goetveld doesn't create CL's, only LP reviews [02:03] niemeyer: try another zone ? [02:03] i ofter use southeast-1 [02:04] https://codereview.appspot.com/6573050 [02:09] https://codereview.appspot.com/6579043 [02:25] error: Failed to update merge proposal log: EOF [02:25] Maybe it's my connection that is in a bad state [02:25] https://codereview.appspot.com/6567048 [02:25] davecheney: Last for the day ^ [02:26] niemeyer: i haen't seen that one for a while, the EOF [02:26] reviewing last CL now [02:37] Shower.. will pass by in a bit.. [02:37] davecheney: Thanks for the reviews! [02:39] kk [02:39] niemeyer: no worries [02:50] davecheney: Okay, I guess I'll merge this stuff [02:50] davecheney: So people can make progress tomorrow on top of it [02:50] davecheney: How's config-get going there? [02:51] niemeyer: dunno, i should pull that branch and see if it compiles [02:51] davecheney: I mean the stuff that you were pushing forward since the sprint [02:51] davecheney: Or what have you been up to? [02:53] i've been on swap days since the sprint [02:53] Today as well? [02:54] davecheney: ? [02:56] niemeyer: getting yelled at from downstairs, back in a while === davecheney is now known as davecheney-a6faf [05:56] morning [05:57] morning === davecheney-a6faf is now known as davecheney-bb1ad [06:57] mornings [07:01] fwereade: http://paste.ubuntu.com/1228005/ [07:01] did I just break transactions ? [07:02] davecheney-bb1ad, huh, not seen that before [07:02] davecheney-bb1ad, er, maybe? :) [07:02] fwereade: shitter [07:02] davecheney-bb1ad, but I actually have no idea [07:02] davecheney-bb1ad, what did you do? :) [07:03] just what I typed [07:03] that is trunk [07:03] fwereade: morning [07:03] TheMue, heyhey === davecheney-bb1ad is now known as davecheney [07:03] davecheney, bah, I'll try myself [07:05] https://bugs.launchpad.net/juju-core/+bug/1056642 [07:05] for posterity [07:21] mroning rogpeppe [07:22] fwereade: hiya [07:22] rogpeppe: morning [07:24] fwereade: running those commands in a loop [07:52] fwereade: another failure [07:52] davecheney, oh dear :( same one? [07:52] different [07:52] http://paste.ubuntu.com/1228059/ [07:53] davecheney, hmm, is that just ec2 taking too long? [07:54] possibly, but we should be able to cope with that [07:55] lucky(~/src/launchpad.net/juju-core/cmd/juju) % juju destroy-environment [07:55] lucky(~/src/launchpad.net/juju-core/cmd/juju) % bash -x stress.bash [07:55] + set -e [07:55] + true [07:55] + juju bootstrap --upload-tools [07:55] + juju deploy mongodb [07:55] error: no instances found [07:55] that happened in 2 minutes [07:55] less [07:55] our timeout is 10 minutes [08:01] lucky(~/src/launchpad.net/juju-core/cmd/juju) % bash -x stress.bash [08:01] + set -e [08:01] + true [08:01] + juju bootstrap --upload-tools [08:01] + juju deploy mongodb [08:01] + juju status [08:01] error: instance i-bbd62fc6 for machine 1 not found [08:02] https://bugs.launchpad.net/juju-core/+bug/1056679 [08:08] bizarre :( [08:12] ha! [08:12] catched the service bug in firewaller, phew [09:07] hmm, still sometimes have a race *sigh* [09:49] rogpeppe: http://paste.ubuntu.com/1228172/ [09:49] ^ evil, or very evil ? [09:50] moin [09:50] davecheney: why? [09:51] the py tests fro juju ssh were using this weird mocking thing [09:51] I needed a way to catch the ssh arguments, without running the command [09:52] davecheney: the way i was doing that before was by changing $PATH [09:52] davecheney: is that not reasonable? (it doesn't involve changing the actual code at all) [09:53] i guess I want to test that the code is generating the correct invovation [09:53] davecheney: i realise it's more work though [09:53] so I need the args [09:53] Aram: hi [09:53] davecheney: you can get the args. just make the ssh into a shell script that saves the args. [09:54] rogpeppe: sure, that would work, but then I have to write that stuff to disk, blah blah [09:54] davecheney: i thought we were doing something like this anyway [09:55] davecheney: we do, in TestSSHErrors [09:55] fair point [09:56] davecheney: i don't think it's much more hassle to get the script to save its args. [09:56] davecheney: or just use a different script template [09:56] davecheney: i'm not that keen on mocking Command. [09:57] fair enough' [09:59] davecheney: just change the script to do something like this: for i in "$@"; do echo $i >> {{.Dir}}/args; done [10:34] fwereade: do you find the "JUJU:DEBUG watcher: loading new events from changelog collection..." messages useful? [10:34] fwereade: i'm inclined to delete them - they're so noisy. [10:34] rogpeppe, sometimes; I would be quite keen on a package-specific debug flag that defaults to false though [10:35] fwereade: how would you turn it on? [10:35] rogpeppe: good catch about nil != empty on golang-dev. I was like "wtf is this" but didn't say anything because I assumed I missed some subtlety and I was wrong. [10:35] rogpeppe, watcher.SetDebug(true) in the setups of tests that I thought it would help in debugging [10:35] :) [10:36] yes, delete the damn verbosity. [10:37] fwereade: or even just watcher.Debug = true :-) [10:37] rogpeppe, indeed :) [10:41] Aram, btw, what's the priority on the confignode read-missing change? [10:42] fwereade: ? [10:42] I'm missing something :) [10:42] what's wrong [10:42] Aram, sorry, I thought I overheard you discussing it with niemeyer with a vague view to doing it soon [10:43] Aram, I mean changing CondfigNode such that an attempt to read an empty one returns an error [10:43] Aram, sorry, a *missing* one [10:43] Aram, empty is fine [10:43] hmm. [10:44] I'll take a look. [10:44] the behavior used to be like you described. [10:44] Aram, honestly I'm easy either way, I think it would be a good change but not an overwhelmingly awesome one [10:44] Aram, I just wanted to know the plan because I'm about to fiddle around with one of the clients [10:45] the behavior should be like you described, if it is not, I'll do it after I do the machine units watcher. [10:46] which I'll start after lunch. [10:46] which is now [10:46] :L) [10:59] fwereade: what's the current status of the uniter w.r.t. lifecycle? does it watch for Dying? [10:59] rogpeppe, nope, no handling at all [10:59] rogpeppe, that would be a good thing to add, actually :) [11:00] lunchtime (did i already say i hate testing for races *grmpf*) [11:00] fwereade: where would you fit the watcher in? Uniter.loop is sooo neat currently... [11:01] fwereade: hmm, i suppose it could just be a separate thing that kills the watcher, perhaps with a known error. [11:01] rogpeppe, I was expecting all the steady state modes to watch for it [11:01] rogpeppe, maybe I can have occasional checks at transition time too [11:02] * fwereade shrugs [11:03] fwereade: is there anything mode-specific that needs to be done at shutdown time? [11:03] rogpeppe, yes, all sorts of watcher stops, generally [11:03] fwereade: those will happen anyway when the uniter tomb is killed, no? [11:05] rogpeppe, s/anyway/assuming the existence of a new entity whose responsibility it is to do that on unit death/ [11:05] fwereade: ah, of course, the watchers are independent of the modes [11:05] rogpeppe, sorry I think I am miscommunicating [11:06] rogpeppe, yes, when we return from a mode, its watchers will be cleaned up; there are no watchers outside modes at the moment [11:06] fwereade: that's what i thought originally [11:06] fwereade: so... [11:06] rogpeppe, but 2/3 of the steady state modes already watch the unit [11:07] fwereade: why do we need a new entity to stop watchers on unit death? [11:07] rogpeppe, we don't [11:07] rogpeppe, we just need to use unit watchers in the modes [11:07] rogpeppe, no call at all to mess around with uniter.loop IMO [11:07] fwereade: i'm wondering if there's any reason to clutter up every single mode with dying logic [11:08] rogpeppe, well, yes [11:08] rogpeppe, usually Dying is of no concern at all [11:08] fwereade: when we can have an entirely separate entity that just kills the uniter when life changes to state.Dying [11:08] rogpeppe, whoa how do yo propose to do that? [11:08] rogpeppe, we don;t want to *kill* the uniter [11:08] fwereade: no? [11:08] rogpeppe, we enter an extended and detailed shutdown sequence in which lots of things happen [11:09] rogpeppe, when we hit Dead is a different matter [11:09] fwereade: ah, ok. that's what i meant by "is there anything mode-specific that needs to be done at shutdown time?" [11:09] rogpeppe, but if you're in a hook error state it matters not one whit whether you're alive or dying [11:09] rogpeppe, you wait for that user resolution, like it or not [11:09] fwereade: does that mean we can't remove units that are in an error state? [11:10] rogpeppe, there will be mode-specific subtleties though [11:10] rogpeppe, yes [11:10] rogpeppe, unless you -force [11:10] rogpeppe, eg, a hook error state should probably just ignore charm upgrades while dying [11:10] fwereade: ok, this all makes sense. not a trivial change then :-) [11:11] rogpeppe, afraid not :) [11:12] rogpeppe, I am slightly worried that all the lifecycle logic will rather dirty up the nice clean uniter [11:13] fwereade: i *think* that it'll just result in another couple of modes [11:13] fwereade: and some logic in the existing modes to make that transition [11:13] rogpeppe, I'm talking more about responses to life checks in the various watchers within the modes [11:14] rogpeppe, I worry that they will obscure the rest ;) [11:14] fwereade: i guess we'll find out... [11:15] rogpeppe, yeah :) [11:31] fwereade: do you think it's reasonable that Refresh return a *NotFoundError when the entity has been removed? [11:32] rogpeppe, hmm, yeah, I think so [11:32] rogpeppe, we'll see how it is in practice [12:47] fwereade: rogpeppe: since now we are only returning Ids, what do you say about the idea of only returning two kinds of changes, one for ints and one for strings, instead of a custom change type for each watcher. [12:47] * fwereade kinda has a sad about this [12:48] generally speaking, when I get a change I really *like* being able to do something useful with it [12:48] so you don't like that we are only returning ids? [12:49] Aram, I guess I need more context; which watchers? [12:49] all of them. [12:49] :) [12:49] look at how machines watcher is now. [12:50] it returns [12:50] type MachinesChange struct { [12:50] Alive []int [12:50] Dead []int [12:50] } [12:50] all the others will return struct { Alive, Dead []string } [12:50] Aram, yeah, I do appreciate the new structure... but, hmm, nothing needs Dying? [12:51] other watchers sure need Dying. [12:51] unsure about this. [12:51] niemeyer wrote it [12:51] Aram, I thought we agreed a while back that we'd be supplying a single list of those entities whose life status had changed? [12:52] Aram, I'm not 100% sure that's right, but I'm sketchy on just having Alive/Dead fields [12:52] yeah, I remember we agreed on that long ago, but I also remember we agreed on this more recently. [12:53] "agreed" [12:53] Aram, blast, I missed that conversation [12:53] Aram, (sure, I am aware that agreement is a fluid that changes its nature as it flows within a group) [12:53] if it were my choice, I'd just return map[Life]Id, because we might add more life states in the future [12:54] map[Life][]Id [12:54] so they are grouped by life [12:54] and you can have as many life states as you want [12:54] Aram, that sounds good to me [12:54] but it's not my choice :). [12:54] Aram, ...although... [12:55] Aram, I kinda feel that reporting by status is somewhat against the spirit of the ids-only change [12:55] Aram, really just `Changed []Id` seems best [12:56] I agree in principle. [12:56] Aram, ok back to concrete :) [12:56] Aram, I want to watch a service's relations, and know when they are dying [12:57] Aram, am I now expected to set up an individual Dying watch, per relation that I detect alive? [12:57] no [12:57] Aram, how can I avoid this? [12:58] you'll get a ServiceRelationWatcher which will return struct { Alive, Dying []string } [12:59] Aram, ok, so each change type is tailored to its clients, and the Alive/Dead thing is not a brook-no-exceptions Agreement [12:59] Aram, that SGTM then :) [13:00] but struct { Alive, Dying []string } is hardly tailored to its client, almost all watchers will use it. [13:01] Aram, I still don't like the amount of reloading I'll be doing on doc watches... [13:02] yeah, I agree in principle. simpler watcher are good, but when they put a burden on the client the fact that they are simple means nothing. [13:02] Aram, on the basis that the ids change will be really ugly and inconvenient for the doc watchers' only clients... as it will be I think for ConfigNode watches... would you hold off on doing them until you've changed the other ones? [13:02] sure. [13:02] Aram, I will try to remember to bring it up with niemeyer this pm [13:03] Aram, cheers [13:03] Aram, and for the relations, I should be thinking in terms of lists of relation IDs only? [13:03] Aram, (grouped as Alive/Dying ofc) [13:04] whatever you want, but the primary key is the string formed from the endpoints and that has the advantage that you can extract information from it without having to load the document. [13:05] Aram, hmm; do we have a Relation accessor on state that accepts that key? [13:06] Aram, I don't think we do... [13:07] surprisingly no, though State.Relation is close in spirit [13:07] since the key is the stringified endpoints [13:07] Aram, I think we have some thinking do do [13:08] Aram, *lossily* stringified endpoints, I think [13:08] Aram, we can't reconstruct the originals without hitting state [13:08] yeah, we can't ATM [13:08] Aram, and that will involve figuring stuff out by getting information from the charms [13:09] Aram, which I'm pretty sure we don't want to do because it will break all our tests [13:10] Aram, (because most tests just slap a dummy charm into state, which doesn't declare any relations) [13:11] Hello all! [13:11] hi [13:11] niemeyer, heyhey [13:13] niemeyer: hiya [13:15] How's the day looking? [13:15] GOod stuff? [13:16] > db.mycoll.insert({_id: 1}) [13:16] duplicate key insert for unique index of capped collection [13:16] > db.foo.insert({_id: 1}) [13:16] E11000 duplicate key error index: test.foo.$_id_ dup key: { : 1.0 } [13:17] It's unfortunate that these are different errors [13:18] * niemeyer reports upstream [13:30] * TheMue still fights with a race condition [13:40] niemeyer: do we return dying machines in the initial event of the machine units watcher? [13:40] or do we now. [13:40] s/now/not/ [13:40] returning dying machines makes it consistent with Machine.Units [13:40] Aram: Dying, yes [13:41] Aram: The only thing that cares about an Alive => Dying transition is the entity agent itself [13:41] Aram: For all other purposes, the thing is still alive [13:42] niemeyer: so you're fine with UnitsChange being [13:42] type UnitsChange struct { [13:42] Alive []string [13:42] Dying []string [13:42] } [13:42] ? [13:42] Aram: s/Dying/Dead/? [13:42] Aram, surely that's an Alive/Dead one [13:42] Yeah [13:43] Aram, ServiceRelations is Alive/Dying, because while the watching entity is not istelf a relation, it *is* responsible for responding to the relation's lifecycle changes [13:44] Aram, sorry that was not a helpful sentence [13:44] how does one get Alive -> Dying if we don't deliver this event here? one watcher per each unit? [13:44] Aram, I think the question here is entirely situational, and dependent on what the client needs to use [13:45] Aram, the MPW only needs to about Alive (to deploy a container) and Dead (to destroy it) [13:45] ok, that seems sensible. [13:46] Aram, the Uniter is responsible for watching the unit for Dying, and then shutting itself down in an orderly fashion before making itself Dead [13:46] Weird.. that was an abrupt "disconnection by peer" [13:46] Aram, the Uniter is also responsible for handling everything about relations, and itself sets Dead, so it should only need Alive/Dying [13:47] Aram, from SRW [13:47] fwereade: That said, hmm [13:47] niemeyer_, I personally would prefer just []ChangedLife [13:48] niemeyer_, and handle it appropriately at the various call sites [13:48] fwereade: Wouldn't it be weird.. let's say.. for a machine watcher to report a unit as alive when it's dying.. [13:48] niemeyer_, which will I suspect contain a number of subtle differences [13:48] fwereade: just so it starts up, and dies [13:48] it seems kind of wrong to return Dying units inside the Alive field of UnitsChange though. [13:49] niemeyer_, so a unit that is initiall seen to be Dying should be ignored, and should not generate a Dead [13:49] fwereade: Kind of.. [13:49] fwereade: That's even more incorrect [13:49] fwereade: imagine the same situation, but the unit is actually still running [13:50] niemeyer_, ha [13:50] niemeyer_, fwereade: fairly trivial: https://codereview.appspot.com/6564054 [13:50] rogpeppe: Thanks [13:50] niemeyer_, wait, surely the machine agent *knows* what is running anyway? [13:50] fwereade: I think something along your ChangedLife idea might be plausible [13:50] fwereade: Forces acknowledgement of the possibilities [13:50] niemeyer_, but regardless, yeah, I feel that's the way to go for now [13:51] fwereade: Or even Alive/Dying/Dead fields [13:51] fwereade: Which makes the need for handling even more explicit [13:51] niemeyer_, I'd prefer to keep it just as "changed!", rather than risk a lie (when the id is loaded and revealed to be in a different state) [13:52] fwereade: Good point [13:52] niemeyer_, on a related note: ids from watchers [13:52] fwereade: ok [13:53] so Changed is the consesnsus? [13:53] niemeyer_, I like it in principle, and (I think) in practice for the collection watchers; I feel that it's going to be unhelpful when applied to the document watchers [13:53] Aram: It may be just a slice of ids, I guess, but let's wait until the end of the conversation [13:53] niemeyer_, as the only client so far, every time I get a changed service or unit, I want it refreshed [13:54] fwereade: There are a few different details that have been going unperceived [13:54] niemeyer_, can I handwave single document watchers to be "different enough" as to be sent as objects? [13:54] niemeyer_, go on [13:55] fwereade: 1) We're faking the data for the entity; since the entity might not be there by the time we try to fetch it, we change its field to Dead and send a cached version [13:56] fwereade: Which is quite wrong.. the unit may have died in an entirely different state [13:56] fwereade: and such a Dead + arbitrary state may never have happened.. the justification for the death is in the unit that we didn't see (unit being just an example here) [13:57] fwereade: 2) If we get 100 changes between the last time we've observed the unit, and the next time we're able to handle the change because e.g. the hook returned, [13:57] fwereade: we're reloading the unit 100 times within the watcher, for absolutely no reason [13:58] niemeyer_, re (1), I dunno -- for what clients is the distinction important? [13:58] fwereade: I don't know.. we'll likely find out when something explodes in our face [13:58] niemeyer_, re (2), hmm, true [13:58] fwereade: 3) The cost of loading the unit is minimal in all cases I've ported [13:59] niemeyer_, I mean, I've been thinking about it, and I can't see any -- doesn't Dead mean "the only safe way to interact with this document is to destroy it"? [13:59] niemeyer_, yeah, it is only a few lines of code [13:59] fwereade: and, interesting, in some cases it cleaned up.. [13:59] fwereade: The firewaller, for example, didn't really care much about that machine object [13:59] fwereade: It ended up not using it at all in most cases [13:59] fwereade: It was using it just because it was being handed off anyway [13:59] niemeyer_, I'm 100% behind it on the collections [14:00] niemeyer_, and I think I'm now convinced on the documents side [14:00] fwereade: 4) It makes the watchers simple (!) :-) [14:00] niemeyer_, ;) [14:02] fwereade, Aram: So, slice of ids? [14:02] niemeyer_, +1 [14:02] Changed []string? [14:02] Aram: []string [14:02] * rogpeppe wishes that machine ids were strings too [14:02] hmm [14:03] Aram: <-chan []string [14:03] rogpeppe++ [14:03] niemeyer_: ok, that seems fine [14:05] rogpeppe: We should talk about that someday when we're not in the middle of a big change [14:05] niemeyer_: yeah [14:10] rogpeppe: Reviewed === niemeyer_ is now known as niemeyer [14:10] niemeyer: thanks [14:11] niemeyer: i chose FitsTypeOf because then if the test fails it'll say what the failing error actually is [14:11] rogpeppe: If the test fails it's trivial to find out what's wrong [14:11] niemeyer: ok [14:11] rogpeppe: That's how it's being done in the rest of the code already [14:12] rogpeppe: We have a test for IsNotFound that verifies it actually works as intended, and the rest is relying on it workng [14:12] niemeyer: that's fine. i generally to try to make test failures as informative as i can, but i'm happy to use IsNotFound too [14:14] rogpeppe: That's a great approach, but there's value in testing the semantics we want.. we'll generally be using IsNotFound in client code, and it should work [14:14] niemeyer: ok, but i thought that's what the IsNotFound test is for. [14:14] niemeyer: anyway, i've changed it already [14:15] rogpeppe: Nevermind. Thank you! [14:15] and one other occurrence too [14:23] have to leave due to an emergency, bbl [14:27] TheMue: Ouch.. hope it's all good there [15:06] niemeyer, hey! available for a chat about versions of mongodb in quantal? I understand 2.2.0 is desired.... [15:06] niemeyer, whoops, I need to leave early today... I might have a CL for you later, but nothing yet I'm afraid [15:07] jamespage: Yo, here [15:07] fwereade: np, have a pleasant time there [15:08] niemeyer, Aram: ooo, one important thought: to make the ids change work with RelationUnitsChange et al, we will need a map[unit-name]txn-revno [15:08] fwereade: Hmm [15:08] Aram, niemeyer, actually, I don;t think we do [15:08] niemeyer, I had a request to up the version of mongodb in quantal to 2.2.0 to support go-juju [15:09] fwereade: That's good I guess, because I have no idea about what you have in mind yet :D [15:09] niemeyer, we just clear the settings out when we get a change [15:09] niemeyer, currently we will be shipping 2.0.6 (maybe 2.0.7 if I get time) [15:09] jamespage: Col [15:09] jamespage: Cool [15:09] jamespage: 2.2.0 would be good indeed [15:09] niemeyer, I was thinking we'd need to keep track of revnos at the watcher level because they're used by the relation context level [15:09] jamespage: Very good, in fact [15:09] niemeyer, at the moment I'm pushing back - its very late in the cycle [15:09]