[01:48] niemeyer: testing/mgo_test.go wasn't being run [01:48] i'll roll that fix into my next proposal [02:14] davecheney: Oops.. thanks! [02:19] niemeyer: https://code.launchpad.net/~niemeyer/goetveld/trunk/+merge/126136 [02:20] ^ not propsed with lbox, sorry [02:22] urgh, change list is empty as well ... [02:24] ahh, because i did it ass backawards [02:25] niemeyer: https://code.launchpad.net/~dave-cheney/goetveld/001-add-darwin-termios-support/+merge/126137 [03:16] davecheney: LGTM [03:16] davecheney: I've just changed the project so it's trunk is owned by the gophers team [03:16] s/it's/its/ [03:16] davecheney: Please feel free to repropose against the new trunk and submit it straihgt [03:16] straight [03:24] * niemeyer => bed [07:52] ha! my first Refresh-related bug with a long-lived-entity [07:52] fwereade: morning! [07:52] fwereade: it was bound to happen [07:52] rogpeppe, what would you think about putting Refresh in at the start of Watch on Unit and Service? [07:53] rogpeppe, that way the initial event actually is the current state, rather than potentially being whatever aged rubbish you started to watch? [07:53] fwereade: what are the events that those watchers produce? [07:54] rogpeppe, something-changed [07:54] rogpeppe, send fresh units back [07:54] fwereade: so won't the initial event be a fresh unit? [07:54] fwereade: and hence have all the currently correct settings? [07:54] rogpeppe, AFAICT it'll correspond to *some* transaction more recent than the revno of the old document [07:55] rogpeppe, hey wait that makes no sense [07:55] fwereade: hmm, that seems wrong [07:55] rogpeppe, ok, time seemed to be flowing in reverse, and a Refresh fixed it, but clearlymore thought is required [07:55] fwereade: the initial event should be the most recent revno [07:56] rogpeppe, ah, no, the initial event is just the unit yu originally watched [07:56] fwereade: that seems wrong to me [07:56] rogpeppe, yeah, agreed [07:56] fwereade: then there's no point in having the initial event [07:57] rogpeppe, well, yeah; we could either Refresh it or reconstruct a new one and send that back [07:57] fwereade: i'd incline towards the latter [07:57] fwereade: we don't do any implict Refresh anywhere else [07:58] TheMue: yo! [07:58] rogpeppe, ok, sgtm [07:58] fwereade: and every other event on the channel is a fresh object [07:58] TheMue, heyhey [07:58] rogpeppe, yeah, feels neat [07:58] morniing rogpeppe and fwereade [08:32] rogpeppe, TheMue: should be trivial: https://codereview.appspot.com/6571047 [08:35] fwereade: LGTM [08:36] rogpeppe, cheers; trivial enough to submit directly, do you think? [08:37] fwereade: i *think* so, but it is a significant change in one sense. [08:37] fwereade: even though the code is trivial [08:38] rogpeppe, in the sense that it causes the uniter to work properly, it is significant ;p [08:39] rogpeppe, and I think it is unambiguously (1) correct, because watches should send current state and then deltas; and (2) safer, because we don't ever send the originating entity over to some unknown client over a channel [08:39] fwereade: it's that last thing that i think is the killer argument [08:39] rogpeppe, indeed, it is only secondary in my mind because it wasn't the thing that bit me [08:40] fwereade: i'd say go for it [08:40] * fwereade cheers [09:41] moin. [09:42] hello and goodbye [09:42] time to make the dinner [09:42] then i'll see ya'll for the hangout [09:46] ok launchpad.net/juju-core/worker/uniter 117.284s [09:46] ok launchpad.net/juju-core/worker/uniter/charm 7.835s [09:46] ok launchpad.net/juju-core/worker/uniter/hook 0.007s [09:46] ok launchpad.net/juju-core/worker/uniter/relation 8.826s [09:46] fwereade: did you see my comment to the list about i/o usage [09:46] that might explain your excessive test times [09:46] davecheney, I did, makes a lot of sense [09:46] davecheney, thanks [09:46] fwereade: maybe aram can help [09:46] i had a look today, but I wasn't much chop for anything [09:47] invoking mgo the way the tests do, doesn't make it create that big _tmp [09:47] flie [09:47] but then again, I never connected a client to the manually invoked one [09:47] 20MB [09:47] make it smaller [09:47] 1MB [09:47] on my machine _tmp was 256mb [09:47] hmm? [09:47] perhaps testing/mgo had leaked? [09:48] or was this only from one test? [09:48] aaah [09:48] I know [09:48] niemeyer made the capped collection 20MB [09:48] 200 [09:48] aram what is _tmp ? [09:48] and it needs contiguous space for that [09:48] (see email) [09:48] Aram, aaaahhh, and it's only made little in *some* tests [09:48] ok, let me read the email [09:48] Aram, right? [09:49] fwereade: only made little in cloudinit -> deploy to ec2 [09:50] oh, that explains why my had started version didn't use any disk space [09:50] it won't do that until someone connects to it and does ensure index ... [09:53] davecheney, StartMgoServer specifies everything that addMongoToBoot does, doesn't it? [09:54] fwereade: make the capped collection 1MB and see if tests take less. [09:54] in open.go [09:55] wtf is wrong with launchpad today, so slow. [09:56] actually, my internet is slow. [09:56] * Aram reboots router [10:15] lunchtime, see you in the hangout [10:59] hmm, my build complains about an undefined bson.SetZero. i already update mgo. or is it a different problem? [10:59] bzr pull [10:59] go get -u is not enough [11:00] Aram: thx, will try [11:01] Aram: it pulled nothing, but i'm still using 1.0.2. may this be the problem? [11:01] no [11:01] you pulled mgo, right? [11:02] yes [11:02] what does bzr log --line -r -1 say? [11:03] so, how about that meeting ... [11:04] funny, 172 with the added SetZero [11:04] TheMue: maybe a fast path is to [11:04] rm -rf .../v2/mgo [11:04] go get ... [11:04] the cd v2/mgo [11:05] bzr pull [11:05] that is what I did [11:05] i'm pretty sure it will work [11:05] will try that too [11:05] please double check you don't have another mgo snuck away somewhere else [11:05] davecheney: but it's always funny to find new bzr misfeatures and broken behaviors to complain about them. [11:06] * davecheney smiles at Aram [11:06] its like bzr has set the revision to a revision in a branch [11:07] hm, should we be meeting now? [11:08] indeed [11:08] we should [11:12] we should have a meeting about the lack of the meeting. [11:13] Aram: please propose an agenda first [11:13] a user story. [11:13] which we have to send to a program manager. [11:13] shall i sent out the invite? [11:14] * davecheney is texting mrarmm [11:15] TheMue, please do [11:16] done [11:17] sorry I was late to the meeting [11:17] s'ok [11:17] lets get going [11:20] I'm in the meeting, called by frank [11:20] err, nobody else is [11:21] we are [11:21] :) [11:21] davecheney: https://plus.google.com/hangouts/_/74ad471b07ee1c4e3215133bbf54c62e79ff5c7b?authuser=1&hl=en [11:21] ta [11:23] rogpeppe: ^ ^ [11:24] oops, didn't see it [11:24] will be there in a moment [11:29] 16:06 niemeyer: Woah [11:29] 16:06 niemeyer: services: [11:29] 16:06 niemeyer: builddb: [11:29] 16:06 niemeyer: charm: builddb [11:29] 16:06 niemeyer: exposed: false [11:29] 16:06 niemeyer: units: [11:29] 16:06 niemeyer: builddb/0: [11:29] 16:06 niemeyer: agent-version: 0.0.0 [11:29] 16:06 niemeyer: machine: 1 [11:29] 16:06 niemeyer: public-address: ec2-204-236-223-223.compute-1.amazonaws.com [11:29] 16:06 niemeyer: status: started [11:33] davecheney, sorry, did you paste it somewhere I missed? [11:33] sorry, email [11:33] irc is on another machine, sorry [11:35] davecheney, np, ty [11:59] lunch bbl [12:48] Morning juju masters! [13:06] niemeyer: hello [13:06] davecheney: Heya! [13:06] sleep it would seem, eludes me [13:10] niemeyer: hi [13:14] davecheney: Wow, yeah, you've been here for a while :) [13:14] TheMue: Yo [13:30] niemeyer, heyhey [13:30] fwereade: Heya [13:30] fwereade: Just reviewing the relation units stuff [13:30] fwereade: Superb so far [13:31] niemeyer, for some reason I made the make-uniter-work branch depend on that one [13:31] niemeyer, jolly good, hopefully I won't need to reparent the followup ;) [13:31] niemeyer, I submitted one trivial this morning btw that was necessary [13:32] fwereade: No, certainly not.. I think there were a few opportunities to have split the branches, as you noted yourself in the description, but it's alright.. happy to have it in whatever form. [13:33] niemeyer, cheers [13:33] fwereade: Super, thanks re. trivial [13:34] niemeyer, now I come to think of it, the timeout bumps in the followup are surely only necessary because I wasn't using a tmpfs [13:34] niemeyer, I will adjust them down as seems fit [13:35] fwereade: Hmm [13:35] fwereade: The timeouts on the sad cases may be high [13:35] niemeyer, I was thinking "conservatively", at first, anyway -- just to what they were originally [13:35] fwereade: Those prevent spurious breaks when the machine happens to get some load or when tests are running on a slow machine [13:36] fwereade: I haven't seen the branch [13:36] fwereade: What were they? [13:36] niemeyer, 5s to 10s is the notable one, in a couple of places [13:36] niemeyer, and I really don't think it deserves that long [13:36] fwereade: wow [13:36] fwereade: Yeah, that's a lot [13:37] fwereade: How come we're getting such long timeouts? Are we missing a resync? [13:37] niemeyer, that was pessimistic, in practive it only missed the 5s mark by a few milliseconds even under load [13:37] niemeyer, but I was in no mood to mess around ;) [13:37] niemeyer, almost certainly it is because the mongo tests stress the hell out of my machine unless I have a tmpfs [13:37] fwereade: Hmm [13:38] fwereade: I wonder why [13:38] Well, I do have an SSD [13:38] niemeyer, I think Aram and davecheney have a better developed understanding of it -- the stuff I've been working on has not generally been affected enough to get really worked up about when you're running a few tests at a time [13:39] niemeyer, that said, it feels *much* nicer now than it ever did before, not sure is this is psychosomatic [13:39] niemeyer, is anyone working on jujud btw? [13:39] fwereade: not that I know of [13:40] fwereade: I was about to ask what else has broken tests atm [13:40] niemeyer, that seems to be the one remaining eyesore in the test output [13:40] fwereade: Anything obvious there? [13:40] niemeyer, barely looked tbh [13:40] niemeyer, focus has been elsewhere [13:41] niemeyer, schema error in bootstrap_test is the first [13:41] niemeyer, the remaining failures can probably be mostly attributed to collateral damage [13:42] fwereade: That rings a bell somehow [13:57] fwereade: Review sent [13:57] niemeyer, cheers [14:00] Aram: ping [14:00] pong [14:01] Aram: Heya [14:01] Aram: How're things going there? [14:02] niemeyer, the resyncing is to pick up Alive changes, without which Unit.Status will report "down" regardless of content [14:02] niemeyer: yesterday I've taken a swap day and today I'm working on the missing watchers and lifecycle in watchers. but a little bit later, ATM I'm running some rrands. [14:02] errand [14:02] s [14:03] fwereade: Sorry, I'm still slow today.. can you cover it a bit more piecemeal? [14:03] Aram: Okay, let's please talk once you're back [14:04] niemeyer, ah, sorry, ok: I'm actually on crack, that's a different piece of code [14:04] niemeyer, no reason for the resyncing you correctly complain about [14:04] fwereade: Is it just a matter of refreshing the unit itself? [14:04] niemeyer, yeah, that is the sane and easy way [14:05] fwereade: Super.. I'm really just trying to understand these new patterns and see where/whether they fall short [14:05] niemeyer, that was almost certainly an unremoved attempt at working around the weirdness in the entity watchers [14:11] niemeyer, fwereade: just checking, do you think the provisioner is currently working in trunk? [14:11] rogpeppe, STM to be, just deployed a unit [14:12] rogpeppe, not from trunk, but I haven't touched the provisioner [14:12] fwereade: hmm, my live tests aren't working [14:12] rogpeppe: It was yesterday [14:12] niemeyer: the live test? [14:12] rogpeppe: The provisioner [14:14] niemeyer: the provisioner test suite runs fine for me locally - just not live [14:14] rogpeppe: I've used the provisioner live yesterday [14:15] rogpeppe: Multiple times [14:15] niemeyer: i've probably broken the live tests somehow then [14:15] rogpeppe: Or they are just not working after the state migration [14:16] niemeyer: entirely possible. but it's not relying on watchers, so i *thought* it should work ok [14:19] niemeyer: did juju status report you the correct agent version for the bootstrap machine? [14:20] rogpeppe: Yep [14:21] rogpeppe: I have a paste somewhere, hold on [14:23] rogpeppe: Sorry, can't find it [14:23] rogpeppe: I can easily fire a new env, though [14:23] niemeyer: i think i saw it actually [14:23] niemeyer: i'm trying again, but not in the test this time [14:27] fwereade: I think we'd benefit from adding back service.CharmURL, btw [14:27] fwereade: We can probably just revert it from somewhere in history [14:28] niemeyer, I can only think of one place it's be a real boon... quite often we want the charm itself as well [14:28] fwereade: In hindsight, it was the SetCharmURL bit that was ill-conceived.. CharmURL is fine [14:28] niemeyer, but yeah no actual objections [14:28] fwereade: Yeah, but often we grab the charm and throw it away.. we might grab the charm only if [14:29] niemeyer, fair enough [14:29] fwereade: Not an issue by any means.. just sharing brain state :) [14:36] niemeyer, ok, re MachineUnitsWatcher: what is the mechanism by which it guarantees it does not send events with the same unit both Added and Removed? [14:37] fwereade: I think there isn't any right now.. [14:38] fwereade: I've pondered about the same thing in the context of the MachinesWatcher [14:38] niemeyer, I guess that was true both before and after the loop change [14:38] fwereade: Right [14:38] fwereade: I've actually made a comment regarding this in the CL [14:38] fwereade: Dave suggested it wasn't an issue, but I'm not yet sure [14:39] fwereade: I'm tempted to drop them too [14:39] niemeyer, it feels to me like it is [14:39] fwereade: If nothing else, we must handle the situation where it doesn't show up correctly [14:39] fwereade: Because it is just a perspective of timing [14:39] fwereade: If we made the same query moments afterwards, we'd see nothing [14:40] niemeyer, yeah, indeed [14:40] niemeyer, I guess it's best to use that style though, and fix the watchers so that mergeChanges always does the Right Thing [14:41] fwereade: +1 [14:41] niemeyer, the advantage of the double-loop style in RUW is that I think it *did* allow me to make useful guarantees of that nature [14:41] niemeyer, anyway, shouldn't be too tricky :) [14:41] fwereade: I've been doing a few changes, btw: renaming mergeChanges to merge; getInitialEvent to initial, and pass the change itself onto initial; plus the w.out change I've mentioned [14:42] fwereade: Hmm [14:42] niemeyer, those all sgtm [14:42] fwereade: I don't think so [14:42] fwereade: I mean, I don't think there were any guarantees before either [14:42] fwereade: Because events were aggregated regardless [14:43] niemeyer, I *think* that because the loops in which I send and the loops in which I get scope changes are different, it all Just Works [14:43] niemeyer, I don;t pick up a new scope event until I've sent the change derived from the first one [14:43] fwereade: I don't see why it makes a difference.. the code that may be executed or not is exactly the same, with the new convention or with the old one [14:43] niemeyer, that change may have had any number of settings changes merged in but those are safe [14:43] fwereade: The second loop is exactly a copy with the first one with the send disabled [14:43] niemeyer, not in RUW [14:44] fwereade: Which is exactly what we do in the new convention, but with less code [14:44] fwereade: Oh? [14:44] fwereade: /me looks again [14:44] niemeyer, although it is just one more var to track really [14:47] fwereade: I see.. I think it's actually preferable to teach merge about how to handle the situation [14:47] fwereade: Otherwise it's unnecessarily buffering changes arbitrarily [14:47] niemeyer, yeah, sounds sane [14:48] niemeyer: ah, i think i *may* have discovered the reason why live tests are failing [14:48] rogpeppe: Sweet! [14:48] niemeyer: just a hunch as yet [14:49] niemeyer: but... [14:49] niemeyer: what's the name of the public bucket containing the mongo binaries? [14:49] rogpeppe: juju-dist.. why does it matter? [14:50] niemeyer: is it hard-coded? [14:50] rogpeppe: yes [14:50] drat [14:50] rogpeppe: As of today [14:50] rogpeppe: It shouldn't be, but this avoids yet another refactoring without all tests green [14:50] niemeyer: that's fine. hrmph. [14:52] fwereade: Follow up reviewed too [14:52] niemeyer: test of removals in firewaller always showed an effect. the fw dislikes the service removal when it is stopped. stopping the service watcher returns an error then. [14:52] niemeyer, ta [14:53] TheMue: More details please? [14:54] TheMue: If the firewaller is stopped, how can it dislike anything? [14:54] niemeyer, ok, so *that* was the bit with a weird StartSync that I think is justified [14:54] fwereade: Yeah, I imagined it when reading [14:54] niemeyer, Unit.Status does an alive check on the UA [14:54] fwereade: I'm just curious about how it unrolls into it [14:54] fwereade: Aha, ok.. is that all? [14:55] niemeyer, Status returns "down" when the pinger is not there [14:55] niemeyer, yeah, that's it [14:55] fwereade: So a suggestion: let's move the sync down to right before Status, and call Sync rather than StartSync [14:55] niemeyer, ah, yeah, sounds sensible [14:55] niemeyer, cheers [14:56] fwereade: Thanks! [14:56] TheMue? [15:02] niemeyer: sorry, my wife called me ;) [15:02] niemeyer: fw.Stop() is expected to return no error [15:02] niemeyer: but if a service is removed it returns a service not found [15:03] niemeyer: because during the stop all internal go routines are stopped, and there the service watchers [15:03] niemeyer: and this stopping does state.Service(name) for the non-existing service [15:03] TheMue: This is a bug that should be fixed [15:04] niemeyer: indeed [15:04] niemeyer: so you had the right feeling about testing it [15:05] TheMue: Would you mind to review all fw call sites that grab objects, and consider the appropriate action when the respective entity isn't found? [15:06] TheMue: Happy to have that done in tiny branches that follow each other so that we have a good way to coordinate as we go [15:06] niemeyer: could you please rephase "fw call sites"? thx. [15:06] TheMue: call site == a point in the code that calls something [15:07] niemeyer: ah, yes. ok, will do. [15:07] TheMue: Once you have that first issue fixed, please push for review so we can talk a bit about it with more concrete logic [15:08] niemeyer: ok [15:26] niemeyer: hmm, live tests fail even though bootstrapping and deploying live work ok. looks like the provisioner never sees the new machine. [15:27] rogpeppe: How can the provisioner never see the new machine if it is the one firing off new machines [15:28] niemeyer: it's not the one that creates the new Machine though [15:28] rogpeppe: Exactly.. and if it didn't see that machine, it wouldn't fire an instance fo rit [15:29] rogpeppe: Also, tests pass [15:29] niemeyer: the live deploy tests are currently disabled [15:29] rogpeppe: Provisioner tests pass [15:30] niemeyer: sure. i'm just saying what i see. [15:30] rogpeppe: Of course, and I'm just explaining that "provisioner never sees the new machine" can't be true on its own [15:30] rogpeppe: The builddb charm works [15:30] rogpeppe: Firing new machine [15:30] rogpeppe: With machiner, uniter, etc [15:30] niemeyer: absolutely. i see that working too. [15:30] rogpeppe: That doesn't happen with a provisioner that never sees a new machine [15:31] niemeyer: but my live test that used to work no longer works. so *something* is up. [15:31] niemeyer: it might be the fact that the second machine is added before the provisioner comes up [15:31] rogpeppe: I'm sure.. [15:32] rogpeppe: Again, tests pass.. [15:32] rogpeppe: You seem to argue that the provisioner is completely broken.. I doubt it's as simple as that [15:32] niemeyer: i'm not arguing that at all [15:32] niemeyer: i'm saying that live tests fail that used to pass, that's all [15:33] niemeyer: and i don't see the provisioner see the new machine, which seems to be a symptom [15:33] rogpeppe: Sorry, okay.. I don't know how to help then [15:33] niemeyer: i'm not asking for help, just giving a status update [15:33] rogpeppe: That live tests are broken, ok :) [15:34] niemeyer: yeah, it's weird [15:35] fwereade, rogpeppe: Regarding https://codereview.appspot.com/6571047, I really want to stop sending entities down the pipe at all [15:36] Everything is a lot simpler and more correct if we just sent the entity identification [15:36] niemeyer: just send a change notification? [15:36] rogpeppe: Yeah, with the id/name [15:36] niemeyer: i tend to agree. [15:36] The machines watcher was significantly simpler and more obvious to handle correctly on the other side with it [15:37] That said, I've been resisting suggesting this for now, just so we can get it all working as it used to [15:37] After we're comfortable again, I think we should change [15:37] niemeyer, no strong feelings myself [15:38] This also avoids the spurious reads we have nowadays within the watcher itself [15:38] when the entity changes repeatedly without being consumed [15:38] and it also simplifies the Dead-handling logic.. [15:38] Anyway, lunch time here [15:39] rogpeppe: If you're still struggling by the time I'm back, let's try to pair on it [15:39] niemeyer: i think i might be getting somewhere now. just expressing the problem was helpful. [15:39] niemeyer: aaargh! [15:39] niemeyer: found it [15:41] so frickin simple [16:02] Greets [16:03] I'd like to suggest that, on the https://juju.ubuntu.com/ page, under the "Try it for yourself" heading, the instructions to install Juju from the PPA don't work out of the box with Ubuntu 12.04.1 [16:04] gregkrsak@brewmaster:~$ add-apt-repository ppa:juju/pkgs [16:04] The program 'add-apt-repository' is currently not installed. You can install it by typing: sudo apt-get install python-software-properties [16:04] (thanks!) [16:18] platypusfriend: kthxbye [16:19] rogpeppe: What was it? [16:19] niemeyer: well, one problem was that i was assuming that the machine received from the machine watcher had no information content, so was reading agent tools from the original. [16:20] niemeyer: but... that doesn't appear to have fixed the problem. we will see. [16:20] niemeyer: am currently running a live test again [16:24] niemeyer: hmm, looks like the machine watcher might not be sending an initial event. [16:25] rogpeppe: Indeed, I don't think it is [16:25] niemeyer: i thought we tested for that [16:27] rogpeppe: I can fix that.. what else is using it? [16:28] Hmm [16:28] * rogpeppe checks [16:28] Apparently only used in tests [16:28] In two places only, one of them being the live tests [16:28] niemeyer: that sounds about right [16:29] rogpeppe: I'm on it.. will take the chance to simplify the watcher according to the latest conventions [16:29] niemeyer: sounds good [16:30] rogpeppe: Hah, curious [16:30] rogpeppe: If we send just the id, the initial event becomes a bit silly [16:31] niemeyer: i'm not sure [16:31] niemeyer: it's actually still useful, even though it carries no info content [16:32] niemeyer: because it for a very simple pattern inside various watcher clients [16:32] s/it/it makes/ [16:32] rogpeppe: Cool, I'm not suggesting we change the convention right now either way, since we're in the middle of a huge transition [16:32] niemeyer: +1 [16:32]