[00:00] fark [00:00] I thought this was a small branch [00:00] $ git diff master | wc -l [00:00] 973 [00:00] 15 files changed, 467 insertions(+), 174 deletions(-) [00:00] hmm... menno is reviewer [00:00] no worries then [00:01] * thumper goes to write more tests :) === kadams54 is now known as kadams54-away === kadams54-away is now known as kadams54 === kadams54 is now known as kadams54-away [00:39] thumper: screw you hippie :) [00:40] * thumper bows [00:41] oh fuck [00:41] hmm... [00:42] * thumper found an interesting bug^Wfeature [01:01] FINE [01:01] IF THAT IS THE WAY IT'S GOING TO BE... [01:01] * thumper beats the code into submission [01:19] fwereade: if you get a chance in the morning could you review http://reviews.vapour.ws/r/1181/ ? [01:20] fwereade: I've updated with good feedback from perrito666 and dimitern [01:34] menn0: got a few minutes to discuss a testing approach? [01:43] * thumper grumbles [01:44] davecheney: can I pass around an unbound method as a func to call on a target? [01:46] * thumper recalls the following [01:46] type foo struct {} [01:47] func (f *foo) method(param string) error {...} [01:47] foo.method -> func (*foo, string) error [01:47] ... [01:47] that'll work [01:48] thumper: back now. standup hangout? [01:48] yea [01:54] Bug #1433384 was opened: utopic unit-test failure: TrackerSuite.TestWaitMinionBecomeMinion

[02:07] Bug #1433384 changed: utopic unit-test failure: TrackerSuite.TestWaitMinionBecomeMinion

[02:13] Bug #1433384 was opened: utopic unit-test failure: TrackerSuite.TestWaitMinionBecomeMinion

[02:17] menn0: what do you think about having the function api.OpenWithLoginV1 exported publicly? [02:17] menn0: that way we can call it from the apiserver package [02:19] thumper: it's a little ugly but it doesn't bug me that much [02:19] * thumper nods [02:19] thumper: perhaps it should be OpenWithVersion [02:20] but that implies it takes an arg [02:20] but it doesn't [02:20] thumper: would that make sense? then there'd be just one call to let people get at any version. [02:21] hmm... [02:21] perhaps [02:21] thumper: but could it take an arg [02:21] ? [02:21] yes [02:21] we could switch on the version [02:21] and error on unknown [02:21] thumper: why do you really need to export it from apiserver? [02:21] it is an api client function right now [02:22] and I want to use it in the apiserver package [02:22] * thumper hunts for some books of equal height [02:22] thumper: right I see. [02:22] * thumper is trying standing up [02:23] Beginning GIMP is the same thickness as Generative Programming [02:23] * thumper is winning [02:23] :) [02:23] hmm... that's better [02:23] keyboard was a little low before [02:24] thumper: just thought some more about it [02:24] yus... [02:24] thumper: and it seems ok [02:25] which bit? [02:25] thumper: esp if there's a docstring saying that it's mainly for testing [02:25] exporting one that takes a version [02:25] * thumper nods [02:25] will do [02:25] thumper: either is ok [02:25] and I'll have it take a version number [02:25] I think it is a little more flexible [02:25] thumper: but i prefer the version taking form [02:25] ack [02:26] * thumper takes a deep breath and ignores a little mess [02:26] * thumper makes a note to clean it up next [02:30] menn0: I'm not sure I'm going to get all this done before 6 today [02:30] menn0: as I have a parent-teacher interview at 4 [02:30] and a class at 5 [02:30] but I'll finish it off tongiht [02:31] menn0: I don't expect you to review it today (maybe tomorrow) :) [02:32] thumper: np [03:48] wallyworld: http://reviews.vapour.ws/r/1193/ [03:48] ok [03:49] anyone able to review this? http://reviews.vapour.ws/r/1192/ [03:50] it's a fix for a problem I introduced that is preventing compilation on i386 [03:50] axw: i've done the go-amz mods ti support volumes, writing tests. had to update v4-unstable branch with latest v3 changes, conflicts due to stupid paths in imports. that shits me no end with Go [03:50] menn0: looking [03:50] menn0: class is at 6 not 5 [03:50] so I'm about to push this branch up [03:50] thumper: ok, bring it on [03:51] 20 files changed, 588 insertions(+), 192 deletions(-) [03:51] * menn0 warms up his review issue fingers [03:54] menn0: i just left a suggestion, not that important [03:54] wallyworld: good idea. i'll do that. [03:57] hmm... [03:57] why is reviewboard not picking that up [03:57] menn0: https://github.com/juju/juju/pull/1868 [03:58] thumper: cool. i'll do that next. [04:06] looks like CI is blocked at the moment [04:06] bug 1433384 [04:06] Bug #1433384: utopic unit-test failure: TrackerSuite.TestWaitMinionBecomeMinion

[04:07] thumper: your PR didn't make it to RB? [04:08] * thumper shrugs [04:08] NFI [04:10] thumper: i'll review on GH [04:13] this standing desk thing has my shoulders aching [04:13] less hunching and more hanging I guess [04:19] thumper: time to go to walking desk :) [04:20] thumper: FYI, that RB problem is a bug in my hook code where it cannot handle unicode properly (yay 7 year old Python version) [04:20] thumper: from time to time Github sends unicode in the request (or we have some in our code somewhere) [04:21] thumper: once the vivid stuff settles down I'm planning on fixing it [04:29] axw: reviewed, sorry about delay, pool man was here [04:29] wallyworld: nps, thanks [04:34] ericsnow: ah... I was wondering what it was [04:35] thumper: yeah, it's the most common reason why the hook fails [04:35] ok, handy to know [04:35] I'm going to head off to my boxing class now [04:35] time to hit things hard [04:35] thumper: review done [04:35] thumper: the other is when a branch get's rebased against an incompatible base revision to the original [04:35] menn0: awesome thanks [04:35] menn0: I was wondering about the restricted root ordering [04:35] it seemed weird to do work that you were going to throw away [04:36] but I'll look for the method first [04:36] for consistency [04:36] with the other methods [04:36] menn0: thanks for pointing out filepath.EvalSymlinks [04:36] menn0: I've updated the patch [04:36] thumper: yeah it does looks weird but I did put some thought into it for upgradingRoot [04:36] laters peeps [04:36] menn0: ack, will address tomorrow [04:36] ericsnow: no problems [04:36] thumper: o/ [04:37] ericsnow: i've got to finish up in a sec so i might not be able to get back to your PR until tomorrow [04:37] menn0: no worries [04:38] menn0: thanks for the reviews [04:38] menn0: we kept you busy today [04:39] menn0: I'll probably have someone else wrap up the review so I that we can maybe release 1.23b1 tomorrow [04:39] menn0: thanks for the review, and the kind words :) [04:40] ericsnow: np [04:40] jw4: np [04:44] ericsnow: i just managed to sneak another review in [04:44] ericsnow: one problem, but otherwise ship it [04:44] \o/ [04:44] menn0: a pity review :) [04:58] ericsnow: what are you still doing here? [05:07] wwitzel3: just checking on reviews before going to bed :) [06:14] wallyworld: I've proposed another couple of branches. I'll now look at the last bits required to populate filesystem info for filesystems-on-volumes [06:14] can't land anything atm, as trunk is blocked [06:17] axw: ok, i'm still plowing through writing tests for the go-amz work, have to go to soccer soon, may look after i get back. i expect tmorrow we'll have EBS volume source working [06:18] wallyworld: awesome :) [06:18] axw: oh, and that will include attach, detach [06:18] great [06:18] wallyworld: I haven't removed VolumeParams.Attachment yet, will defer that for now [06:19] but it's still necessary to have separate attach/detach anyway [06:19] axw: sure, np, yep [06:19] and it's not much extra work to do it up front [06:35] wallyworld, hey there, do you know if thumper's on leave? [06:48] dimitern: he was here before, but it's past his EOD now [06:49] wallyworld, yeah, I was wondering about that i386 issue, but I've seen menn0 proposed a fix for it.. and now we have another blocker [06:49] i saw the i386 fix (reviewed it) but hadn't seen the other blocker [06:51] #1433384 - some leadership fallout it seems [06:51] Bug #1433384: utopic unit-test failure: TrackerSuite.TestWaitMinionBecomeMinion

=== urulama|afk is now known as urulama [08:36] fwereade, hey [08:37] dimitern, heyhey [08:37] fwereade, I wonder if you can clarify the logic around worker/leadership/tracker_test.go - the durations chosen [08:38] fwereade, seems like refreshes() picks a halfRefreshes in the order or a few nanoseconds, which leads to frequent failures in TestWaitMinionBecomeMinion [08:39] fwereade, that's related to the current blocker for 1.23 - bug 1433384 [08:39] Bug #1433384: utopic unit-test failure: TrackerSuite.TestWaitMinionBecomeMinion

[08:40] dimitern, hmm, did I screw up the logic? it ought to always pick (2n+1)/2 * coretesting.ShortWait for integer N [08:42] fwereade, hmm so that looks like it needs to yield a duration around 1.5 - 2.5 x shortWait [08:42] dimitern, yeah, it shouldn't ever return anything less than half shortWait [08:43] dimitern, and looking at the code I can't immediately see how it's not doing that [08:43] fwereade, depending on N ofc; the trouble is if you run the worker/leadership tests a few times - that same test (and only it so far I can see) fails - it happens to me about 50% of the time [08:45] fwereade, hmm, actually it's a lot less than 50% [08:45] fwereade, can you try running this on 1.23 in worker/leadership?: time go test -c && for i in `seq 50`; do ./leadership.test & sleep 0.01; done [08:46] dimitern, yeah, it looks like I *can*induce failures, but I'm not seeing the one you mentioned above [08:47] fwereade, but is the ultimate failure the same - got unexpected readiness: true ? [08:47] morning all [08:47] mattyw, o/ [08:47] dimitern, afraid not -- I'm seeing occasional failures to unblock the release goroutine in TestFailGainLeadership [08:48] mattyw, o/ [08:49] dimitern, and they're not guaranteed to trigger, whether running with seq 50 or seq 500 [08:50] fwereade, :) ok, there goes my one clue of the cause of the failure [08:51] fwereade, I'll experiment on i386 to see if it makes a difference [08:55] dimitern, hmm, seq 5000 seems to be stressing it a bit more, I managed to get 21 failures -- but *all* of them are in unblockRelease :/ [08:55] fwereade, are you using go 1.2.1 from the archive? [08:56] dimitern, hmm, looks like I'm on 1.2 actually [08:57] dimitern, (I think the unblockRelease *is* an issue actually -- we ought to have a generous timeout there instead of the default: case) [08:57] fwereade, right, sounds fair [09:00] dimitern, yeah, with coretesting.LongWait in unblockRelease I can do seq 5000 without failures [09:04] fwereade, nice! well, if I have success with that other test on i386 I can propose a fix for both [09:08] dimitern, cool, thanks [09:32] fwereade, it gets weirder.. on a i386 machine even with seq 5000 I couldn't reproduce the TestWaitMinionBecomeMinion failure [09:33] dimitern, blargh [09:33] fwereade, I had ~25 failures and all of them related to unblockRelease [09:55] * fwereade out, laura's school, bbs [10:02] dimitern, voidspace, TheMue: hangout? [10:02] dooferlad: yes [10:02] dooferlad, sorry, omw [10:41] I need to be able to alias juju destroy-enviroment to juju destroy-environment [10:41] ...or learn to type/spell [11:27] natefinch, juju burn-more-fossil-fuels? [11:36] natefinch, would you mind taking another look at http://reviews.vapour.ws/r/1005/? I'm going to fix the test up - but I'll do it in a follow up [12:29] Bug #1433566 was opened: Precise unit test failure discoverySuite.TestDiscoverInitSystemScript

[12:36] virt-manager changes the owner of iso images you use to create kvm images from! [12:37] voidspace, yeah, because they're in added to the libvirt storage pool [12:37] dimitern: the iso image? [12:37] that seems unfriendly [12:37] voidspace, indeed [12:38] dimitern: I can now create a storage pool - but can't read the iso image. Which is odd. I think it's because the iso image is behind a symlink [12:39] well, copying elsewhere worked - for whatever reason [12:39] voidspace, :) go figure [12:41] dimitern: I think the type of access it uses doesn't like symlinks [12:41] block level [12:42] voidspace, why are there symlinks in there? [12:43] dimitern: I moved the directory and left a symlink in the old location [12:43] dimitern: ~/Downloads [12:43] dimitern: it's where the iso image is [12:44] dimitern: for the storage volumes I used bind mount to move them [12:44] dimitern: which may or may not be necessary - but works [12:44] dimitern: I just needed to set the right owner/group on the files as well [12:46] voidspace, hmm bind mounts should work better than symlinks [12:47] voidspace, can't you specify default owner/perms at mount time of the bind? [12:49] dimitern: the bind mount is working fine [12:49] dimitern: the ~/Downloads symlink is also fine normally [12:50] dimitern: it's just that because it's a symlink virt-manager can't get block level access to iso images there [12:50] dimitern: I should have specified the full path instead of via the symlink - just habit [12:50] voidspace, ah, I got you [12:50] fwereade, are you around? [12:59] Bug #1433577 was opened: Vivid unit tests need to pass

[13:05] Bug #1433577 changed: Vivid unit tests need to pass

[13:11] Bug #1433577 was opened: Vivid unit tests need to pass

[13:35] mattyw: that severity is different than the severity you were using before... is that right? [13:53] natefinch, it's changed a little bit, the logic had changed with it though yes [13:53] natefinch, I think this order makes more sense [13:57] dimitern: FYI, bug 1433577 looks like the same issue as bug 1433384, but on vivid [13:57] Bug #1433577: Vivid unit tests need to pass

[13:57] Bug #1433384: unit-test failure: TrackerSuite.TestWaitMinionBecomeMinion

[13:58] ericsnow, well part of it is [13:58] dimitern: right [14:00] ericsnow, fwereade is already on #1433384 btw [14:00] Bug #1433384: unit-test failure: TrackerSuite.TestWaitMinionBecomeMinion

[14:00] dimitern: ah, great [14:01] ericsnow, I wanted to check something with you re systemd support [14:02] dimitern: sure [14:02] ericsnow, looking at https://github.com/juju/juju/blob/master/container/lxc/clonetemplate.go#L129-L172 [14:03] dimitern: yep [14:03] ericsnow, and the corresponding test - https://github.com/juju/juju/blob/master/container/lxc/lxc_test.go#L811-L841 [14:04] ericsnow, do you think this will work for systemd? I mean for upstart it does (it's live tested), but I wasn't sure if you can have multiple commands in ExecStart (provided all of them have abs. paths) [14:06] dimitern: systemd only allows a single command for ExecStart (or any Exec*) [14:06] dimitern: however, the systemd code in juju turns a multi-line ExecStart into a separate shell script that ExecStart then calls [14:08] ericsnow, so it will work? [14:09] dimitern: it should work fine; I'll look more closely at it in a minute [14:09] ericsnow: wwitzel3 ? [14:09] ericsnow, thanks! [14:29] Bug #1433615 was opened: Manual provider fails to bootstrap: failed to check provisioned status ... /: Is a directory

[14:45] natefinch, ericsnow, fwereade, dimitern thank you all for tackling critical bugs for 1.23 quickly [14:46] alexisb: blame my feeling of guilt :) [14:47] ericsnow, given the number of features we landed in 1.23 we will be seeing a fair amount of guild :) [14:48] we will just need to tackle the bugs quickly and get things hardened and stable, ready for the release [14:48] ericsnow, :) guilt usually works for me as well [14:49] alexisb, thanks! [15:06] fwereade: you free to meet briefly? [15:07] ericsnow, oops, sorry, with you in 2 [15:07] fwereade: no worries === kadams54 is now known as kadams54-away === kadams54-away is now known as kadams54 === kadams54 is now known as kadams54-away [16:02] mgz, sinzui: FATAL: Unable to delete script file /tmp/hudson7105716500738528055.sh ? [16:02] gah, why the heck is machine 0 running without the environmanager job when it first boots up? :/ [16:04] oops, nevermind... figured out what I was doing wrong :) === kadams54-away is now known as kadams54 [16:41] man I hate it when people type assert stuff and then if the type assert fails... don't tell me what type it actually was in the error - https://github.com/juju/juju/blob/master/apiserver/watcher.go#L90 [16:49] natefinch: at least log the type right? [16:50] natefinch: not sure if it's a potential security issue to reveal the type in the error? [16:51] * jw4 out for a couple hours [17:22] dimitern, can you clarify http://reviews.vapour.ws/r/1165/diff/3/?file=42926#file42926line396 ? [17:23] dimitern, that's a comment I added (in what I thought was the right place) to address my remembering someone had once asked why we did that, and that it wasn't immediately clear from context [17:24] dimitern, if I explained it nearer the top of the method it'd be further away from the code it's explaining and likely to rot even faster than most comments [17:48] fwereade: I hate that this code doesn't differentiate between "not found" and "wrong type" :/ https://github.com/juju/juju/blob/master/apiserver/watcher.go#L90 === kadams54 is now known as kadams54-away === kadams54-away is now known as kadams54 [18:00] natefinch, I think the error is reasonable for both cases, but yes, when debugging you'd probably want to know more [18:01] abentley, if you're around I appear to have broken the bot with http://juju-ci.vapour.ws:8080/job/github-merge-juju/2552/ [18:09] fwereade: looking... [18:14] fwereade: do you have an idea what the cause is here? I don't understand why it can't delete the script. I can (try to) delete the script, but I imagine the same thing will happen with the next run. [18:14] abentley, I have no idea, I'm afraid [18:16] fwereade: AFAICT, the script is being deleted by jenkins, and it has 0644 with ownership jenkins:jenkins, and /tmp (the parent directory) is 0777 [18:17] fwereade: I might as well try [18:17] fwereade: I've deleted the script and triggered a rebuild. [18:19] fwereade: It seems to be getting further: http://juju-ci.vapour.ws:8080/job/github-merge-juju/2553/console [18:19] abentley, cool, thanks [18:29] dimitern: ping [18:29] dimitern: you shouldn't be here even if you are :-) [18:29] dimitern: but just in case... [18:30] dimitern: running upgrade with loggging shows that the life field is correctly set to "dead" for a destroyed container IP address [18:30] dimitern: so I'm proposing the logging branch (just two logging lines) [18:30] dimitern: and moving onto the worker === arosales_ is now known as arosales [19:11] you know it's bad when I can't even tell WTF is going on with a full traceback. [19:11] this watcher code is a twisted mess [19:21] natefinch: there are worse, beware of what you wish [19:21] I have been around watcher when changin state, need a hand? [19:23] perrito666: yeah... just trying to debug why this watcher is causing an error [19:24] perrito666: brb [19:24] heh, seems my help causes people to flee :p [19:28] perrito666: back [19:28] perrito666: let's jump in moonstone === kadams54 is now known as kadams54-away [19:38] wwitzel3: you around? === kadams54 is now known as kadams54-away [20:04] natefinch: yep [20:06] wwitzel3: was the watcher working for you? I keep getting a problem where the state server says it's trying to find a watcher with an empty id... but I can't figure out why, because the watcher code is all so twisty [20:07] wwitzel3: nope. that is the same error I left it at last night, I'm working on that no, trying to get it fixed. [20:07] natefinch: [20:08] natefinch: the setup runs, the handle runs (which is our usual throw away the first run stuff), then I get that unknown watcher id [20:09] wwitzel3: ok, cool (sorta). I made a branch with more logging on my github called nate-ww3-ha-to [20:10] it also has some of the work to actually do stuff inside the watcher when it triggers... but yeah, I haven't been able to figure out the unknown watcher ID thing either [20:10] juju local behaves much like the ring from lotr, when you most need it it leaves you stranded [20:10] wwitzel3: I know that it's because it's somehow getting an empty id in newNotifyWatcher, but can't figure out why [20:10] natefinch: right, and the Next attempt throws the error [20:11] abentley, ping [20:11] abentley, are you able to target https://bugs.launchpad.net/juju-core/+bug/1423936 to 1.22 ? [20:11] Bug #1423936: Juju backup fails when journal files are present

[20:15] niedbalski: I can. [20:16] o perrito666 look a backup and restore bug, your favorite [20:16] lol [20:16] alexisb: it is my favorite [20:16] because natefinch has a patch for it [20:16] :p [20:17] looks like niedbalski has fixed it for you though [20:17] and by natefinch I meant niedbalski [20:17] haha I was going to say... news to me [20:17] heh exactly [20:17] natefinch: you should have seen your face [20:17]