[00:11] wallyworld: I think I figured it out (notes logged to the lp issue) [00:12] ericsnow: sorry, still in meeting [00:12] wallyworld: the worker closes the channel when its loop finishes; so if the runner restarts the worker then it's using a closed channel [00:12] wallyworld: np :) [00:12] wallyworld: and I'm sorry you're stuck in meetings :/ [00:48] ericsnow: ah, good pickup. i now have 40 mins till next meeting :-) [01:13] wallyworld: yeah, I'm not aware of the precedent for dealing with that situation and simple solution isn't coming to mind [01:13] wallyworld: perhaps we should only close the channel when the worker's tomb is killed [01:16] Bug #1494542 opened: unit does not go to error state [01:19] ericsnow: maybe, i need to look atthe code in detail, haven't had a chance yet [01:20] wallyworld: np, I've got to EOD so if you don't pick it up I will tomorrow === natefinch-afk is now known as natefinch [02:17] waigani: hey, merge my PR! https://github.com/waigani/GoOracle/pull/19 [02:18] natefinch: yikes, I haven't looked at this in a while! thanks :) [02:18] waigani: :) Figured as a coworker, I had the right to poke you, especially since it's just docs ;) [02:20] natefinch: done [02:20] waigani: awesome :) [02:20] I know how it is. I have a couple outstanding PRs on my projects I need to attend to. [03:27] wallyworld: just managed to bootstrap :) I manually added a public IP to machine-0, but it's not hard to do that in code [03:27] axw: whoohoo [03:27] awesome [03:27] wallyworld: I'll write up my feedback email now. [03:28] tyvm [03:28] be gentle with them :-) [03:28] wallyworld: :) [03:38] gah, our inconsistent use of Id vs ID is sooo annoying [04:23] wallyworld: sent a big wall of text. lunch time [04:40] axw: np, ty, was just otp [05:29] well this week has pretty much sucked [05:29] glad it's over [05:29] time for wine [05:29] * thumper hopes for better next week === urulama__ is now known as urulama [07:36] TheMue: (not sure if message arrived) when you have time, could you please take a look at http://reviews.vapour.ws/r/2628/ ? note that this work is proposed for merging into a feature branch currently. thanks! [07:38] frankban: yep, will take a look [07:38] TheMue: ty === akhavr1 is now known as akhavr [08:05] frankban: you've got a review [08:06] TheMue: thanks! [09:03] dimitern, TheMue: standup time [09:04] frobware, we're there [09:04] frankban: we're, retrospective and planning [09:04] frobware, ah, sorry - it's a different link today [09:05] oops, frobware [09:05] https://plus.google.com/hangouts/_/canonical.com/sapphire [09:05] aha [09:16] TheMue: could you also take a look at the followup branch in http://reviews.vapour.ws/r/2630/ (still proposed against the feature branch)? [09:17] frankban: currently in retrospective HO, but then, yes [09:17] TheMue: ty! [09:38] Bug #1494661 opened: Old logs are not compressed or removed [11:33] dimitern, reviewed http://reviews.vapour.ws/r/2593/ at last, you might have thoughts re some of my comments [11:35] fwereade, sweet! thanks! [11:35] morning all [11:37] fwereade: could you re-look at http://reviews.vapour.ws/r/2592/ [11:37] pretty please? [11:37] perrito666, looking [11:46] * perrito666 really wishes he wasn't freezing [12:10] dimitern: frobware: TheMue: dooferlad: morning! [12:13] voidspace: heya, you missed a pretty long meeting today. ;) [12:18] TheMue: full team meeting? [12:18] voidspace: retro/planning [12:18] TheMue: ah! shame [12:18] TheMue: I should check the board [12:18] TheMue: it's just past 8am here, I still need coffee! [12:19] voidspace: no, too early for your time zone. you're still in the US? [12:19] TheMue: I'm back in Europe on Monday [12:20] TheMue: yep, still at Wayne's house [12:20] TheMue: he's now married and the big celebration is tomorrow [12:20] voidspace: relaxing after the wedding party :D [12:20] voidspace: oh, tomorrow, ic [12:22] frankban: you've got a review [12:22] TheMue: ty! [12:23] voidspace, hey there :) welcome back! [12:26] TheMue: great suggestion Errorf, I have been looking for something like that but did not find it for some reason [12:27] frankban: I would have called it Newf, so it's more easy to find. *bg* [12:27] TheMue: yeah [12:44] Bug #1494726 opened: TestGetUsesCache different mime-type centos

[12:44] Bug #1494729 opened: Panic created by net/http.fileTransport.RoundTrip

[12:47] voidspace, hiya - let's catch up monday as I have a couple of meetings RSN [12:55] dimitern, want to sync up? [12:58] frobware, I started to summarize my thoughts around the needed changes - how about I finish them and send you a mail with those? [12:58] dimitern, sure [12:59] frobware: ok, no problem [12:59] frobware, cheers [13:01] fwereade, ping? [13:01] mattyw, pong [13:02] fwereade, do you have 10 minutes to receiving a whinge? [13:02] mattyw, certainly, just between reviews :) [13:02] Bug #1494734 opened: Panic in jujud/agent on ppc64el [13:08] Bug #1494734 changed: Panic in jujud/agent on ppc64el [13:14] Bug #1494734 opened: Panic in jujud/agent on ppc64el [13:21] Bug #1494743 opened: TestMachineAgentRunsCertificateUpdateWorkerForStateServer timeout

=== tvansteenburgh1 is now known as tvansteenburgh [13:49] TheMue: time for a quick third and last one along the same line? http://reviews.vapour.ws/r/2633/ [13:50] Hello all. I have a service comprised of three units (all on different machines). If one of those machines disappears, should I expect to see one or both of cluster-{broken,departed} to be triggered on the other units? [13:51] Bug #1494749 opened: TestWorkerRetriesOnPublishError fails on wily

[13:51] Bug #1494754 opened: TestActionsReceived failed on precise

[13:51] (tvansteenburgh: ^^^) [13:52] Odd_Bloke: yeah, i'm eagerly awaiting the answer :) [13:56] is this the only place where users can look in order to choose a tools version to upgrade to? https://streams.canonical.com/juju/tools/releases/ [13:58] regarding #1494661, didn't we fix log rotation (including compressing/deleting) already? [13:58] Bug #1494661: Old logs are not compressed or removed [14:02] pmatulis: yes, those are the latest stable tools, there's also a devel folder with newer, but usually alpha or beta versions [14:13] fwereade: you have a few minutes (in a little while) to talk about how to fix #1493123? [14:13] Bug #1493123: Upgrade in progress reported, but panic happening behind scenes

[14:13]

[14:14] ericsnow, sure [14:14] fwereade: OTP, but I'll ping you soon :) [14:22] fwereade: you free? [14:23] ericsnow, yeah [14:24] Bug #1494765 opened: TestWorkerRemovesDeadAddress fails on ppc64el

[14:24] Bug #1494774 opened: MetricSenderSuite fails on ppc64el panic

[14:25] my juju is in a state where [14:25] "juju bootstrap -e local" hangs forever [14:25] after "Bootstrapping Juju machine agent" [14:25] anyone have an idea what might be going on? [14:26] tried --debug? [14:26] i can't even interrupt the bootstrap [14:26] it's hung at "Interrupt signalled: waiting for bootstrap to exit" [14:27] bogdanteleaga: so i *would* try --debug if i could get it to stop [14:27] C^Z and kill process? :) [14:27] yeah, i think i'll sigquit it [14:27] might have to clean up after it if you do that tho [14:27] bogdanteleaga: any idea what i might need to clean up? [14:28] any machines/networks it created I'm thinking [14:28] I usually just power off maas machines [14:28] not sure about local [14:28] bogdanteleaga: lxc-ls shows one entry [14:28] juju-trusty-lxc-template [14:29] * rogpeppe runs lxc-destroy [14:30] bogdanteleaga: i've got an lxcbr0 network interface [14:30] bogdanteleaga: do you think i should remove that too? [14:31] rogpeppe, hmm, not sure, I'm really not familiar with lxc [14:31] rogpeppe, the only lxc machine I have seems to not create that [14:32] bogdanteleaga: alright thanks [14:33] ha, "destroy-environment -y local" still leaves me with an entry in environments/cache.yaml [14:33] with an empty key [14:33] try --force [14:33] bogdanteleaga: i did [14:33] bogdanteleaga: (sorry, i didn't mention that) [14:33] * rogpeppe removes cache.yaml [14:33] rogpeppe, I never actually saw a cache.yaml before [14:34] bogdanteleaga: you only get it with the jes feature flag [14:34] rogpeppe, that makes sense :) [14:36] ericsnow: natefinch: ready? [14:36] yep [14:40] bogdanteleaga: for future reference: looking in ~/.juju/local/log/machine-0.log is dead useful :) [14:41] rogpeppe, I'll cherish the day I get to use local and not worry about boringly long deployment times [14:42] bogdanteleaga: :) [14:46] bogdanteleaga: filed https://bugs.launchpad.net/juju-core/+bug/1494787 [14:46] Bug #1494787: bootstrap cannot be interrupted if machine agent fails to start [14:57] Bug #1494782 opened: should *-broken *-departed hooks run when a unit goes AWOL? [14:57] Bug #1494787 opened: bootstrap cannot be interrupted if machine agent fails to start [15:15] Bug #1494774 changed: MetricSenderSuite fails on ppc64el panic

[15:15] Bug #1494798 opened: Juju fails to report it cannot create buckets

[15:18] Bug #1494798 changed: Juju fails to report it cannot create buckets

[15:18] Bug #1494774 opened: MetricSenderSuite fails on ppc64el panic

[15:21] Bug #1494774 changed: MetricSenderSuite fails on ppc64el panic

[15:21] Bug #1494798 opened: Juju fails to report it cannot create buckets

[15:27] Bug #1455625 opened: TestStateWatcherTwoEnvironments fails

[15:30] Bug #1455625 changed: TestStateWatcherTwoEnvironments fails

[15:39] Bug #1455625 opened: TestStateWatcherTwoEnvironments fails

[15:56] natefinch: hey can you update bug 1486553 with the latest status? [15:56] Bug #1486553: i/o timeout errors can cause non-atomic service deploys

[16:09] review anyone for a critical fix? katco, ericsnow, etc? http://reviews.vapour.ws/r/2636/ [16:09] breaking for lunch === natefinch is now known as natefinch-afk [16:10] natefinch-afk: I'll take a look [16:35] https://bugs.launchpad.net/charms/+source/openstack-dashboard/+bug/1494829 [16:35] Bug #1494829: Support for custom panels

[16:42] Bug #1494831 opened: Windows instances on GCE will have the same hostname [16:45] Bug #1494831 changed: Windows instances on GCE will have the same hostname [16:48] Bug #1494831 opened: Windows instances on GCE will have the same hostname [17:12] Bug #1494848 opened: 1.24+ cannot upgrade in canonistack

[17:24] Bug #1494848 changed: 1.24+ cannot upgrade in canonistack

[17:27] Bug #1494848 opened: 1.24+ cannot upgrade in canonistack

=== natefinch-afk is now known as natefinch [17:39] ericsnow: lol @ comment about using a struct for the params... i have code doing exactly that on my machine, but getting this fix in first. [17:39] natefinch: np :) [17:40] natefinch: it's just less churn overall if you do it in this patch [17:40] ericsnow: yeah, but this patch needs to be in ASAP. I was going to do it all together, but it turned out this fix was needed faster than we expected. [17:40] natefinch: k [17:41] natefinch: +1 [17:41] ericsnow: making a test for it right now [17:41] natefinch: thanks [18:04] katco: gah, 1.25 is blocked. [18:05] natefinch: hmm [18:07] mgz: ping? [18:09] katco: hey [18:09] mgz: we'd like to land a fix for bug 1486553 [18:09] Bug #1486553: i/o timeout errors can cause non-atomic service deploys

[18:10] mgz: but master is blocked [18:10] mgz: thoughts on jfdi? [18:10] yeah, please don't [18:10] katco: 1.25 [18:10] look at the history of OS-deployer job [18:10] katco: (I mean, master is also blocked, but the bug is for 1.25) [18:10] natefinch: mgz: oops sorry, for 1.25 [18:10] fine on 1.24 and feature branches [18:10] screwed on 1.25 and master [18:11] it's already too hard to pin down cause because too much stuff was piled on [18:11] mgz: this is a fairly hot bug [18:11] katco: if 1.25 is broken currently, getting my fix in won't help get it to customers [18:11] sure, but we can't release in the current state [18:11] so it's not like that fix on 1.25 will get anywhere [18:12] mgz: I guess we're past the point of being able to revert whatever it is? [18:12] natefinch: master can be reverted back to some kind of sanity [18:12] Bug #1494864 opened: TestBlockChangeServiceUpdate fails on windows

[18:12] Bug #1494868 opened: TestAzureWindows fails on wily and centos

[18:12] natefinch: how hard would it be to get into 1.24? [18:12] Bug #1494870 opened: TestMeterStatusEvents fails on wily and vivid

[18:13] but yeah, as no one is taking my suggestion on backing out broken changes we now have regressions without clear blame [18:15] mgz: ty for the input... we won't land in 1.25 right now. natefinch, just learned this fix is needed in 1.24.6, so let's land there [18:15] katco: even better. [18:15] mgz: http://cdn.meme.am/instances2/500x/1820980.jpg [18:23] very small review for a critical bug on 1.24.6: http://reviews.vapour.ws/r/2638/ [18:23] mgz: are wily and centos tests running with another go version? [18:24] bogdanteleaga: wily uses go 1.5 [18:25] sinzui: what version of go is on centos? [18:25] bug 1494868 above is really interesting [18:25] Bug #1494868: TestAzureWindows fails on wily and centos

[18:25] it should fail everywhere [18:25] but it fails only on wily and centos [18:25] because the test doesn't get ran on other platforms [18:27] bogdanteleaga: that is interesting. [18:27] mgz: bogdanteleaga looking. I thought the test log printed that for us [18:28]