[00:11] <ericsnow> wallyworld: I think I figured it out (notes logged to the lp issue)
[00:12] <wallyworld> ericsnow: sorry, still in meeting
[00:12] <ericsnow> wallyworld: the worker closes the channel when its loop finishes; so if the runner restarts the worker then it's using a closed channel
[00:12] <ericsnow> wallyworld: np :)
[00:12] <ericsnow> wallyworld: and I'm sorry you're stuck in meetings :/
[00:48] <wallyworld> ericsnow: ah, good pickup. i now have 40 mins till next meeting :-)
[01:13] <ericsnow> wallyworld: yeah, I'm not aware of the precedent for dealing with that situation and simple solution isn't coming to mind
[01:13] <ericsnow> wallyworld: perhaps we should only close the channel when the worker's tomb is killed
[01:16] <mup> Bug #1494542 opened: unit does not go to error state <juju-core:New> <https://launchpad.net/bugs/1494542>
[01:19] <wallyworld> ericsnow: maybe, i need to look atthe code in detail, haven't had a chance yet
[01:20] <ericsnow> wallyworld: np, I've got to EOD so if you don't pick it up I will tomorrow
[02:17] <natefinch> waigani: hey, merge my PR!  https://github.com/waigani/GoOracle/pull/19
[02:18] <waigani> natefinch: yikes, I haven't looked at this in a while! thanks :)
[02:18] <natefinch> waigani: :)  Figured as a coworker, I had the right to poke you, especially since it's just docs ;)
[02:20] <waigani> natefinch: done
[02:20] <natefinch> waigani: awesome :)
[02:20] <natefinch> I know how it is. I have a couple outstanding PRs on my projects I need to attend to.
[03:27] <axw> wallyworld: just managed to bootstrap :)  I manually added a public IP to machine-0, but it's not hard to do that in code
[03:27] <wallyworld> axw: whoohoo
[03:27] <wallyworld> awesome
[03:27] <axw> wallyworld: I'll write up my feedback email now.
[03:28] <wallyworld> tyvm
[03:28] <wallyworld> be gentle with them :-)
[03:28] <axw> wallyworld: :)
[03:38] <natefinch> gah, our inconsistent use of Id vs ID is sooo annoying
[04:23] <axw> wallyworld: sent a big wall of text. lunch time
[04:40] <wallyworld> axw: np, ty, was just otp
[05:29] <thumper> well this week has pretty much sucked
[05:29] <thumper> glad it's over
[05:29] <thumper> time for wine
[05:29]  * thumper hopes for better next week
[07:36] <frankban> TheMue: (not sure if message arrived) when you have time, could you please take a look at http://reviews.vapour.ws/r/2628/ ? note that this work is proposed for merging into a feature branch currently. thanks!
[07:38] <TheMue> frankban: yep, will take a look
[07:38] <frankban> TheMue: ty
[08:05] <TheMue> frankban: you've got a review
[08:06] <frankban> TheMue: thanks!
[09:03] <frobware> dimitern, TheMue: standup time
[09:04] <dimitern> frobware, we're there
[09:04] <TheMue> frankban: we're, retrospective and planning
[09:04] <dimitern> frobware, ah, sorry - it's a different link today
[09:05] <TheMue> oops, frobware
[09:05] <TheMue> https://plus.google.com/hangouts/_/canonical.com/sapphire
[09:05] <frobware> aha
[09:16] <frankban> TheMue: could you also take a look at the followup branch in http://reviews.vapour.ws/r/2630/ (still proposed against the feature branch)?
[09:17] <TheMue> frankban: currently in retrospective HO, but then, yes
[09:17] <frankban> TheMue: ty!
[09:38] <mup> Bug #1494661 opened: Old logs are not compressed or removed <juju-core:New> <https://launchpad.net/bugs/1494661>
[11:33] <fwereade> dimitern, reviewed http://reviews.vapour.ws/r/2593/ at last, you might have thoughts re some of my comments
[11:35] <dimitern> fwereade, sweet! thanks!
[11:35] <perrito666> morning all
[11:37] <perrito666> fwereade: could you re-look at http://reviews.vapour.ws/r/2592/
[11:37] <perrito666> pretty please?
[11:37] <fwereade> perrito666, looking
[11:46]  * perrito666 really wishes he wasn't freezing
[12:10] <voidspace> dimitern: frobware: TheMue: dooferlad: morning!
[12:13] <TheMue> voidspace: heya, you missed a pretty long meeting today. ;)
[12:18] <voidspace> TheMue: full team meeting?
[12:18] <TheMue> voidspace: retro/planning
[12:18] <voidspace> TheMue: ah! shame
[12:18] <voidspace> TheMue: I should check the board
[12:18] <voidspace> TheMue: it's just past 8am here, I still need coffee!
[12:19] <TheMue> voidspace: no, too early for your time zone. you're still in the US?
[12:19] <voidspace> TheMue: I'm back in Europe on Monday
[12:20] <voidspace> TheMue: yep, still at Wayne's house
[12:20] <voidspace> TheMue: he's now married and the big celebration is tomorrow
[12:20] <TheMue> voidspace: relaxing after the wedding party :D
[12:20] <TheMue> voidspace: oh, tomorrow, ic
[12:22] <TheMue> frankban: you've got a review
[12:22] <frankban> TheMue: ty!
[12:23] <dimitern> voidspace, hey there :) welcome back!
[12:26] <frankban> TheMue: great suggestion Errorf, I have been looking for something like that but did not find it for some reason
[12:27] <TheMue> frankban: I would have called it Newf, so it's more easy to find. *bg*
[12:27] <frankban> TheMue: yeah
[12:44] <mup> Bug #1494726 opened: TestGetUsesCache different mime-type centos <centos> <ci> <test-failure> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1494726>
[12:44] <mup> Bug #1494729 opened: Panic created by net/http.fileTransport.RoundTrip <ci> <panic> <test-failure> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1494729>
[12:47] <frobware> voidspace, hiya - let's catch up monday as I have a couple of meetings RSN
[12:55] <frobware> dimitern, want to sync up?
[12:58] <dimitern> frobware, I started to summarize my thoughts around the needed changes - how about I finish them and send you a mail with those?
[12:58] <frobware> dimitern, sure
[12:59] <voidspace> frobware: ok, no problem
[12:59] <dimitern> frobware, cheers
[13:01] <mattyw> fwereade, ping?
[13:01] <fwereade> mattyw, pong
[13:02] <mattyw> fwereade, do you have 10 minutes to receiving a whinge?
[13:02] <fwereade> mattyw, certainly, just between reviews :)
[13:02] <mup> Bug #1494734 opened: Panic in jujud/agent on ppc64el <juju-core:New> <https://launchpad.net/bugs/1494734>
[13:08] <mup> Bug #1494734 changed: Panic in jujud/agent on ppc64el <juju-core:New> <https://launchpad.net/bugs/1494734>
[13:14] <mup> Bug #1494734 opened: Panic in jujud/agent on ppc64el <juju-core:New> <https://launchpad.net/bugs/1494734>
[13:21] <mup> Bug #1494743 opened: TestMachineAgentRunsCertificateUpdateWorkerForStateServer timeout <ci> <i386> <test-failure> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1494743>
[13:49] <frankban> TheMue: time for a quick third and last one along the same line? http://reviews.vapour.ws/r/2633/
[13:50] <Odd_Bloke> Hello all.  I have a service comprised of three units (all on different machines).  If one of those machines disappears, should I expect to see one or both of cluster-{broken,departed} to be triggered on the other units?
[13:51] <mup> Bug #1494749 opened: TestWorkerRetriesOnPublishError fails on wily <ci> <intermittent-failure> <lxc> <unit-tests> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494749>
[13:51] <mup> Bug #1494754 opened: TestActionsReceived failed on precise <ci> <intermittent-failure> <precise> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1494754>
[13:51] <Odd_Bloke> (tvansteenburgh: ^^^)
[13:52] <tvansteenburgh> Odd_Bloke: yeah, i'm eagerly awaiting the answer :)
[13:56] <pmatulis> is this the only place where users can look in order to choose a tools version to upgrade to? https://streams.canonical.com/juju/tools/releases/
[13:58] <ericsnow> regarding #1494661, didn't we fix log rotation (including compressing/deleting) already?
[13:58] <mup> Bug #1494661: Old logs are not compressed or removed <juju-core:New> <https://launchpad.net/bugs/1494661>
[14:02] <bogdanteleaga> pmatulis: yes, those are the latest stable tools, there's also a devel folder with newer, but usually alpha or beta versions
[14:13] <ericsnow> fwereade: you have a few minutes (in a little while) to talk about how to fix #1493123?
[14:13] <mup> Bug #1493123: Upgrade in progress reported, but panic happening behind scenes <landscape> <landscape-release-29> <upgrade-juju> <juju-core:In Progress by ericsnowcurrently>
[14:13] <mup> <juju-core 1.24:In Progress by ericsnowcurrently> <juju-core 1.25:In Progress by ericsnowcurrently> <https://launchpad.net/bugs/1493123>
[14:14] <fwereade> ericsnow, sure
[14:14] <ericsnow> fwereade: OTP, but I'll ping you soon :)
[14:22] <ericsnow> fwereade: you free?
[14:23] <fwereade> ericsnow, yeah
[14:24] <mup> Bug #1494765 opened: TestWorkerRemovesDeadAddress fails on ppc64el <ci> <intermittent-failure> <ppc64el> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1494765>
[14:24] <mup> Bug #1494774 opened: MetricSenderSuite fails on ppc64el panic <ci> <intermittent-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1494774>
[14:25] <rogpeppe> my juju is in a state where
[14:25] <rogpeppe> "juju bootstrap -e local" hangs forever
[14:25] <rogpeppe> after "Bootstrapping Juju machine agent"
[14:25] <rogpeppe> anyone have an idea what might be going on?
[14:26] <bogdanteleaga> tried --debug?
[14:26] <rogpeppe> i can't even interrupt the bootstrap
[14:26] <rogpeppe> it's hung at "Interrupt signalled: waiting for bootstrap to exit"
[14:27] <rogpeppe> bogdanteleaga: so i *would* try --debug if i could get it to stop
[14:27] <bogdanteleaga> C^Z and kill process? :)
[14:27] <rogpeppe> yeah, i think i'll sigquit it
[14:27] <bogdanteleaga> might have to clean up after it if you do that tho
[14:27] <rogpeppe> bogdanteleaga: any idea what i might need to clean up?
[14:28] <bogdanteleaga> any machines/networks it created I'm thinking
[14:28] <bogdanteleaga> I usually just power off maas machines
[14:28] <bogdanteleaga> not sure about local
[14:28] <rogpeppe> bogdanteleaga: lxc-ls shows one entry
[14:28] <rogpeppe> juju-trusty-lxc-template
[14:29]  * rogpeppe runs lxc-destroy
[14:30] <rogpeppe> bogdanteleaga: i've got an lxcbr0 network interface
[14:30] <rogpeppe> bogdanteleaga: do you think i should remove that too?
[14:31] <bogdanteleaga> rogpeppe, hmm, not sure, I'm really not familiar with lxc
[14:31] <bogdanteleaga> rogpeppe, the only lxc machine I have seems to not create that
[14:32] <pmatulis> bogdanteleaga: alright thanks
[14:33] <rogpeppe> ha, "destroy-environment -y local" still leaves me with an entry in environments/cache.yaml
[14:33] <rogpeppe> with an empty key
[14:33] <bogdanteleaga> try --force
[14:33] <rogpeppe> bogdanteleaga: i did
[14:33] <rogpeppe> bogdanteleaga: (sorry, i didn't mention that)
[14:33]  * rogpeppe removes cache.yaml
[14:33] <bogdanteleaga> rogpeppe, I never actually saw a cache.yaml before
[14:34] <rogpeppe> bogdanteleaga: you only get it with the jes feature flag
[14:34] <bogdanteleaga> rogpeppe, that makes sense :)
[14:36] <katco> ericsnow: natefinch: ready?
[14:36] <natefinch> yep
[14:40] <rogpeppe> bogdanteleaga: for future reference: looking in ~/.juju/local/log/machine-0.log is dead useful :)
[14:41] <bogdanteleaga> rogpeppe, I'll cherish the day I get to use local and not worry about boringly long deployment times
[14:42] <rogpeppe> bogdanteleaga: :)
[14:46] <rogpeppe> bogdanteleaga: filed https://bugs.launchpad.net/juju-core/+bug/1494787
[14:46] <mup> Bug #1494787: bootstrap cannot be interrupted if machine agent fails to start <juju-core:New> <https://launchpad.net/bugs/1494787>
[14:57] <mup> Bug #1494782 opened: should *-broken *-departed hooks run when a unit goes AWOL? <juju-core:New> <https://launchpad.net/bugs/1494782>
[14:57] <mup> Bug #1494787 opened: bootstrap cannot be interrupted if machine agent fails to start <juju-core:New> <https://launchpad.net/bugs/1494787>
[15:15] <mup> Bug #1494774 changed: MetricSenderSuite fails on ppc64el panic <ci> <intermittent-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1494774>
[15:15] <mup> Bug #1494798 opened: Juju fails to report it cannot create buckets <ci> <ec2-provider> <storage> <juju-core:Triaged> <https://launchpad.net/bugs/1494798>
[15:18] <mup> Bug #1494798 changed: Juju fails to report it cannot create buckets <ci> <ec2-provider> <storage> <juju-core:Triaged> <https://launchpad.net/bugs/1494798>
[15:18] <mup> Bug #1494774 opened: MetricSenderSuite fails on ppc64el panic <ci> <intermittent-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1494774>
[15:21] <mup> Bug #1494774 changed: MetricSenderSuite fails on ppc64el panic <ci> <intermittent-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1494774>
[15:21] <mup> Bug #1494798 opened: Juju fails to report it cannot create buckets <ci> <ec2-provider> <storage> <juju-core:Triaged> <https://launchpad.net/bugs/1494798>
[15:27] <mup> Bug #1455625 opened: TestStateWatcherTwoEnvironments fails <ci> <test-failure> <wily> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1455625>
[15:30] <mup> Bug #1455625 changed: TestStateWatcherTwoEnvironments fails <ci> <test-failure> <wily> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1455625>
[15:39] <mup> Bug #1455625 opened: TestStateWatcherTwoEnvironments fails <ci> <test-failure> <wily> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1455625>
[15:56] <katco> natefinch: hey can you update bug 1486553 with the latest status?
[15:56] <mup> Bug #1486553: i/o timeout errors can cause non-atomic service deploys <cisco> <landscape> <juju-core:Triaged> <juju-core 1.25:In Progress by natefinch> <https://launchpad.net/bugs/1486553>
[16:09] <natefinch> review anyone for a critical fix?  katco, ericsnow, etc? http://reviews.vapour.ws/r/2636/
[16:09] <natefinch> breaking for lunch
[16:10] <ericsnow> natefinch-afk: I'll take a look
[16:35] <bdx> https://bugs.launchpad.net/charms/+source/openstack-dashboard/+bug/1494829
[16:35] <mup> Bug #1494829: Support for custom panels <designate> <designate-dashboard> <django> <horizon> <murano> <murano-dashboard> <openstack> <openstack-dashboard (Juju Charms Collection):New> <https://launchpad.net/bugs/1494829>
[16:42] <mup> Bug #1494831 opened: Windows instances on GCE will have the same hostname <juju-core:Incomplete> <https://launchpad.net/bugs/1494831>
[16:45] <mup> Bug #1494831 changed: Windows instances on GCE will have the same hostname <juju-core:Incomplete> <https://launchpad.net/bugs/1494831>
[16:48] <mup> Bug #1494831 opened: Windows instances on GCE will have the same hostname <juju-core:Incomplete> <https://launchpad.net/bugs/1494831>
[17:12] <mup> Bug #1494848 opened: 1.24+ cannot upgrade in canonistack <canonistack> <openstack-provider> <streams> <upgrade-juju> <juju-core:Triaged> <https://launchpad.net/bugs/1494848>
[17:24] <mup> Bug #1494848 changed: 1.24+ cannot upgrade in canonistack <canonistack> <openstack-provider> <streams> <upgrade-juju> <juju-core:Triaged> <https://launchpad.net/bugs/1494848>
[17:27] <mup> Bug #1494848 opened: 1.24+ cannot upgrade in canonistack <canonistack> <openstack-provider> <streams> <upgrade-juju> <juju-core:Triaged> <https://launchpad.net/bugs/1494848>
[17:39] <natefinch> ericsnow: lol @ comment about using a struct for the params... i have code doing exactly that on my machine, but getting this fix in first.
[17:39] <ericsnow> natefinch: np :)
[17:40] <ericsnow> natefinch: it's just less churn overall if you do it in this patch
[17:40] <natefinch> ericsnow: yeah, but this patch needs to be in ASAP.  I was going to do it all together, but it turned out this fix was needed faster than we expected.
[17:40] <ericsnow> natefinch: k
[17:41] <katco> natefinch: +1
[17:41] <natefinch> ericsnow: making a test for it right now
[17:41] <ericsnow> natefinch: thanks
[18:04] <natefinch> katco: gah, 1.25 is blocked.
[18:05] <katco> natefinch: hmm
[18:07] <katco> mgz: ping?
[18:09] <mgz> katco: hey
[18:09] <katco> mgz: we'd like to land a fix for bug 1486553
[18:09] <mup> Bug #1486553: i/o timeout errors can cause non-atomic service deploys <cisco> <landscape> <juju-core:In Progress by natefinch> <juju-core 1.25:In Progress by natefinch> <https://launchpad.net/bugs/1486553>
[18:10] <katco> mgz: but master is blocked
[18:10] <katco> mgz: thoughts on jfdi?
[18:10] <mgz> yeah, please don't
[18:10] <natefinch> katco: 1.25
[18:10] <mgz> look at the history of OS-deployer job
[18:10] <natefinch> katco: (I mean, master is also blocked, but the bug is for 1.25)
[18:10] <katco> natefinch: mgz: oops sorry, for 1.25
[18:10] <mgz> fine on 1.24 and feature branches
[18:10] <mgz> screwed on 1.25 and master
[18:11] <mgz> it's already too hard to pin down cause because too much stuff was piled on
[18:11] <katco> mgz: this is a fairly hot bug
[18:11] <natefinch> katco: if 1.25 is broken currently, getting my fix in won't help get it to customers
[18:11] <mgz> sure, but we can't release in the current state
[18:11] <mgz> so it's not like that fix on 1.25 will get anywhere
[18:12] <natefinch> mgz: I guess we're past the point of being able to revert whatever it is?
[18:12] <mgz> natefinch: master can be reverted back to some kind of sanity
[18:12] <mup> Bug #1494864 opened: TestBlockChangeServiceUpdate fails on windows <blocker> <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1494864>
[18:12] <mup> Bug #1494868 opened: TestAzureWindows fails on wily and centos <centos> <ci> <test-failure> <unit-tests> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494868>
[18:12] <katco> natefinch: how hard would it be to get into 1.24?
[18:12] <mup> Bug #1494870 opened: TestMeterStatusEvents fails on wily and vivid <ci> <race-condition> <test-failure> <unit-tests> <vivid> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494870>
[18:13] <mgz> but yeah, as no one is taking my suggestion on backing out broken changes we now have regressions without clear blame
[18:15] <katco> mgz: ty for the input... we won't land in 1.25 right now. natefinch, just learned this fix is needed in 1.24.6, so let's land there
[18:15] <natefinch> katco: even better.
[18:15] <natefinch> mgz: http://cdn.meme.am/instances2/500x/1820980.jpg
[18:23] <katco> very small review for a critical bug on 1.24.6: http://reviews.vapour.ws/r/2638/
[18:23] <bogdanteleaga> mgz: are wily and centos tests running with another go version?
[18:24] <mgz> bogdanteleaga: wily uses go 1.5
[18:25] <mgz> sinzui: what version of go is on centos?
[18:25] <bogdanteleaga> bug 1494868 above is really interesting
[18:25] <mup> Bug #1494868: TestAzureWindows fails on wily and centos <centos> <ci> <test-failure> <unit-tests> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494868>
[18:25] <bogdanteleaga> it should fail everywhere
[18:25] <bogdanteleaga> but it fails only on wily and centos
[18:25] <bogdanteleaga> because the test doesn't get ran on other platforms
[18:27] <mgz> bogdanteleaga: that is interesting.
[18:27] <sinzui> mgz: bogdanteleaga looking. I thought the test log printed that for us
[18:28] <mgz> sinzui: I see the juju version but not the go version
[18:28] <sinzui> mgz: our python test runner doesn't do a go version like our bash version :(
[18:28]  * sinzui visits the machine
[18:29] <sinzui> mgz: bogdanteleaga go version
[18:29] <sinzui> go version go1.4.2 linux/amd64
[18:29] <mgz> bogdanteleaga: thae windows-public-clouds branch was retested and passed, which further persuades me it's not the cause of recent breakage
[18:30] <mgz> sinzui: ta!
[18:31] <bogdanteleaga> http://reviews.vapour.ws/r/2640/
[18:32] <katco> natefinch: http://reviews.vapour.ws/r/2639/diff/# is the same as the one that was targetted to 1.25?
[18:32] <bogdanteleaga> mgz: well, it can't be since you got the same failures on master
[18:32] <bogdanteleaga> mgz: and, afaik that didn't get in master yet
[18:32] <mgz> bogdanteleaga: we got so many issues on master I was unpersuaded of anything
[18:32] <bogdanteleaga> :)
[18:33] <voidspace> fwereade: thanks for the review
[18:33] <voidspace> fwereade: I tried to make the changes without any changes to the external api - e.g. the allwatcher
[18:33] <voidspace> fwereade: where empty address was used to signal no address set yet
[18:33] <voidspace> fwereade: I think that's specifically been changed on master already though
[18:34] <voidspace> fwereade: so the same work on master looks better
[18:34] <natefinch> katco: yep
[18:34] <voidspace> fwereade: a bunch of good suggestions though (especially the transaction testing)
[18:34] <voidspace> fwereade: I have left a couple of replies, working through the others
[18:34] <katco> natefinch: go for it
[18:34] <katco> natefinch: and i could use a review for: http://reviews.vapour.ws/r/2638/
[18:35] <natefinch> katco: ok looking
[18:35] <katco> that goes for anyone ^^ 22 line change
[18:35] <katco> critical bug for 1.24.6
[18:36] <mgz> bogdanteleaga: renderers.ToBase64 returns a string?
[18:36] <bogdanteleaga> fwiw, you ought to get https://bugs.launchpad.net/juju-core/+bug/1493887 fixed on windows first
[18:36] <mup> Bug #1493887: statusHistoryTestSuite teardown fails on windows <blocker> <ci> <regression> <test-failure> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1493887>
[18:36] <bogdanteleaga> since it might break other tests
[18:36] <natefinch> katco: lgtm
[18:36] <bogdanteleaga> I had a mail about this kind of failure on the ml before
[18:36] <katco> cherylj: hey what's the status on bug 1494356 ?
[18:36] <mgz> bogdanteleaga: yeah, there's a bunch of things on master I'm afraid
[18:36] <mup> Bug #1494356: OS-deployer job fails to complete <blocker> <ci> <regression> <juju-core:Triaged by cherylj> <juju-core 1.25:Triaged by cherylj> <https://launchpad.net/bugs/1494356>
[18:36] <bogdanteleaga> mgz: it very well could return a string
[18:37] <bogdanteleaga> mgz: I just wanted to use deepequals everywhere for consistency
[18:37] <mgz> I mean, does it? it's a function with a call value
[18:37] <bogdanteleaga> mgz: now the tests runs on all platforms and it's good
[18:37] <bogdanteleaga> mgz: yeah, think of it like a transformer
[18:37] <mgz> just want to avoid the ugliness of a mismatch on []uint8
[18:38] <bogdanteleaga> mgz: that needs everything to move from jc.DeepEquals to str(x), gc.Equals, str(y)
[18:38] <bogdanteleaga> mgz: because some functions in those suites give back strings and some give back byte arrays
[18:38] <bogdanteleaga> mgz: can't really remember which do which right now
[18:39] <mgz> ;_;
[18:39] <mgz> bogdanteleaga: lgtmed
[18:39] <bogdanteleaga> mgz: wasn't the best idea in hindsight :)
[18:39] <bogdanteleaga> mgz: 1.25 is blocked though, right?
[18:40] <natefinch> bogdanteleaga: what you need to do is write a jc.StringEquals that'll do the string conversion for you :)
[18:40] <mgz> katco: I think she's hoping for more data on lxc specifics still, I'm going to rerun against 1.25 with that shortly
[18:40] <katco> mgz: ah ok
[18:40] <mgz> bogdanteleaga: yep
[18:41] <bogdanteleaga> natefinch: that sounds useful, yeah
[18:42] <natefinch> bogdanteleaga: it's 50% useful and 50% horrible.... since it produces a very squishy test that can pass when it really shouldn't.
[18:42] <cherylj> katco: yeah, what mgz said
[18:42] <cherylj> :)
[18:42] <katco> cherylj: k ty :)
[18:43] <bogdanteleaga> natefinch: I'm actually wondering if contains would work if they're equals
[18:43] <mgz> cherylj: note that the test just passed on the windows-pulbic-clouds branch which is 1.25 as-of-a-few-revisinos-back
[18:43]  * bogdanteleaga goes to the playground
[18:44] <cherylj> mgz: I'm wondering if the test is just taking a long time, as we've seen some of the containers come up in some of the test runs.
[18:44] <mgz> cherylj: that is certainly part of it.
[18:45] <cherylj> mgz: I'm running some manual testing in canonistack and it took over an hour to download the image to create the template container
[18:45] <cherylj> don't know if it could be related or not
[18:45] <bogdanteleaga> natefinch: deepequals still doesn't test that they have the same type though
[18:46] <mup> Bug #1494876 opened: TestFatalErrors fails on wily <ci> <test-failure> <unit-tests> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494876>
[18:46] <mgz> but compare the 1.24 runs. they're either done in 35-40 mins, or hit the *internal* deployer timeout
[18:46] <mup> Bug #1494887 opened: uniterV1Suite.SetUpTest <ci> <test-failure> <unit-tests> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494887>
[18:46] <cherylj> mgz: hmm, good point
[18:46] <mgz> the master runs are *still deploying units* an hour in when our external job timeout hits
[18:47] <cherylj> ouch
[18:47] <mgz> so like, at minimum it's a 50% perf regression due to either our box, the network, (but not as of today), or the code in master
[18:49] <cherylj> mgz: Is there any way to access the machines once the test completes?  I'm wondering if we need to examine the containers to see what's going on
[18:50] <cherylj> or are they immediately released?
[18:50] <mgz> cherylj: let me check the job, most of them have a flag we can set to not destroy-environment after
[18:53] <mgz> cherylj, sinzui: I am setting --keep-env on OS-deployer job
[18:54] <sinzui> mgz: I hope you remember to undo it before you sleep :)
[18:55] <mgz> also setting failure-threshhold to 1
[18:55] <sinzui> mgz: I've dont that. Once the job starts, you can restore the script., then wai 40 minutes for the job to get to your point of investifation
[18:55] <mgz> as this one ties up most of our maas we really can't be doing other things at the same time
[18:59] <voidspace> fwereade: hmmm... and yes, setting the preferred addresses on set rather than on read is better
[18:59] <voidspace> fwereade: only nuisance is we have two setters for addresses. Ah well, still better
[19:04] <cherylj> mgz: hopefully it won't take too long to see what's going on with the containers.
[19:19] <mup> Bug #1494894 opened: TestWatchInterfaces fails on wily <ci> <test-failure> <unit-tests> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494894>
[19:23] <mgz> katco: can I persuade one of you to revert the last change on master before the weekend?
[19:23] <mgz> the dep change broke like four things, I don't see a good reason to leave it in place
[19:27] <katco> mgz: sec meeting... was it cmars patch?
[19:28] <cmars> mgz, commit hash please?
[19:28] <natefinch> katco: I can pick that up ^^ if you want.  Pretty trivial... from Rogpeppe & frankban
[19:28] <katco> natefinch: yep +1... go for it
[19:29] <mgz> katco, cmars: http://reviews.vapour.ws/r/2606/
[19:29] <cmars> mgz, what tests fail? can I see a log?
[19:30] <mgz> bug 1494441 bug 1494864 two more unrelated windows test failures and who knows what else after the build succeeds
[19:30] <mup> Bug #1494441: ppc64el: cannot find package "encoding" <blocker> <ci> <ppc64el> <regression> <unit-tests> <juju-core:Triaged> <gccgo-go (Ubuntu):Invalid> <gccgo-go (Ubuntu Trusty):In Progress> <https://launchpad.net/bugs/1494441>
[19:30] <mup> Bug #1494864: TestBlockChangeServiceUpdate fails on windows <blocker> <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1494864>
[19:30] <cmars> mgz, ty
[19:31] <natefinch> cmars: can you rubber stamp the revert? http://reviews.vapour.ws/r/2641/
[19:31] <natefinch> it's just a PR created with the revert button in github
[19:31] <cmars> natefinch, i'm going to look at the test failures first
[19:31] <mgz> cmars: so, the ppc64 build failure is 'not your fault' but prevents us doing proper testing
[19:32] <natefinch> cmars: I've looked at the 1494441: ppc64el: cannot find package "encoding"  one
[19:32] <mgz> we had a fix to ggcgo that for whatever reasons didn't get back into trusty-updates
[19:34] <natefinch> cmars: what he said ^  also, the windows one looks like a pretty standard "Someone doing file moves/deletes before they close the file" in windows
[19:34] <perrito666> mgz: still here?
[19:35] <cmars> natefinch, mgz done
[19:35] <mgz> perrito666: I am
[19:35] <perrito666> mgz: how much do you know about mong ca files and stuff?
[19:36] <mgz> cmars: I will file other bugs from the results we got, but can address and reland easily enough next week
[19:37] <mgz> perrito666: not much of the detail
[19:37] <perrito666> mmmpf, bad luck
[19:37]  * perrito666 tries to setup mongo3 and it seems to want to authenticate with ssl certs
[19:38] <perrito666> aghh finally
[19:38] <perrito666> --sslWeakCertificateValidation
[19:39] <perrito666> and by weak they mean nil
[19:40] <mgz> perrito666: well, that sounds better than what I think we had before which is just a password-style key file
[19:40] <mgz> of some number of random bytes
[19:40] <perrito666> mgz: I am trying to get juju to run on a minimal mongo 3
[19:40] <perrito666> we can implement certs later on
[19:40] <perrito666> but with that option it should be the same as we have now
[19:47] <natefinch> I really hate that JFDI requires __ JFDI__ because the underscores get hidden by markdown
[19:47] <natefinch> I screw it up every time
[19:48] <natefinch> WTF
[19:49] <natefinch> I guess having __JFDI__ in a previous comment is not good enough. sigh
[19:52] <mup> Bug #1494912 opened: TestHookContextEnv fails on windows <blocker> <ci> <test-failure> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1494912>
[19:52] <mgz> natefinch: this isn't a jfdi, it's a fixes-1494441
[19:55] <natefinch> mgz: it doesn't fix anything, it just backs out the change... I mean, I guess it sorta does, but I figured it might be better to leave the bug around for when the code actually gets fixed.
[19:55] <perrito666> mgz: since we are in it, can we have fixes be a bit more permissive?
[19:56] <perrito666> mgz: fixes does not remove the block, it just lets your merge pass
[19:56] <perrito666> I meant natefinch
[19:56] <perrito666> mgz: could we have it also take fix- fix_ and fixes_ ?
[19:57] <mgz> we can still use the bug
[19:58] <natefinch> mgz: I guess I find the wording of "fixes" to be inaccurate in the case of just backing out a change... but maybe I'm being too pedantic.
[19:59] <mgz> that bug is really about the ftb with gccgo, which backing out the change does address. I agree that that thinks can get complicated when using all the features of version control.
[20:07] <mgz> well, this is fun.
[20:13] <fwereade> voidspace, yeah, the two setters are a bit of a hassle
[20:19] <katco> natefinch: did your patch merge?
[20:21] <mgz> katco: both seem to have.
[20:22] <mup> Bug #1494913 opened: TestNoSpoolDirectory fails on windows <blocker> <ci> <test-failure> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1494913>
[20:22] <mup> Bug #1494917 opened: TestEnvSetsPath fails on windows <blocker> <ci> <test-failure> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1494917>
[20:22] <katco> mgz: ty
[20:26] <natefinch> katco: yep
[20:32] <natefinch> katco: what do you think about making the rest of my changes for the add service stuff to 1.25 or master?  Or should I be trying to get the rest of it into 1.24 as well?
[20:40] <natefinch> gotta run for a while, will check in later
[20:40] <katco> natefinch: i'd say no
[20:40] <natefinch-afk> yeah, the other half is trickier, I'd prefer 1.25 or later
[21:16] <mup> Bug #1494936 opened: imageSuite.TestDownloadEnvironmentPath <ci> <intermittent-failure> <panic> <unit-tests> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494936>
[21:16] <mup> Bug #1494938 opened: Panic DeploySuite.TestConfig on wily <ci> <intermittent-failure> <panic> <unit-tests> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494938>
[21:16] <mup> Bug #1494939 opened: Panic backupsSuite.TestAuthRequiresClientNotMachine on wily <ci> <intermittent-failure> <panic> <unit-tests> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494939>
[21:19] <mup> Bug #1494936 changed: imageSuite.TestDownloadEnvironmentPath <ci> <intermittent-failure> <panic> <unit-tests> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494936>
[21:19] <mup> Bug #1494938 changed: Panic DeploySuite.TestConfig on wily <ci> <intermittent-failure> <panic> <unit-tests> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494938>
[21:19] <mup> Bug #1494939 changed: Panic backupsSuite.TestAuthRequiresClientNotMachine on wily <ci> <intermittent-failure> <panic> <unit-tests> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494939>
[21:22] <mup> Bug #1494936 opened: imageSuite.TestDownloadEnvironmentPath <ci> <intermittent-failure> <panic> <unit-tests> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494936>
[21:22] <mup> Bug #1494938 opened: Panic DeploySuite.TestConfig on wily <ci> <intermittent-failure> <panic> <unit-tests> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494938>
[21:22] <mup> Bug #1494939 opened: Panic backupsSuite.TestAuthRequiresClientNotMachine on wily <ci> <intermittent-failure> <panic> <unit-tests> <wily> <juju-core:Triaged> <https://launchpad.net/bugs/1494939>
[21:31] <mup> Bug # opened: 1494947, 1494948, 1494949, 1494951
[21:34] <mup> Bug # changed: 1494947, 1494948, 1494949, 1494951
[21:37] <mup> Bug # opened: 1494947, 1494948, 1494949, 1494951
[21:43] <mup> Bug # changed: 1494947, 1494948, 1494949, 1494951
[21:46] <mup> Bug # opened: 1494947, 1494948, 1494949, 1494951
[21:54] <benji> f/quit
[22:06] <ericsnow> anyone still around that could give me a quick review?  http://reviews.vapour.ws/r/2644/
[22:16] <katco> ericsnow: tal
[22:18] <katco> ericsnow: so for now the channel never gets closed, but it doesn't matter b/c its lifecycle is the same as the process?
[22:18] <katco> *agent process
[22:29] <ericsnow> katco: pretty much
[22:29] <katco> ericsnow: what is the harm in having a defer close?
[22:29] <ericsnow> katco: also, william indicated that not closing the channel would be fine
[22:29] <katco> ericsnow: ah wait, that would close it as the function exited
[22:30] <katco> ericsnow: not as the workers died
[22:30] <ericsnow> katco: yep :)
[22:30] <katco> ericsnow: +2, ship it. good work
[22:30] <ericsnow> katco: thanks
[22:32] <ericsnow> blech, currently 7 blockers on master :(
[22:33] <perrito666> wtf happened
[22:33] <perrito666> hi btw
[22:33] <perrito666> :)
[22:35] <katco> perrito666: o/
[22:35] <katco> ericsnow: wow
[22:35] <katco> ericsnow: i thought there was only 1 and nate had removed it
[22:36] <mgz> katco: you got us back to our previous blockers :)
[22:36] <katco> mgz: face palm
[22:36] <mgz> which was why I was keen on the revert
[22:36] <katco> mgz: understood... ty for pushing us in the rigth direction
[22:37] <mgz> trying to unpick which were due to the dep update and which were due to maltese-falcon was causing me headaches
[22:37] <katco> fun fact: the rigth direction is an early Gaelic belief that an undiscovered direction existed. later, it was recoined as the "right" direction
[22:37] <katco> or, alternatively, i just can't type.
[22:38] <mgz> now you've removed the dep update from the confusion
[22:38] <katco> good to hear