[00:12] <a123> yesterday cherylj help me fix a hung jujud process. I'm wonder how to go about debugging that if it happens again. Is there any info that can be extracted by attaching gdb to the pid?
[00:16] <mup> Bug #1579059 changed: MainSuite.TestFirstRun2xFrom1x fails on windows <blocker> <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Fix Released by natefinch> <https://launchpad.net/bugs/1579059>
[00:16] <mup> Bug #1579173 changed: wily configSuite.TestNewModelConfig test failure lxd <blocker> <ci> <lxd> <regression> <test-failure> <unit-tests> <wily> <juju-core:Fix Released by reedobrien> <https://launchpad.net/bugs/1579173>
[00:22] <davecheney> a123: https://github.com/juju/juju/wiki/pprof-facility
[00:23] <a123> davecheney: great. thanks
[00:51] <davecheney> thumper: https://bugs.launchpad.net/juju-core/+bug/1580391
[00:51] <mup> Bug #1580391: juju/errors tests do not pass with go 1.6 <juju-core:New> <https://launchpad.net/bugs/1580391>
[00:51] <davecheney> looks like this never worked
[00:51] <davecheney> maybe I should delete the feature
[01:01] <mup> Bug #1580391 opened: juju/errors tests do not pass with go 1.6 <juju-core:New> <https://launchpad.net/bugs/1580391>
[01:11] <mup> Bug #1580391 changed: juju/errors tests do not pass with go 1.6 <juju-core:New> <https://launchpad.net/bugs/1580391>
[01:17] <mup> Bug #1580391 opened: juju/errors tests do not pass with go 1.6 <juju-core:New> <https://launchpad.net/bugs/1580391>
[01:19] <redir_> easy review http://reviews.vapour.ws/r/4800/
[01:21] <davecheney> https://bugs.launchpad.net/juju-core/+bug/1580400
[01:21] <mup> Bug #1580400: cmd/juju/charmcmd is untested <juju-core:New> <https://launchpad.net/bugs/1580400>
[01:31] <thumper> davecheney: I can look at errors if you like
[01:32] <mup> Bug #1580400 opened: cmd/juju/charmcmd is untested <juju-core:New> <https://launchpad.net/bugs/1580400>
[01:44] <mup> Bug #1580400 changed: cmd/juju/charmcmd is untested <juju-core:New> <https://launchpad.net/bugs/1580400>
[01:47] <mup> Bug #1580400 opened: cmd/juju/charmcmd is untested <juju-core:New> <https://launchpad.net/bugs/1580400>
[02:21] <thumper> davecheney: it did work...
[02:22] <thumper> davecheney: however, I think I'd rather remove it...
[02:22] <thumper> davecheney: it is a bit bollocks IMO
[02:26] <davecheney> thumper: it probably did work
[02:26] <davecheney> but it stopped working a looooooooooooooooong time ago
[02:26] <davecheney> and nobody noticed
[02:27] <thumper> well... 1.2
[02:29] <davecheney> yup, so when I said "loooong time ago" and "nobody noticed", those statements are still correct :)
[02:32] <anastasiamac> davecheney: o/
[02:33] <davecheney> anastasiamac: hello!
[02:35] <natefinch> davecheney: what, 1.2 has only been out of date for 23 months
[02:35] <anastasiamac_> davecheney: r u coming?
[02:36] <davecheney> anastasiamac_: where ?
[02:36] <natefinch> davecheney: it's not like we can just build a binary that'll work without the right version of go installed
[02:36] <davecheney> anastasiamac_: coming
[02:37] <anastasiamac_> davecheney: the meeting u've acepted :D sent u a link into pm
[02:37] <davecheney> anastasiamac_: i'm in the hangout
[02:37] <davecheney> nobodyu is talking
[02:50] <mup> Bug #1580417 opened: Allow a different VPC ID to be used for hosted models <juju-core:Triaged> <https://launchpad.net/bugs/1580417>
[03:00] <mwhudson> i don't *think* i want to put go 1.2 in my ~gophers ppa :)
[03:08] <davecheney> mwhudson: that'll be a paddling
[03:13] <axw> wallyworld: we need yet another endpoint in clouds.yaml: https://bugs.launchpad.net/juju-core/+bug/1580423
[03:13] <mup> Bug #1580423: provider/azure: add identity endpoint to clouds.yaml <juju-core:Triaged> <https://launchpad.net/bugs/1580423>
[03:20] <mup> Bug #1580423 opened: provider/azure: add identity endpoint to clouds.yaml <juju-core:Triaged> <https://launchpad.net/bugs/1580423>
[05:29] <wallyworld> axw: osrry, missed ping, yeah saw bug
[05:30] <mup> Bug #1580418 opened: Cached local charms should be deleted when their service is removed <juju-core:In Progress by wallyworld> <juju-core 1.25:In Progress by wallyworld> <https://launchpad.net/bugs/1580418>
[05:49] <davecheney> thumper https://github.com/juju/errors/pull/25
[05:57] <menn0> davecheney, wallyworld or axw: easy fix for one of the blockers: http://reviews.vapour.ws/r/4804/diff/
[05:57] <menn0> took me ages to get my windows VM working again to be able to repro it though
[05:57] <menn0> and quite a while to confirm the problem
[05:57] <wallyworld> lgtm
[05:57] <wallyworld> thanks :-)
[05:58] <menn0> wallyworld, axw: that was fast. thanks :)
[05:58] <axw> np
[08:01] <axw> menn0 fwereade_: tech board
[08:10] <voidspace> hmmmm... when I install XChat-Gnome it still has the config from the lost XChat
[08:10] <voidspace> although under Xenial XChat-Gnome looks even worse than XChat did
[08:11] <frobware> jam: ping
[08:12] <jam> frobware: pong
[08:12] <frobware> jam: want to sync after standup?
[08:12] <jam> frobware: sure
[08:14]  * frobware notes that swap days do lead to a huge accumulation of unread email...
[08:24] <mup> Bug #1580485 opened: juju/tools/lxdclient: TestUsingTCP and TestUsingRemoteTCP not isolated from the host and failing <lxd> <test-failure> <unit-tests> <juju-core:New> <https://launchpad.net/bugs/1580485>
[08:29] <thumper> voidspace: ping
[08:30] <dimitern> thumper: o/
[08:31] <dimitern> thumper: is this even implemented: `juju add-user --share model username` ?
[08:31] <dimitern> error: flag provided but not defined: --share
[08:32] <dimitern> hmm apparently not
[08:37] <voidspace> thumper: pong
[08:41] <thumper> dimitern: something like that should work
[08:54] <mup> Bug #1580497 opened: juju2 on maas permanent " connection is shut down" msg and loss of connection <juju-core:New> <https://launchpad.net/bugs/1580497>
[08:54] <mup> Bug #1580501 opened: cloudimg-base-url parameters not in Juju2 anymore <juju-core:New> <https://launchpad.net/bugs/1580501>
[08:56] <voidspace> dimitern: babbageclunk: frobware: last of the test changes for AllocateContainerAddresses
[08:56] <voidspace> http://reviews.vapour.ws/r/4805/
[08:56] <voidspace> a nice easy one to review :-)
[08:58] <dimitern> thumper: I figured it out - first `juju add-user --models admin --acl read bob Bob` then running the suggested `juju register ...`
[08:58] <dimitern> voidspace: looking
[09:01] <hoenir> :w
[09:02] <voidspace> dimitern: standup?
[09:02] <dimitern> the plot thickens: ERROR storing charm for URL "cs:ubuntu-0": delegatable macaroon is not obtainable using admin credentials
[09:02] <dimitern> voidspace: omw, sorry
[09:04] <axw> jam thumper fwereade_ menn0: just stumbled across http://opentracing.io/
[09:04] <axw> looks interesting, I'll dig a bit
[09:05] <menn0> axw: that does look interesting
[09:12] <dimitern> voidspace: reviewed
[09:12] <voidspace> dimitern: thanks
[09:34] <frobware> jam: standup running over
[09:35] <jam> frobware: k
[09:35] <jam> not unexpected :)
[09:41] <TheMue> morning
[09:53] <jam> axw: interesting. though looking at it only 3 actual implementations and 2 of those are private
[09:53] <jam> axw: appdash has direct support, and zipking has private support, lightstep has a private beta
[10:31] <frobware> jam: I can see you on the hangout, not sure if you can hear me though.
[10:42]  * dimitern is loving the $JUJU_HOME/aliases support!
[10:43] <dimitern> juju bootstrap dimitern aws/us-east-1 --upload-tools -> juju bud aws/us-east-1
[10:51] <babbageclunk> frobware: I meant to ask, now you're back - any idea why I get -ve times from strace -cf?
[10:51] <babbageclunk> frobware: http://pastebin.ubuntu.com/16357965/
[11:30] <babbageclunk> dooferlad: ha, running the tests with tmpfs is heaps faster, unsurprisingly - the expensive bits in mongo 3.2 (index creation and DB teardown) are only 10x as slow then.
[11:31] <dooferlad> babbageclunk: :-)
[11:46] <hoenir> could anyone review my pr ? http://reviews.vapour.ws/r/4809/
[12:31] <alexisb> jam, fwereade__, frobware, cherylj ping
[12:32] <jam> alexisb: omw
[12:48] <babbageclunk> hoenir: hi - I'm looking at it
[13:25] <mup> Bug #1276444 changed: juju switch -l should look at jenv files to add to available environments <switch> <juju-core:Invalid> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1276444>
[13:25] <mup> Bug #1282217 changed: Specifying bootstrap-host requires editing environments.yaml <bootstrap> <ci> <manual-provider> <juju-core:Fix Released> <https://launchpad.net/bugs/1282217>
[13:34] <mup> Bug #1276444 opened: juju switch -l should look at jenv files to add to available environments <switch> <juju-core:Invalid> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1276444>
[13:34] <mup> Bug #1282217 opened: Specifying bootstrap-host requires editing environments.yaml <bootstrap> <ci> <manual-provider> <juju-core:Fix Released> <https://launchpad.net/bugs/1282217>
[13:37] <mup> Bug # changed: 1276444, 1282217, 1564511, 1580314
[13:38] <babbageclunk> hoenir: reviewed
[13:40] <mup> Bug #1564511 opened: cmd/jujud/reboot tests fail with lxd container running <blocker> <ci> <regression> <s390x> <tech-debt> <test-failure> <unit-tests> <xenial> <juju-core:Fix Released by cox-katherine-e> <https://launchpad.net/bugs/1564511>
[13:40] <mup> Bug #1580314 opened: MainSuite.TestFirstRun2xFrom1xOnUbuntu Exec function not called on windows <blocker> <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Fix Released by menno.smits> <https://launchpad.net/bugs/1580314>
[13:45] <hoenir> babbageclunk, thanks, found a bug , need to work on it more
[13:55] <mup> Bug #1564511 changed: cmd/jujud/reboot tests fail with lxd container running <blocker> <ci> <regression> <s390x> <tech-debt> <test-failure> <unit-tests> <xenial> <juju-core:Fix Released by cox-katherine-e> <https://launchpad.net/bugs/1564511>
[13:55] <mup> Bug #1580314 changed: MainSuite.TestFirstRun2xFrom1xOnUbuntu Exec function not called on windows <blocker> <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Fix Released by menno.smits> <https://launchpad.net/bugs/1580314>
[13:57] <alexisb> katco, morning
[13:57] <katco> alexisb: morning
[13:58] <alexisb> katco, I need someone to pickup this bug today: https://bugs.launchpad.net/juju-core/+bug/1580184
[13:58] <mup> Bug #1580184: Timeout in github.com/juju/juju/apiserver/service on windows <blocker> <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1580184>
[13:58] <katco> alexisb: ack. we'll see who's doing what and get it done
[13:58] <alexisb> katco, thank you
[13:58] <katco> alexisb: np
[14:25] <natefinch> fwereade__: interesting thought.... is there a way to list all machines from all models on a controller?
[14:26] <natefinch> fwereade__: just got an AWS alert for high usage, and wanted to see how many machines I was using from my aws controller
[14:43] <mup> Bug #1580626 opened: provider/openstack: Don't use a hard-coded clock.WallClock in deleteSecurityGroup <tech-debt> <testing> <juju-core:Triaged> <https://launchpad.net/bugs/1580626>
[15:14] <alexisb> perrito666, ping
[15:14] <perrito666> alexisb: pong
[15:15] <alexisb> heya perrito666 I am available early for our 1x1 if you are also available
[15:15] <perrito666> I am
[15:15] <perrito666> usual channel?
[15:16] <alexisb> yep
[15:16] <perrito666> already there
[15:31] <redir_> natefinch: any pointers on how to reproduce races?
[15:32] <natefinch> redir_: go test -race ? :)
[15:32] <natefinch> redir_: link me to the bug?
[15:35] <redir_> natefinch: https://bugs.launchpad.net/juju-core/+bug/1580186
[15:35] <mup> Bug #1580186: Race in github.com/juju/juju/worker/instancepoller/aggregate_test.go <blocker> <ci> <intermittent-failure> <race-condition> <regression> <unit-tests> <juju-core:In Progress by reedobrien> <https://launchpad.net/bugs/1580186>
[15:36] <redir_> natefinch: they seem to hit it on every CI run, but I can't make it happen here.
[15:36] <redir_> bbiab
[15:36] <natefinch> redir_: k
[15:50] <katco> fwereade__: hey sorry, i have to move on from that PR and pick up another blocker. i'll circle back when/if i get some free time
[15:53] <alexisb> natefinch, just fyi, for the blocker bug you picked up... if you need a repo case sinzui would be happy to provide access to the machine where the test is failing consistently
[15:53] <natefinch> alexisb: consistent repros are my favorite thing in the world
[15:54] <alexisb> :)
[16:12] <katco> ericsnow: hey i'm ocr today. what was the impetus for http://reviews.vapour.ws/r/4812/ ?
[16:13] <katco> ericsnow: looks like maybe so we can stop consuming the loggo package just for a testing utility?
[16:14] <ericsnow> katco: it's a precursor I had kicking around for a follow-up cleanup patch (itself to be followed by a patch that may be useful for the syslog work)
[16:14] <katco> ericsnow: ahh
[16:14] <katco> ericsnow: well looks like you have a shipit at any rate
[16:14] <perrito666> bbl
[16:14] <ericsnow> katco: trying to avoid moving code around in the same patch where I change stuff :)
[16:14] <katco> ericsnow: excellent idea :)
[16:24] <natefinch> katco: would like a review on http://reviews.vapour.ws/r/4801/ if you get a chance
[16:24] <katco> natefinch: tal
[17:04] <katco> ericsnow: errr... not even sure my comment to you went through, but if so, it looks like it was a connection issue on my end
[17:05] <ericsnow> katco: apparently it didn't go through as I don't know what you're talking about :)
[17:05] <katco> ericsnow: ha. i was complaining that review board was acting up again
[17:05] <ericsnow> katco: great :(
[17:05] <katco> ericsnow: then my irc bouncer kicked me from all the channels at once and i wised up :)
[17:06] <katco> ericsnow: no, like i said i'm pretty sure it was just a connection problem on my end
[17:06] <ericsnow> katco: oh, phew :)
[17:11] <katco> natefinch: you have a review. mostly just some questions
[17:11] <natefinch> katco: cool, thanks
[17:53] <mup> Bug #1580714 opened: ubuntu user not in 'lxd' group on xenial instances <juju-core:Triaged> <https://launchpad.net/bugs/1580714>
[18:23] <mup> Bug #1580717 opened: Juju agent stuck in pending if charm storage's minimum-size exceeds available disk space on machine <juju-core:New> <https://launchpad.net/bugs/1580717>
[19:27] <natefinch> oh boy, interesting.... package launchpad.net/tomb: cannot download, http://bazaar.launchpad.net/~niemeyer/tomb/trunk/ uses insecure protocol
[19:35] <katco> natefinch: i think he moved tomb over to github didn't he?
[19:36] <katco> natefinch: https://gopkg.in/tomb.v2
[19:36] <natefinch> katco: we still seem to use launchpad.net/tomb somewhere in juju though
[19:36] <katco> natefinch: time to standardize :)
[19:37] <natefinch> katco: actually, we don't use the gopkg.in version at all
[19:39] <perrito666> we could standardize oooor, we could make a script that uses launchpad, github or gopkg randomly
[19:39] <perrito666> super fun to debug
[19:39] <natefinch> only 3 deps in juju on launchpad, and 1 already exists in github....
[19:40] <natefinch> niemeyer3: do you know if there's a launchpad-side fix possible for the "package foo: cannot download, uses insecure protocol" error?  Is there something the project owner can do to enable https?
[19:41] <katco> natefinch: i believe he is on a plane
[19:41] <natefinch> katco: ahh boo
[19:42] <perrito666> katco: well if he can tweet a picture in flight he might very well have irc :p
[19:42] <natefinch> lol true
[19:42] <katco> perrito666: that's an excellent point. where are his priorities! ;p
[19:44] <natefinch> heh: C:\Users\Administrator\src\github.com\juju\juju\apiserver\service>go test
[19:44] <natefinch> 2016-05-11 22:44:03 WARNING juju.testing mgo.go:186 failed to start mongo: exec: "/usr/local/bin/mongod": file does not exist
[19:45] <perrito666> ahhh how fun
[19:45] <perrito666> natefinch: beware, that might be a red herring
[19:45] <perrito666> I recall an error somewhere having the path hardcoded even though it was looking in the right place
[19:45] <perrito666> but I am pretty sure I fixed all of those
[19:46] <natefinch> perrito666: the windows tests require mongo?
[19:46] <perrito666> natefinch: some of them
[19:46] <natefinch> perrito666: we don't support windows as a controller
[19:46] <perrito666> and if you are using the instructions provided by cloudbase you should have it installed
[19:46] <natefinch> perrito666: I thought I was, but their instructions either have changed or I was looking at the wrong page
[19:47] <perrito666> natefinch: well no, but our unit tests are little unit-ish and need a mongo to try the most elemental things
[19:47] <natefinch> perrito666: it really boggles my mind that we don't actually support running mongo on windows or controller code running on windows... but the tests assume it works ok
[19:48] <perrito666> natefinch: the tests run a fake controller
[19:48] <perrito666> that sort of works
[19:48] <perrito666> http://wiki.cloudbase.it/juju-testing
[19:48] <perrito666> http://wiki.cloudbase.it/juju-testing#getting_a_ssl_enabled_version_of_mongodexe
[19:48] <natefinch> (╯°□°)╯︵ ┻━┻
[19:48] <perrito666> to be accurate
[19:48] <natefinch> yeah, I was there... I guess I just didn't scroll down all the way before
[19:49] <natefinch> man, I can't get any of their powershell to run on this machine... I don't know what I'm doing wrong, if it's a different version of powershell or what
[19:50] <natefinch> oops, well, that one was a typo
[20:03] <mup> Bug #1564577 changed: Add MAAS 2.0 support in Juju 2.0 <juju-core:Fix Released by thumper> <juju-core 1.25:Invalid> <https://launchpad.net/bugs/1564577>
[20:03] <babbageclunk> perrito666: ping?
[20:05] <perrito666> babbageclunk: sorry was having afternoon tea
[20:05] <perrito666> how can I help you?
[20:06] <babbageclunk> alexisb said that you knew about Mongo - I'm looking at bug 1573294
[20:06] <mup> Bug #1573294: state tests run 100x slower with mongodb3.2 <juju-release-support> <mongo3> <mongodb> <juju-core:Triaged> <https://launchpad.net/bugs/1573294>
[20:06] <babbageclunk> perrito666: just wanted to ask if you had any advice or suggestions.
[20:07] <babbageclunk> perrito666: Basically digging into it it seems like everything is slower in Mongo3.2.
[20:08] <perrito666> babbageclunk: you have to be kidding me
[20:08] <perrito666> what could have changed since early versions of 3 :(
[20:09] <perrito666> babbageclunk: that is very sad news, 2.x is EOL
[20:10] <babbageclunk> perrito666: Well, from rereading the bug it sounds like it's really the wiredtiger storage engine that's the problem
[20:11] <perrito666> babbageclunk: it is, but mmapv1 is also EOL
[20:11] <babbageclunk> perrito666: Ah, that was what I was about to ask.
[20:11] <perrito666> babbageclunk: it sounds almost as if mongo sucked :p
[20:14] <babbageclunk> perrito666: Maybe it's because the tests are essentially sequential, while WT is more tuned for concurrent writes?
[20:15] <perrito666> I reached a similar conclusion that the one in the bug, michael was nice enough to put data behind that
[20:15] <bdx> is it possible to have multiple lxd hypervisor as resources under one juju lxd controller?
[20:17] <perrito666> babbageclunk: that is an interesting theory, I also suggested that the fixed cost of bootstraping stuff in WT was high and since our tests seem to do that a lot that causes the pain
[20:18] <perrito666> babbageclunk: I wonder if we can design a few cheap benchmarks to see how can we better use wt?
[20:18] <babbageclunk> perrito666: I've hacked out the index creation in the tests, and while that cuts out some of the cost they're still much slower under 3.2.
[20:19] <perrito666> mwhudson suggests that he did a deep trace of the db run, do we have numbers of there are the other 10X?
[20:22] <babbageclunk> perrito666: I was running it under strace, but that was giving nonsense numbers (negative call times in the child processes).
[20:22] <babbageclunk> perrito666: I've hacked up something that can get the same numbers from perf trace, and it lets me split out the syscals in mongo from the ones in our test code.
[20:23] <babbageclunk> perrito666: Actually, would you be able to run the strace command for me to see if you get the same weird numbers?
[20:23] <perrito666> certainly
[20:23] <perrito666> lets go priv
[20:28] <niemeyer3> natefinch: Hey
[20:28] <niemeyer3> natefinch: Sorry, what's the question again?
[20:29] <natefinch> niemeyer3: go get as of 1.5?  definitely 1.6 will refuse to download from launchpad without the -insecure flag, because it goes over http, not https
[20:30] <natefinch> niemeyer3: sorry, gotta run, but will read responses.  No rush, just wondering if there was a way for a package author on launchpad to enable https to avoid that problem
[20:33] <mup> Bug #1564577 opened: Add MAAS 2.0 support in Juju 2.0 <juju-core:Fix Committed by thumper> <juju-core 1.25:Invalid> <https://launchpad.net/bugs/1564577>
[20:33] <niemeyer3> natefinch3: ECONTEXT, feel free to reach out when you have time.. o/
[21:03] <redir_> katco: http://reviews.vapour.ws/r/4813/
[21:03] <redir_> pls
[21:06] <katco> redir_:
[21:06] <katco> redir_: sec
[21:06] <redir_> katco: np
[21:06] <redir_> easy one too
[21:24] <katco> redir_: running into meetings, but i want to give this one more time. something looks fishy here
[22:01] <alexisb> wallyworld, ping
[22:18] <mup> Bug #1580184 changed: Timeout in github.com/juju/juju/apiserver/service on windows <blocker> <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Fix Released by natefinch> <https://launchpad.net/bugs/1580184>
[22:48] <mup> Bug #1580802 opened: NoContextWithLock fails on windows because of another process <blocker> <ci> <intermittent-failure> <regression> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1580802>
[23:02] <axw> jam: (bit delayed) yeah, and I tried appdash and its UI is a little underwhelming. I modified jujud to create spans for each API connection, and subspans for each API call that comes in. easy enough, tho it's just a hack. it would be nice to be able to drill into timings for provider API calls too, so we'd probably end up exposing the tracing API to provisioners, firewaller, etc.
[23:14] <wallyworld> axw: anastasiamac_: redir_: perrito666: dtuck in meeting, will be a few minutes late
[23:15] <perrito666> wallyworld: tis ok, we love you anyway
[23:15] <anastasiamac_> wallyworld: ping when u r ready
[23:15] <anastasiamac_> so melodramatic perrito666 :-P
[23:20] <wallyworld> axw: anastasiamac_: there now
[23:55] <mwhudson> natefinch: heh i just ran into https://github.com/coreos/go-systemd/issues/156 (again)