[00:17] <menn0> axw: I said in the standup that the blobstore changes to allow it support io.{ReaderAt,Seeker} turned out to be bigger than I thought they would be
[00:17] <menn0> axw: I just pushed through with it and it's not too bad
[00:18] <menn0> axw: although the requirements for the ReaderAt interface are a little tricky to get right, especially efficiently
[00:18] <menn0> axw: I've gone for correctness over performance
[00:18] <menn0> axw: parking for now though, will hopefully pick it up later today
[00:19] <veebers> thumper: dumb question, if I create a model, deploy a charm and add a unit, if I just destroy the model it'll cleanup that unit right? Or do I need to specifically remoev that unit myself?
[00:22]  * thumper afk for lunch
[00:22] <thumper> yes
[00:22] <thumper> it will clean up that unit
[00:22] <veebers> awesome, thanks th
[00:22] <veebers> thumper*
[00:47] <redir> axw: https://github.com/juju/juju/pull/6308 when you have some time
[00:49] <redir> anastasiamac: I also applied your recommendation wrt --reset as []string in https://github.com/juju/juju/pull/6308
[00:50] <redir> anastasiamac: apologies though, as I squashed it in with the other changes.
[00:50] <anastasiamac> redir: \o/ u r wonderful! tyvm :D
[00:51] <axw> menn0: ok, thanks. no drama if you don't get to it until later - it's just a nice to have
[00:51] <axw> redir: looking
[00:51] <redir> np , I'm on to make dinner, so won't be back until this eve.
[00:51] <menn0> axw: it definitely would be a lot better that way, and I am going to get to it
[00:52] <redir> Hopefully, this is really close with trivials or less:)
[01:57] <anastasiamac> thumper: axw: menn0: if u have any ideas/suggestions for "server not reachable" and/or "replicaset" intermittent failures that we are seeing, it'd be great to share them now... i'd love to improve this area as part of the sprint :D
[02:11] <lazyPower> interesting, is there a known bug currently where you have to model-config enable-os-update && enable-os-refresh-update to false?   I'm seeing this on both gce and aws, investigating further as it pops up while testing.
[02:24] <natefinch> lazyPower: yes
[02:24] <natefinch> lazyPower: some lxd update is failing
[02:25] <lazyPower> natefinch: ta, i was looking for it and didn't find a bug. i was about to reproduce and file one
[02:26] <natefinch> lazyPower: it's like the apt upgrade of lxd is failing or something, I forget the details, but it was not our fault, per se.
[02:27] <lazyPower> right, bit by an SRU or something
[02:27] <natefinch> right
[02:27] <lazyPower> not even mad just trying to be helpful :)
[02:28] <natefinch> :)
[02:28] <natefinch> anastasiamac: you around?
[02:29] <anastasiamac> natefinch: it depends on how difficult ur question is :D
[02:30] <natefinch> heh
[02:30] <natefinch> https://bugs.launchpad.net/juju/+bug/1625243
[02:30] <natefinch> mup: https://bugs.launchpad.net/juju/+bug/1625243 ?
[02:30] <mup> natefinch: I apologize, but I'm pretty strict about only responding to known commands.
[02:30] <natefinch> mup help
[02:30] <anastasiamac> natefinch: yep, i'llb mup and got it :D
[02:31] <natefinch> anastasiamac: anyway.... do you know what they're talking about? :)
[02:31] <anastasiamac> natefinch: mostly about format of simplestreams files
[02:31] <natefinch> juju should fix its code to know that https://identity.api.rackspacecloud.com/v2.0 Is a global URL expected to be accessed first before going to any regionalized service.
[02:31] <anastasiamac> natefinch: tech board had fun discussing format this week too :D
[02:31] <natefinch> what does "accessed first" mean?
[02:32] <anastasiamac> natefinch: if u look at endpoint url, the one we understand have v2
[02:33] <anastasiamac> without region in it
[02:33] <anastasiamac> the data that was supplied had region
[02:34] <anastasiamac> comment 10 :)
[02:34] <natefinch> yes
[02:34] <natefinch> the comment that says "we could fix this, but nah"
[02:34] <anastasiamac> yep :(
[02:34] <natefinch> so, what I don't understand is their suggested fix
[02:34] <anastasiamac> so rackpsace provider need to be modified to handle region in endpoint url ..
[02:35] <natefinch> I guess I don't understand. It's a URL.  Are we parsing it?  What does it matter if the URL has a region in it?
[02:35] <anastasiamac> my biggest disappointment with htis, if they've fixed data, all our current implementations (including 1.25) would work
[02:36] <anastasiamac> since we ar fixing the provider only release with the fix will work :(
[02:36] <natefinch> right
[02:37] <anastasiamac> dunno more than that.. u'll need to try it out with both our current (residing in CI) data without region and supplied data with regions.. not fun but this what I would have been doing
[02:37] <anastasiamac> to see what's wrong and why it matters...
[02:37] <natefinch> the thing I don't understand is that the URLs they're listing in the bug don't have any data in them
[02:37] <natefinch> for example https://iad.images.api.rackspacecloud.com/v2
[02:39] <natefinch> and the url that curtis says is correct (https://identity.api.rackspacecloud.com/v2.0) just returns some xml that does not look pertinent to juju at all
[02:40] <natefinch> I also don't see a difference between the rackspace data and the rest of the clouds' data here: https://cloud-images.ubuntu.com/releases/streams/v1/index.json
[02:41] <natefinch> they *all* have region in the endpoint url
[02:41] <natefinch>      "region": "eu-central-1",
[02:41] <natefinch>      "endpoint": "https://ec2.eu-central-1.amazonaws.com"
[02:41] <natefinch> If I shrug any harder I'm gonna pull a muscle
[02:42] <anastasiamac> natefinch: this looks like ec2 endpoint not rackspace.. curtis is the best person to talk about this bug :)
[02:42] <anastasiamac> oh no - don't pull muscles \o/
[02:42] <anastasiamac> talk to curtis :D
[02:42] <natefinch> no no, I know... I was showing an endpoint from another cloud that also had region in the endpoint, which they were saying was bad somehow
[02:43] <natefinch> I will talk to curtis when he gets back on.
[02:43] <anastasiamac> \o/
[02:43] <anastasiamac> m sooo happy that u r being thrown into simplestreams too :)
[02:43] <natefinch> rofl
[02:43] <natefinch> misery loves company
[02:44] <anastasiamac> :D
[03:52] <axw> thanks thumper
[03:52] <thumper> np
[03:52] <axw> thumper: I replied to your q, but was perhaps a bit terse. we don't care about the lifecycle state of the units, just that none have been added or removed
[03:53] <thumper> ok
[03:53] <thumper> fair enough
[03:53] <thumper> I was just wondering
[03:53] <thumper> you are good to land
[03:53] <axw> ta
[04:08] <menn0> axw: https://github.com/juju/juju/pull/6309 pls
[04:08] <menn0> axw: reviewing your other PR now
[04:19] <veebers> menn0, thumper: list-disabled-commands now returns "''" instead of "[]" if there are no disabled commands, I imagine this is a bug and not expected?
[04:20] <thumper> no
[04:20] <thumper> I think I changed that on purpose
[04:20] <veebers> thumper: sorry, this is in format yaml/json too
[04:20] <veebers> s/too//
[04:21] <thumper> not sure now, sorry
[04:21] <natefinch> that's a bug.  It should only be for tabular
[04:21] <veebers> natefinch: cool, filing now. Thanks :-)
[04:29] <natefinch> veebers: did you get that bug filed, 'cause I have the fix ready ;)  thumper?  +8 -2 https://github.com/juju/juju/pull/6310
[04:30] <axw> menn0: looking
[04:30] <thumper> natefinch: lgtm
[04:31] <natefinch> thumper: thanks
[04:33] <anastasiamac> natefinch: bug 1626824
[04:33] <mup> Bug #1626824: list-disabled-commands  returns "''" instead of "[]" for json/yaml format <ci> <regression> <ui> <juju:Triaged> <https://launchpad.net/bugs/1626824>
[04:33] <axw> menn0: LGTM
[04:33] <menn0> axw: thanks. I now it's a backport but it was different enough that I thought it needed a review
[04:36] <veebers> natefinch: yep, https://bugs.launchpad.net/juju/+bug/1626824
[04:36] <mup> Bug #1626824: list-disabled-commands  returns "''" instead of "[]" for json/yaml format <ci> <regression> <ui> <juju:Triaged> <https://launchpad.net/bugs/1626824>
[04:36] <veebers> anastasiamac: hah, I'm to slow
[04:38] <anastasiamac> veebers: :) nps
[04:40] <menn0> axw: just a few minor things for yours. ship it!
[04:40] <axw> menn0: thanks
[04:42] <axw> menn0: you really want me to add docs to each method on Backend and friends? can I just say "go look at the state package"?
[04:43] <menn0> axw: just describe the interfaces briefly (referring to state if that makes sense)
[04:43] <axw> menn0: ok
[04:43] <menn0> axw: and maybe "<blah> implements <blerg>" for the method implementation (although I'm fairly +0 on that)
[04:43] <menn0> implementations
[04:44] <axw> menn0: okey dokey
[04:59] <natefinch> veebers: fix landed, will be in next release
[05:11] <thumper> I get this periodically
[05:11] <thumper> ERROR cannot resolve charm URL "cs:xenial/ubuntu": cannot get "/xenial/ubuntu/meta/any?include=id&include=supported-series&include=published": Get https://api.jujucharms.com/charmstore/v5/xenial/ubuntu/meta/any?include=id&include=supported-series&include=published: dial tcp: lookup api.jujucharms.com on 127.0.1.1:53: server misbehaving
[05:11] <thumper> I think we should have a retry around the charmstore access
[05:14] <thumper> why does the ubuntu charm install g++ ?
[05:14] <thumper> it is getting build-essential
[05:15] <thumper> and python3.5-dev
[05:15] <thumper> seems weird
[05:16]  * thumper waits for them all to finish
[05:20] <thumper> well that didn't work
[05:22] <thumper> ah fuck...
[05:40]  * thumper calls it a day
[05:40] <thumper> later folks
[05:47] <axw> anastasiamac: do you have time for a review?
[05:48] <anastasiamac> axw: i wish but not really ;( m only marginally here - ghossting..
[05:48] <axw> anastasiamac: np
[05:48] <axw> menn0: are you EOD yet?
[05:48] <menn0> axw: not quite
[05:49] <axw> menn0: if you have time, would appreciate a review on https://github.com/juju/juju/pull/6311.
[05:49] <axw> doh, doc string updates snuck in
[05:49] <menn0> axw: looking
[05:55] <menn0> axw: done
[05:55] <menn0> axw: all good
[05:59] <axw> menn0: thanks
[07:41] <menn0> axw: Clean ups of uniter temporary download files: https://github.com/juju/juju/pull/6314
[07:42] <axw> menn0: cool, looking
[07:45] <menn0> axw: I just added some comments to the change about possible improvements. pls review anyway though.
[07:50] <mup> Bug #1626878 opened: ERROR juju.worker.dependency engine.go <juju-core:New> <https://launchpad.net/bugs/1626878>
[07:58] <axw> menn0: reviewed
[08:05] <mup> Bug #1626878 changed: ERROR juju.worker.dependency engine.go <juju-core:New> <https://launchpad.net/bugs/1626878>
[08:08] <mup> Bug #1626878 opened: ERROR juju.worker.dependency engine.go <juju-core:New> <https://launchpad.net/bugs/1626878>
[09:53] <babbageclunk> hey menn0, I think I found it!
[09:54] <babbageclunk> https://github.com/juju/juju/blob/master/state/cleanup.go#L155
[09:54] <babbageclunk> menn0: ^
[09:59]  * menn0 is about to watch something :)
[09:59] <menn0> babbageclunk: that's awesome though!
[10:00] <menn0> I don't see the problem though :-/
[10:05] <babbageclunk> menn0: It deletes settings non-transactionally. If you're unlucky, you get this sequence: cleanup queued, undertaker runs, makes remove settings txn, cleanup runs, txn gets stuck.
[10:05] <babbageclunk> menn0: I think
[10:06] <babbageclunk> menn0: Making a binary now to get Jason to try it.
[10:07] <babbageclunk> menn0: Isn't it a bit late to start watching something!? Kids don't sleep later on weekends.
[10:07] <babbageclunk> :)
[10:08] <babbageclunk> I guess he went to bed
[10:16] <voidspace> frobware: ping
[10:16] <voidspace> perrito666: ping
[10:16] <perrito666> voidspace: pong
[10:17] <perrito666> voidspace: be kind I am barely conscious
[10:17] <voidspace> perrito666: babbageclunk says you had an issue with maas2Instance.Status being effectively cached and not updated per call
[10:17] <voidspace> perrito666: heh
[10:17] <voidspace> perrito666: are you fixing that or working around it?
[10:17] <perrito666> voidspace: i thought it was that, but it might be something else, I fixed the call to be not cached anyway
[10:18] <voidspace> perrito666: I wonder if it is behind the problem at Telefonica that frobware, macgreagoir and I have been looking at
[10:18] <frobware> voidspace: pong
[10:18] <voidspace> perrito666: is it landed?
[10:18] <voidspace> frobware: I'm wondering whether an issue that perrito666 found is behind the telefonica issue
[10:18] <voidspace> frobware: it doesn't *entirely* fit with the logs, so I'm still trying to dig into it
[10:18] <perrito666> voidspace: it is not landed, can you describe the problem?
[10:19] <voidspace> perrito666: basically a maas2Instance.Status call always returns the same value (the status when the maas2Instance was created)
[10:19] <voidspace> frobware: basically a maas2Instance.Status call always returns the same value (the status when the maas2Instance was created)
[10:19] <voidspace> perrito666: oops, that was meant to go to frobware
[10:19] <voidspace> perrito666: we're seeing delayed provisioning on maas
[10:19] <voidspace> perrito666: making a large delay very slow
[10:20] <voidspace> perrito666: frobware: so if somewhere in the code is polling Status waiting for it to change, then it won't
[10:20] <perrito666> well I have both problems, not enough polling and the status being cached anyway
[10:20] <perrito666> so I started by fixing the not enough polling
[10:20] <frobware> voidspace: heh, interesting. in trying to repro locally I think my MAAS setup is truly borked atm
[10:20] <voidspace> perrito666: frobware: I can't find any polling in the code yet, but the code paths are quite convoluted
[10:20] <voidspace> frobware: :-(
[10:20] <voidspace> perrito666: when do you think your fix will land?
[10:21] <frobware> perrito666: happy to be a guinea pig before then
[10:21] <voidspace> perrito666: I wonder if we can get binaries to them to check if this fixes the issue
[10:21] <voidspace> frobware: it's provider/maas/maas2Instance.go:#L68
[10:21] <perrito666> voidspace: let me push up my changes
[10:22] <voidspace> frobware: perrito666: it looks like this is called by the provisioner_task in classifyMachine
[10:22] <voidspace> frobware: hard to see if it's polling though
[10:23] <voidspace> frobware: the Status is checked if InstanceId errors, which it does if there is no real maas machine yet (so no instance data)
[10:23] <voidspace> frobware: and we do have "found machine pending provisioning" log lines from that code path
[10:23] <perrito666> voidspace: its something cherilj added some time ago
[10:23] <voidspace> perrito666: what, polling?
[10:24] <perrito666> polling of the underlying status for useful return iirc
[10:24] <voidspace> right
[10:24] <voidspace> so it *could* be the problem
[10:24] <perrito666> voidspace: that is half of my problem solved https://github.com/perrito666/juju/tree/fix_1604965
[10:24] <perrito666> I am now trying to figure where are we exactly refreshing since my problem is that failed machines never gets marked as so
[10:25] <voidspace> right
[10:30] <voidspace> perrito666: you probably want to delete the TODO as well.
[10:30] <voidspace> frobware: worth trying that branch to see if it fixes the issue
[10:31] <frobware> voidspace: yep, need to get a working MAAS2 atm
[10:31] <voidspace> cool
[10:32] <perrito666> voidspace: true, sorry was not thiking on uploading that :p
[10:32] <voidspace> heh
[11:04] <babbageclunk> frobware: You've done this lots - is sending someone a binary to run just zipping up juju and jujud? Anything else to send?
[11:05] <frobware> babbageclunk: I do
[11:05] <frobware> babbageclunk: cd ~/go
[11:05] <frobware> babbageclunk: tar -cvf ~/juju-2-0-rc1-for-brad.tar --transform='s!^!juju-2.0-rc1-for-brad/!' bin/juju*
[11:05] <frobware> babbageclunk: at least then it has a path in it when you untar
[11:06] <babbageclunk> frobware: Awesome, thanks.
[11:07] <frobware> babbageclunk: it's also handy to s/bard/for-git-commit-sha1/
[11:07] <frobware> babbageclunk: and to complete the metacircular evaluator, s/bard/brad/
[11:08] <babbageclunk> frobware: lol. In words, you mean, it's handy to include the git hash so you know what code they're running?
[11:08] <frobware> babbageclunk: yup.
[11:08] <babbageclunk> frobware: Will do
[11:09] <frobware> babbageclunk: we should change our build to include this
[11:31] <babbageclunk> frobware: Do I need to tell the recipient to use -build-agent? Will that work without the source?
[11:32] <frobware> babbageclunk: I haven't mentioned it for binaries I've given away
[11:33] <babbageclunk> frobware: So hopefully it just does the right thing? I'm a bit nervous because I'm building off the beta18 tag and in my testing it kept trying to use the stream version unless I passed --build-version
[11:34] <frobware> babbageclunk: if you bootstrap with debug does it not show that it's using the juju from your $PATH?
[11:38] <babbageclunk> No, it says http://paste.ubuntu.com/23219686/
[11:39] <babbageclunk> frobware: ^. I might rebase onto tip just to be sure.
[11:41] <frobware> babbageclunk: doesn't it say all the anyway; I think if you do: $(which jujud) -- and grep for that in the debug output
[11:48] <babbageclunk> frobware: No, that definitely doesn't appear.
[11:48] <frobware> babbageclunk: hmm. so, with -build-agent?
[11:49] <babbageclunk> frobware: The sha1sums don't match either (although I guess they might not with --build-agent either).
[11:49] <babbageclunk> frobware: Huh, with build-agent they do.
[11:50] <frobware> babbageclunk: TIL
[11:50] <babbageclunk> frobware: That could just be that go didn't touch the binary because everything was up to date.
[11:57] <babbageclunk> frobware: Ok, a quick scan of the code suggests that it'll use jujud from beside the running juju binary as long as it matches the version. So I think --build-agent is the right thing.
[13:18] <mup> Bug #1627015 opened: no way to specify additional storage for controller(s) <juju-core:New> <https://launchpad.net/bugs/1627015>
[13:24] <mup> Bug #1627015 changed: no way to specify additional storage for controller(s) <juju:Triaged> <https://launchpad.net/bugs/1627015>
[14:01] <rick_h_> frobware: voidspace natefinch ping for standup
[14:01] <voidspace> omw
[14:02] <voidspace> rick_h_: gah, browser hang
[14:03] <mgz> hangout just dropped me...
[14:04] <voidspace> rick_h_: browser hang again - I can hear everyone but currently can't unmute!
[15:41] <redir> morninf
[15:41] <redir> had one of those mornings where system wouldn't display fonts after resume.
[15:41] <natefinch> lol
[15:41] <redir> then wouldn't resolve dns after reboot
[15:42] <redir> it was a two reboot morning
[15:42] <natefinch> dang
[15:43] <natefinch> speaking of which, my "hangs for 7 minutes after reboot" problem mysteriously went away after 2 months of me never rebooting. :)
[15:43] <alexisb> redir, you system is trying to tell you it is friday
[15:43] <redir> right. I better update and reboot everything before I leave tomorrow
[15:44] <redir> I feel a little lenient toward this machine not always working right. I't has been getting every upgrade since 2013, including betas
[15:45] <redir> speaking of, i it time to upgrade to yakkety yet?
[15:53] <redir> also have this ongoing issue where chromium doesn't display jpegs, on multiple different systems. Works on arch though.
[15:53]  * redir quits whinging and gets to being productive
[16:44] <voidspace> frobware: ping
[16:45] <frobware> voidspace: pong - otp
[16:46] <voidspace> frobware: ok, answer when you get a chance (you may not know anyway)
[16:46] <voidspace> frobware: I'm trying to deploy a lxd (to the juju controller machine for convenience) and I get this error in the logs:
[16:46] <voidspace> machine-0: 17:39:57 ERROR juju.worker.proxyupdater lxdbr0 has no ipv4 or ipv6 subnet enabled
[16:46] <voidspace> It looks like your lxdbr0 has not yet been configured. Please configure it via:
[16:46] <voidspace> 	sudo dpkg-reconfigure -p medium lxd
[16:46] <voidspace> and then bootstrap again.
[16:46] <voidspace> frobware: there is no lxdbr0 of course
[16:47] <voidspace> frobware: and so the container is forever stuck in pending
[17:02] <frobware> voidspace: i'm back
[17:02] <frobware> voidspace: on MAAS?
[17:03] <frobware> voidspace: are you using rc1/
[17:36] <rick_h_> natefinch: ping, did you get your info and unblocked on things?
[17:38] <natefinch> rick_h_: yeah, I got it.  The descriptions in the bug were evidently only one side of a conversation, more or less :)
[17:38] <rick_h_> natefinch: ok cool, thanks
[18:16] <mup> Bug #1627138 opened: unable to bootstrap on openstack provider <juju-core:New> <https://launchpad.net/bugs/1627138>
[18:31] <mup> Bug #1627138 changed: unable to bootstrap on openstack provider <juju:New> <https://launchpad.net/bugs/1627138>
[18:40] <mup> Bug #1627138 opened: unable to bootstrap on openstack provider <juju-core:New> <https://launchpad.net/bugs/1627138>
[18:46] <mup> Bug #1627138 changed: unable to bootstrap on openstack provider <juju:New> <https://launchpad.net/bugs/1627138>
[19:42] <natefinch> good lord bzr + launchpand is slow.
[19:46] <natefinch> anyone online know how to set up credentials for rackspace?  It's asking for domain name, and that isn't described in our docs
[19:49] <natefinch> gah, when you bootstrap rackspace, all the error messages talk about Openstack.
[20:22] <natefinch> holy hell.  bootstrapCommand.Run is an almost 500 line function.
[20:23] <redir> only 500?
[20:24] <natefinch> :/
[20:25] <redir> sorry natefinch I don't know about bootstrapping rackspace.
[20:25] <natefinch> redir: it's ok.... either it doesn't matter or my bug is occurring before that field is checked :)
[20:26] <natefinch> I am using delve for the first time ever, and it's amazing.  Running it in Visual Studio Code is almost as good as debugging C# with the real Visual Studio.... and that's saying a lot for me.
[21:48] <redir> natefinch: that is saying something.
[21:48] <redir> but I've not debugged C# or used visual studio
[21:51] <rick_h_> natefinch: leave off the domain name? thought that was not needed
[21:52] <rick_h_> natefinch: search our bugs for domain name, thought there was one about removing it