[00:12] menn0: I can take a look later (after I put the kid to bed) [00:13] tych0 - running lxd init; systemctl restart lxd.service worked to get the lxd provider to bootstrap [00:13] cherylj: thank you (no rush) [00:14] tych0: will juju need to do that when provisioning lxd containers (using deploy --to lxd)? [00:46] cherylj: is there a kanban board with the helpdocs updates on it? [00:47] redir: i think bug squad is still it.. [00:47] redir: i'll pm u :D [01:03] redir: I haven't been keeping up to date on all the helpdocs bugs and their comments [01:03] redir: might be a good idea to change the titles of them once they've been acked and are ready to merge.... makes it easy to quickly search for them in lp [01:04] cherylj: anastasiamac got me to the board. So I can move cards appropriately. [01:04] redir: they're not all there, btw [01:04] feel free to add if you'd like [01:05] cherylj: will do when I get to one in lp that isn't on the board. [01:06] redir: if you're interested, bug 1566237 is an easy non-help docs bug [01:06] Bug #1566237: juju ssh doesn't work with multiple models [01:09] cherylj: k [01:21] Bug #1567161 opened: juju2 beta3, cannot download charm, failed to download, 400

[01:33] Bug #1566843 changed: juju add-credential should set default when there is only one credential (when using interactive mode)

[01:33] Bug #1567169 opened: juju deploy bundle does not honor existing machine definitions

[01:57] so if I want to run one test a la `go test . -name TestThatIWantToRun` do I need to do something special for gocheck to honor that? [01:57] axw: I wonder if TerminateInstances call should be re-written to use retry too instead of shortAttempt... [01:59] anastasiamac: probably [01:59] redir: go test -gocheck.v -gocheck.f TestYouWant [02:00] redir: which I think is a regular expression, so u can have both 'myTestSuite' or 'individualTestName' or whatever matching... [02:00] thank you [02:00] if it is a pass through then it should be a regexp [02:02] and thank you [02:02] u r welcome \o/ [02:03] Bug #1567170 opened: Disallow upgrading with --upload-tools for hosted models

[02:15] Bug #1567175 opened: Service start is not reliable with any network initialization issues === redir is now known as redir_eod [02:22] see you all tomorrow sometime [02:23] o/ redir_eod === redir_eod is now known as redir_afk [02:36] Bug #1567179 opened: ec2 terminate instances should retry with exponential delay on err [02:36] Bug #1567182 opened: Fallback to installing mongo 2.4 if no 3.2 doesn't work

[02:39] menn0: while i'm waiting for other thigns [02:39] i started a set of PR's about testing that we talked about [02:39] https://github.com/juju/juju/pull/5023 === natefinch-afk is now known as natefinch [02:40] davecheney: taking a look [02:43] davecheney: very nice. land it! [02:45] menn0: that lxd failure has been hit on other branches, so it's not unique to model migrations [02:46] cherylj: ok good [03:00] menn0: thanks [03:00] lots more where that came from [03:04] oh god [03:04] I hacked distro-info to return xenial as the lts [03:04] to see what fails in unit tests [03:04] so many failures [03:05] was gonna say, maybe you should catalog what passes. Might be faster. [03:05] lol [03:06] Bug #1567169 changed: juju deploy bundle does not honor existing machine definitions

[03:08] * cherylj cries [03:08] we're going to be screwed in 2 weeks [03:08] better to know now? [03:08] cherylj: more time for wine between now and then? [03:09] rick_h_: I wish. We gotta fix these or else we will not be able to release anything once xenial is our official lts [03:10] cherylj: understand, it's 11pm have to go light hearted [03:10] cherylj: really really smart to check that.... sorry the results weren't more in our favor. [03:12] Bug #1567169 opened: juju deploy bundle does not honor existing machine definitions

[03:13] 92 test failures so far [03:13] well, to be fair, there are some lxd failures because I don't have lxd on my laptop [03:13] so maybe 80 [03:14] man, I really hate that our code cares what series exist and relies on that data never to change and never to be out of date. [03:15] 2 years is forever, man [03:15] trusty 4 lyfe [03:15] cherylj: 2 years my butt. This happens every 6 months when a new series comes out and also as a bonus, whenever we get a new windows or OSX release... I suppose probably centos as well. [03:16] Bug #1567169 changed: juju deploy bundle does not honor existing machine definitions

[03:16] haha [03:17] and what unit test run would be complete without a test timeout!! [03:19] I wonder how many would pass if we set testing.FakeDefaultSeries = xenial [03:19] or, what we should do is testing.FakeDefaultSeries = actualDefaultSeries [03:19] and people can patch [03:19] if they need to [03:20] anyway that's a problem for tomorrow Cheryl [03:20] night cherylj [03:20] poor tomorrow Cheryl [03:20] yeah, yesterday Cheryl is terrible [03:20] :) [03:20] lol... yeah, I hate yesterday Nate. That guy's an idiot. [03:44] func (s *BaseRepoSuite) SetUpSuite(c *gc.C) {} [03:44] func (s *BaseRepoSuite) TearDownSuite(c *gc.C) {} [03:44] great [03:44] that's just great [03:46] davecheney: we should have this on the t-shirt "juju, just great" [03:51] thumper: easy review pls http://reviews.vapour.ws/r/4466/ [04:03] davecheney: http://reviews.vapour.ws/r/4466/ pls [04:07] lookin' [04:07] LGTM, mostly harmless [04:12] davecheney: ta [04:13] menn0: https://github.com/juju/juju/pull/5025 [04:13] more of the same [04:16] ahhh... damn macaroons === thumper is now known as thumper-afk [05:25] Bug #1567228 opened: "juju destroy-controller" can leak hosted models [07:09] dimitern, if you need an immediate fix for the broken /e/n/i try this http://pastebin.ubuntu.com/15653945/ [07:10] frobware, cheers, I'll try this on the next test I'm doing shortly [07:11] dimitern, it's obviously not the complete story... but is in essence the fix [07:12] dimitern, any joy with the case where the first add-machine lxd:0 fails? [07:40] frobware, sorry, I had to step afk [07:41] frobware, well, add-machine lxd:0 only works (post-bootstrap), if you first juju switch admin, is that what you mean? [07:41] frobware, otherwise it says no such machine 0 [07:41] but apart from that, yes - it still fails, still digging.. [07:42] dimitern, nope. if you switch to :admin and add-machine lxd:0 the first one fails to work until the agent is restarted on the node hosting the container. [07:44] dimitern, I think we're saying the same things (give or take the model switch) [07:44] frobware, yeah [07:44] frobware, :) I was about to write the same thing === thumper-afk is now known as thumper [08:58] frobware, dimitern: why is there no standup listed for today? [08:59] voidspace, babbageclunk: care to hangout to discuss maas2? [09:00] I was awaiting a standup that doesn't appear to be scheduled [09:03] thumper: team meeting today at 11am instead [09:03] thumper: but sure [09:03] I'm not going to hang around for that [09:03] thumper: that's our loss [09:04] ow, ow, ow.... cramp [09:04] * thumper stretches [09:04] dimitern: we could use you to talk to babbageclunk and thumper about the link layer model and what information we need from maas [09:04] * thumper is in the sapphire normal standup hangout [09:04] dimitern: currently the maas 1 code uses stuff like link.Parent() and IPAddresses that don't seem to exist in MAAS 2 [09:05] well it does [09:05] thumper: omw [09:05] .. [09:05] hmm [09:09] oops sorry guys, omw [09:10] voidspace, babbageclunk, thumper, standup HO? [09:10] dimitern: we're there [09:10] dimitern: hello [09:21] babbageclunk, dooferlad, frobware, join us guys for standup? [09:21] dimitern: sure. URL? [09:21] Coming now [09:34] Bug #1567296 opened: Plugin API fails with multiple juju binaries [10:02] anyone know what the best way to shut down a juju lxd provider controller is when juju won't let me (probably because i started it with an old juju version) ? [10:12] whoopse, that was rather aburpt [10:12] sorry about that [10:17] rogpeppe: the best way has been build that old env and kill-controller with that [10:17] mgz: unfortunately i've no idea what version i used to bootstrap the old env [10:17] mgz: i think i need to go behind the scenes [10:18] mgz: but my naive approach ("lxd shutdown") doesn't seem to have got me to a good place [10:19] mgz: i just removed $HOME/.local/share/juju and now i'm getting "ERROR invalid config: no addresses match" [10:19] mgz: when i try "juju bootstrap local lxd" [10:19] mgz: :-( [10:19] mgz: AFAIK i haven't *got* a config [10:22] rogpeppe: yeah, it's a joy [10:23] mgz: i've just proposed https://github.com/juju/cmd/pull/35 to make it easier to debug that kind of error message [10:26] mgz: that makes the error somewhat more informative... 2016-04-07 10:25:39 DEBUG cmd supercommand.go:449 error details: [{github.com/juju/juju/cmd/juju/commands/bootstrap.go:406: } {github.com/juju/juju/environs/open.go:104: } {github.com/juju/juju/environs/open.go:175: } {github.com/juju/juju/provider/lxd/provider.go:44: } {github.com/juju/juju/provider/lxd/environ.go:43: invalid config} {github.com/juju/juju/provider/lxd/config.go: [10:26] 182: } {github.com/juju/juju/provider/lxd/config.go:255: } {github.com/juju/juju/tools/lxdclient/config.go:65: } {github.com/juju/juju/tools/lxdclient/remote.go:199: } {no addresses match}] [10:31] didn't that used to have prettier formatting? [10:36] frobware: dimitern: babbageclunk: doing a merge of master onto maas2 [10:36] frobware: dimitern: babbageclunk: some fixing to do as anastasiamac fixed a maas bug that changes the maasInstance interface [10:37] voidspace: nothing too nasty I hope? [10:37] mgz: possible. i don't care much :) [10:38] babbageclunk: maasInstance.zone() now returns an error - so the interface definition and our maas2Instance implementation need updating [10:38] mgz: FWIW the error is because juju's looking for netif lxdbr0 but my machine only has lxcbr0 [10:38] babbageclunk: plus there are a bunch of new tests that do a type cast to maasInstance that should now be maas1Instance [10:38] babbageclunk: plus another bugfix by davecheney in maas storage [10:38] babbageclunk: nothing too nasty, no - easy enough [10:52] morning [10:55] mgz: ha, i upgraded lxd and now there *is* an lxdbr0 interface but it doesn't have any ipv4 addresses so i get the same failure [10:56] anyone used the lxd provider with the latest juju, by any chance? [10:56] morning perrito666 [10:56] voidspace: do you know anything about this issue, by any chance? [11:04] rogpeppe: there's an email thread.on the juju list around this i think [11:04] rogpeppe: from yesterday [11:04] rick_h_: yeah, just looking for it [11:05] rick_h_: found [11:07] rick_h_: i don't think it helps me though [11:08] rick_h_: there's no "lxd-bridge" service on my machine AFAICS. [11:08] rogpeppe: no, afraid not [11:09] rogpeppe: I just saw an upgrade do the same thing (well - change the bridge name) on my server too though [11:09] rogpeppe: jam might know if he's still around [11:09] voidspace: yeah, it seems to break everything [11:09] :-( [11:09] or maybe frobware I know he's done a lot of work with lxd [11:09] voidspace: i guess i need to create an ipv4 address on the bridge somehow [11:09] rogpeppe: it should *get* an ipv4 address [11:10] so it's odd [11:10] voidspace: it really seems that it shouldn't *need* an ipv4 address anyway [11:10] voidspace: aren't we supposed to work with ipv6? [11:10] maybe look in /etc/network/interfaces and see how the bridge is defined [11:10] voidspace: good idea [11:10] rogpeppe: I'm not convinced I think we've *tried* to be compatible but not been thorough [11:10] voidspace: FWIW ifconfig shows this: http://paste.ubuntu.com/15667206/ [11:10] rogpeppe: and ifdown/ifup iit [11:11] voidspace: well, utils.GetAddressForInterface discards all ipv6 addresses... [11:11] rogpeppe: no packets dropped and some traffic [11:12] rogpeppe: we have a preferIPv6 setting somewhere - does that code not honour it [11:12] voidspace: almost nothing in /etc/network/interfaces [11:12] voidspace: just two lines: [11:12] auto lo [11:12] iface lo inet loopback [11:12] voidspace: nothing in interfaces.d either [11:12] oh, odd [11:12] I wonder where it is - this is xenial? [11:13] voidspace: nope, trusty [11:13] ah [11:13] I've not worked with trusty networking - I can commission a trusty box though [11:13] * rogpeppe has been meaning to find the time to reinstall from scratch for *ages* [11:14] rogpeppe, for trusty you need to add "trusty-backports" to /e/apt/sources.list && sudo apt-get update && sudo apt-get install -t trusty-backports lxd [11:15] dimitern: like: add-apt-repository trusty-backports ? [11:15] rogpeppe, voidspace, prefer-ipv6 is no longer respected (always false), but the setting itself is not gone yet (will be soon though) [11:15] dimitern: hiya BTW :) [11:15] rogpeppe, it's not a repo, it's a cloud pocket or something like that [11:15] rogpeppe, ;) hey [11:15] dimitern: what exact line(s) do i need to add for /etc/apt/sources.list ? [11:16] rogpeppe, if you paste yours, I can show you - can remember off-hand [11:16] dimitern: paste what? [11:16] rogpeppe, /etc/apt/sources.list from the trusty machine in question [11:16] dimitern: the current contents of my sources.list [11:17] dimitern: https://pastebin.canonical.com/153655/ [11:18] rogpeppe, actually - here's a wiki page: https://help.ubuntu.com/community/UbuntuBackports [11:18] rogpeppe, so you need a line like this somewhere: deb http://archive.ubuntu.com/ubuntu trusty-backports main restricted universe multiverse [11:19] rogpeppe, but looking at your paste you seem to have it enabled already [11:22] dimitern: yeah [11:23] rogpeppe, so just apt-get update && apt-get install lxd/trusty-backports should work now [11:24] dimitern: still same problem after doing that [11:24] rogpeppe, the container still did not start? [11:25] dimitern: juju refuses to do anything because the interface has no ipv4 addresses [11:25] rogpeppe, lxdbr0 ? [11:25] dimitern: yeah [11:28] rogpeppe, hmm I'm not that familiar with lxdbr0 specifics, but you should be able to use "lxc-bridge: lxcbr0" in your bootstrap config [11:28] rogpeppe, provided lxc1 package is also installed and lxcbr0 exists [11:28] dimitern: frobware: babbageclunk: dooferlad: merge master onto maas2 http://reviews.vapour.ws/r/4473/ [11:28] dimitern: i'd prefer to be able to bootstrap without using that tbh [11:29] dimitern: but i'll try it [11:30] rogpeppe, well, on trusty lxd is still kinda flaky; btw you might instead need to change /etc/default/lxd to enable IPv4 addresses [11:30] rogpeppe, check this, it might help - https://github.com/lxc/lxd/pull/971/files [11:30] dimitern: unknown config field "lxc-bridge" [11:42] rogpeppe, ah, sorry - that's no longer supported yeah :/ - I guess the second option then - changing /e/d/lxd (or lxd-bridge) there to provide IPv4 addresses [11:42] dimitern: frobware: dooferlad: babbageclunk: branch adding a feature flag for the MAAS 2 work http://reviews.vapour.ws/r/4474/ [11:42] dimitern: /etc/default/lxd doesn't exist for me [11:43] dimitern: ah, but lxd-bridge does [11:43] rogpeppe, right - there [11:43] voidspace, looking [11:46] dimitern: hmm, i added a bunch of ipv4 stuff in lxd-bridge, restarted lxd (sudo initctl restart lxd), but no ipv4 addresses seem to be arriving [11:46] dimitern: FWIW my lxd-bridge file now looks like this: http://paste.ubuntu.com/15667827/ [11:48] rogpeppe, try adding a new container [11:48] dimitern: is that easy? i've not used lxd directly before. [11:48] voidspace: in fact, if possible, it'd be awesome if any property accessors from maas instance also returned errors... it was very sad that we were swallowing that one :/ [11:49] dimitern: is the lxd command the way to do it? [11:49] dimitern: (i don't see anything that talks about creating a new container in its help) [11:49] * rogpeppe googles [11:50] rogpeppe, lxc launch ... [11:50] dimitern: not lxd? [11:51] rogpeppe, lxc is the cli client, lxd is the server daemon [11:51] voidspace, LGTM [11:51] dimitern: ok, now i have to work out how to fetch an image... [11:52] https://linuxcontainers.org/lxd/getting-started-cli/ << nice guide [11:52] rogpeppe ^^ [11:52] dimitern: ah, launch ubuntu failed but launch ubuntu:14.04 worked [11:53] dimitern: still no ipv4 address [11:53] rogpeppe, I think juju adds an alias "ubuntu" for the current series image [11:54] rogpeppe, ah :/ bad luck - folks in #server @c might be able to help [11:54] dimitern: i'll just give up and use ec2 i think. i've already spent too much time chasing this. [11:54] dimitern: thanks for your help. [11:55] dimitern: thanks [11:55] anastasiamac: the way we interact with maas has completely changed [11:55] anastasiamac: isn't it horrifically late for you now? [11:56] rogpeppe, np [11:56] dimitern: shall I just land the master merge? === urulama|eod is now known as urulama [11:56] dimitern: http://reviews.vapour.ws/r/4473/ [11:58] voidspace: i had a preview of new maas abstraction (peeked at thumper's branch)! it's awesome \o/ thank you for doing it! [11:58] anastasiamac: yeah, it's very cool - all thumper's work, we're just using [11:58] voidspace: it's 10pm... for normal ppl probably not a work hour... but who here respects that? :D [11:59] anastasiamac: ah, I thought it was later [11:59] heh [12:00] voidspace: r u re-writting maas19 interactions in similar way too? [12:00] or it stays the same? [12:01] voidspace, does it build and pass make check? [12:37] voidspace: sorry, been talking to frobware, not doing much typing [12:41] Bug #1567396 opened: tools/lxdclient should not be using a global variable for several functions

[12:41] dooferlad, ping [12:42] frobware: pong [12:43] dooferlad, can we HO about the bond issue - was interested in your repro case [12:43] anastasiamac: probably not, too much work :-( [12:43] frobware: sure, but I really need lunch [12:43] anastasiamac: MAAS 2 is new code so worth doing right [12:43] frobware: can it wait 30 minutes? [12:43] dooferlad, whenever works [12:43] frobware: thanks! [12:43] dimitern: yep, builds and passes [12:43] dimitern: I had some conflict resolution to do - nothing too difficult though [12:44] voidspace: sounds sensible [12:45] babbageclunk: my feature flag change has landed on MAAS2 [12:46] voidspace: Sweet [12:46] babbageclunk: so after you update you'll need to set the MAAS2 feature flag to bootstrap against MAAS 2.0 [12:46] voidspace, lgtm [12:46] anastasiamac: it would certainly be possible, and nicer... so if anyone has any spare time :-) [12:46] dimitern: ta [12:48] voidspace: well, i don't think it's worth it... but the decision should b based on maas.19 life expectancy ... if it's to be short-lived, I'd say that what we have now will do... [12:48] anastasiamac: yeah [12:48] nobody ever has any spare time anyway [12:49] so true [13:28] could i have a review on this almost-entirely-trivial one line change, please? it should make some things easier to debug in the future. https://github.com/juju/cmd/pull/35 [13:29] frobware: back [13:40] rogpeppe: would prefer not to log the error twice [13:40] rogpeppe: can you do an if debug enabled... [13:40] dooferlad, was your repro case with h/w? [13:41] frobware: yes [13:41] dooferlad, as opposed to KVM [13:41] natefinch: what would you do by preference? it's *really* useful to be able to see the error details. [13:41] natefinch: you'd print the error differently if debug enabled? [13:42] natefinch: i think it's kinda reasonable to have a separate debug line exposing details of the error without changing the usual error printing [13:42] rogpeppe: yes... details if debug enabled, otherwise just the string..... otherwise you're printing the error twice, which could be confusing, make it look like we did something twice [13:43] natefinch: tbh i think we'd want the error details on a separate line for readability anyway [13:43] natefinch: perhaps a better error message string would felp? [13:43] help [13:44] rogpeppe: I just worry that two log entries would be confusing. Maybe if debug is enabled, append the details to the single errorf log message? [13:45] natefinch: if you do that, that line becomes very unwieldy [13:45] natefinch: i think it should be reasonably obvious - one's at debug level and they both contain the same text. [13:46] natefinch: perhaps surround it all in brackets? "(error details: xxxx)" and print after the main error message? [13:46] rogpeppe: then put a comment in the code that states you're doing it on purpose, otherwise someone's going to come along and "clean up" the "duplicate" log output. [13:46] rogpeppe: I do think after is better than before [13:47] rogpeppe: and brackets will make it more obvious that it's not just another instance of the error. [13:47] rogpeppe: and maybe use a sans-serif font, since that'll make it stand out more. [13:47] natefinch: :) [13:56] Bug #1567458 opened: destroy-controller message and failure is not user friendly [14:10] natefinch: better now? https://github.com/juju/cmd/pull/35/files [14:15] rogpeppe: lgtm [14:15] natefinch: thanks [14:15] morning all [14:18] morning alexisb [14:18] hi alexisb [15:20] hey tych0, does your lxdbr0 fix always return 'lxdbr0', or do we check elsewhere for whatever the user configured? (https://github.com/juju/juju/pull/5004/files#diff-d0829a0ea5ffc060343eba40ecd40aa8R66) [15:24] kwmonroe: yes, that only gets called when the user hasn't configured anything [15:24] kwmonroe: expand the last hunk of the diff and you can see [15:25] babbageclunk: StartInstance can allocate and start a machine [15:25] babbageclunk: and I've done as much of StartInstance as is possible without further work in gomaasapi [15:26] babbageclunk: now to test it all... [15:26] babbageclunk: don't forget to write up what you've done in the doc [15:27] cool tych0, thx [15:56] babbageclunk, http://juju-sapphire.github.io/ [16:02] babbageclunk, https://github.com/dimitern/testcharms [16:04] cherylj: ping [16:04] voidspace: nice! I've got the storage stuff done, although not tested yet. Need to merge your master merge. [16:04] hey perrito666, what up [16:04] babbageclunk: cool [16:05] voidspace: will write up now [16:05] babbageclunk: I found another undocumented parameter to AllocateMachine [16:05] babbageclunk: storage.... [16:05] voidspace: yay [16:05] hey, I am trying to test my fix using lxd provider but I get a permission denied on my instances :( when juju tries to ssh, did you enconter this issue testing with xenial? [16:05] babbageclunk: so storage and spaces are not yet supported, but trivially easy to add them [16:06] voidspace: why not spaces? [16:07] babbageclunk: because interfaces parameter is missing on AllocateMachineArgs [16:07] that's the first undocumented parameter, storage is the second [16:07] cherylj: that was for you [16:07] they haven't changed since 1.9 - but they weren't documented there either... [16:07] voidspace: ah, right - thought you meant because storage isn't done yet [16:08] perrito666: I'm thinking of what might be causing that.... [16:08] babbageclunk: no, you specify storage to AllocateMachine (as well as the storage specific code) [16:08] babbageclunk: I'm off to do some exercise - bbiab [16:08] perrito666: when are you running into this? during bootstrap? [16:09] perrito666: do you have a --debug output you could paste? [16:09] babbageclunk: what happened with StopInstances - is it done? [16:09] babbageclunk: I need that too really, it's used by StartInstance [16:09] cherylj: during bootstrap everything seems fine until the moment where it has to try logging int [16:09] which is odd because the ubuntu user has the right credentials in authorized keys [16:09] perrito666: what's your host? xenial? [16:10] wily [16:10] voidspace: It's done but not tested - it needed storage so I parked it while I was doing that. [16:11] voidspace: Once storage is merged and pushed I'll merge storage into stop instances and push that too. [16:11] Then you can use both/ [16:11] Bug #1567518 opened: Payload commands don't work [16:13]