[00:58] Bug #1552021 opened: lxd not enough arguments in call to manager.CreateContainer

[01:26] Sso login is really painful on a cell phone [01:51] thumper: menn0, a small driveby https://github.com/juju/juju/pull/4591 [02:09] davecheney: will look in a sec once I've finished the other review I'm doing [02:20] davecheney: ship it [02:39] Can I get a super quick review? http://reviews.vapour.ws/r/4032/ [02:40] cherylj: looking [02:41] cherylj: ship it [02:41] thanks, menn0! [02:47] I desperately want someone to write a new test provider [02:47] to replace dummy [02:47] one that doesn't try to be too special [02:47] just an in memory db of machines etc... [02:47] thumper: friday afternoon job :-) [02:48] ha [02:48] menn0: not sure if you have time, this pr changes create-model. you may have backgroud jes knowledge that may be useful http://reviews.vapour.ws/r/4033 [02:48] it works live on ec2 [02:49] I want a base suite that has a db, with an environment, but no apiserver etc... [02:49] is there one such base suite? [02:49] thumper: state/testing.StateSuite [02:49] menn0: cheers [02:52] menn0: heh, that doesn't actually give me a valid provider... [02:52] hmm... [02:59] wallyworld: looking now (I'm OCR anyway) [02:59] thats's why i picked on you :-) [03:00] wallyworld: I thought it was because of my JES super powers :) [03:00] that too - i was sucking up [03:02] wallyworld: "--credential" refers to an entry in credentials.yaml? [03:02] yep [03:03] same as for bootdttrap [03:03] same arg name [03:03] nice [03:03] and bootstrap too :-p [03:03] nice because as admins testing stuff on aws, we don't need to supply credentials at all anymore for create-model [03:04] qa folks were very happy === natefinch-voting is now known as natefinch [03:42] wallyworld: no major issues, just minor stuff === bruno is now known as Guest56910 [03:42] menn0: you rock, tyvm [04:28] menn0: i did improve the credential handling to account for different types of credentials being passed in. if the user has specified any valid set of credential values for a given auth type, they will be used in preference to the controller ones [04:29] we won't copy across controller values to result in different values all smooshed together [04:31] wallyworld: awesome that sounds great. [04:31] yeah, thanks for raising it [05:01] wallyworld: could you take a quick look at this one: http://reviews.vapour.ws/r/4035/ [05:02] wallyworld: it's a mechanical only change [05:03] sure [05:05] menn0: much nicer [05:10] wallyworld: thanks [05:31] menn0: here's a small one for you https://github.com/juju/juju/pull/4596 [05:51] davecheney: +1 [05:52] anastasiamac: if you get a chance, i'd love a review of http://reviews.vapour.ws/r/4037/ which adds detail to list-controller, plus a driveby fix of old env names [05:52] wallyworld: sure [05:52] looking \o/ [05:52] ty :-) [05:52] bbiab, school pickup [05:52] wallyworld: have fun :D === Guest56910 is now known as BrunoR [09:26] dimitern: ping [09:27] voidspace, pong [09:27] dimitern: I'm moving ConvertSpaceNames out of the discoverspaces worker so that the provider can use it for constraints [09:28] dimitern: is it ok for it to live in apiserver/networkingcommon and for the maas provider to depend on the apiserver [09:28] dimitern: ? [09:28] dimitern: I can't see why not but I thought I'd check [09:28] voidspace, I'd prefer to put it in network/ [09:29] dimitern: d'oh [09:29] dimitern: yeah, that's better [09:29] thanks [09:29] voidspace, to minimize the import loop / possible such issues [09:57] dimitern, voidspace, dooferlad: please could we meet at 10past the hour for standup. [09:57] jam: ^^ [09:58] frobware: +1 [10:01] sgtm [10:03] sure :-) [10:12] voidspace, jam, standup? [10:25] dimitern: oops, sorry [10:34] dooferlad, frobware, I look how CI just proved me wrong - we have a #3689 curse on maas-spaces2 [10:34] s/I/now/ [10:54] Who do we ping for APT details? [10:55] I'm trying to understand why "apt-get install lxc/trusty-backports" can find "liblxc1/trusty-backports" but "apt-get install lxd/trusty-backports" can't find "lxc/trusty-backports" [11:02] voidspace, dimitern: meeting [11:03] coming [11:07] frobware: dimitern: omw, sorry *again* [11:41] Hi, lads. When jujud (one that's running on the state servers) connects to mongodb replicaset (assuming i'm running juju-ha), is it always connecting to PRIMARY mongodb only, or sometimes reads can go from the SECONDARYes too? That is, what is the readPreference that juju uses when reading from mongo? [11:44] If I'm reading the code correctly it seems it's using .SetMode(mgo.Monotonic), which means sometimes reads can go from the SECONDARYes, but I'd like a confirmation, if possible :) [11:47] mgz: if you're around, what machine would be running "run-unit-tests-xenial-ppc64el" because it has a test failing that I can't reproduce on stilson-05 [12:01] mgz: also, do we have any problems with clock skew on stilsons? I just saw a test fail in 400ms when it should have been waiting for LongWait to expire, which is 10s. [12:20] dimitern, voidspace, dooferlad: I think we should cherry-pick the "can't bootstrap on xenial" change into maas-spaces2. We talked about this yesterday but this would help remove one of the voting failure we see in the maas-spaces2 CI results. === admcleod1 is now known as admcleod [12:24] mgz: looks like the time thing was a red herring. it took 400ms to setup the assert, but the assert did fail after 10s [12:26] frobware, sounds good, yeah let's do it [13:02] frobware, ping [13:06] jam: ppc64-slave is stilson-06 at present [13:08] jam: are you running the tests throught the same run-unit-tests script when trying on -05? the specific setup of the lxc etc seems more likely to influence results than the (generally minor) difference between the machines [13:09] dooferlad, fwiw "defaultsMap" in state.mergeBindings is no in any way related to network.DefaultSpace [13:10] not anymore that is [13:10] mgz: I'm just doing "go test -check.v" in a given subdir. [13:10] currently investigating a test that is flaky [13:10] passes about 9 in 10 times [13:10] but when it fails it was waiting 10s when it fails [13:11] if you can do the subset like that it is the best way [13:29] jam, frobware, dimitern, alexisb: hey can we postpone or catchup by 1hr? I have a conflict I can't miss... [13:30] jamespage, yep I can make that happen, jam can you make that? [13:31] jamespage, ok, sure [13:32] jamespage, we have good news though ;) [13:33] * jamespage is a bit excited... [13:36] dimitern: since that change of mine is quite big and you had plenty of comments, would you like to have a hangout to discuss it? I expect it would move things along more quickly. If so, what time works for you? [13:37] dooferlad, sure, how about in an hour, after the openstack call? [13:38] jamespage: dimitern: frobware: does 3.5hrs from now work for you guys? Its getting into dinner-and-family time for me. [13:39] dimitern: Sure. I don't have an invite to the OpenStack call so I can get on with another card until then. [13:39] jam: I could do right now if that's helpful? [13:39] works for me [13:39] sure [13:39] dimitern, frobware, alexisb? ^^ [13:40] jamespage: alexis and I are in the hangout [13:40] jamespage, jam, alexisb, a bit late, but I can manage [13:41] dimitern: it's on now [13:41] dimitern, can you do now? [13:41] sure [13:41] dimitern: we're there now [13:51] jam, frobware, http://reviews.vapour.ws/r/4038/ [14:00] Bug #1552237 opened: socket.error: [Errno 110] Connection timed out while calling status()

[14:25] dimitern: reviewed [14:26] frobware, sweet! thanks [14:26] frobware, and I've just confirmed network-get works live with extra-bindings [14:27] dimitern: \o/ [14:28] dimitern: so this (https://github.com/juju/charm/pull/192) has landed too. anything else right now to review? [14:33] frobware, there's one more, which I can propose now for preliminary review until I sort out it's prereq [14:34] dimitern: point me at it before I hunker down with the bridge (br-eth1) problem. [14:35] frobware, proposing now, just a sec [14:37] frobware, http://reviews.vapour.ws/r/4042/ [14:39] dimitern: wow, that touched a lot of files. [14:39] frobware, it includes the PR you've reviewed already, so looking at individual commits might be useful [14:40] dimitern: is it worth changing dooferlad's demo script and verifying this...? [14:41] frobware, definitely - the changes should be simple to add [14:41] dimitern: do you have a branch that I could use that has all of this in one place? [14:41] frobware, I can paste the charms and bundle I used in case you want to give it a go later? [14:42] frobware, yeah, the maas-spaces2-extra-bindings2 includes all [14:42] dimitern: I just thought a different (and existing) test scenario may help [14:42] dooferlad, hey, should we do the HO now? [14:42] dimitern: can you paste the charms/bundle anyway [14:43] frobware, give me a sec [14:43] frobware, http://paste.ubuntu.com/15267040/ this is the bundle [14:45] dimitern: 10 mins? [14:46] frobware, this is the client-nc metadata: http://paste.ubuntu.com/15267060/, tree: http://paste.ubuntu.com/15267048/, and hooks: http://paste.ubuntu.com/15267071/ [14:48] frobware, and here's the same for server-nc: http://paste.ubuntu.com/15267062/ (metadata.yaml), http://paste.ubuntu.com/15267049/ (tree), http://paste.ubuntu.com/15267081/ (hooks) [14:53] dimitern: could you dump all that in a github repo? seems like the nc would be another useful demo. [14:54] frobware, can do yeah - later tonight [14:54] dimitern: thx, appreciated. [14:59] dimitern: still doing some debug, but can stop if now is a good time. [15:05] dooferlad, well, now is as good a time as any :) [15:06] dimitern: ring ring... [15:06] dooferlad, ah, let's use the standup HO as that invite came to my phone :) [15:06] dimitern: doh! [15:06] dimitern: there now [15:10] I love it when I find exactly the right function that I need in the standard library [15:10] natefinch: oh, you're writing python? [15:10] mgz: lol [15:11] :P [15:11] mgz: does python have a mime.FormatMediaType / ParseMediaType in the standard library? [15:11] FormatMediaType serializes mediatype t and the parameters param as a media type conforming to RFC 2045 and RFC 2616. [15:11] yeah, I think it's hidden in email though [15:11] nice [15:17] voidspace: ping. Please could you help out with a review of http://reviews.vapour.ws/r/4042/ - bug #1549545 is critical for master atm. [15:17] Bug #1549545: Bundle deploys fail at lxc-start when bridge br-eth1 is created

[15:18] voidspace: the review and bug are not related. I'm just trying to make progress with the bug and that is a large PR. [15:21] Bug #1552274 opened: juju list-credentials inconsistencies between format output [15:42] dimitern: can you jump into sapphire HO regarding your extra-bindings changes... [15:42] frobware, sure === psivaa_ is now known as psivaa [16:26] dammit.... I hate it when I inevitably use a stdlib function that doesn't exist in our version of go :/ [16:27] natefinch: hey, never got that estimate for the current card you're working on [16:28] katco: oh, sorry [16:29] natefinch: gj landing the last card btw [16:29] katco: thanks [16:29] katco: tomorrow for the new card, I think. Hopefully early tomorrow.. doesn't seem to bad. It sucks, go 1.5 has a bunch of improvements in exactly the area this card needs. [16:30] katco: (mime type stuff for setting http headers etc) [16:30] natefinch: we don't need mime stuff do we? [16:31] natefinch: this is just comparing the name specified with the name in the metadata? [16:31] katco: there's functions for formatting the content-disposition, which is how you should set the filename [16:31] katco: we could set the filename some other way, but it would be non-standard [16:32] natefinch: ericsnow: moonstone plx, need to chat anyway [16:32] kk [16:52] frobware: dammit, missed that - sorry [16:52] frobware: looking now [17:15] ericsnow: natefinch: oh, also forgot to mention. urulama has an open invitation to our standup while this collaboration is going on [17:15] ericsnow: natefinch: so it will be fun to see him from time to time :) [17:15] katco: cool [17:20] frobware: I'm seeing that I cannot deploy to a node with wily on 1.25.4 with or without your patch. I can't get the cloud-init-output.log file from the node because networking doesn't seem to be working. [17:21] I'm trying it with 1.25.0 now to see if it's just a problem in my env [17:21] cherylj: ack [17:22] cherylj: so I /think/ I had the same problems, but with precise. It was late last night when I tried so I assumed it was finger trouble. I did have success with trusty and xenial. [17:23] frobware: I was in the same boat last night :) Had issues with precise, but not trusty [17:23] cherylj: I'm also using daily images, not sure if that's a factor. [17:24] cherylj: I didn't see an obvious way in MAAS in having released and daily. Removing daily is a pain because it has to reimport all images again. [17:25] frobware: I don't even know which I'm using. I just used the gui to import precise and wily (no option for xenial) [17:27] cherylj: so that sounds like you're using released. It's in "Settings" -> "Boot Images". Mine says: http://maas.ubuntu.com/images/ephemeral-v2/daily/ [17:28] frobware: ah, yeah, I see: http://maas.ubuntu.com/images/ephemeral-v2/releases/ [17:29] thanks! [17:32] frobware: I was able to deploy with 1.25.0, so seems to be a problem with 1.25.4 [17:32] :( [17:33] I'm not sure how I can grab the cloud-init as the console I get with the libvirt tools doesn't seem to provide a way to capture the output [17:33] cherylj: do you fancy bisecting at .minor releases? [17:33] and there's no password set [17:33] frobware: sure, I can do that [17:33] I'll try 1.25.2 now [17:33] cherylj: so when I run into these problems during development I change the bridge script so that the first thing it does is 'passwd -d ubuntu' [17:33] oh nice [17:35] cherylj: ... don't forget if you change add-juju-bridge.py you need to run make in provider/maas, then rebuild. [17:36] cherylj: this particular light bulb went off once I tried to guess 1 to many times what butchering I had done to /e/n/i... :) [17:36] hehe [17:42] * katco lunches === natefinch is now known as natefinch-lunch [17:47] cherylj: so with these same changes that I landed on master I am able to bootstrap precise [17:47] frobware: I was able to bootstrap wily too, the problem came when I tried to deploy [17:48] cherylj: do you still have wily deployed? care to HO if so? [17:48] frobware: I just destroyed the environment and I'm trying out a deploy with 1.25.2 [17:49] ok [17:49] dimitern: LGTM (modulo that one logging tweak) [17:49] voidspace: many thanks!!! [17:49] dimitern: although it looked big it's mostly tweaking stuff [17:51] voidspace, thanks! [17:51] frobware: not that this helps much, but I saw that it couldn't download tools because of no route to host: https://private-fileshare.canonical.com/~cherylj/images/node-deploy-output-2.jpg [17:53] cherylj: as a first pass I would verify that MAAS can deploy a precise node. let's take juju out of the equation. [17:53] I'm working on wily right now [17:53] But I was able to deploy wily with 1.25.0 [17:53] and it also works with 1.25.2 [17:54] wait, might've spoken too soon [17:54] im having problems deploying via juju to openstack compute nodes. Im having to use juju-core 1.25.4 due to bug 1532167. However, the orchestration node has the x86 code installed and cant push that to ibm power... so doing juju add-machine fails for the power nodes [17:54] Bug #1532167: maas bridge script handles VLAN NICs incorrectly

[17:54] cherylj: why it it trying to download 1.25.99.1? [17:55] frobware: I changed the version number to distinguish that it was running with your patch [17:55] frobware, updated http://reviews.vapour.ws/r/4038/ it should be ready to land so the follow-up can as well [17:55] aha [17:55] dimitern: is this post voidspace's LGTM? [17:56] dimitern: ah, no. Was my review comments. :) [17:56] frobware: well this is interesting. with 1.25.2, I was able to ssh into the node, but then networking died as it was finishing cloud init [17:56] :) [17:56] I like how much simple the parseBind tests have become [17:56] simpler even [17:57] ok, time to go celebrate ;) [17:57] neato! the passwd -d ubuntu worked and I can see the cloud init output log [17:57] cherylj: try again, it's possible the ifdown -a; ifup -a was running [17:58] frobware: I saw as the cloud init log was scrolling by that it failed to download tools [17:58] cherylj: what's the state of /e/n/i? [18:00] frobware: https://private-fileshare.canonical.com/~cherylj/images/eni.jpg [18:01] cherylj: looks innocuous [18:01] cherylj: ip route? [18:02] cherylj: but this is dead without my changes, correct? [18:02] frobware: right, this is on 1.25.2 [18:02] cherylj: you're still testing 1.25.2? [18:02] ack [18:03] * frobware wonders why he doesn't have a jenkins instance that continually validates the simples [18:05] frobware: https://private-fileshare.canonical.com/~cherylj/images/ip-route.jpg [18:08] cherylj: so don't see anything wrong [18:08] cherylj: and your virbX is NAT'd? [18:09] frobware: yes [18:10] cherylj: easier to HO maybe [18:11] frobware: sure, give me just a minute [18:11] cherylj: I would still deploy via MAAS, validate that that deployed node can see the internet [18:11] frobware: okay, let me kick that off [18:11] cherylj: at least it helps separate juju specific issues [18:14] cherylj: I have a fix for bug #1549545 too, but could do with some 3rd party validation (hint hint) :) [18:14] Bug #1549545: Bundle deploys fail at lxc-start when bridge br-eth1 is created

[18:15] frobware: haha :P let's figure out this one first [18:16] cherylj: sure. but I got bored with that one. :P [18:16] heh [18:16] frobware: the node deployed in maas seems to be fine [18:16] cherylj: so just to confirm 1.25.2 cannot deploy wily, from MAAS 1.9.0 [18:17] frobware: seems to be the case [18:17] although surprising that we wouldn't have a test for this in CI [18:17] cherylj: now that I pushed my branch ^^ let me try as well [18:21] cherylj: yeah, this was my "simples" case above. Just a smoke test that you can bootstrap and fetch http://kernel.org/ [18:23] toolstorage lacks a way to delete the tools? [18:28] frobware: I think bug 1551959 is the same issue I'm running into [18:28] Bug #1551959: juju-tools 1.25.0 with juju-agent 1.25.3 has networking issues [18:29] cherylj: ahhh [18:29] cherylj: without looking I'm guessing this is how we parse lines like dns-nameservers...' [18:29] cherylj: yep [18:29] cherylj: can reproduce. :( :( :( [18:33] cherylj: so a known and fixed issue: https://bugs.launchpad.net/juju-core/+bug/1534795 [18:33] Bug #1534795: unit loses network connectivity during bootstrap: juju 1.25.2 + maas 1.9

[18:39] cherylj: I think we will have upgrade problems with the script. [18:39] frobware: how so? [18:40] cherylj: so if I understood the bug you just referenced then 1.25.3 will come along and try to bridge something that may well be bridged already. We did discuss this one writing the script but ... :( [18:40] frobware: ah, I see. However, in my case, I was not doing an upgrade [18:41] cherylj: we talked about putting a sentinel in there but there's no guarantee that something else doesn't remove it. [18:41] cherylj: I think we're gonna have to be a lot smarter... [18:41] cherylj: no, 1.25.2 is busted because of 1534795 [18:42] cherylj: but 1551959 is something we should address for maas-spaces2, master and 1.25... [18:43] cherylj: we should really really get MAAS to model bridges. we're endlessly futzing. [18:43] cherylj: patch upon patch [18:43] frobware: :( [18:43] cherylj: let's talk about some priorities. [18:44] frobware: in a call now, let me ping you [18:44] ok [18:44] * frobware sulks off as far as his teapot [18:44] heh [18:55] is there a way to upload juju tools for multiple arches when bootstraping ie juju bootstrap --upload-tools? === natefinch-lunch is now known as natefinch [19:15] who writes a decoder that can't handle all the outputs of the corresponding encoder? Sheesh [19:16] natefinch: you do realize that we work in a project that had a set that did not take the output of get until recently :p [19:16] heh... yeah, and that's terrible [19:16] here's the context to my rant: https://groups.google.com/forum/#!topic/golang-nuts/ovxD01JNoBQ [19:36] cherylj: I just bootstrapped precise on my change (1.25.4) [19:37] cherylj: I changed two/three things: [19:37] 1) add the passwd -d ubuntu in anticipation ... [19:38] 2) changed enable-os-refresh-update from true to false [19:38] 3) ditto for enable-os-upgrade [19:54] sigh [19:55] there's no juju destroy-controller --force? [19:56] ....so I guess we changed bootstrap and/or destroy-controller such that we can't destroy old controllers anymore (and by old, I mean like, a couple days old)? [19:56] http://pastebin.ubuntu.com/15269548/ [19:57] ERROR invalid config: can't connect to the local LXD server: json: cannot unmarshal number into Go value of type string .... dude, UX, geez. [19:59] wait wiat wait... it's bootstrap ? really? [19:59] not ? So someday when we want to improve the UX, we can make controllername optional? [20:00] also, that's the opposite of deploy [20:00] rick_h__: ^^ [20:01] natefinch: reading backlog [20:01] natefinch: kill-controller is the new destroy-controller --force [20:01] cherylj: ahh [20:02] natefinch: right, it's controller-name cloud because you can do controller-name cloud/region [20:02] natefinch: or just leave off cloud I think if you have a default, plus the credential and such [20:02] rick_h__: why couldn't you do bootstrap cloud/region controllername? [20:02] natefinch: but controller-name is always required [20:04] rick_h__: will there ever be an option to just default to the same name as the cloud? Seems like a lot of people will only have one controller [20:04] natefinch: no, because we don't want that to happen [20:04] natefinch: it makes the UI confusing when the cloud name and controller name are the same [20:04] natefinch: it's why we're making the default model "admin" [20:05] natefinch: because seeing the controller name the same as a model name is confusing [20:05] natefinch: we want to shoot for more meaningful names vs shallow ones like that [20:05] rick_h__: I'm pretty sure everyone who has ever used juju before is going to be doing juju bootstrap amazon amazon [20:05] (or whatever) [20:05] natefinch: we'll see, but I'd like to discourage that. [20:05] if you only have one, the name is meaningly [20:05] meaningless [20:06] natefinch: it's not, the cloud name is meaningless :) You never interact with it other than bootstrap [20:06] I still think it's unfortunate that the order is backwards from deploy [20:06] natefinch: the controller one is very meaningful as it's the thing you talk to all day [20:06] natefinch: understand, but it's back to the optional arg is at the end [20:06] rick_h__: I hope never to have to type either except during bootstrap [20:07] natefinch: well one turns into two turns into ... if we do this well at all [20:07] natefinch: so I do hope you end up typing it as you move between controllers in different areas with different purposes [20:07] that means we're successful in making juju essential to your use of things out there [20:09] rick_h__: you say cloud is optional... what does it pick if I don't specify a cloud? How can I see what the default is? [20:09] natefinch: see PM [20:10] natefinch: there's work we need to do that if you only have one credential setup that it just uses that [20:10] natefinch: not setup that way currently [20:10] natefinch: as you say, no sense asking you to tell me what cloud if I only know credentials for one [20:11] yeah [20:11] natefinch: I completely see your point and there are times it feels backwards to me as well [20:11] but we did think through a bit farther out and so have $reasons which may or may not turn out to be ideal [20:12] really, the mismatch with deploy is my biggest concern. I just know I'm going to get the args backwards all the time [20:13] rick_h__: did you just give reasons in php, not cool mate [20:13] natefinch: understand, thanks for the feedback [20:13] perrito666: oh come on...maybe it was...crap [20:16] ahh crap.... were there fixes landed in master to work with the new LXD? [20:16] natefinch: yes [20:16] natefinch: there's a bug folks are hitting that's getting chased down, but there's a work around in the bug I believe [20:17] rick_h__: I had reverted my version of LXD from the ppa to wily stable when they broke the ppa... nwo I guess they broke wily stable :/ [20:17] natefinch: yea, wily won't work because it's got to work with lxd betas [20:18] anyone have the name of the ppa handy? [20:19] natefinch: https://launchpad.net/~ubuntu-lxc/+archive/ubuntu/lxd-stable [20:19] rick_h__: yeah, just found that with google, was going to ask if stable was the right one. Cool. [20:22] rick_h__: are you going to grace the onyx standup? [20:23] cherylj: I can, linky? [20:23] rick_h__: n/m everyone left :) [20:23]