[00:53] thumper: SSHClient facade. Review pls. http://reviews.vapour.ws/r/4713/ [01:14] menn0: i have a couple of simple PRs https://github.com/juju/juju/pull/5252 https://github.com/juju/juju/pull/5241 [01:14] menn0: reviewboard doesn't seem to have picked them up [01:21] * redir is eod. See you tomorrow juju-dev [01:46] Bug #1575448 opened: trusty juju 1.25.5 HA availability issues

[02:00] mwhudson: looking [02:00] menn0: looks like tim merged them [02:00] mwhudson: ok cool. I was having lunch. [02:01] or well [02:01] menn0: https://github.com/juju/juju/pull/5252 hasn't merged, can you tell why? [02:01] oh still processing [02:01] mwhudson: yep, still merging [02:02] hey perrito666 - your window tests PR fixed this bug, right? bug 1571783 [02:02] Bug #1571783: Windows unit tests cannot setup under go 1.6

[02:03] * perrito666 checks [02:03] yes [02:03] cool, thanks === redir_ is now known as 16WAADNFF === 16WAADNFF is now known as redir [02:06] having trouble building juju(cb347bb7). Following the README.md results in: cannot find package "github.com/Azure/azure-sdk-for-go/Godeps/_workspace/src/github.com/Azure/go-autorest/autorest/mocks" [02:06] a123: did you run godeps first ? [02:07] cd $GOPATH/github.com/juju/juju && godeps -u dependencies.tsv [02:08] no I did not. The README.md only says to run 'make install-dependencies' [02:08] that should do the same thing [02:08] did that process work ? [02:08] it might be easier to run the command I suggested [02:08] and raise a bug if our install docs are out of date [02:09] hmm. godeps: command not found. I'm using: go version go1.6.1 linux/amd64 [02:09] I'm new to go, if that's not obvious. [02:10] please raise a bug that make install-dependencies does not install godeps, which is a dependency :) [02:11] for the momement [02:11] go get launchpad.net/godep [02:11] go get launchpad.net/godeps [02:11] you will need to install bzr, sorry [02:12] ha. yep. [02:12] that may have been while make isntall-deps failed [02:13] odd, I thought make install installed bzr [02:13] hmm. odd. bzr already installed. here's the output of get get launchpad.net/godeps [02:13] # cd .; bzr branch https://launchpad.net/godep /home/ubuntu/proj/gojuju/src/launchpad.net/godep bzr: ERROR: Not a branch: "https://launchpad.net/godep/". package launchpad.net/godep: exit status 3 [02:13] a123: its godeps [02:13] go get launchpad.net/godeps [02:13] yep. [02:13] with an s at the end [02:13] you are missing it [02:14] yes, sorry, i typed it incorrectly the first time [02:14] working.... [02:14] there is a related tool with a very similar name [02:16] godeps -u dependencies.tsv looks like it did its thing. [02:19] now running go install -v github.com/juju/juju/... and that looks like it's doing its thing. Thanks for the help. [02:20] no worries [02:21] BTW, anyone get juju bootstrap to work from behind a proxy? At home, no issues at all. Different story behind a proxy. [02:21] juju _should_ pickup the various http_proxy variables if they are defined in your shell [02:22] a123: 1.25, or 2.x? [02:22] 1.25 works for me [02:22] yes. It does look like it does that at first.... Yes, 1.25 worked for me too, now trying 2.0 [02:24] don't know if this is related to our proxy or not, but doing lxc remote list produced a table where images url looked like: [02:24] https://images.linuxcontainers.org [02:25] I could not download any images from there. but when I redefined that url to include the port(8443) the image download was successful. [02:28] if you've change something (i'm not sure what you've changed) then you'll need to be explicit [02:28] it sounds like the thing your downloading from expects 443 as the default [02:29] right. I ran: lxc remote set-url images https://images.linuxcontainers.org:8443 [02:29] this redefined the remote images url. I then ran: lxc image copy images:ubuntu/xenial/amd64 local: --alias ubuntu-xenial [02:30] w/out changing the URL, the lxc image copy command would not work behind our proxy [02:30] umm, juju 2.0 doesn't support lxc [02:30] only lxd [02:30] i hope this statement is helpful, not frustating [02:31] laughing... I thought lxd2.0 was built on top of lxc. Your point is taken though. [02:32] well it is [02:32] so your point is technically correct, which is the best kind of correctness [02:34] before changing the image url and running: juju bootstrap --config default-series=xenial lxd-test lxd --debug produced: [02:42] sorry guys. I'm seeing different debug output now than earlier. lots of connection refused to the 10.0.3.0/24 network. Does the no_proxy setting take CIDR? [02:44] that network is attached to lxdbr0 [02:47] What I should ask is, does juju understand CIDR if used in the no_proxy env var? ie. export no_proxy=10.0.3.0/24 [02:48] a123: no, no_proxy doesn't take a cidr [02:49] thanks. [02:49] i _think_ no_proxy is just a sort of match string [02:49] export no_proxy=10. [02:49] ^ not tested [02:50] just a comma seperated list of "domain extensions" [02:50] so, it's suffixes, not prefixes [02:51] valid: no_proxy=.com [02:51] no_proxy=.255 [02:51] I've found this to be application dependent in the past. [02:52] ah. nice. [02:52] not valid: no_proxy=10. no_proxy=10.* etc etc [02:52] we plug this stuff into wget in some places so are limited by what wget supports [02:52] oh. It wasn't clear to me if wildcards would work. [02:54] ok. wget is the driver. I think I know why my debug is different now. I used the --keep-broken flag when running the bootstrap in hopes of finding answers. How do you destroy the model? [02:54] juju kill-controller $controller [02:54] from memory [02:55] kill is the more finite form of destroy [02:55] which tends to not actually destroy things 'cos it's a wimp [02:58] so much I don't know. Is the controller a different concept than a model? It looks like a controller never got created when using the --keep-broken flag. [03:02] so. juju list-controllers returns an empty table. When I then run: juju list-models I get: error: controller local.lxd-test not found. What exactly does --keep-broken do? [03:08] thanks for the help everyone. I'm not confident my environment is in a good state. I'm going to tear down the VM, bring up a fresh one and try again tomorrow. [03:21] menn0: a few questions on your review [03:21] * thumper goes to make coffee [03:21] thumper: ok, looking [03:24] thumper: good point... these APIs copy the existing APIs used by juju ssh/scp exactly but it's probably worth making sure they at least work for IPv6 too [03:28] thumper: hangout to discuss? [03:31] menn0: sure, gimmie 5? [03:31] yep [03:32] * thumper wants to enjoy his coffee first [03:55] Bug #1575463 opened: buildSuite.TestGetVersion* CryptAcquireContext: Provider DLL failed to initialize

[04:09] menn0: are you looking at this bug https://bugs.launchpad.net/juju-core/+bug/1458585 [04:09] Bug #1458585: SSHGoCryptoCommandSuite.TestCommand fails

[04:09] or a duplicate of it ? [04:09] davecheney: no and no [04:11] davecheney: I'm working on making juju ssh/scp use the actual SSH host keys of the machine being connected to [04:11] davecheney: close to being done [04:14] ok, related, but not the same issue [04:14] thanks [04:21] wallyworld: when do we add things into the cloud metadata storage? [04:21] wallyworld: is it "normal" behaviour for clouds [04:21] or more used for custom openstack stuff? [04:21] thumper: you mean keystone? [04:21] or the metadata url [04:22] will still allow the use keystone for simplestreams [04:22] but i don't think we've used the metadtaa url for ages [04:22] we now bootstrap differently [04:23] it used to be a way to pass bootstrap instance info across [04:25] Bug #1575469 opened: liveSuite.TestBootstrapMultiple invalid character \"\\\\\" in host name

[04:25] Bug #1575472 opened: Data Race github.com/juju/juju/environs/tools/build.g

[04:32] wallyworld: I mean the state cloudmetadataC collection [04:34] thumper: we cache the simplestreams metadata there at bootstrap and whenever we poll cloud-images [04:42] k [04:59] menn0: https://github.com/juju/juju/pull/5289 [04:59] i heard u were on call review [06:09] davecheney: looking [06:11] davecheney: looks fine. what's changed in x/crypto that fixes the isssue? [06:19] menn0: who knows it's been a year since we updated that dependency [06:19] and there have been heaps of bug fixes to the crypto repo [06:19] i'm deliberately phrasing it like this because I don't want to backport anything [06:19] davecheney: ah right... so you didn't find a specific upstream change that fixed the issue [06:19] i didn't even look [06:19] upgrading to tip fixed the problem [06:19] davecheney: well you have a ship it [06:45] Do we need to maintain comptabilty with Go 1.2 ? [06:45] the build bot just failed to land a branch 'cos it tried to build with Go 1.2 [07:42] axw: are you around by any chance? [07:43] or wallyworld ? [07:43] hey, otp [07:43] dimitern: yo, I am [07:43] axw, wallyworld: hey, just a quick question, if you happen know [07:44] in provider/ec2 can we now only filter instances by model tags rather than secgroups ? [07:44] axw: I know you did something around that lately [07:44] dimitern: not yet, tried to land today but master is blocked [07:45] I'm working on bug 1321442 and it will be a lot easier not to have to make group filters work with explicit VPC ID [07:45] Bug #1321442: Juju does not support EC2 with no default VPC

[07:46] axw: I see, ok - I'll look into your changes [07:46] dimitern: hmm you know, I just remembered a reason why we may not want to do away with the group filtering. [07:46] dimitern: tags aren't added immediately after creating an instance... [07:46] axw: ah, bummer [07:46] sorry [07:47] dimitern: I mean, they're not added until after they're created [07:47] dimitern: so there's a window where they could leak [07:47] axw: np, it makes sense in an AWS world I guess [07:48] axw: ok, cheers, I'll do group filtering then as well [07:48] dimitern: and I'll revert that bit in my PR :) [07:48] cheers [07:51] you gotta love how inconsistent AWS API is at times :[ [07:53] lol `deleteSecurityGroupInsistently` [08:03] jam, tech board? [08:32] dimitern: ping [08:32] voidspace: pong [08:33] dimitern: I've addressed two of your comments from your review of the devices branch [08:33] dimitern: you suggest just adding a test for the multi nic case as it's basically done [08:33] dimitern: however, adding a test means some test infrastructure work as the multi-nic path calls additional provider methods [08:34] dimitern: not difficult but not nothing [08:34] voidspace: it looked like that, but yeah - can be a follow up [08:34] dimitern: in addition there is the known bug with the multi-nic case [08:34] dimitern: I'd rather land as is - with working and tested single nic containers [08:34] voidspace: what did you find about the missing subnet? [08:34] dimitern: not uncovered it yet - still digging [08:34] dimitern: should be done and dusted today though, can't be *that* hard to find :-) [08:35] dimitern: so I'll land as is and land multi-nic tests along with the bug fix (assuming it's not a maas bug) [08:35] dimitern: ok? [08:35] voidspace: you know what occurred to me yesterday: it might be due to unchecked 0 ids for fabrics or vlans [08:35] dimitern: possibly [08:36] dimitern: I'm going to try always setting VLAN ID even if it isn't changing in that update call [08:36] voidspace: e.g. fabric-0 is always there, but we might not have meant that [08:36] yeah [08:36] voidspace: ok, if you don't mind let me do a quick bootstrap with your branch now [08:37] dimitern: sure [08:37] I'm doing the same, with modified gomaasapi [08:39] ok [08:43] forcing the VLAN to be specified didn't help [08:44] adding more debugging [08:44] dimitern: anyway, I'm on it [08:45] voidspace: can you paste the outputs of 'maas subnets read', 'fabrics read', and 'machines read' for your maas2? [08:47] dimitern: http://pastebin.ubuntu.com/16076113/ [08:49] voidspace: thanks, here's mine for comparison: http://paste.ubuntu.com/16076129/ [08:50] dimitern: I'm going to try allocating the machines and making the maas calls manually [08:51] dimitern: after this bootstrap to confirm it's the CreateDevice call that fails [08:51] voidspace: sgtm, I'm bootstrapping now [08:51] dimitern: I'd still like to land single nic support [08:53] voidspace: I don't disagree with that :) [08:53] cool [08:54] voidspace: are both of your first 2 fabrics managed? (0 and 1) [08:55] i.e. subnets 172.16 and 172.17 [08:55] dimitern: gah, master blocked [08:55] dimitern: subnets are managed if that's what you mean [08:55] I don't know what a managed fabric is [08:56] both subnets [08:56] dimitern: ok, I was wrong it gets past CreateDevice and UpdateInterface [08:57] dimitern: so I need to add more instrumentation and redo [08:57] dimitern: I've got this anyway [08:57] dimitern: I'll ping you when I've made progress [08:57] voidspace: ok, your maas2 setup looks fine - so the issue must be related to how we're calling the api [08:58] yep [09:00] voidspace: still the same thing - no params (with vlan) passed to update the primary NIC of the device [09:02] dimitern: standup old boy [09:21] morning [09:34] babbageclunk: it would appear you're the luck OCR today, would you please take a look at http://reviews.vapour.ws/r/4718/diff/#? [09:34] fixes master blocker [09:39] dimitern: I see vlan=5001 in the params of the PUT [09:40] voidspace: and I can't :/ [09:41] voidspace: and with the vlan set do you still get 500 ? [09:42] dimitern: wait, that's the wrong vlan though [09:42] dimitern voidspace: can either of you take a look at the tiny PR above? it should unblock master [09:42] well, with the juju/juju change coming after [09:42] dimitern: I do get 500 [09:42] dimitern: but I think that's the wrong vlan [09:42] dimitern: interesting [09:42] axw: LGTM [09:42] ta [09:43] voidspace: aha! [09:43] voidspace: 5001 should be the untagged vlan of your fabric-0 [09:43] voidspace: and it should be the same as the host's first NIC's VLAN [09:44] dimitern: however this subnet comes straight from MAAS [09:44] but "spaces read" seems to give the right thing [09:46] dimitern: so in gomaasapi it should be args.Subnet.VLAN() not iface.VLAN() [09:47] dimitern: and in the single NIC case it works because there's only one [09:47] dimitern: trying that [09:47] voidspace: exactly! [09:48] I think that's it [09:48] trying it now [09:48] voidspace: the iface is the device interface, while args.Subnet.VLAN is the one we want on iface [09:48] yep [09:48] gaah, that took a long time [09:48] at least we get to blame Tim [09:48] :) [09:48] and a chunk of the time was getting a multi-nic MAAS setup which is worthwhile work [09:49] voidspace: \o. [09:49] \o/ even [09:50] voidspace: and it also helped your maas2 setup is a bit different than mine [09:50] yep [09:51] voidspace: are you using 2.0.0 beta4 more recent than bzr 4936 ? [09:52] dimitern: 4941 [09:52] dimitern: beta3 though! [09:53] dimitern: I've got past that point with no error [09:53] dimitern: and I see a device with two nics and two IP addresses [09:54] voidspace: hmm, well you *might* hit this bug 1572070 [09:54] Bug #1572070: MAAS 2.0 cannot link physical device interfaces to tagged vlans, breaking juju 2.0 multi-NIC containers [09:54] axw: Sure - sorry, missed this until now. [09:54] voidspace: sweet! then it should work the rest of the way [09:54] dimitern: yep, so I'll propose a fix for gomaasapi and then tests for multi-nic [09:55] frobware: babbageclunk: it turned out to be a bug in gomaasapi [09:55] voidspace: but that bug will be relevant with mult-nic only [09:55] dimitern: ok, thanks [09:55] dimitern: the linking is already done (my vlans are untagged) [09:55] dimitern: so I think I've got past that [09:55] voidspace: nice [09:57] voidspace: well since the physical nics of the host are on untagged VLANs it only will become an issue trying to create a second physical NIC of the device linked to a tagged VLAN [09:57] dimitern: container is running fine [09:57] voidspace: no WARNINGs in the log around provisioning / PrepareContainerInterfaceInfo ? [09:58] dimitern: only about no DNS settings found [09:58] dimitern: nothing else [09:58] axw: dimitern: d'oh, should've reloaded [09:58] dimitern: and I can ssh fine [09:58] dimitern: case closed on that one [09:59] voidspace: awesome! so the device has eth0 and eth1 linked to br-eth0 and br-eth1 on the host? [10:00] dimitern: this is e/n/i on the container http://pastebin.ubuntu.com/16076575/ [10:02] voidspace: slightly odd to see `dns-nameservers 172.16.0.2` on eth0 which has address 172.17.0.4/24 [10:02] dimitern: it adds the nameservers to the first stanza [10:02] but otherwise looks solid [10:02] yeah [10:02] voidspace: yeah - and only the .16 subnet has dns_servers set? [10:02] dimitern: this is the host http://pastebin.ubuntu.com/16076591/ [10:03] dimitern: you only need one nameserver entry [10:03] they're global [10:04] neither dns is set in maas (both 0.0.0.0) and both are managed [10:04] voidspace: I see, ok - does 'ping bbc.co.uk' work while inside the container? [10:05] dimitern: yep [10:06] voidspace: we're nearly done then :) [10:09] dimitern: can't test that change easily in gomaasapi - you can specify responses in the test server it uses but not verify requests it seems [10:10] you can fetch the last request, but it makes another after the update [10:10] looking to see if I can get at the one before last :-) [10:11] hah, no [10:11] ah, I can check the VLAN [10:11] should be ok [10:12] voidspace: sure, on it [10:18] dimitern: are you doing it? [10:18] dimitern: I have a branch with a fix [10:18] voidspace: haven't started [10:18] doing a few things at once as usual [10:19] dimitern: a proper test involves a lot of json (new interface, vlan and subnet in json) [10:19] dimitern: I have this though https://github.com/juju/gomaasapi/compare/master...voidspace:maas2-create-device-vlan [10:19] as there's only one interface on the test machine the test doesn't fail without the fix [10:20] I suggest we land this, I'll work on juju and come back for a multi-nic test for gomaasapi later [10:20] voidspace: looks good [10:20] babbageclunk: https://github.com/juju/gomaasapi/pull/47 [10:22] coffee [10:34] voidspace: I like it [10:34] voidspace: ! === akhavr1 is now known as akhavr === dimitern is now known as dimitern-afk [11:26] voidspace: ahh, missed your discussion about the test above. [11:48] babbageclunk: yeah, it needs a better test - but creating a machine with multiple interfaces, subnets and vlans for the gomaasapi test harness is a pain [11:48] babbageclunk: I'll do it later [11:48] babbageclunk: it's no worse tested than it was before ;-) === dimitern-afk is now known as dimitern [12:37] dimitern: I've updated http://reviews.vapour.ws/r/4700/ [12:37] dimitern: includes gomaasapi revision bump and a happy path test for multi-nic [12:37] is master unblocked yet [12:37] dimitern: still more tests needed [12:39] voidspace: still looks good to land, and I'll do another quick live test with it [12:39] dimitern: that would be good, see if it works for you [12:46] master is still blocked [13:08] voidspace, dimitern, frobware, babbageclunk please JFDI any maas2 related PRs [13:09] master is blocked but maas2 stuff is an exception here [13:36] alexisb: thanks! [13:37] voidspace: it looks a lot better: bootstrap ok, switch to admin, add-machine, then add-machine lxd, lxc, kvm to both :0 and :1 [13:38] voidspace: but only machine-1 containers all came up ok and have expected addresses, the ones on machine failed with host machine device "br-eno2" has no address, and I was digging into the logs to figure out why [13:56] Bug #1575676 opened: Hard to use non-default LXD bridge

[13:57] thank you katco ! [13:57] alexisb: np... happy that we're giving capacity planning a little more attention [13:57] alexisb: also revealing future targets :) [13:58] * katco begins her morning routine [14:17] alexisb: ok [14:18] dimitern: machine-0 ones didn't work? [14:18] picking up daughter from school [14:18] back in 15mins [14:20] voidspace: nope, something's odd, still investigating and adding more logging === deathpip is now known as ivyharriet [15:00] dimitern: alexisb: frobware: babbageclunk: container support has landed on master [15:01] voidspace: nice work! [15:01] voidspace: awesome! === redir_ is now known as redir [15:03] voidspace: great! [15:03] I'm still debugging the issue on machine-0 [15:03] dimitern: ok [15:03] dimitern: I'll try adding a node to my maas with two nics and try that [15:04] dimitern: I've *only* deployed to machine-0 [15:06] voidspace: in the admin model? [15:06] dimitern: yep [15:08] dimitern: commissioning an additional node now [15:09] \o/ [15:12] dimitern: when I commission a node with two nics the second nic comes up as "unconfigured" (no address) whereas the primary nic is "auto assign" [15:12] dimitern: I can manually change it to auto assign [15:12] dimitern: if I don't do that then I think I only get a single nic on the container [15:13] voidspace: otp [15:17] successfully created a container with two nics on a new machine not in the admin model [15:24] Bug #1574809 changed: "to: lxc:0" ignored in bundle