[00:02] <cholcombe> ok. looks like you can't specify the series in the config.yaml.  It has to be on the cmd line
[00:04] <Budgie^Smore> lazyPower so I am running into a problem with my infra-in-a-box maas environment right now hence why I haven't come back to the issues with the k8s cluster :-/
[00:05] <Budgie^Smore> I am thinking of switching out the virt tech I am using for it
[00:50] <lutostag> Cynerva: any charm out there using a debug-layer currently?
[00:50] <lutostag> (one I can easily test-deploy to see if new feature in crashdump works with it)?
[00:51] <lazyPower> lutostag: kubernetes-master/worker/etcd
[00:51] <lazyPower> however etcd may not have landed yet.... i'd need to check the charm store
[00:55] <lutostag> lazyPower: that'll work, thanks
[03:12] <KeyboardSquid> Anyone online? Looking for some help troubleshooting a bootstrap issue
[03:14] <KeyboardSquid> Juju Version: 2.0.2-xenial-amd64
[03:14] <KeyboardSquid> When running "juju bootstrap lxd lxd-juju" I get the following error:
[03:14] <KeyboardSquid> error: cannot load ssh client keys: open /home/keyboardsquid/.local/share/juju/ssh: permission denied
[03:17] <stormmore> KeyboardSquid, what's the output from ls -la /home/keyboardsquid/.local/share/juju/ssh ?
[03:17] <KeyboardSquid> "Cannot open directory /home/keyboardsquid/.local/share/juju/ssh"
[03:18] <KeyboardSquid> looks like the owner is listed as User: Root, Group:root
[03:19] <KeyboardSquid> looking online, this seems to be an issue in earlier versions of juju, but was supposed to be fixed after version 1.8.0
[03:20] <KeyboardSquid> I am connected to this machine via SSH if that matters
[03:23] <stormmore> you can do a "chown -R keyboardsquid. /home/keyboardsquid/.local/
[03:23] <stormmore> you can do a "chown -R keyboardsquid. /home/keyboardsquid/.local/"
[03:23] <stormmore> that should fix the permissions for all the folders in the directory and should fix your problem
[03:24] <KeyboardSquid> ok, i can try that. Is there any reason .local would be owned by root in the first place?
[03:24] <stormmore> no
[03:24] <stormmore> nothing in your home dir should really be owned by root
[03:24] <stormmore> root can walk in regardless anyway
[03:25] <KeyboardSquid> Boom, looks like im good to go. Thanks!
[03:25] <stormmore> no worries :)
[08:14] <kjackal> Good morning Juju world!
[08:52] <eeemil> How does Juju decide which subnet to use within a specific space? I'm deploying to MaaS where each machine has 2 NIC:s, one NIC is exposed to Internet and one NIC is for internal communication. I want to deploy openstack-base. If I have 2 separate spaces (one external, one internal), Juju seems to become confused. If I have 1 space with both external and internal subnet, I can't communicate with some units as
[08:52] <eeemil>  they get internal IP:s listed as public...
[09:59] <cnf> meuwning
[10:12] <cnf> lazyPower: so it seems socks support missing isn't a bug ^^;
[10:55] <anrah> Has anyone been able to bootstrap controller with 2.1 using OpenStack as cloud provider?
[11:04] <Zic> hello, is NGINX Ingress Controller deployed by CDK support this https://github.com/nginxinc/kubernetes-ingress/tree/master/examples/customization#using-annotations ?
[11:09] <cnf> hmm, wow
[11:09] <cnf> can you bootstrap a juju controller manually?
[11:10] <cnf> because if you use maas, it claims an entire machine for this
[11:33] <cnf> hmm
[11:44] <zeestrat> cnf: regarding juju controllers with maas. to reduce footprint on smaller deployments, we've used kvm's on maas controllers to colocate juju controllers
[11:45] <cnf> hmm
[11:45] <cnf> and you set the power mode to manual?
[11:46] <zeestrat> we use the Virsh power mode. see https://docs.ubuntu.com/maas/2.1/en/installconfig-add-nodes#kvm-guest-nodes
[11:46] <cnf> hmm, interesting
[11:46] <cnf> thanks
[11:47] <zeestrat> no problem
[11:50] <cnf> can you use a local socket as the power address?
[11:53] <cnf> hmm, maas GUI really doesn't give a lot of feedback when it doesn't like some entry :P
[11:53] <zeestrat> cnf: not sure, I've only used qemu+ssh with an IP.
[11:53] <cnf> zeestrat: ssh to the localhost?
[11:54] <zeestrat> yeah, can do that too I imagine.
[11:55] <zeestrat> the #maas guys can probably help you more if you get into the nitty gritty :)
[11:56] <cnf> yeah, i'll ask there :P
[11:56] <cnf> i haven't used kvm in ages, either
[12:31] <cnf> zeestrat: how did you create the kvm guest node? and make it pxe boot?
[12:52] <Zic> -> to anwser my question of earlier, the nginx-ingress-controller used by CDK does not support Annotations like the one from NGINX Inc., but it's not important, I used this howto and it works: https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx/examples/custom-configuration
[12:53] <Zic> TL;DR, just use a ConfigMap instead of Annotations
[13:38] <cnf> hmm, i always get stuck on "Fetching Juju GUI 2.4.2"
[13:43] <cnf> hmz
[13:43] <cnf> where is that gui downloaded? is that a remote command?
[13:43] <cnf> how do i tell it to use a remote proxy?
[13:50] <cnf> does juju expect direct ssh access to hosts it creates?
[13:52] <marcoceppi> cnf: you can set the proxy values when bootstrapping, so it can use that during bootstrap
[13:52] <marcoceppi> cnf: ssh access is needed during the bootstrap process, at the moment
[13:52] <cnf> bah
[13:52] <cnf> that's just not feasable here
[13:53] <cnf> why can't is just respect my ssh config
[13:53] <cnf> it*
[13:53] <marcoceppi> cnf: not sure about the ssh portion - as in if it does use a config or not
[13:54] <marcoceppi> Also, I'm not sure why we even need SSH on first launch
[13:54] <cnf> i am about 4 hops separated from direct ssh access
[13:54] <marcoceppi> cnf: what provider are you using?
[13:54] <cnf> MAAS
[13:54] <marcoceppi> let me check
[13:54] <cnf> my main goal is to install and manage openstack
[13:54] <marcoceppi> sounds reasonable
[13:55] <cnf> it's been a rocky road so far...
[14:01] <marcoceppi> cnf: lets see if I can help curb that
[14:01] <cnf> that would be nice :P
[14:05] <cnf> marcoceppi: btw, https://bugs.launchpad.net/juju/+bug/1668727 is also mine
[14:05] <mup> Bug #1668727: juju does not support socks5 as a proxy <juju:Triaged> <https://launchpad.net/bugs/1668727>
[14:06] <marcoceppi> cnf: There's a Juju Show later today, I'll make sure to bring it up
[14:06] <cnf> oh, nice
[14:13] <cnf> marcoceppi: if you need ssh, it'd be nice if i could specify an ssh wrapper, or at least get my ssh_config used
[14:14] <marcoceppi> cnf: yeah, I think so
[14:14] <disposable2> have 4 machines ready in MAAS. i want to control it by juju BUT i don't want to sacrifice 1 of the 4 machines to be the controller. is it possible to have the controller in lxd?
[14:15] <cnf> i'm going to take a break, and think on how i can bypass this temporary
[14:15] <disposable2> s/control it/control maas
[14:16] <cnf> disposable2: i'm putting the controller in kvm on the maas machine
[14:16] <cnf> well, i;m trying to, juju isn't cooperating much atm :P
[14:16] <cnf> disposable2: maas can work with KVM just fine
[14:17] <cnf> disposable2: https://docs.ubuntu.com/maas/2.1/en/installconfig-add-nodes#kvm-guest-nodes
[14:17] <disposable2> cnf: thanks, but the question is somewhat different. i want to know if the controller HAS to be in the same 'cloud' as the cloud it is controlling
[14:18] <cnf> it has to run somewhere
[14:18] <cnf> if you don't want it on the metal, AND you don't want it on the MAAS controller
[14:18] <cnf> where would you put it?
[14:18] <disposable2> cnf: into lxd 'cloud'
[14:18] <cnf> lxd is just a local instance
[14:19] <cnf> on your laptop
[14:19] <cnf> or on something running
[14:19] <disposable2> i'm not going to sacrifice a 60K server with 1TB ram to be a controller.
[14:19] <marcoceppi> disposable2: the controller has to be reachable by other machines
[14:19] <cnf> the only thing running is the maas controller
[14:19] <cnf> disposable2: so, as i said, run it in KVM on the MAAS controller
[14:19] <cnf> disposable2: you said you have a MAAS working, run it on the same machine the MAAS controller is on
[14:20] <marcoceppi> ^ +1
[14:20] <marcoceppi> I run three KVM on my 5 node maas cluster to just get more density
[14:20] <disposable2> marcoceppi: so if my lxd machines are using a bridged interface on a common network, then bootstrapping my maas-controller in lxd will work?
[14:21] <cnf> disposable2: you have nothing to run lxd on, atm
[14:22] <disposable2> marcoceppi: my main problem is with the extremely superficial juju documentation."man juju" doesn't clearly explain what i can or cannot do with 'juju bootstrap'
[14:22] <marcoceppi> disposable2: it's possible, but if you bootstrap lxd juju will expect lxd as your model, let me chekc if you can bootstrap one cloud and point at another
[14:22] <disposable2> marcoceppi, cnf: i guess, i can add a cheap node to maas and run kvm on it if that's the only solution.
[14:23] <cnf> disposable2: ...
[14:23] <cnf> disposable2: use the MAAS server...
[14:23] <cnf> you already HAVE that
[14:23] <cnf> disposable2: what is the maas controller running on, atm?
[14:26] <disposable2> cnf: i understand what you're trying to say, my question was meant to be a generic one - i.e. "can the controller of 1 cloud backend be running on another cloud backend?" (if networking isn't an issue)
[14:27] <marcoceppi> disposable2: I know you can do things like, regions across a controller cloud
[14:27] <marcoceppi> disposable2: not sure about cross controller, trying now
[14:28] <rick_h> marcoceppi: no, can't do that atm
[14:28] <marcoceppi> ah :(
[14:29] <rick_h> marcoceppi: too much chance for things to go boom with networking and dupe 10.XXX addresses and such
[14:29] <rick_h> you can do manual machines if you want to
[14:29] <rick_h> e.g. add-machine ssh... and point at a machine on another cloud but it's kind of a mess
[14:30] <marcoceppi> rick_h: right.
[14:33] <disposable2> rick_h, marcoceppi: thanks
[15:23] <aisrael> rick_h, how is manual provider a mess right now? iow, what should we be keeping an eye out for?
[15:24] <rick_h> aisrael: I just mean that mixing a manual machine into an AWS model
[15:24] <rick_h> aisrael: e.g. it's not as nice as "juju add-unit"
[15:24] <aisrael> rick_h, Gotcha. It's "complicated". :D
[15:24] <rick_h> aisrael: since to get a machine from another cloud you'd need to get that instance, setup SSH, and then add-machine your new GCE instance into your AWS model so that you can cross cloud bits
[15:25] <rick_h> aisrael: yea, it's a messy UX I guess. Sorry for the confusion there.
[15:51] <brandor5> Hello everyone: I've tried asking my question in #openstack-charmers and #openstack but haven't gotten any replies, so I'm going to try here... I'm trying to set up an openstack POC using maas/juju and I'm hitting a snag when trying to use vlan networks (for floating and provider networks)... flat networks work fine but when using a vlan network for a floating external pool, i don't see traffic jumping from the router namespace out to
[16:00] <cnf> hmm
[16:00] <cnf> marcoceppi: so any suggestions on how to get going for now?
[16:15] <dakj> Hello guys, I need help. I try to deploy Landscape Dense-Maas on Ubuntu 14.04Lts but I've received that error on haproxy: "Status: error - hook failed: "config-changed"
[16:15] <dakj> IP address: none
[16:15] <dakj> Public address: none". Someone can help me, please?
[16:39] <hackedbellini> Hey guys! I have a setup here using lxd. I'm trying to make some changes to cloud-init's user-data (more specifically, add a dns search option in it), but it doesn't seem that the it is being propagated
[16:40] <hackedbellini> I'm doing the changes like this: lxc config edit juju-449b90-6
[16:40] <hackedbellini> after that, I try to restart the lxc (I even tried restarting the host itself), but it is still using the old config
[16:41] <hackedbellini> when I look at /var/lib/cloud/instance/user-data.txt, it is still in the old version too
[16:41] <hackedbellini> am I missing something here?
[16:42] <jrwren> i do not think you can change user-data once a system is booted. its a one time thing AFAIK.
[16:42] <jrwren> lxc is behaving like other clouds in this regard. AFAIK you can't change it on AWS, Azure or GCE either.
[16:43] <hackedbellini> jrwren: hrm, so is there any other way of doing that?
[16:43] <hackedbellini> I really need to make those changes
[16:44] <jrwren> hackedbellini: there is some way to copy cloud-config to a host and make cloud-init evaluate and execute its modules based on it. I don't recall how, but IIRC it is on stack overflow.
[16:45] <hackedbellini> jrwren: ok, I'll check it. Do you have any idea of what kind of keywords I should use for the search? I just don't know how exactly to describe the problem to search it on google =P
[16:46] <jrwren> hackedbellini: run cloud-init with cloud-config after boot?
[16:47] <jrwren> hackedbellini: http://stackoverflow.com/questions/6475374/how-do-i-make-cloud-init-startup-scripts-run-every-time-my-ec2-instance-boots#10455027 :)
[16:47] <hackedbellini> jrwren: oh thanks! I'll take a look
[16:58] <hackedbellini> jrwren: I don't know if those will help me. Let me ask you a question and maybe the answer to it will point me in the right direction:
[16:59] <hackedbellini> the config from /etc/hosts are generated by cloud-init. They are based on /etc/cloud/templates/hosts.tmpl. When I look at that template, I see that cloud-init will use the fqdn/hostname from its configuration. In this case, how can I change the hostname?
[17:00] <hackedbellini> since my changes are not propagated, what would be the way to change it? Changing it on /var/lib/cloud/instance/user-data.txt would do the trick?
[17:01] <jrwren> yes, I'd overwrite that file.
[17:01] <hackedbellini> jrwren: ok, I did. So, how can I "trigger" the update of /etc/hosts now? I updated that file and restarted the lxc, but nothing
[17:02] <hackedbellini> because if I know how I can trigger that I probably can trigger the dns changes if I make them on that file too
[17:03] <dakj> Anyone can help me?
[17:05] <lazyPower> dakj: what substrate and juju version are you using?
[17:05] <lazyPower> *cloud
[17:05] <dakj> Lazy power: 2.0
[17:06] <lazyPower> dakj: and what cloud?
[17:06] <dakj> lazypower: MAAS
[17:06] <lazyPower> dakj: can you pastebin the logs from haproxy?  juju ssh haproxy/0 && pastebinit /var/log/juju/unit-haproxy-0.log
[17:07] <dakj> I've opened a post here (http://askubuntu.com/questions/886533/deploy-landscape-status-error-hook-failed-install-ip-address-none-public)
[17:08] <lazyPower> dakj: ah actually your debug-log --replay command would work as well
[17:09] <dakj> lazypower: on post I had a problem with postgresql, but re-make everything the same issue is on haproxy
[17:09] <lazyPower> dakj: is it possible the vm is configured without proper networking?
[17:10] <dakj> lazypower: look that http://askubuntu.com/questions/881208/deploy-landscape-gui-via-juju-gui-on-ubuntu-14-04lts-server
[17:10] <hackedbellini> jrwren: btw, if I change /var/lib/cloud/instance/cloud-config.txt or /var/lib/cloud/instance/user-data.txt, when I restart the container, it gets reverted to the previous version...
[17:10] <dakj> Lazy power: all node are configured well on MAAS and their status is correct
[17:13] <jrwren> hackedbellini: right. its not really designed for this. you are off the rails. Maybe better to find a different way to acomplish what you need?
[17:14] <hackedbellini> jrwren: ok, but now I'm even more lost hahaha
[17:15] <jrwren> hackedbellini: when you first asked your question, I didn't realize you were using juju, I thought this was lxcontainers irc channel, so I gave a very poor response for a juju context.
[17:15] <jrwren> hackedbellini: I thought you were using lxc/lxd directly.
[17:17] <hackedbellini> jrwren: oh, probably my fault, I think I wasn't very clear about that. So, since I'm using juju, that makes what I want to do easier or harder?
[17:19] <jrwren> hackedbellini: neither, it just makes it different, or possibly unsupported.
[17:20] <jrwren> hackedbellini: I think the juju-way would be to use a juju charm to do whatever custom server config you desire.
[17:23] <hackedbellini> jrwren: hrm I see. Just to be sure, one last question then: What I'm doing right now is hardcode changing the templates in /etc/cloud/templates. I need some changes on /etc/resolv.conf and /etc/hosts. Are you sure there's no way to change those variables anywhere easy?
[17:26] <jrwren> hackedbellini: AFAIK there is no juju easy way. Depending on the behavior of your models, you could do things like use `juju run` to keep them up to date.
[17:27] <jrwren> hackedbellini: I think resolvconf may be what is overwriting /etc/resolv.conf editing an /etc/resolvconf/ template may be enough to persist that. the /etc/hosts file, I'm surprised is overwritten.
[17:28] <hackedbellini> jrwren: I see. Thanks anyway =P. And yeah, it is on every boot
[17:28] <jrwren> hackedbellini: to what end are these changes important? does this effect the workload of those units and the charms which they run?
[17:32] <hackedbellini> jrwren: kind. The hosts one is because the "localdomain" domain that juju used was being rejected by out mail server. It was expecting a certain domain or no domain, and because of that I needed to change it. The dns one is because we have a local dns server here and we need to use it for search or else some of our services will not find some hostnames using
[17:32] <hackedbellini> some local domains
[17:36] <jrwren> hackedbellini: is there a reason DHCP isn't pointing you to local DNS by default?
[17:40] <dakj> Lazy power: sorry a lost the connection
[17:41] <dakj> lazyPower: any sugget
[17:41] <dakj> Suggest?
[17:42] <hackedbellini> jrwren: It is pointing to the host machine, but the dns server is running on another one
[17:43] <jrwren> hackedbellini: maybe the host machine is misconfigured then? its DNS upstream could be that important other one.
[17:44] <hackedbellini> jrwren: hrm, probably. Is it something I can configure on /etc/default/lxc-bridge?
[17:44] <hackedbellini> this is my current content:
[17:44] <hackedbellini> https://www.irccloud.com/pastebin/1Z1Oovem/
[17:46] <jrwren> hackedbellini: I'm not sure. does `dpkg-reconfigure -p medium lxd` ask about DNS at all?
[17:47] <hackedbellini> jrwren: no, but there's a "Path to an extra dnsmasq configuration file" in that file. Maybe it is for that?
[17:48] <jrwren> hackedbellini: sounds great, yes. then find the dnsmasq config to point to upstream DNS
[17:52] <hackedbellini> jrwren: lets see if that will do the trick
[18:01] <lazyPower> dakj: not sure, sorry. Sounds like there's an issue with one of the units config in maas though. if its consistent but changes application on each deployment
[18:01] <lazyPower> dakj: i suspect you need to assign the networks the unit is part of
[18:01] <lazyPower> eg: edit the unit in question "precipitating-petunia.maas" for example, and check those interfaces, then assign each interface to a fabric.
[18:11] <bdx> brandor5: hey, whats up
[18:11] <brandor5> bdx: hello :)
[18:11] <bdx> brandor5: I don't think vlan type networks are supported as external provider network type in openstack
[18:12] <brandor5> bdx: that's weird, I'm pretty sure that in a hand rolled (not maas/juju) install I had them working without issue :/
[18:12] <bdx> brandor5: I have tried to get that working a while back (mitaka), as I couldn't find any info on it either, and had no luck
[18:13] <bdx> brandor5: when you create an "external" network in openstack .... you can't also set it as type vlan
[18:13] <bdx> the api commands just won't take ....
[18:14] <bdx> brandor5: please let us know, if you get this working, as I think many in the community have difficulty here
[18:14] <bdx> brandor5: I'll give it a shot again if  have time in the next day or so
[18:14] <brandor5> bdx: I'm almost 100% that I've done it in the past when not using the charms
[18:15] <bdx> brandor5: the "charms" have nothing to do with the way openstack networking works once it is setup really
[18:16] <bdx> brandor5: the charms let you specify at a higher level what traffic should go over what interface, so that when openstack is deployed you have the capability to jump right in and start using it
[18:16] <brandor5> bdx: when you say 'external' you mean network that provides floating ips?
[18:17] <bdx> brandor5: yes, in the context of an traditional openstack "external" network
[18:17] <brandor5> bdx: ok, that's fair enough. I can't remember if I had an external network using vlans... but I did definitely have 'provider' networks using vlans... is that supported with the charms?
[18:20] <bdx> brandor5: yes, out of the box, the openstack charms support vxlan,gre,vlan, and flat network types using neutron-openvswitch/neutron-gateway
[18:20] <brandor5> bdx: maybe that's what is causing some of my confusion? I'm not able to get provider vlan networks functioning either
[18:21] <bdx> brandor5: are you specifying that network type when you create your networks via openstack api?
[18:21] <brandor5> bdx: yes
[18:21] <bdx> brandor5: can I see that command?
[18:22] <brandor5> neutron net-create --provider:network_type vlan --provider:segmentation_id 4002 --provider:physical_network physnet1 --router:external dtint
[18:25] <hackedbellini> jrwren: it worked! Btw, in the process, when I was trying to rerun my cloud-init config, I tried something that made my juju agent get lost. The container is working, I can even "juju ssh" to it, bu on "juju status" while others show as "started" it shows as "down". Is there something I can do about that?
[18:25] <bdx> brandor5: what is the result of that command?
[18:25] <brandor5> bdx: it creates a network
[18:25] <jrwren> hackedbellini: restart the juju machine agent. there should be a juju service for it.
[18:25] <bdx> oh ... interesting
[18:26] <bdx> brando5: have you ensured all of your physical routing up to the interface is correct, e.g. the vlan is getting there?
[18:26] <hackedbellini> jrwren: probably it is worse than that. I restarted the server hosting the lxcs
[18:26] <hackedbellini> jrwren: Let me tell you what I did
[18:26] <bdx> brandor5: the interface to which physnet1 is assignedb
[18:27] <brandor5> bdx: yes, i have... tcpdump at multiple locations
[18:27] <bdx> ok
[18:27] <bdx> brandor5: so, what is not working?
[18:28] <brandor5> bdx: I would except that when I spin up an instance that's attached to that network that I would be able to see traffic and communicate with out real-world systems on it
[18:28] <brandor5> expect, rather
[18:28] <rick_h> REMINDER juju show coming in 33 minutes!!! Some big stuff coming!
[18:30] <rick_h> hatch: bac arosales marcoceppi lazyPower mbruzek kwmonroe and anyone else reminder ^
[18:30] <hackedbellini> jrwren: I was trying to find the answer on stackoverflow that I followed. Buyt well, I removed the "/var/lib/cloud/instances/juju-449b90-6/" directory and run "cloud-init init" again. After that the problem started. Like I said, the container is working fine, but juju is not able to connect to its agent
[18:30] <bdx> brandor5: you spin up instances in an internal network which uses that same router as its gateway right?
[18:30] <hatch> oh I'll be there :)
[18:30] <bdx> brandor5: I'm not aware that the dhcp mechanism is running on the external network by default
[18:31] <jrwren> hackedbellini: that... might be very bad. rahter than try to recover can you remove that juju unit and add a new one for that application?
[18:31] <brandor5> bdx: sorry, I've sent you the wrong command above... that's from when I was trying to set up a vlan external network
[18:32] <hackedbellini> jrwren: and one even stranger thing. I was trying to do a restart on the service but it doesn't seem to exist. There's only a "juju-clean-shutdown" service, it should have 2 more for the machine and the unit
[18:32] <bdx> which should be ok though ... as there is nothing stating that creating vlan external provider networks isn't a thing
[18:32] <brandor5> bdx: now I'm confused
[18:32] <hackedbellini> jrwren: as a last resort I can do that, but I would rather not have to. The service is easy to deploy, but the data migration would take me some time to do
[18:33] <jrwren> hackedbellini: right, and its the juju defined cloud-config which creates those, but if you someone removed that... you are really off the rails :)
[18:33] <bdx> brandor5: I haven't been able to do it, but I'm finding docs that say its a thing
[18:33] <jrwren> hackedbellini: ah, the data is local to the unit? ugh.
[18:34] <hackedbellini> jrwren: yeah, unfortunatly. Its a jenkins unit btw, lots of configurations for lots of jobs
[18:34] <brandor5> bdx: you just told me that i can't create an external network that is of vlan type
[18:34] <hackedbellini> jrwren: can I copy some file from other container and maybe make that work? hahaha
[18:34] <brandor5> bdx: sorry for the delay there, wanted to scroll back up and make sure I'm not really losing it... :D
[18:36] <bdx> ha, brandor, I tried very hard to get that working .... it very well may be possible, but its an area that entirely stumped me .... its possible that it wasn't supported in kilo/mitaka and now it is though
[18:36] <bdx> its an amorphous beast
[18:37] <mimizone> hi all.
[18:38] <brandor5> bdx: like I said earlier, I'm almost 100% that I've at least done provider networks that way... and I'm also fairly confident that I had external networks set up like that too... and this would have been on kilo and liberty
[18:38] <arosales> rick_h: unfortunately I wont be able to make the live show today
[18:38] <mimizone> is there a way to give a hint to juju 2.1 on which IP to use for the node/machines? instead of trying to guess by connecting to all interfaces, and then be at the mercy of timing.
[18:38] <rick_h> arosales: booo :P ok, thanks for the heads up
[18:38] <hackedbellini> jrwren: you know what, I'll recreate that machine :)
[18:38] <rick_h> mimizone: on maas you can use endpoint bindings for this
[18:39] <rick_h> ming: what substrate are you looking at currently?
[18:39] <arosales> rick_h: looking forward to the reply
[18:40] <bdx> rick_h: when are you guys going to rev the beta controller?
[18:40] <mimizone> rick_h: not sure what your mean.
[18:40] <rick_h> bdx soon, there was an email yesterday about if we should go forward with 2.1 controllers. so I think it's getting ready to happen
[18:42] <rick_h> juju show hangout url: https://hangouts.google.com/call/w3ojiephyrbslk44jhq4wbfb2me and the streaming url: https://www.youtube.com/watch?v=RdWCtGk_4rU
[18:48] <bac> hey rick_h, is it almost time for the Juju Show?
[18:48] <hatch> 12m bac :)
[18:48] <rick_h> bac: yes, links above ^
[18:49] <bac> thanks
[18:53] <lazyPower> bac: in case it was missed: https://www.youtube.com/watch?v=RdWCtGk_4rU
[18:53] <rick_h> well that was rude google
[18:53] <rick_h> juju show hangout url: https://hangouts.google.com/call/w3ojiephyrbslk44jhq4wbfb2me and the streaming url: https://www.youtube.com/watch?v=RdWCtGk_4rU
[18:53] <rick_h> I got booted and had to relogin/etc again
[19:00] <mimizone> so I am not clear on how network bindings would help picking the right network interface/IP when adding machines to a juju controller.
[19:01] <rick_h> mimizone: sec, starting the show but can chat after
[19:02] <mimizone> oh ok sorry
[19:13] <zeestrat> rick_h: Any updates to the Juju charm store to show the bindings available now that they need to be set explicitly for containers in 2.1?
[19:13] <stormmore> o/ juju world
[19:13] <zeestrat> rick_h: bindings for each charm that is
[19:14] <hackedbellini> hey guys! Any reason why my mediawiki deployment is listed like this on juju status: mediawiki/1*                  blocked   idle   13       10.67.62.191    80/tcp              Database required
[19:14] <hackedbellini> it is saying that "Database required", but it is working fine
[19:16] <lazyPower> heyo stormmore. You're just in time for the juju show https://www.youtube.com/watch?v=RdWCtGk_4rU
[19:18] <stormmore> wonder if the "work across multiple clouds" will come to regular juju
[19:19] <mimizone> using juju 2.1, I see error as such in one of the machines, blocking the creation of lxd containers.
[19:19] <mimizone> container provisioner for lxd: setting up container dependencies on host machine: not found
[19:19] <mimizone> ERROR juju.worker runner.go:210 exited "1-container-watcher": worker "1-container-watcher" exited: setting up container dependencies on host machine: not found
[19:20] <lazyPower> Hooo look at that hot kubernetes action
[19:20] <mimizone> ERROR juju.provisioner container_initialisation.go:116 starting container provisioner for lxd: setting up container dependencies on host machine: not found
[19:20] <stormmore> lazyPower, now now, no need to get a big head :P
[19:21] <lazyPower> stormmore: i'm not entirely certain thats possible... my head is already pretty big
[19:22] <stormmore> lazyPower, ah but there is physical and egoistical size :P
[19:24] <mimizone> good stuff I see on the show by the way :)
[19:24] <stormmore> I wish I was working with juju and not maas right now :-/
[19:29] <lazyPower> stormmore: pack a 1/2 punch and do both
[19:29] <stormmore> lazyPower, I am but I am fighting an inconsistency in the maas installer :-/
[19:30] <lazyPower> doh
[19:34] <stormmore> lazyPower, yeah it is weird one where installing maas doesn't register the rack controller with the region controller intermittently
[19:37] <stormmore> hmm youtube / canonical disconnect?
[19:38] <rick_h> stormmore: ?
[19:39] <stormmore> I lost connection to the video and can't get it back
[19:44] <rick_h> kwmonroe: any link for your CI thing to note?
[19:44] <rick_h> kwmonroe: docs, CI charm, etc
[19:45] <kwmonroe> rick_h: the latest cwr charm has the workflow documented in the readme (CWR on Charm Source Pull Requests):  https://jujucharms.com/u/juju-solutions/cwr/
[19:46] <kwmonroe> rick_h: a better synopsis/blog is coming this afternoon, but it's not quite ready yet
[19:46] <rick_h> ty kwmonroe
[19:46] <rick_h> kwmonroe: k, will look out for that and update the readme later on. This is enough to get a first drive at
[19:46] <kwmonroe> cool
[19:47] <stormmore> e2e sounds really cool
[19:49] <stormmore> when is nagios going to die!
[19:50] <rick_h> stormmore: hah, one day
[19:50] <rick_h> mimizone: ping, so where were we?
[19:51] <mimizone> rick_h: thanks for coming back to me :)
[19:51] <mimizone> I have 2 issues.
[19:51] <rick_h> mimizone: so https://jujucharms.com/docs/2.1/network-spaces is the basics of it
[19:52] <rick_h> mimizone: basically, you can tell Juju to place an application on a machine that has access to different network spaces. Juju then makes sure that the machine that it's placed on has devices on the right spaces, and handles making sure the charm is given the right information about what network device to use for communication/etc
[19:53] <mimizone> thanks I 'll into the spaces stuff. FYI. I am deploying the OPNFV bundle called JOID. It creates the bundle itself with a bunch of spaces/bindings  already for the application.
[19:53] <mimizone> my question regarding that is more about the dns-name showing up in the juju-status I guess
[19:53] <rick_h> mimizone: ok, aisrael might be able to help with that bundle. He's got some experience with it more.
[19:53] <rick_h> mimizone: ok, so what's up with the dns-name?
[19:54] <mimizone> it picks one of the IP address of the machine (they have 7 IPs in my setup).
[19:54] <mimizone> I would like it to be predictable and pick only for instance what we call the admin network IP.
[19:55] <rick_h> mimizone: so I think there is to set the default binding
[19:55] <mimizone> right now for instance, some of the machines have a dns-name sitting on the 1G nic card, and others on the 10G ports.
[19:55] <rick_h> mimizone: so that the default interface used is the one on the admin network you're looking for
[19:56] <rick_h> mimizone: check out https://lists.ubuntu.com/archives/juju-dev/2017-February/006313.html
[19:57] <mimizone> rick_h: cool I read that.
[19:57] <mimizone> rick_h: can I bug you on the second question/issue?
[19:59] <rick_h> mimizone: what's up?
[20:00] <mimizone> it's regarding one of the machines not creating the lxd containers. I see error in machine-1.log about missing dependencies. but I don't see any errors in cloud-init-output.log about a bad installation
[20:01] <mimizone> rick_h: stuff lke this ERROR juju.provisioner container_initialisation.go:116 starting container provisioner for lxd: setting up container dependencies on host machine: not found
[20:02] <rick_h> mimizone: hmm, anything else in there?
[20:03] <rick_h> mimizone: the lxd stuff won't be on the clout-init as containers are setup on demand
[20:03] <rick_h> mimizone: so it's juju's job. Now, is this on xenial? lxd is there ootb on xenial and I wonder if this is some issue working backwards on trusty?
[20:03] <mimizone> yep xenial
[20:04] <mimizone> rick_h: lxd is there and running as a service
[20:05] <mimizone> rick_h: is there a way to retrigger a provisioning of the machine?
[20:07] <mimizone> a few more lines from the machine-1.log file in case it's useful
[20:07] <mimizone> 2017-03-01 20:06:26 DEBUG juju.provisioner container_initialisation.go:185 release lock "machine-lock" for container initialisation
[20:07] <mimizone> 2017-03-01 20:06:26 WARNING juju.provisioner container_initialisation.go:134 not stopping machine agent container watcher due to error: setting up container dependencies on host machine: not found
[20:07] <mimizone> 2017-03-01 20:06:26 ERROR juju.provisioner container_initialisation.go:116 starting container provisioner for lxd: setting up container dependencies on host machine: not found
[20:07] <mimizone> 2017-03-01 20:06:26 INFO juju.worker runner.go:262 stopped "1-container-watcher", err: worker "1-container-watcher" exited: setting up container dependencies on host machine: not found
[20:07] <mimizone> 2017-03-01 20:06:26 DEBUG juju.worker runner.go:190 "1-container-watcher" done: worker "1-container-watcher" exited: setting up container dependencies on host machine: not found
[20:07] <mimizone> 2017-03-01 20:06:26 ERROR juju.worker runner.go:210 exited "1-container-watcher": worker "1-container-watcher" exited: setting up container dependencies on host machine: not found
[20:09] <rick_h> mimizone: hmm, sorry not sure. Would have to file a bug and look into it. It works on other machines?
[20:09] <mimizone> rick_h: yes that the disturbing thing. same hardware, same config push by the same maas.
[20:10] <mimizone> what can juju do when a machine / app is in pending mode?
[20:11] <mimizone> can it be reinstalled  without destroying everything?
[20:11] <rick_h> mimizone: so you can remove-machine.
[20:11] <rick_h> mimizone: and you can --retry-provisioning
[20:11] <rick_h> sorry, juju retry-provisioning
[20:11] <rick_h> mimizone: check the output of juju status --format=yaml
[20:12] <rick_h> mimizone: for that machine
[20:12] <rick_h> mimizone: or juju show-machine X where X is the machine number the container is on
[20:13] <mimizone> machine is said to be running / deployed and all the containers are pending
[20:14] <mimizone> think I can force a remove-machine and retry-provisioning? seems I can't retry if the machine is not in an error state
[20:18] <mimizone> oh well, I force the remove-machine, but can't reprovision it then... :)
[20:29] <hackedbellini> Guys, I'm trying to build a custom livecd inside a lxc container (on my jenkins deployment). I'm having some problems with mounts though
[20:29] <hackedbellini> for example, when I try to do this: https://help.ubuntu.com/community/LiveCDCustomization#Prepare_and_chroot
[20:30] <hackedbellini> this happens:
[20:30] <hackedbellini> https://www.irccloud.com/pastebin/XtnkwFmL/
[20:30] <hackedbellini> I found that the major problem is that the bind mount to dev is not working as it should
[20:31] <hackedbellini> for example, I tried creating a /tmp/foobar and "mount --bind /dev /tmp/foobar". It works and I can see the content inside it, but they are all empty
[20:31] <hackedbellini> for example, if I try to "cat /tmp/foobar/urandom" I get nothing instead of the random content that I get from "/dev/urandom"
[20:32] <hackedbellini> am I missing something here?
[20:48] <hackedbellini> nevermind, the problem was that I had to use "mount --rbind" instead of "mount --bind"
[20:49] <lazyPower> arosales: I think i missed context? is this for charmbox?
[20:49] <arosales> lazyPower: yes
[20:49] <lazyPower> arosales: so long as its still published to pypi, we should have it https://github.com/juju-solutions/charmbox/blob/master/charmbox-setup.sh#L26
[20:49] <lazyPower> however matrix would be missing
[20:50] <arosales> ya I was thinking the same that matrix would be missing
[20:51] <lazyPower> arosales: do you happen to konw the official distribution pipeline for matrix? is it installable from pypi?
[20:52] <arosales> lazyPower: pip install
[20:52] <arosales> sorry may be a clone, and not pypi
[20:53] <lazyPower> arosales: thats fine, i'm installing charm-tools from MASTER in here as well, we can certainly add matrix as well
[20:53] <lazyPower> we're goign to keep bloating the base image size, but its an acceptable tradeoff
[20:54] <arosales> lazyPower: I think Matrix is a worthy bit add
[20:55] <arosales> lazyPower: I am going to confirm with cory_fu and tvansteenburgh on which bundletester release supports native 2.0 deploy and matrix
[20:55] <arosales> lazyPower: from there perhaps I could work with you to make sure the latest bundletester is in charm box and install matrix
[20:56] <tvansteenburgh> arosales: 0.11.0 supports native 2.0 deploy
[20:56] <lazyPower> arosales: sure
[20:57] <arosales> tvansteenburgh: awesome, cory_fu do you know if 0.11 supports matrix as well?
[20:57] <tvansteenburgh> arosales: matrix support was added in 0.10.0
[20:57] <tvansteenburgh> arosales: it does
[20:57] <arosales> ah ok
[20:57] <arosales> 0.11.0 has all the goodness then
[20:57] <arosales> tvansteenburgh: thanks
[20:57] <lazyPower> tvansteenburgh: i'm pretty sure thats whats in pypi right?
[20:58] <tvansteenburgh> lazyPower: yep
[20:58] <lazyPower> sorry split-attention with a customer meeting. i could go look myself...
[20:58] <lazyPower> thanks
[20:58] <arosales> lazyPower: so as long as you are pulling in bundletester >= 0.11 we should be good to add in Matrix
[20:58] <lazyPower> arosales: sounds like we just need to fetch matrix and we're in like flynn
[20:58] <arosales> lazyPower: agreed, current instructions are at https://github.com/juju-solutions/matrix#running-matrix
[20:59] <arosales> lazyPower: but cory_fu said he would be open to putting Matrix on pypi
[20:59] <lazyPower> tvansteenburgh: i see matrix is python3, should we move deployer to python3 as well?
[20:59] <arosales> deployer
[20:59] <arosales> ?
[20:59] <lazyPower> er
[20:59] <lazyPower> bundletester
[20:59] <arosales> deployer for 1.x/
[21:00] <lazyPower> sorry, split attention
[21:00] <arosales> ah bundletester
[21:00] <lazyPower> my mistake
[21:00] <arosales> no worries :-)
[21:01] <arosales> do we need bundletester at py 3?
[21:02] <arosales> or nice to have
[21:02] <lazyPower> thats my open ? - its just moving between pip install and pip3 install
[21:02] <lazyPower> simple on my end, just wanted confirmation
[21:02] <arosales> gotcha
[21:02] <arosales> cory_fu: and tvansteenburgh ^
[21:03] <arosales> cory_fu: petevg if you guys do get matrix on pypi could you ping the juju list?
[21:03] <arosales> I think for charmbox we can still clone and install from there for the time being
[21:03]  * arosales thinks
[21:03] <cory_fu> lazyPower: I don't think BT will work in py3 just yet, but it would be good to work toward that.
[21:03] <lazyPower> ok
[21:03] <lazyPower> i'll leave BT in py2 for now until otherwise notified
[21:04] <petevg> There's a bug blocking it -- it's noted in a docstring inside of bundletester.
[21:04] <petevg> arosales: packaging matrix is on my list of TODOs for the near future. Will ping the list when I release it.
[21:06] <lazyPower> kicking off a build locally to include matrix
[21:18] <catbus1> Hi, I am trying to juju bootstrap to openstack, when setting up simplestreams, juju metadata: unrecognized command. using juju 2.1.0
[21:19] <cholcombe> is there a way to set the lxd profile with juju?
[21:20] <jhobbs> catbus1: can you show your full command and output?
[21:21] <arosales> petevg: thanks!
[21:23] <catbus1> jhobbs: nm, apt-cache policy showed juju wasn't installed, but I could juju status fine. weird. I installed juju again, and don't get the error message anymore.
[21:29] <skayskay> hi, I'm seeing 'agent lost' messages, but juju show-status-log doesn't find any history
[21:30] <skayskay> and I've restarted jujud on my machines to no avail
[21:31] <jhobbs> should that be https://jujucharms.com/beta ?
[21:33] <hatch> jhobbs thx ;)
[21:47] <skayskay> I thought restarting jujud on machine 0 would bring back the agent
[21:48] <catbus1> $ juju bootstrap demo-openstack demo-openstack-controller --config tools-metadata-url=http://172.16.0.26:80/swift/v1 --config network=8b53cb07-fdb0-4fc9-9b02-6ea4ccad141e
[21:48] <catbus1> WARNING unknown config field "tools-metadata-url"
[21:48] <catbus1> Creating Juju controller "demo-openstack-controller" on demo-openstack/RegionOne
[21:48] <catbus1> ERROR failed to bootstrap model: no image metadata found
[21:49] <catbus1> jhobbs: ^^^ do you happen to know what might go wrong?
[22:09] <jhobbs> catbus1: use image-metadata-url rather than tools-metadata-url
[22:09] <jhobbs> also it's easier to put the metadata in a local directory than on a http server
[22:10] <catbus1> jhobbs: I tried both, image-metadata-url and --metadata-source <local file path>, now the error is no image metadata found
[22:11] <catbus1> oh wait
[22:11] <catbus1> I think I may get the keystone ip wrong in the cloud config
[22:12] <catbus1> no, I have it right.
[22:14] <jhobbs> whats the local path, can you provide the output of tree from it?
[22:20] <catbus1> jhobbs: I provided the wrong keystone ip in the simplestream index.json.
[22:25] <catbus1> jhobbs: I can confirm that it is the wrong keystone ip I put in both json files that caused the problem.
[22:25] <jhobbs> cool catbus1
[22:25] <jhobbs> glad you gat it sorted
[22:26] <catbus1> --debug is a good friend.
[22:26] <jhobbs> yeah it's absolutely necessary when using the openstack provider
[22:29] <kwmonroe> welp hatch, i'm never bootstrapping again :)
[22:44] <cholcombe> axw you around?