[01:12] <Teranet> question can you move in juju a container from 1 machine to the next via command line by chance ?
[01:13] <Budgie^Smore> so I am about to put a PoC using Juju onto AWS and am trying to figure out what the min instance size should be?
[02:18] <marcoceppi_> Budgie^Smore: t2.medium is pretty good for most things, depends on the owrkload
[02:19] <Budgie^Smore> well I was just going to put the juju controller on it and maybe a kubernete-master node
[02:20] <Budgie^Smore> it is probably only going to manage a few other slave nodes
[02:26] <marcoceppi_> Budgie^Smore: so, t2.medium should be okay for controller, m3.medium might be more stable
[02:27] <marcoceppi_> as for kubernetes-master / worker the master doesn't need too much if you're going to be doing a small number of worker nodes. The workers are really up to you depending on how many continaers you want to pack in per node
[02:30] <Budgie^Smore> yeah I am looking at doing c4.4xlarge or c4.8xlarge for the slaves
[02:31] <Budgie^Smore> I should be able to put the master on the same instance as the juju controller right/
[02:31] <Budgie^Smore> ?
[03:57] <Budgie^Smore> lol managed to blue screen the work laptop! new windows 10!
[06:53] <Budgie^Smore> is there a way for models to share machines?
[07:43] <Budgie^Smore> hey marcoceppi_ just came across your name on a GitHub "issue" in relation to useing AWS ELB as a substitue for kubeapi-load-balancer, defintely would get thumbs up from for that :)
[08:32] <kjackal> Good morning Juju world
[12:36] <fang64> I have been trying to deploy 10 nodes with Juju/MAAS and I've gotten to the point where I bootstrap juju onto MAAS as a cloud, but it doesn't indicate machine status when HA is enabled is this normal?
[12:38] <marcoceppi_> fang64: could you elaborate on machine status?
[12:39] <fang64> marcoceppi: when I type juju machines it indicates ha status is 1/3
[12:39] <fang64> this is after I've enabled ha, and the hosts are done being deployed.
[12:40] <fang64> marcoceppi: I mean controllers sorry, wrong command
[12:41] <fang64> it does show the controller has machines 3 but HA is yellow with 1/3
[12:42] <fang64> I assume it means high availability isn't working? because it's not indicating 3/3? maybe it's my lack of understanding
[12:44] <fang64> marcoceppi_: this is what I see, http://i.imgur.com/onIDdtL.png
[12:46] <rick_h> fang64: yes, it should work its way to 3/3 as HA kicks in and the db is replicated/etc
[12:46] <fang64> it's been a day
[12:46] <fang64> I don't think it takes that long for ha to enable?
[12:46] <rick_h> fang64: k, so something is up. You'll have to check the debug-log or at this point the logs on the machines
[12:47] <rick_h> fang64: no, definitely not
[12:47] <fang64> ok, so it's broken
[12:47] <fang64> I just wanted to make sure I wasn't misunderstanding,.
[12:47] <rick_h> no, definitely not. That's why it shows yellow in that you've asked for 3 controller nodes in HA, but only have one functioning
[12:48] <fang64> alright well I'll take a look, really I am trying to get to a point where I can deploy openstack, but I kept running into issues with bootstrap I suspect I have some network issues.
[12:48] <rick_h> fang64: oh, ok.
[12:50] <fang64> I don't know if anyone can answer this question, if I am deploying to MAAS as I am now, is juju using lxc containers or is it just installing the charms to physical hosts?
[12:50] <rick_h> fang64: so it depends on how you're installing
[12:51] <fang64> I just created maas as a cloud
[12:51] <rick_h> fang64: the bundles that the openstack folks use use lxd containers on the physical hosts to help spread things out
[12:51] <fang64> in juju and then told it to bootstrap
[12:51] <rick_h> fang64: so nothing there is lxd centric
[12:51] <fang64> ah ok
[12:51] <rick_h> fang64: you'll see in the bundle things getting told to go to 0:lxd or the like
[12:52] <rick_h> fang64: to help colocate the openstack services on deploy, but until you do that nothing is lxd unless told to be
[12:52] <fang64> ok, so in my case with juju bootstrap as it stands now it's provisioning the hosts as the failover
[12:52] <fang64> no containers being used to do that?
[12:52] <rick_h> so juju boostrap will go ask MAAS for one node, download the jujud binary to it, and start the service
[12:53] <rick_h> fang64: no, no containers being used to do that
[12:53] <fang64> ah ok, that's something I didn't understand initially when I looked at this, so juju is running on the host os, and when I enabled ha it grabbed 2 more
[12:53] <rick_h> fang64: exactly
[12:54] <rick_h> fang64: to have HA you need three machines so that you're not going to fall over if a disk dies/etc
[12:54] <fang64> that makes more sense, I was just a little confused what it's actually doing with MAAS
[12:55] <fang64> Another thing I was curious about which is responsible for networking configuration? Juju or MAAS?
[12:55] <fang64> or is it a combination of both, or it depends on what bundle or charm you are using?
[12:57] <fang64> rick_h: I appreciate the help, I'm going to try and figure out what's borked on my networking
[13:10] <anrah> Is there a way to react when controller losts connection to agent?
[13:11] <anrah> for example now it says: agent lost, see 'juju show-status-log my-instance/3'
[13:23] <stub> anrah: If it doesn't recover, I think it means either your network is blocked between the controller and that unit, or one or more jujud agents has crashed and you have a Juju bug to deal with
[13:24] <stub> anrah: If you find a dead juju service, you can try restarting it to see if sorts itself out.
[13:25] <anrah> I mean that if i have a case where the server dies for reason X
[13:25] <anrah> meaning some sort of selfhealing that juju would spin up replacament unit
[13:26] <anrah> I think that requires external monitoring and some magic with jujulib for python..
[13:27] <stub> Yes. I don't think juju ever spins up new units without user input.
[13:29] <stub> If you run juju recursively, you can even charm that :) I think some people do it for autoscaling.
[13:31] <stub> (probably easier now with 2.0 - you just need your monitoring charm given credentials for the main controller and it can administer itself and all the other models)
[13:31] <tvansteenburgh> anrah: there's a basic autoscale/autoheal demo here: https://github.com/juju/python-libjuju/blob/autoscaler/examples/autoscale.py
[13:32] <tvansteenburgh> you could extend it to deal with more healing conditions
[13:32] <anrah> stub: Yeah, I have build something like that for autoscaling
[13:33] <anrah> next step is to make it work with autohealing
[13:34] <stub> tvansteenburgh: Do you know if there is a best practice to adding unit tests to layers that don't mess up the main charm's tests?
[13:35] <tvansteenburgh> stub: sorry, i dunno
[13:36] <stub> I think I either need to exlude them from charm build, or stick them in a non-standard directory
[15:20] <beisner> stub, should be able to exclude unit/tests/file-by-name iirc
[15:20] <beisner> or some such
[17:12] <Mmike> Hi, lads. How do I use 'Login wih USSO' in juju2's gui? I bootstraped 2,0 environment, did 'juju gui --show-credentials', opened the juju-gui URL in the browser, and when I click on 'Login with USSO' i get 'authentication failed: no credentials provided'
[17:13] <Mmike> i am logged within ubuntu single-sign-on
[17:13] <Mmike> I am also logged into charmstore
[17:13] <lazyPower> Mmike thats a great question.
[17:14] <lazyPower> rick_h - is there any feedback/guidance here re USSO login support? or who should i be pinging to find out for mmike?
[17:14] <rick_h> Mmike: hmm, that's only available if it's configured to use an external idenity provider.
[17:15] <rick_h> Mmike: not useful for most cases unless you've bootstrapped that way
[17:15] <Mmike> rick_h: oh
[17:15] <rick_h> Mmike: the --show-credentials should show you the username/password ot use in the gui
[17:15] <Mmike> rick_h: yup, those work, I just wanted to see how 'USSO' would work
[17:15] <rick_h> Mmike: honestly, I think that button should be hidden unless the controller supports it.
[17:15] <rick_h> Mmike: I'll bring it up with the team
[17:15] <Mmike> rick_h: that was my thinking too!
[17:16] <Mmike> rick_h: how do I bootstrap with external identity provider configured?
[17:16] <Mmike> that's a controller option, or?
[17:16] <rick_h> Mmike: it's a config. I'll have to see if I can find it sec
[17:17] <Mmike> thnx!
[17:17] <rick_h> Mmike: https://lists.ubuntu.com/archives/juju/2016-September/007843.html
[17:17] <rick_h> Mmike: beware "here be dragons" as it works but has side effects and such
[17:17] <rick_h> Mmike: we were just working on things like show-model listing users with access/etc this morning
[17:18] <rick_h> Mmike: so it's there but there's some things that it causes to act a bit wonky
[17:18] <Mmike> I see
[17:18] <Mmike> rick_h: thank you for that info
[17:18] <rick_h> Mmike: np, hope that helps
[17:18] <lazyPower> thanks rick_h
[17:18] <Mmike> I'm asking because on https://blog.jujugui.org/ it's mentioned that if you log in with sso you get additinoal options, etc...
[17:18] <Mmike> so I was wondering how to get there
[17:19] <Mmike> but, yea - disabling that button if controller doesn't support it would be the excellent
[17:19] <rick_h> Mmike: what additional options?
[17:19]  * rick_h skims
[17:20] <rick_h> Mmike: not sure I can think of any additional options you get tbh
[17:20] <rick_h> it's the same juju/gui/etc
[17:21] <Mmike> rick_h: the video says that 'you can log in into controller using sso, and then you get additional canvas to select models, etc, etc'
[17:21] <rick_h> Mmike: oh hmm, will have to watch that I guess.
[17:21] <rick_h> Mmike: I mean it just shows models you have access to/etc
[17:23] <Mmike> rick_h: yup, maybe the video is confusing or giving information that's specific for a particular type of a controller
[17:25] <aisrael> What's the bash equivalent of charmhelper.config's .changed?
[17:34] <cory_fu> aisrael: You using reactive?  There is a state set that you can check with `charms.reactive is_state config.changed.foo`
[17:34] <cory_fu> aisrael: I don't know if there's a CLI for the actual method on Config.changed
[17:39] <aisrael> cory_fu, Yeah, this is a pure-bash charm, but I'm going to convert it to a layer, I think
[17:40] <Zic> lazyPower: (I prefer IRC over Slack :D) the last time you said to me that Canonical Kubernetes was not compatible with Helm, is that so?
[17:41] <Zic> lazyPower: about my issue with Vitess posted in their Slack, the guys from Vitess pointed me to https://github.com/youtube/vitess/tree/master/helm/vitess (work-in-progress)
[17:41] <Zic> it's a Helm package :s
[17:41] <Zic> help charts* don't know the right word :)
[17:41] <lazyPower> Zic  - we certainly do support helm, but there is a known issue with teh current incantation of the kube-api-loadbalancer
[17:42] <lazyPower> the supported work around is to either update or clone your kubeconfig and point it directly at one of your kubernetes-master units, and expose the kubernetes master unit. Proxying through the master load balancer will cause you heartburn and trigger false positive failures with helm.
[17:42] <Zic> oh, I can do that
[17:42] <lazyPower> Zic - we have an open PR to actually put that in the upstream docs for the CDK
[17:43] <lazyPower> thanks to SaMnCo for that submission *hattip*
[17:43] <Zic> I already access directly to my kubernetes-master on my LAN
[17:43] <Zic> and even the kube-api-loadbalancer is exposed only on LAN
[17:43] <lazyPower> ok, if you're already pointing directly at your master, you should be g2g
[17:43] <Zic> so I just need to upgrade my kubeconfig
[17:43] <lazyPower> and if thats not the case, i want all your bugs and feedback around this so we can triage accordingly
[17:43] <Zic> hehe :D
[17:43] <lazyPower> yep, just point kubeconfig at the correct port/ip for a master and you should be g2g
[17:44] <lazyPower> Zic - not sure if you saw but we just landed *everything* upstream yesterday around 7pm CST
[17:45] <lazyPower> https://github.com/kubernetes/kubernetes/pull/40324
[17:45] <Zic> lazyPower: as my Canonical Kubernetes cluster runs perfectly fine, I throw all of my ninja-power to make Vitess works in K8s... except I discovered today that their config (used in their official deployment guide) is not resiliant at all \o/
[17:46] <Zic> so... no, I didn't see anything from tech-world except my rage and tears with Vitess this last 2 days :)
[17:46] <Zic> but noted, I will take a look )
[17:46] <lazyPower> well, i'm sorry to hear about the tears, but its awesome to hear that we are empowering you
[17:46] <lazyPower> makes my own day to day tears worth while :)
[17:46] <Zic> huhu :)
[17:48] <Zic> simply puts: canonical-kubernetes does the job perfectly, and when I saw some guys raging against kubeadm which can't do the same, or even try to build K8s from scratch, I thank Juju :)
[17:48] <Zic> building K8s from scratch is very instructive anyway
[17:48] <Zic> but I need a tool to industrialize it in my company
[17:49] <lazyPower> I appreciate that feedback, we've been cycling hard to remove the barrier to going to prod with kubernetes
[17:49] <Zic> that's for the right part o/
[17:49] <lazyPower> so much so that we're going hard in the paint, my nice grey paintjob has streaks of red and white all up and down the side
[17:49] <Zic> for the wrong part (and that's totally offtopic here): Vitess is not ready for production-grade experience at this time
[17:49] <Zic> it will need some tricks to be
[17:49] <Zic> the good news is that this K8s cluster is not entirely dedicated to Vitess :p
[17:50] <lazyPower> we're looking to leave beta soon
[17:50] <lazyPower> so, be prepared for bulletproofing in the coming iterations
[17:50] <lazyPower> Zic - did you perhaps update to 1.5.2 with last weeks update?
[17:51] <Zic> my main coming-feature is haproxy replacing nginx in kube-api-loadbalncer :)
[17:51] <lazyPower> we just had a brief meeting about that this morining, we're looking to do layer 4 routing instead of layer7 via nginx. so that haproxy replacement cant come soon enough
[17:51] <Zic> nope, I think I'm on 1.5.1 for now, I didn't have the time to upgrade with my Vitess problems :(
[17:51] <lazyPower> Ack. when you do i'm highly interested in a) which approach you took to do the upgrade and b) how that experience went for you
[17:52] <Zic> lazyPower: layer 4 is OK also, in fact it's just the possibility to make a master offline easily which I need
[17:52] <lazyPower> so capturing any feedback for us would be highly useful in that context
[17:52] <Zic> (without touching the nginx vhost)
[17:52] <lazyPower> thats our goal, to not have the master exposed at all, so you can isolate it in some network segment and sleep easy
[17:54] <Zic> to describe our infra for now : I'm using 3 master, 5 etcd, 1 kube-api-loadbalancer (with easyrsa charms also on it), 6 physical workers, 3 EC2 AWS instances, 2 DRBD filers for PV with NFS, all in private-LAN
[17:54] <Zic> to expose publicly, I set up a public haproxy which have Ingress as backend
[17:54] <Zic> (3 haproxy, with a heartbeat VIP)
[17:54] <lazyPower> oh hey thats a nice setup
[17:54]  * lazyPower nods
[17:55] <lazyPower> i also see you took etcd durability very serious and gave it proper fault tolerant pooling
[17:55] <Zic> yes, as you advice me the first time I came here :)
[17:55] <lazyPower> :D
[17:55] <Zic> (was for the first pizza i owe you :p)
[17:56] <Zic> but yeah, I don't use public Ingress because if I bring-up a public ethernet interface on my kubernetes-worker, all my NodePort will be exposed, as K8s does not have a setting like "NodePort only bind on private address"
[17:56] <Zic> so I prefered to put public LB in front of Ingress and NodePorts
[17:56] <lazyPower> right
[17:56] <lazyPower> i think there was discussion on the k8s project around that
[17:56] <lazyPower> NodePort on Interface
[17:56] <lazyPower> but i haven't been tracking that too closely, and it may have been tabled
[17:57] <Zic> yes, I saw an issue on GitHub but not so much news since 2015
[17:58] <lazyPower> yeah, that sounds about right
[17:58] <lazyPower> might be worth poking it to see if there can be some renewed interest around it
[17:59] <Zic> the only negative effect of that is that our customer had full capability with kubectl to expose his service privately through Ingress and NodePorts
[17:59] <Zic> but for public parts, it's HAProxy VMs wich is fully managed through Puppet
[18:00] <lazyPower> so, we're going to augment ingress with configmaps, which should make it more durable for you
[18:00] <Zic> so our customer needs to fill at ticket for every public expose
[18:00] <lazyPower> as in, you can expose interesting things like ssh services for a private gogs instance
[18:00] <Zic> (not so important as he only expose 80 and 443, and we do the ssl offloading with a wildcard cert)
[18:00] <lazyPower> and it'll proxy that ssh connection through the ingress controller
[18:00] <Zic> cool :)
[18:00] <lazyPower> as it stands today we're only concerned with web traffic on that ingress controller
[18:00] <Zic> yep
[18:00] <lazyPower> but we do realize and understand there is another class of workloads that need to be supported
[18:00] <Zic> I use NodePorts for all other concerns
[18:01] <lazyPower> and its a bit slower going but i'm tracking that
[18:01] <lazyPower> yeah
[18:01] <lazyPower> i do the same in my homelab
[18:01] <Zic> but nos as practicle as Ingress
[18:01] <Zic> practical*
[18:02] <lazyPower> yep
[18:02] <Zic> I discovered something like https://traefik.io/ also
[18:02] <lazyPower> there's been some effort around an haproxy ingress controller as well
[18:02] <lazyPower> i've used traefik, its great
[18:02] <lazyPower> i'm not positive that its been updated withs ocket support yet though
[18:02] <Zic> which can enpower my customer for public exposing
[18:02] <lazyPower> are you aware if they added that in recent revisions?
[18:02] <Zic> no not at all, I'm at the point of just reading their homepage :D
[18:02] <lazyPower> i'd like to include that in a workloads repository so end users can mix/match their ingress controllers via namespace
[18:03] <lazyPower> eg: namespace=customer  you get the default nginx controller,    namespace=alpha1  - you launch and use a traefik ingress controller until you're ready to promote to namespace=customer
[18:04] <lazyPower> and can move all that stuff with it, its just kind of a nice to have, and would be great to be tuneable with some curated manifests
[18:04] <lazyPower> but thats all pie in the sky at this time, i haven't had time to dedicate to it
[18:05] <Zic> I'm planning to finish this resilient Vitess cluster, upgrading to K8s 1.5.2 through Juju and after that, poweroff/rebooting randomly all the parts :D
[18:05] <SaMnCo> @lazyPower pleasure :) just the first of (I hope) a long list
[18:06] <SaMnCo> @Zic, let me know if you have issues around helm
[18:06] <Zic> if all stay OK (I already did some tests :p) I will owe you 3 pizzas total
[18:06] <Zic> if all is wrong, I will jump through the window of the ground-floor
[18:06] <Zic> (and then, debug :p)
[18:06] <SaMnCo> also, if you are looking into scaling SQL, we have a Charm Partner partner (ScaleDB) who does just that
[18:07] <SaMnCo> not sure they have a lot of k8s stuff, but still worth looking at.
[18:08] <Zic> noted :)
[18:08] <SaMnCo> and, last but not  least, I'm trying to get good low level sysadmin feedback on Juju usage to document
[18:09] <SaMnCo> so I'd be happy to discuss your XP, in French or English ;)
[18:13] <Teranet> anyone here who knows about ceph a bit how to create a directory without creating a drive mount point : http://paste.ubuntu.com/23864768/
[18:14] <Teranet> I try to get ceph-disk for some reason push a drive creation but I do want only a directory been done
[18:17] <lazyPower> ping cholcombe and icey  ^
[18:19] <icey> Teranet: can you share a bundle that you're using to deploy that? from the bit of logs, it looks like it _should_ just work so would be a bug
[18:20] <Teranet> it's not a bug I was looking now on the container and I do see /srv/osd is created and has data already in it
[18:20] <Teranet> I am thinking it's looking still for an empty one
[18:31] <Teranet> ok I am deploying a bigger OS.yaml which I cutom build already in a lot of ways
[18:31] <Teranet> want me to share the hole OS.yaml ?
[18:36] <icey> ah Teranet, ceph-osd doesan't behave too well inside of a container
[18:37] <Teranet> so I should redploy it outside of the container ?
[18:38] <Teranet> or can I just do redeploy the 3 ceph-osd somehow to the boxes instead of containers ?
[18:39] <jrwren> what is wrong with ceph-osd in a container? if your lxc/lxd is in zfs, make sure to set use-direct-io: false and don't expect /dev osd-devices to work. Other than that it seems to work for me for dev/test.
[18:41] <jrwren> Teranet: is your lxd zfs or btrfs backed?
[18:42] <jrwren> Teranet: I get those errors with zfs backed container and the default use-direct-io value. Set the use-direct-io to false for the ceph-osd charm.
[18:44] <icey> jrwren: I suppose that's relevant too; I wouldn't run ceph-osd inside of a container unless it _is_ just for testing
[18:45] <Teranet> let me doublecheck
[18:54] <Teranet> how can I check that if it's zfs again ?
[18:54] <Teranet> fdisk don't help much there
[18:56] <jcastro> lxc info | grep storage
[18:56] <jcastro> should tell you
[18:57] <Teranet> zfs
[19:02] <Teranet> ok so if I put them now to the host how can I move them ?
[19:04] <Teranet> or do I need to kill those 3 containers with ceph-osd on it first and than redeploy them ?
[19:06] <jrwren> Teranet: probably, yes, or you could set the config, check in /etc/ceph for 'journal dio = false'
[19:17] <bdx> concerning the hosted controller
[19:17] <bdx> I don't seem to be able to connect to a model on the hosted controller via libjuju
[19:17] <bdx> see http://paste.ubuntu.com/23865066/
[19:18] <bdx> ^ succeeds when ran against my own controller, but fails when ran against the hosted controller
[19:18] <bdx> tvansteenburgh
[19:18] <bdx> ^^
[19:18] <bdx> tvansteenburgh: have you tried using libjuju against the hosted controller?
[19:19] <tvansteenburgh> bdx: yeah. it *can* work, but it relies on valid macaroons existing on the host
[19:20] <tvansteenburgh> bdx: do you have juju cli installed on the host?
[19:21] <bdx> tvansteenburgh: yea, does the fact that juju cli can access the model validate that the macaroons are valid?
[19:21] <tvansteenburgh> yeah
[19:22] <tvansteenburgh> bdx: can you deploy to the model with the cli?
[19:22] <tvansteenburgh> if you can, then libjuju should work too
[19:22] <jcastro> heya Zic
[19:22] <jcastro> we should find time this week or next to have you sync with the team in a hangout, would love to get a laundry list of feedback from you
[19:22] <tvansteenburgh> bdx: this is a limitation of libjuju until we add support for obtaining and discharging macaroons
[19:24] <bdx> tvansteenburgh: I run that script ^ against a lxd controller and it does not error, then I switch controllers via `juju switch jujucharms.com` following which select one of my models, and run the script again and it fails
[19:25] <tvansteenburgh> bdx: is there a traceback?
[19:25] <bdx> due to auth errors, see http://paste.ubuntu.com/23865120/
[19:27] <tvansteenburgh> bdx: are you logged in, i.e. juju login
[19:27] <bdx> yea
[19:28] <bdx> tvansteenburgh: http://paste.ubuntu.com/23865139/
[19:28] <tvansteenburgh> bdx: do you have a ~/.go-cookies file?
[19:28] <bdx> yeah ... want me to squash it and try again?
[19:29] <tvansteenburgh> not yet
[19:31] <stormmore> so I just created a controller and ran "juju ssh -m controller 0". while it appears that it is logged in, it doesn't give me a prompt, and hitting enter gives me "-bash: line 1: $'\r': command not found". any ideas what is going on?
[19:32] <tvansteenburgh> bdx: http://pythonhosted.org/juju/narrative/model.html#connecting-with-macaroon-authentication
[19:36] <bdx> tvansteenburgh: YES! that did it
[19:36] <lazyPower> stormmore - that's most definiately a bug, but i'm not certain what happened
[19:36] <bdx> tvansteenburgh: thank you
[19:37] <icey> stormmore: are you on windows?
[19:37] <tvansteenburgh> bdx: \o/ np
[19:37] <stormmore> lazyPower I came across https://bugs.launchpad.net/juju-core/+bug/1468752 so you are right it is bug :)
[19:37] <stormmore> icey yes
[19:37] <lazyPower> oo fantastic find
[19:37] <lazyPower> bummer that its a bug, but glad its not pioneering territory
[19:37] <icey> stormmore: yeah, that was the bug I was curious about :)
[19:45] <stormmore> so what its telling me is that this is a stupid Windows bug without a workaround but the fix is in 2.2 which is currently in alpha!
[19:47] <stormmore> I am going to see if I can install and bootstrap from bash / Ubuntu on Windows 10
[19:50] <stormmore> that is if I can figure out how to get it upgrade to Xenial
[20:10] <bdx> tvansteenburgh: I may have been a bit premature in my rejoice
[20:10] <bdx> tvansteenburgh: I'm getting "Fatal error on SSL transport" now ...
[20:12] <tvansteenburgh> bdx: that's almost always a side effect of another problem, got a traceback?
[20:12] <bdx> tvansteenburgh: my simple script http://paste.ubuntu.com/23865377/
[20:12] <bdx> tvansteenburgh: traceback <- http://paste.ubuntu.com/23865383/
[20:16] <tvansteenburgh> bdx: can you set logging level to DEBUG and paste the full output?
[20:16] <bdx> tvansteenburgh: http://paste.ubuntu.com/23865408/
[20:18] <tvansteenburgh> bdx: did you change line 41 or 43?
[20:18] <bdx> tvansteenburgh: do you think I need to be supplying a 'cacert'?
[20:18] <bdx> tvansteenburgh: both
[20:20] <tvansteenburgh> bdx: yeah, trying sending the cert
[20:20] <tvansteenburgh> i'm not convinced that's the problem but it won't hurt
[20:21] <bdx> tvansteenburgh: http://paste.ubuntu.com/23865441/
[20:21] <bdx> sad
[20:21] <tvansteenburgh> heh
[20:22] <tvansteenburgh> nevermind!
[20:22] <vmorris> any word on this bug? https://bugs.launchpad.net/juju/+bug/1614364
[20:27] <bdx> tvansteenburgh: does libjuju use the asyncio ssl protocol for ssl connections?
[20:29] <tvansteenburgh> bdx: no
[20:29] <tvansteenburgh> well it might indirectly
[20:30] <bdx> tvansteenburgh: this https://github.com/python/asyncio/blob/master/asyncio/sslproto.py
[20:30] <tvansteenburgh> no
[20:30] <bdx> tvansteenburgh: can you replicate this on your end?
[20:31] <tvansteenburgh> bdx: not yet. it works for me. still looking
[20:40] <tvansteenburgh> bdx: change line 41 to: logging.basicConfig(level=logging.DEBUG)
[20:40] <tvansteenburgh> then paste me the full output
[20:41] <tvansteenburgh> bdx: also, print the value of your model_uuid to make sure it's actually set
[20:41] <bdx> tvansteenburgh: I'm setting MODEL_UUID inline in my testing
[20:42] <tvansteenburgh> ok i thought you were getting it from the env
[20:46] <bdx> http://paste.ubuntu.com/23865573/
[20:46] <bdx> thats better
[20:48] <tvansteenburgh> bdx: yeah, so that means whatever macaroons you have locally aren't sufficient
[20:51] <tvansteenburgh> bdx: which is strange since you can run juju status on the model
[20:57] <bdx> tvansteenburgh: I launch a fresh xenial container `lxc launch ubuntu:16.04 libjujutest`, exec in, `su ubuntu`,  install juju, libjuju, asyncio via `sudo apt install juju python3-pip && sudo -H pip3 install juju asyncio`, run the script and get the error
[20:58] <bdx> tvansteenburgh: should the jujuclient version <-> juju controller version mismatch have anything to do with this?
[21:00] <tvansteenburgh1> bdx: when using macaroon auth (shared controller), libjuju can currently only connect if there valid macaroons in ~/.go-cookies already
[21:00] <bdx> * login to the controller:model via juju cli, then run the script and still get the error
[21:00] <bdx> correction
[21:00] <bdx> my bad
[21:00] <bdx> I am logging in first
[21:02] <tvansteenburgh1> bdx: i'm trying to figure out what's different. when i run your script against jimm, it works. on every model i try
[21:02] <tvansteenburgh> bdx: try `juju deploy ubuntu`, then run your script
[21:06] <stormmore> I keep getting "ERROR cannot update credentials for aws: timeout acquiring mutex" when trying add credentials or change the default aws region :-/
[21:07] <bdx> tvansteenburgh: deployed ubuntu, no change, except that now the script will error every other try, and just hangs on 50% it seems ... when I Ctrl+C out of what seems to be a hang, I get this http://paste.ubuntu.com/23865667/
[21:07] <bdx> which has an interesting error at the bottom "juju.errors.JujuAPIError: unknown version (1) of interface "Client""
[21:08] <tvansteenburgh> yeah
[21:16] <tvansteenburgh> bdx: my models are 2.0.1, yours are 2.0.2.1
[21:17] <tvansteenburgh> that shouldn't matter, but...
[21:18] <tvansteenburgh> nope, works for me on 2.0.2.1 too
[21:31] <mskalka> anyone familiar with the openstack base bundle available to answer a few questions?
[22:00] <stormmore> looks like I am going to just spin up an Ubuntu VM since Bash on Windows has problems with mutex when trying to do stuff with juju and the windows version of juju has problems with ssh to the controller
[22:14] <zeestrat> mskalka: What's up? I've got some minutes before EOD.
[22:15] <zeestrat> mskalka: If no one is around here, make sure to check out #openstack-charms
[22:19] <mskalka> zeestrat: thanks for the reply. I'm running into an odd issue deploying openstack on aws. It looks like the deployment is hung up cinder and glance connecting to the mysql db
[22:20] <mskalka> I had it working last week just as a POC after a few tweaks to where it landed the charms (the ceph-mon charms did not like being in lxd for example) but never ran into this issue
[22:25] <zeestrat> mskalka: Hmmm. Unfortunately I only have experience with deploying it to MAAS on bare metal so I won't probably be too helpful. Ping the guys in #openstack-charms with the output of "juju status --output yaml" and perhaps some logs.
[22:25] <mskalka> zeestrat: No worries! Thanks for the help, I'll ping #openstack-charms tomorrow morning
[22:26] <zeestrat> Cool. I think most of them are on a EMEA timezone.
[22:27] <mskalka> copy, thanks
[23:07] <Teranet> do we have an JUJU OPENSTACK guys here ? I am looking for some logging from OpenStack but now since it's all on JUJU / MAAS not sure where it would be looging too for neutron
[23:43] <marcoceppi_> Teranet: #openstack-charms channel might be best