[09:02] <armaan> jamespage: hello, could i ask your opinion on https://github.com/juju/1.25-upgrade ? Can i use this for upgrading OpenStack environments?
[09:13] <Ting_> Hi, could people use other authentication-type for juju aws instead of access-key? We are using aws federation account seems very hard to get access key...
[11:57] <rick_h> Ting_: we need to support that. There's an open bug that could use some real user pressure. Let me see if I can find it.
[11:59] <Ting_> rick_h: thx :)
[12:00] <rick_h> Ting_: on my phone ATM and not able to find it. Let me look when I get to the office.
[12:04] <Ting_> rick_h: sure, no hurry
[13:45] <tychicus> just deployed the microbot application in CDK 1.7 to get a better understanding of how the ingress controller and load balancer work.
[13:45] <tychicus> kubectl get ingress does not return an address
[13:46] <tychicus> the only values returned are NAME, HOSTS, PORTS, AGE
[13:48] <tychicus> I can validate everything is working correctly by creating a /etc/hosts entry for microbot.10.148.0.105.xip.io and using the IP address provided in the hostname,
[13:51] <SimonKLB> tychicus: did you create it using the microbot action?
[13:52] <tychicus> SimonKLB: yes juju run-action kubernetes-worker/0 microbot replicas=3
[13:52] <SimonKLB> did you check the action status and/or output?
[13:53] <SimonKLB> also, is it only the ingress that is missing or is the deployment also not there?
[13:54] <tychicus> https://gist.github.com/roll4life/2cbaad645379f3db6998d2e08f70f664
[13:55] <tychicus> here is the output of kubectl get services,endpoints and kubectl get ingress
[13:55] <SimonKLB> ah my bad, i thought you meant the ingress was never created
[13:56] <tychicus> sorry, ingress is created, but the address is not displayed
[13:56] <SimonKLB> is the kubernetes-worker exposed?
[13:57] <tychicus> yes
[13:57] <tychicus> kubernetes-worker      1.7.0    active      3  kubernetes-worker      jujucharms   40  ubuntu  exposed
[13:58] <SimonKLB> hmm, can you check the logs of the ingress-controller?
[14:12] <tychicus> SimonKLB: updated the gist with the ingress-controller log output https://gist.github.com/roll4life/2cbaad645379f3db6998d2e08f70f664
[14:12] <tvansteenburgh> tychicus: what's the output of kubectl get pods
[14:12] <tvansteenburgh> is there an ingress controller running?
[14:14] <tychicus> yes 3
[14:15] <SimonKLB> tychicus: just tried it out on my end and works fine, wierd..
[14:15] <tychicus> ok gist updated
[14:16] <SimonKLB> can you check which version of the ingress controller youre running? `kubectl get rc nginx-ingress-controller -o yaml | grep image:`
[14:16] <tychicus> image: gcr.io/google_containers/nginx-ingress-controller:0.8.3
[14:17] <SimonKLB> same here
[14:17] <SimonKLB> so it's definately that "has no active endpoints" that's the issue
[14:18] <SimonKLB> or atleast that is what looks different from mine
[14:20] <tychicus> ok thanks I'll look into that and report back what I find, thank you for your help
[14:22] <SimonKLB> tychicus: which provider are you running on btw?
[14:23] <tychicus> local openstack juju/maas deployment
[14:29] <tychicus> updated the gist with the output of kubectl get ing microbot-ingress -o yaml
[14:30] <SimonKLB> that's wierd, it looks like it's adding ips, but they are empty
[14:30] <SimonKLB> actually, looking at the log again it says "Updating loadbalancer default/microbot-ingress with IP *nothing*"
[14:31] <tychicus> right, just noticed that as well
[14:34] <SimonKLB> tychicus: can you paste your juju status output?
[14:34] <SimonKLB> if the workers doesn't have public ips that might be it
[14:35] <tychicus> updated, they do have "public addresses"
[14:36] <tychicus> they are rfc 1918 addresses, but they a "public" internally
[14:36] <SimonKLB> yea
[14:44] <SimonKLB> tychicus: check if kube-proxy is running ok on the workers
[14:45] <SimonKLB> you can run this for example: `juju run --unit kubernetes-worker/0 systemctl status snap.kube-proxy.daemon.service`
[14:46] <tychicus> ok, thanks, I was just getting ready to ask how to check the status :)
[14:49] <tychicus> they all show as running all have these errors with ID 1,5,7,9,11
[14:49] <tychicus> Unable to decode an event from the watch stream: stream error: stream ID 1; INTERNAL_ERROR
[14:51] <SimonKLB> tychicus: you can get the full log if you run `juju run --unit kubernetes-worker/0 "journalctl -u snap.kube-proxy.daemon.service"`
[14:51] <SimonKLB> or ssh into the machine
[14:51] <tychicus> gladly
[15:45] <Cynerva> tychicus: i think the behavior you're seeing with microbot is pretty typical
[15:45] <Cynerva> seems like ingress only gets an address assigned to it if you have external loadbalancer support
[15:46] <Cynerva> which AFAIK requires cloud integration
[15:46] <Cynerva> we only get that on AWS when deployed via conjure-up, i think
[15:46] <Cynerva> but, the ingress is still usable without a loadbalancer
[15:47] <tychicus> cynerva: ok, thanks
[15:51] <SimonKLB> Cynerva: wierd, because it's working fine here and im running it on LXD
[15:53] <Cynerva> huh
[15:53] <Cynerva> that is weird
[15:53] <SimonKLB> yea :D
[15:56] <Cynerva> i just had a go on AWS w/o native integration, got no address but it's working the way i'd expect
[15:58] <SimonKLB> % juju status | head -n 2
[15:58] <SimonKLB> Model                        Controller           Cloud/Region         Version       SLA
[15:58] <SimonKLB> conjure-kubernetes-core-531  localhost-localhost  localhost/localhost  2.3-alpha1.1  unsupported
[15:58] <SimonKLB> % kubectl get ing
[15:58] <SimonKLB> NAME               HOSTS                           ADDRESS            PORTS     AGE
[15:58] <SimonKLB> microbot-ingress   microbot.10.212.38.141.xip.io   10.212.38.141...   80        1h
[15:58] <SimonKLB> :o
[15:59] <tychicus> yeah, that is pretty much what I expected to see :)
[16:02] <SimonKLB> whats even more interesting is that my nodes does not have external ips:
[16:02] <SimonKLB> % kubectl get no -o wide
[16:02] <SimonKLB> NAME            STATUS    AGE       VERSION   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION
[16:02] <SimonKLB> juju-cf3e22-3   Ready     16h       v1.7.0    <none>        Ubuntu 16.04.3 LTS   4.10.0-21-generic
[16:02] <SimonKLB> juju-cf3e22-4   Ready     16h       v1.7.0    <none>        Ubuntu 16.04.3 LTS   4.10.0-21-generic
[16:03] <SimonKLB> Cynerva: do you know if kubernetes or the ingress-controller somehow aware of it's environment and accknowledge the internal ips as external when running inside LXD or something?
[16:04] <SimonKLB> since the ingress controller is actually adding the interal ip of the node:
[16:04] <SimonKLB> % kubectl get no juju-cf3e22-3 -o jsonpath='{.status.addresses}'
[16:04] <SimonKLB> [map[type:InternalIP address:10.212.38.141] map[type:Hostname address:juju-cf3e22-3]]%
[16:06] <Cynerva> SimonKLB: not that i'm aware of, but could be
[17:59] <bdx> rick_h: sup
[17:59] <bdx> rick_h: is there anything on the roadmap for JAAS to support team owned models?
[18:00] <rick_h> bdx: not direction at the moment. There's some precursor work going on though
[18:00] <rick_h> bdx: e.g. not in the next release of work but it's on the radar and some groundwork is landing/going on
[18:00] <bdx> nice nice
[18:00] <bdx> thats good to know
[18:01] <bdx> I'm leaving a trail of models owned by myself across the technology community :)
[18:01] <rick_h> bdx: lol
[18:02] <rick_h> bdx: it's not been pushed because you can add/remove users and such so there's some level of control.
[18:02] <rick_h> so it's longer to do since you have to add each user vs a single team
[18:02] <bdx> I see
[18:03] <kwmonroe> bdx: just share your creds with people you trust.  instant team!  we can put them in relevant irc /title bars if it makes life easier for you.
[18:03] <rick_h> bdx: and juju doesn't really have solid "group" idea on a self-operated controller so it needs some smarts
[18:03] <rick_h> hah
[18:03] <rick_h> kwmonroe: always here to save the day!
[18:03] <kwmonroe> :)
[18:07] <bdx> ^ perfect example of what not to do
[18:07] <bdx> :)