[05:11] <Facso> How do bundles handle exposing ports? or do they not?
[05:11] <Facso> I'm about to test this with a simple web and database charm deploy, but thought I'd ask.
[10:54] <kjackal> marcoceppi and juju at devopsdays Amsterdam in about an hour: http://livestream.com/accounts/20260359/devopsdays-amsterdam-2016
[11:11] <magicaltrout> woop
[11:54] <axino> lp:charms/trusty/ubuntu-repository-cache exists, I prepared a Xenial version at https://code.launchpad.net/~axino/charms/trusty/ubuntu-repository-cache/xenial-ready/+merge/298880, is this MP the right thing to do to make it happen ?
[12:22] <magicaltrout> nice flip flops marcoceppi
[13:33] <neiljerram> lazyPower, good morning!
[13:39] <neiljerram> lazyPower, I've created an etcd-local-proxy charm at https://github.com/neiljerram/etcd-local-proxy (by starting from and hacking layer-etcd).  I wonder if you might take a look at it and see if it looks sane to you?
[13:40] <neiljerram> lazyPower, There's also a needed PR for interface-etcd-proxy at https://github.com/juju-solutions/interface-etcd-proxy/pull/6
[13:43] <neiljerram> lazyPower, However, when I build and deploy etcd-local-proxy, and also cs:~lazypower/etcd-19, I'm seeing TLS auth issues, so it's possible there's something wrong in how the certificates are generated, or in the etcd config at either end.  Have you done successful end-to-end TLS testing with an etcd master and proxy?
[13:56] <marcoceppi>  magicaltrout hah, thanks ;)
[13:58] <magicaltrout> good talk though
[13:58] <magicaltrout> looks like you got a decent crowd
[14:00] <lazyPower> neiljerram - i have not. only verified the data exchange on the wire. If you need server certificates, we can do that but its a different dance over the relation wire. It involves generating a CSR and having it signed by the etcd cluster
[14:02] <lazyPower> s/cluster/leader/
[14:08] <lazyPower> neiljerram - i think i knwo why you're getting ssl errors as well. the fork of etcd there is generating its own certificates
[14:09] <lazyPower> wait no, i jumped the gun i see you removed that from metadata
[14:12] <lazyPower> mbruzek - ping
[14:12] <mbruzek> Hello lazyPower
[14:13] <lazyPower> hey o/  neiljerram has been working with our latest release of etcd, and had some questions. Woudl be good to get your eyes on https://github.com/neiljerram/etcd-local-proxy
[14:13] <mbruzek> neiljerram: What TLS issues are you seeing?
[14:41] <neiljerram> mbruzek, thanks - first I was seeing (IIRC) 'x509 certificate signed by unknown authority'; then a bit later 'remote error: bad certificate'
[14:41] <neiljerram> I will start a redeployment now so I can provide more detail...
[14:43] <mbruzek> neiljerram: OK. Can you describe the model? I have got successful etcd to etcd TLS before (not using proxy)
[14:44] <neiljerram> mbruzek, the model is etcdctl ----- local etcd proxy ----- remote etcd master
[14:45] <mbruzek> And you are running etcdctl on your machine or in the cloud?
[14:45] <neiljerram> mbruzek, there is TLS security for the connection between the proxy and the master; but no TLS security between etcdctl and the local proxy.
[14:45] <neiljerram> mbruzek, etcdctl invocations are on the same machine where the proxy is running
[14:47] <neiljerram> mbruzek, so the etcd master is  cs:~lazypower/etcd-19, which is Charles's latest unreleased version of layer-etcd
[14:47] <neiljerram> mbruzek, and the etcd proxy is https://github.com/neiljerram/etcd-local-proxy, which is a charm that basically just uses the 'requires' side of interface-etcd-proxy
[14:54] <lazyPower> mbruzek - i pushed etcd-19 in my namespace due to the missing etcd-proxy bits. We translated the interface, but missed the reactive code to support that
[14:55] <lazyPower> everything but the reactive code in layer-etcd has merged in master of their respective repositories though, so we can rev ~containers once that lands
[14:55] <lazyPower> if its ready for that
[14:56] <mbruzek> I just diffed etcd-19 and etcd-local-proxy the etcd.py looks quite different.  Let me see if I understand correctly, you are connecting the etcd charm with a different etcd-proxy charm and the tls is not working?
[14:57] <mbruzek> from etcdctl on proxy and the etcd on the master.
[15:05] <neiljerram> mbruzek, That's all correct.   I really just used etcd-19 as a structural starting point, but etcd-local-proxy does a lot less, because it doesn't have to do leader election or registration, or generate its own certificates.  In principle - I think - it just needs to do the 'requires' side of the 'proxy' relation, save the credentials once they're available, update /etc/defaults/etcd, and restart the local etcd (proxy) service.
[15:07] <mbruzek> neiljerram: Yeah that looks right.
[15:07] <mskalka> hey everyone, is there any method for pulling the machine ID or INS-ID inside a hook or action?
[15:07] <neiljerram> My deployment is up now, and here's what I see: http://pastebin.com/Xs4YLDtY
[15:09] <mbruzek> neiljerram: OK please add to the pastebin: openssl x509 -in /tmp/etcd_cert -text
[15:11] <mbruzek> neiljerram: That will print out the text of the certificate to make sure the one on your system is valid.
[15:11] <mbruzek> I suspect it is, but want to start there.
[15:11] <neiljerram> mbruzek, http://pastebin.com/4Dfa2r1V
[15:12] <mbruzek> neiljerram: And these addresses are correct ? IP Address:54.164.27.251, IP Address:172.31.51.7
[15:13] <neiljerram> mbruzek, 54.164.27.251 is the public IP of the AWS instance that contains etcd-19.
[15:14] <neiljerram> and 172.31.51.7 is that same machine's private IP
[15:14] <mbruzek> OK that is good so I believe the certificate is valid and stored correctly on the proxy machine.
[15:14] <neiljerram> (the machine itself only knows its private IP)
[15:14] <mbruzek> neiljerram: Yep
[15:17] <mbruzek> neiljerram: Can you run the following? : etcdctl --debug --cert-file=/tmp/etcd_cert --key-file=/tmp/etcd_key --ca-file=/tmp/etcd_ca and pastebin the result?
[15:17] <mbruzek> wait add the "ls /" to the end
[15:20] <neiljerram> mbruzek, that seems to work: http://pastebin.com/3ze47iCz
[15:21] <mbruzek> sweet!
[15:21] <mbruzek> OK so I think the problem is that etcd-proxy reads the defaults file, but etcdctl does not.
[15:23] <mbruzek> neiljerram: I am not familiar enough with etcdctl to know how to make those values default, some tools use env variables, other tools have a confiuration file.
[15:23] <neiljerram> But I didn't think that the hop from etcdctl to the local proxy should use TLS at all...  So I'm a bit confused!
[15:24] <mbruzek> neiljerram: Are you sure that etcd-proxy is properly configured with these keys/certs?
[15:25] <neiljerram> I was just going to pastebin the proxy's config....
[15:26] <neiljerram> ... http://pastebin.com/ewTBy5mJ
[15:27] <mbruzek> I wonder if "PEER" is not valid in this case, let me do some searching
[15:28] <mbruzek> Also I know they are in the defaults file, but are we sure that etcd is actually getting the values? Do you see in the etcd logs that the key/cert/ca is read in?
[15:28] <mbruzek> https://coreos.com/etcd/docs/latest/security.html
[15:31] <neiljerram> mbruzek, Yes, I think so: http://pastebin.com/jhg1HLve  (I just added ETCD_CA_FILE and other non _PEER_ settings to the config - but that didn't make any difference to our observations so far)
[15:33] <mbruzek> neiljerram: So after doing that, you are still not able to connect using etcdctl without the  cert and key flags?
[15:33] <neiljerram> mbruzek, Perhaps the etcd proxy doesn't really support TLS on one side (towards the cluster) but not on the other (towards the client).  So when it gets a client request it requires TLS auth, even if that client request is over http.  Is that possible?
[15:33] <neiljerram> mbruzek, Yes, correct.
[15:34] <neiljerram> mbruzek, and with those flags, I still _can_ connect
[15:34] <mbruzek> neiljerram: I don't know etcd proxy well enough to answer that.
[15:35] <mbruzek> neiljerram: Great. Still that is a pretty bad user experience, let me see if I can find more information about etcdctl to see if we can eliminate those flags
[15:35] <neiljerram> mbruzek, Do you know who implemented the 'pillowmints' in the etcd charm?  I think they support that hypothesis.  (They set up env vars for etcdctl.)
[15:36] <mbruzek> neiljerram: "pillowmints" was used by lazyPower for lack of a better name. I questioned him about that during a code review.
[15:37] <neiljerram> mbruzek, It is a curious name :-) but nice.
[15:38] <neiljerram> mbruzek, But it sets up these vars, for local etcdctl usage on the machine where the etcd master node is installed:
[15:38] <neiljerram> ETCDCTL_CA_FILE=/etc/ssl/etcd/ca.pem
[15:38] <neiljerram> ETCDCTL_CERT_FILE=/etc/ssl/etcd/server.pem
[15:38] <neiljerram> ETCDCTL_KEY_FILE=/etc/ssl/etcd/server-key.pem
[15:38] <mbruzek> Oh there you go!
[15:39] <mbruzek> export ETCDCTL_CERT_FILE and the others and try etcdctl again
[15:42] <neiljerram> mbruzek, Yes, that works.  But I still don't feel like I understand why.  How does TLS auth mix with doing a request over http:// (i.e. _not_ over https://) ?
[15:46] <mbruzek> neiljerram: According to your pastebin with debug http://pastebin.com/3ze47iCz the http address is getting converted to https and my guess is that the etcdctl is making the direct https connection
[15:47] <mbruzek> neiljerram: etcdctl must be talking http to the proxy but perhaps it is actually talking https to the server as well.
[15:48] <neiljerram> mbruzek, Oh! So the proxy isn't really proxying; but rather redirecting!
[15:53] <mbruzek> neiljerram: That would be my assessment from reading the pastebin you have there.
[15:57] <neiljerram> mbruzek, It appears this is all a behaviour of etcdctl, rather than of the proxy.  'etcdctl --no-sync ls /' works even without those env vars.
[15:58] <mbruzek> neiljerram: Interesting
[15:58] <neiljerram> mbruzek, Thanks so much for helping me to understand all this.  I think I have all the pieces and understanding that I need now.
[15:58] <neiljerram> lazyPower, Thanks also.
[15:59] <mbruzek> neiljerram: Excellent, glad to help. If you need anything else ping me here.
[15:59] <neiljerram> mbruzek, Thanks, I'll do that.
[15:59] <mbruzek> I am the one that wrote the TLS layer so it is my responsibility. What I have found is TLS is hard
[16:09] <lazyPower> mbruzek i try to help :)
[17:12] <xilet> After a reboot a series of my units are stuck with "agent is lost, sorry! See 'juju status-history glance/0'"  The status-history is normal until the reboot and nothing since, any way to kickstart a container back to life? (2.0-beta)
[18:01] <jose> xilet: did IP addresses change?
[18:03] <xilet> No
[18:10] <cholcombe> reactive is doing some odd things i'm having trouble figuring out
[18:11] <cholcombe> i have a base layer that sets a state after it installs everything.  in my layer that uses it i @when() that state and for some reason the base layer never installs what it should but the state is firing
[18:11] <cholcombe> i r confused :)
[18:34] <cholcombe> nvm icey and i figured it out
[19:44] <Yacka> It seems like --config has been removed from deploying bundles?
[19:44] <Yacka> If I create a bundle for a vanillastack application and want to keep the configuration options separate what's the prescribed way in 2.0?
[19:55] <Yacka> ERROR Flags provided but not supported when deploying a bundle: --config
[19:55] <Yacka> Excellent.
[19:55] <Yacka> Why should a deploy of a bundle be any different than a deploy for a charm?
[19:56] <Yacka> They use the same command.
[20:34] <kjackal> kwmonroe: thank you for the cleanup on hbase. I will study all the changes and aling my work based on your pointers. Many thanks!
[20:56] <kwmonroe> np kjackal -- it wasn't too complicated, just doing actions a bit better than we used to.  i also like the collapsed report_status() handler vs individual handlers per missing state.