| === alexlist` is now known as alexlist | ||
| Facso | How do bundles handle exposing ports? or do they not? | 05:11 |
|---|---|---|
| Facso | I'm about to test this with a simple web and database charm deploy, but thought I'd ask. | 05:11 |
| === frankban|afk is now known as frankban | ||
| === gnuoy` is now known as gnuoy | ||
| === spammy is now known as Guest3061 | ||
| kjackal | marcoceppi and juju at devopsdays Amsterdam in about an hour: http://livestream.com/accounts/20260359/devopsdays-amsterdam-2016 | 10:54 |
| === fginther is now known as fginther` | ||
| magicaltrout | woop | 11:11 |
| === bradm_ is now known as bradm | ||
| === fginther` is now known as fginther | ||
| axino | lp:charms/trusty/ubuntu-repository-cache exists, I prepared a Xenial version at https://code.launchpad.net/~axino/charms/trusty/ubuntu-repository-cache/xenial-ready/+merge/298880, is this MP the right thing to do to make it happen ? | 11:54 |
| magicaltrout | nice flip flops marcoceppi | 12:22 |
| neiljerram | lazyPower, good morning! | 13:33 |
| neiljerram | lazyPower, I've created an etcd-local-proxy charm at https://github.com/neiljerram/etcd-local-proxy (by starting from and hacking layer-etcd). I wonder if you might take a look at it and see if it looks sane to you? | 13:39 |
| neiljerram | lazyPower, There's also a needed PR for interface-etcd-proxy at https://github.com/juju-solutions/interface-etcd-proxy/pull/6 | 13:40 |
| neiljerram | lazyPower, However, when I build and deploy etcd-local-proxy, and also cs:~lazypower/etcd-19, I'm seeing TLS auth issues, so it's possible there's something wrong in how the certificates are generated, or in the etcd config at either end. Have you done successful end-to-end TLS testing with an etcd master and proxy? | 13:43 |
| marcoceppi | magicaltrout hah, thanks ;) | 13:56 |
| magicaltrout | good talk though | 13:58 |
| magicaltrout | looks like you got a decent crowd | 13:58 |
| lazyPower | neiljerram - i have not. only verified the data exchange on the wire. If you need server certificates, we can do that but its a different dance over the relation wire. It involves generating a CSR and having it signed by the etcd cluster | 14:00 |
| lazyPower | s/cluster/leader/ | 14:02 |
| lazyPower | neiljerram - i think i knwo why you're getting ssl errors as well. the fork of etcd there is generating its own certificates | 14:08 |
| lazyPower | wait no, i jumped the gun i see you removed that from metadata | 14:09 |
| lazyPower | mbruzek - ping | 14:12 |
| mbruzek | Hello lazyPower | 14:12 |
| lazyPower | hey o/ neiljerram has been working with our latest release of etcd, and had some questions. Woudl be good to get your eyes on https://github.com/neiljerram/etcd-local-proxy | 14:13 |
| mbruzek | neiljerram: What TLS issues are you seeing? | 14:13 |
| === narindergupta is now known as narinder | ||
| neiljerram | mbruzek, thanks - first I was seeing (IIRC) 'x509 certificate signed by unknown authority'; then a bit later 'remote error: bad certificate' | 14:41 |
| neiljerram | I will start a redeployment now so I can provide more detail... | 14:41 |
| mbruzek | neiljerram: OK. Can you describe the model? I have got successful etcd to etcd TLS before (not using proxy) | 14:43 |
| neiljerram | mbruzek, the model is etcdctl ----- local etcd proxy ----- remote etcd master | 14:44 |
| mbruzek | And you are running etcdctl on your machine or in the cloud? | 14:45 |
| neiljerram | mbruzek, there is TLS security for the connection between the proxy and the master; but no TLS security between etcdctl and the local proxy. | 14:45 |
| neiljerram | mbruzek, etcdctl invocations are on the same machine where the proxy is running | 14:45 |
| neiljerram | mbruzek, so the etcd master is cs:~lazypower/etcd-19, which is Charles's latest unreleased version of layer-etcd | 14:47 |
| neiljerram | mbruzek, and the etcd proxy is https://github.com/neiljerram/etcd-local-proxy, which is a charm that basically just uses the 'requires' side of interface-etcd-proxy | 14:47 |
| lazyPower | mbruzek - i pushed etcd-19 in my namespace due to the missing etcd-proxy bits. We translated the interface, but missed the reactive code to support that | 14:54 |
| lazyPower | everything but the reactive code in layer-etcd has merged in master of their respective repositories though, so we can rev ~containers once that lands | 14:55 |
| lazyPower | if its ready for that | 14:55 |
| mbruzek | I just diffed etcd-19 and etcd-local-proxy the etcd.py looks quite different. Let me see if I understand correctly, you are connecting the etcd charm with a different etcd-proxy charm and the tls is not working? | 14:56 |
| mbruzek | from etcdctl on proxy and the etcd on the master. | 14:57 |
| neiljerram | mbruzek, That's all correct. I really just used etcd-19 as a structural starting point, but etcd-local-proxy does a lot less, because it doesn't have to do leader election or registration, or generate its own certificates. In principle - I think - it just needs to do the 'requires' side of the 'proxy' relation, save the credentials once they're available, update /etc/defaults/etcd, and restart the local etcd (proxy) service. | 15:05 |
| mbruzek | neiljerram: Yeah that looks right. | 15:07 |
| mskalka | hey everyone, is there any method for pulling the machine ID or INS-ID inside a hook or action? | 15:07 |
| neiljerram | My deployment is up now, and here's what I see: http://pastebin.com/Xs4YLDtY | 15:07 |
| mbruzek | neiljerram: OK please add to the pastebin: openssl x509 -in /tmp/etcd_cert -text | 15:09 |
| mbruzek | neiljerram: That will print out the text of the certificate to make sure the one on your system is valid. | 15:11 |
| mbruzek | I suspect it is, but want to start there. | 15:11 |
| neiljerram | mbruzek, http://pastebin.com/4Dfa2r1V | 15:11 |
| mbruzek | neiljerram: And these addresses are correct ? IP Address:54.164.27.251, IP Address:172.31.51.7 | 15:12 |
| neiljerram | mbruzek, 54.164.27.251 is the public IP of the AWS instance that contains etcd-19. | 15:13 |
| neiljerram | and 172.31.51.7 is that same machine's private IP | 15:14 |
| mbruzek | OK that is good so I believe the certificate is valid and stored correctly on the proxy machine. | 15:14 |
| neiljerram | (the machine itself only knows its private IP) | 15:14 |
| mbruzek | neiljerram: Yep | 15:14 |
| mbruzek | neiljerram: Can you run the following? : etcdctl --debug --cert-file=/tmp/etcd_cert --key-file=/tmp/etcd_key --ca-file=/tmp/etcd_ca and pastebin the result? | 15:17 |
| mbruzek | wait add the "ls /" to the end | 15:17 |
| neiljerram | mbruzek, that seems to work: http://pastebin.com/3ze47iCz | 15:20 |
| === narinder is now known as naridner | ||
| === naridner is now known as Narinder | ||
| mbruzek | sweet! | 15:21 |
| mbruzek | OK so I think the problem is that etcd-proxy reads the defaults file, but etcdctl does not. | 15:21 |
| mbruzek | neiljerram: I am not familiar enough with etcdctl to know how to make those values default, some tools use env variables, other tools have a confiuration file. | 15:23 |
| neiljerram | But I didn't think that the hop from etcdctl to the local proxy should use TLS at all... So I'm a bit confused! | 15:23 |
| mbruzek | neiljerram: Are you sure that etcd-proxy is properly configured with these keys/certs? | 15:24 |
| neiljerram | I was just going to pastebin the proxy's config.... | 15:25 |
| neiljerram | ... http://pastebin.com/ewTBy5mJ | 15:26 |
| mbruzek | I wonder if "PEER" is not valid in this case, let me do some searching | 15:27 |
| mbruzek | Also I know they are in the defaults file, but are we sure that etcd is actually getting the values? Do you see in the etcd logs that the key/cert/ca is read in? | 15:28 |
| mbruzek | https://coreos.com/etcd/docs/latest/security.html | 15:28 |
| neiljerram | mbruzek, Yes, I think so: http://pastebin.com/jhg1HLve (I just added ETCD_CA_FILE and other non _PEER_ settings to the config - but that didn't make any difference to our observations so far) | 15:31 |
| mbruzek | neiljerram: So after doing that, you are still not able to connect using etcdctl without the cert and key flags? | 15:33 |
| neiljerram | mbruzek, Perhaps the etcd proxy doesn't really support TLS on one side (towards the cluster) but not on the other (towards the client). So when it gets a client request it requires TLS auth, even if that client request is over http. Is that possible? | 15:33 |
| neiljerram | mbruzek, Yes, correct. | 15:33 |
| neiljerram | mbruzek, and with those flags, I still _can_ connect | 15:34 |
| mbruzek | neiljerram: I don't know etcd proxy well enough to answer that. | 15:34 |
| mbruzek | neiljerram: Great. Still that is a pretty bad user experience, let me see if I can find more information about etcdctl to see if we can eliminate those flags | 15:35 |
| neiljerram | mbruzek, Do you know who implemented the 'pillowmints' in the etcd charm? I think they support that hypothesis. (They set up env vars for etcdctl.) | 15:35 |
| mbruzek | neiljerram: "pillowmints" was used by lazyPower for lack of a better name. I questioned him about that during a code review. | 15:36 |
| neiljerram | mbruzek, It is a curious name :-) but nice. | 15:37 |
| neiljerram | mbruzek, But it sets up these vars, for local etcdctl usage on the machine where the etcd master node is installed: | 15:38 |
| neiljerram | ETCDCTL_CA_FILE=/etc/ssl/etcd/ca.pem | 15:38 |
| neiljerram | ETCDCTL_CERT_FILE=/etc/ssl/etcd/server.pem | 15:38 |
| neiljerram | ETCDCTL_KEY_FILE=/etc/ssl/etcd/server-key.pem | 15:38 |
| mbruzek | Oh there you go! | 15:38 |
| mbruzek | export ETCDCTL_CERT_FILE and the others and try etcdctl again | 15:39 |
| neiljerram | mbruzek, Yes, that works. But I still don't feel like I understand why. How does TLS auth mix with doing a request over http:// (i.e. _not_ over https://) ? | 15:42 |
| mbruzek | neiljerram: According to your pastebin with debug http://pastebin.com/3ze47iCz the http address is getting converted to https and my guess is that the etcdctl is making the direct https connection | 15:46 |
| mbruzek | neiljerram: etcdctl must be talking http to the proxy but perhaps it is actually talking https to the server as well. | 15:47 |
| neiljerram | mbruzek, Oh! So the proxy isn't really proxying; but rather redirecting! | 15:48 |
| mbruzek | neiljerram: That would be my assessment from reading the pastebin you have there. | 15:53 |
| neiljerram | mbruzek, It appears this is all a behaviour of etcdctl, rather than of the proxy. 'etcdctl --no-sync ls /' works even without those env vars. | 15:57 |
| mbruzek | neiljerram: Interesting | 15:58 |
| neiljerram | mbruzek, Thanks so much for helping me to understand all this. I think I have all the pieces and understanding that I need now. | 15:58 |
| neiljerram | lazyPower, Thanks also. | 15:58 |
| mbruzek | neiljerram: Excellent, glad to help. If you need anything else ping me here. | 15:59 |
| neiljerram | mbruzek, Thanks, I'll do that. | 15:59 |
| mbruzek | I am the one that wrote the TLS layer so it is my responsibility. What I have found is TLS is hard | 15:59 |
| lazyPower | mbruzek i try to help :) | 16:09 |
| === dpm is now known as dpm-afk | ||
| xilet | After a reboot a series of my units are stuck with "agent is lost, sorry! See 'juju status-history glance/0'" The status-history is normal until the reboot and nothing since, any way to kickstart a container back to life? (2.0-beta) | 17:12 |
| === frankban is now known as frankban|afk | ||
| jose | xilet: did IP addresses change? | 18:01 |
| xilet | No | 18:03 |
| cholcombe | reactive is doing some odd things i'm having trouble figuring out | 18:10 |
| cholcombe | i have a base layer that sets a state after it installs everything. in my layer that uses it i @when() that state and for some reason the base layer never installs what it should but the state is firing | 18:11 |
| cholcombe | i r confused :) | 18:11 |
| cholcombe | nvm icey and i figured it out | 18:34 |
| Yacka | It seems like --config has been removed from deploying bundles? | 19:44 |
| Yacka | If I create a bundle for a vanillastack application and want to keep the configuration options separate what's the prescribed way in 2.0? | 19:44 |
| === zeus is now known as Guest31961 | ||
| === Guest31961 is now known as zeus | ||
| Yacka | ERROR Flags provided but not supported when deploying a bundle: --config | 19:55 |
| Yacka | Excellent. | 19:55 |
| Yacka | Why should a deploy of a bundle be any different than a deploy for a charm? | 19:55 |
| Yacka | They use the same command. | 19:56 |
| === mup_ is now known as mup | ||
| kjackal | kwmonroe: thank you for the cleanup on hbase. I will study all the changes and aling my work based on your pointers. Many thanks! | 20:34 |
| === urulama|___ is now known as urulama|____ | ||
| kwmonroe | np kjackal -- it wasn't too complicated, just doing actions a bit better than we used to. i also like the collapsed report_status() handler vs individual handlers per missing state. | 20:56 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!