=== frankban|afk is now known as frankban [07:59] I have short-live access key to aws which have been confirmed working well by aws command line, but fails when deloying juju on aws with this access key. Anyone have idea about this? [08:00] The error says: authentication failed. please ensure the access key id you have specified is correct. === john is now known as Guest88432 === petevg_ is now known as petevg [12:24] bdx: you around? [14:37] has anyone tested the grafana charm? [14:37] ybaumy: just put up a PR against it yesterday [14:37] ybaumy: will show it off a little bit in the juju show today [14:38] rick_h: juju show? [14:38] ybaumy https://www.youtube.com/watch?v=NUx6kYE60Mc&list=PLW1vKndgh8gI6iRFjGKtpIx2fnJxlr5FF [14:38] ep #20 coming in 3.5hrs [14:39] rick_h: is that you? [14:39] ybaumy: rick is me yes [14:39] tim is tvansteenburgh in that last episode [14:40] rick_h: never seen. will watch it tomorow. would be cool if you could get into grafana [14:41] rick_h: i have a real world case which came in this week so i have to set it up for our network guys [14:41] rick_h: they want it for their switches [14:41] ybaumy: cool yea I'm adding MySQL support to it currently so I can build dashboards on data in mysql [14:41] ybaumy: will talk about it in the show today [14:42] rick_h: great will be looking forward to it [14:45] ybaumy: I did a blog post using grafana for monitoring juju controllers a while ago here as well: http://mitechie.com/blog/2017/3/20/operations-in-production [14:49] rick_h: thanks lad [14:53] rick_h: i hope grafana will be the first charm that reaches production for me. curretly im having a testcluster for kubernetes and openstack. kube is fine but openstack i cant get storage tiering really to work which is the problem currently. [14:54] rick_h: first i was planing using influxdb for grafana but i guess mysql is fine too [15:05] I believe we have at least one grafana instance in production backed by influxdb. Most are talking to prometheus. [15:05] i read influxdb is best for performance [15:05] in this case [15:06] thats why i wanted to use it [15:06] but im open for other stuff [15:07] Unless you are going to hit scaling limits, I'd go with whatever fits with grafana best and lets you write your queries easily. [15:07] (I haven't used influxdb, but most of the grafana docs seem to be based around that backend so it is likely the best fit) [15:08] well i have no experience in scaling this application. we have like circa 100 switches in the datacenters [15:10] i still need information on which metrics the network department want to see there [15:10] so i cannot say who may datasources there will be in the end [15:10] if you are just graphing metrics, 100 switches is small scale. [15:11] ok [15:11] thats good to hear [15:11] Something like prometheus talks about tens of thousands of devices with hundreds of time series on them [15:12] Assuming decent hardware, my guess is any backend is good for you [15:12] ok [15:43] time to watch GoT final episode. then wait until 2019 .. i hope i make it there [15:51] morning /o juju world === frankban is now known as frankban|afk [17:34] Juju Show #20 in 30 minutes! [17:34] are you ready? [17:53] juju show link to join in the fun: https://hangouts.google.com/hangouts/_/vamq45vtirbtrefyry63x4ccsee (tvansteenburgh marcoceppi hml bdx kwmonroe and anyone interested) [17:53] the link to watch the stream https://www.youtube.com/watch?v=iSVd7g0I4pI [17:53] ybaumy: ^ [18:01] arosales: aisrael cory_fu ^ going to chat some charming if you can make it [18:06] lol [18:06] yes [18:09] rick_h: thanks for the invite, but I have a conflict [18:26] rick_h: and now this https://bugs.launchpad.net/bugs/1602192 [18:26] Bug #1602192: when starting many LXD containers, they start failing to boot with "Too many open files"

[18:26] rick_h: production lxd: https://github.com/lxc/lxd/blob/master/doc/production-setup.md [18:26] the fix was released for lxd for the too many open files [18:28] rick_h, kwmonroe: nice work, men [18:29] thanks bdx! [18:30] arosales: np, always want to reach out to folks [18:31] bdx: <3 [18:31] rick_h: I appreciate that :-) [18:32] kwmonroe: hah, camera was at 6% battery life left [18:32] woohoo! perfect! [18:32] note to self, camera as webcam means full battery only [18:32] or, ya know, plug it in. [18:33] kwmonroe: :P [18:34] are you guys tracking ^ bug? [18:35] the "too many open files" one [18:35] bdx: haven't thought about it in a long time tbh. What's up? You still getting it? [18:36] look at the latest in that bug [18:36] from stgrabber [18:36] and yea, I have been [18:36] all the while [18:37] so I was excited to be able to apply the sysctl configs (the production lxd sysctl configs that @kwmonroe listed above) [18:37] and see a significant increase in the # of lxd I could deploy via localhost lxd provider [18:38] but the fact is [18:38] I *really* only care about deploying my lxd to MAAS or AWS [18:39] so the production fix is almost useless unless you want to go around hacking face [18:39] :) [18:39] but now [18:40] with the "The verification of the Stable Release Update for lxd has completed " - per that bug [18:40] we should be seeing resolution for the "too many open files" across all providers [18:40] because its fixed in lxd [18:40] if I'm reading this correctly [18:40] bdx: right [18:41] yessssss [18:42] ok [18:42] so [18:43] how can I get that stable verified lxd from 'updates' to be the lxd that gets installed on all my maas/aws nodes? [18:44] so it just means the -updates repos of xenial need to be enabled. Is that out of the box? /me doesn't recall on the cloud images [18:44] I think they are, but that should be all that's needed. [18:45] you're looking for the specified lxd on there when it comes up: [18:45] This bug was fixed in the package lxd - 2.0.10-0ubuntu1~16.04.2 [18:45] rick_h: this http://imgur.com/a/FZ9B5 ? [18:45] needs to include updates? [18:46] bdx: maybe? I'm not sure on maas. My GCE instances I was using for the demo in the show today have them enabled it looks like [18:47] deb http://us-east1.gce.archive.ubuntu.com/ubuntu/ xenial-updates main restricted [18:47] Version: 2.0.2-0ubuntu1~16.04.1 - from apt-cache show lxd on there [18:47] rick_h: OOOoooo [18:47] so it's not quite there...hmmm [18:47] sudo apt list --upgradable | http://paste.ubuntu.com/25433632/ [18:48] run apt update [18:48] might need to be sync'd out to the stuff out there [18:48] then you will get it [18:48] k, cool [18:48] its just sitting there [18:49] I've been waiting so long for this moment ... I don't even want to install it yet [18:49] :) [18:50] lol, the cake is a lie? [18:51] I'm soaking it in man [18:51] might even take a few days off work [18:52] lol, /me feels like bdx got a new toy for christmas [18:52] ok, I need coffee /me walks away to make some [18:54] definitely ..... not being able to get any density from lxd on my maas nodes has been a thorn no doubt [20:01] Hi all.. If I'm running canonical kube, via juju, inside a vm, where I have a bridged networking interface, how do I get the kube to recognise the bridged network i/f as the external IP for ingress etc ? [20:11] "there should be ~400 cores and 2 TB of memory available for your Kube cluster." finally i get to properly deploy some k8s stuff [20:13] wow, jackpot [20:17] tvansteenburgh / rick_h i see you folks "messing" with grafana, if i want resource stats out of CDK, would that be the way to go? [20:17] i'm used to deploying nagios on juju, not messed with grafana yet [20:18] magicaltrout: our internal clusters use prometheus + grafana [20:18] cool, thanks [20:21] BarDweller: we're not ignoring you, i just don't know the answer [20:22] no probs.. got any general advice for configuring ingress external ip's ? [20:22] * BarDweller kube noob, but learning fast =) [20:23] * tvansteenburgh googles [20:24] BarDweller: this might be what you want: https://stackoverflow.com/questions/40136891/gcloud-ingress-loadbalancer-static-ip/40164860#40164860 [20:24] define a Service with the external IP [20:25] hmm.. mebbe.. I'm way off in the weeds reading about the nginx kubernetes ingress controller [20:28] no, strike that, you don't want type LoadBalancer [20:28] https://kubernetes.io/docs/concepts/services-networking/service/#external-ips [20:29] that looks better ^ [20:29] and map that Service to the ingress [20:29] time to figure out how to charm! [20:32] I mean.. I can bring up kube ok, deploy services ok, but the external ip is always blank even if I tell juju to expose it, I think because the ingress thingy doesn't know it's external ip, and the ingress seems to be nginx-ingress-controller, created by the replication controller nginx-ingress-controller which does setup a config map for the controllers it creates.. am reading thru the docs for those to see if I can spot how to tell it to [20:32] use a particular ip/interface for the external side [20:36] BarDweller: right, and I think the way to do that is the create a Service with an externalIPs entry, and put that Service in front of the nginx-ingress-controller [21:30] tvansteenburgh: other question that seems obvious but i shall ask [21:31] for cdk HA Master we just deploy more? [21:32] scratch that found that github issue that claims that is the case [21:40] magicaltrout: affirmative [21:48] okay other random question [21:48] if i'm doing a manual cloud deployment [21:48] how do I add units to a model thats not "default"? [21:50] or when you add unit are they model specific? [21:51] model specific [21:51] nifty