[03:52] <rmcadams> when you deploy juju, against maas - (juju bootstrap maas controller01) is there a way to specify what server from maas it uses?
[04:37] <stormmore> you could tag the machine in maas and then use --constraints as part of your juju deploy command
[04:45] <rmcadams> duh, why didnt I think of that
[04:45] <rmcadams> good call stormmore
[04:45] <rmcadams> thank youv ery much
[04:45] <stormmore> no prob
[04:49] <rmcadams> watching juju deploy openstack is actually kind of mind bending for anyone who's ever deployed it
[04:49] <stormmore> I remember my first time doing that :)
[04:55] <rmcadams> I can't tell you how many times I've built/rebuild clusters (Piston, RedHat, Canonical etc...) and to watch juju do this... is insane.
[05:01] <stormmore> I am building a kubernetes cluster right now using it
[05:03] <rmcadams> nice
[07:41] <kjackal> Good morning Juju world!
[08:14] <junaidali> Good morning kjackal
[08:14] <kjackal> hello junaidali
[11:49] <cjwatson> I have a weird deployment issue that I don't understand.  Can somebody help?  Freshly-deployed lxd controller fails to deploy with https://pastebin.canonical.com/176151/ even though a different local charm seems to deploy fine; the charm in question is still a work in progress but looks like http://paste.ubuntu.com/23815957/
[11:50] <cjwatson> I believe the "cannot interpret as local bundle" bit is normal (it's not a bundle, and I see this with a successful deployment too).  The weird bit seems to be that it POSTs the charm to its local store and then claims it's not found.
[11:53] <rick_h> cjwatson: is the other one a nested directory like that?
[11:54] <cjwatson> rick_h: Roughly the same structure, but this is the source code structure rather than what the actual charm looks like, and I guess the latter would be more useful to you
[11:55] <rick_h> cjwatson: this seems close, but not from the charmstore, to https://bugs.launchpad.net/juju/+bug/1628786
[11:55] <mup> Bug #1628786: Resources can't be downloaded <rteam> <juju:Fix Released by natefinch> <https://launchpad.net/bugs/1628786>
[11:55] <cjwatson> http://paste.ubuntu.com/23815984/
[11:55] <cjwatson> that looks a bit different, it's not getting as far as uploading resources here AFAICS
[11:57] <rick_h> cjwatson: yea, agree. Local charm, charm isn't found vs resource.
[11:57] <rick_h> cjwatson: the only thing I can think to "try" is to see if you can deploy the charm form the charm directory "juju deploy . --resource ../....."
[11:58] <cjwatson> I could try without the resource first as well I guess
[11:58] <rick_h> cjwatson: but it what you have should be fine, unless something went wrong in the post that's in the controller logs
[11:58] <cjwatson> what should I be looking for in controller logs?
[11:58] <rick_h> cjwatson: fair enough, just to try to narrow down what's up
[11:59] <rick_h> cjwatson: something around getting this post and rejecting it for any reason.
[11:59] <rick_h> 11:36:34 DEBUG httpbakery client.go:246 } -> error <nil>
[11:59] <cjwatson> I see stuff like http://paste.ubuntu.com/23815994/
[11:59] <rick_h> that's about the cookie jar bits used with the macaroon auth
[11:59] <rick_h> cjwatson: so perhaps something is up with auth...not sure
[12:00] <cjwatson> error <nil> looks like AOK though
[12:00] <cjwatson> and I see that in my control case of a successful deployment too - <cjwatson@niejwein ~/src/canonical/launchpad-buildd/charm/charm>$ env -u http_proxy juju deploy --debug --resource=launchpad-buildd=/home/cjwatson/src/canonical/launchpad-buildd/charm/charm/dist/resources/launchpad-buildd/launchpad-buildd.deb ...
[12:00] <cjwatson> ... --resource=python-lpbuildd=/home/cjwatson/src/canonical/launchpad-buildd/charm/charm/dist/resources/launchpad-buildd/python-lpbuildd.deb /home/cjwatson/src/canonical/launchpad-buildd/charm/charm/dist/xenial/launchpad-buildd
[12:00] <cjwatson> oops
[12:00] <cjwatson> http://paste.ubuntu.com/23815999/
[12:01] <rick_h> ah ok cool
[12:01] <rick_h> ignore me then there
[12:02] <cjwatson> deploying from charm directory with no resources, same failure: http://paste.ubuntu.com/23816005/
[12:02] <rick_h> cjwatson: k, yea have to file a bug and see if folks can trace what's different there. Nothing jumping out.
[12:02] <cjwatson> it feels like there must be something wrong with the structure of my charm, but I can't see what
[12:02] <rick_h> cjwatson: has this worked before on other controllers?
[12:03] <cjwatson> this is a new charm, trying it for the first time - so no
[12:03] <rick_h> cjwatson: gotcha, you tried charm proof on it?
[12:03] <cjwatson> no icon.svg but it's otherwise happy
[12:03] <rick_h> k
[12:06] <cjwatson> is there any controller-side debugging that I can quickly bump up to maximum?
[12:13] <stub> cjwatson: Just a wild arse guess, but is there a metadata.yaml in the charm directory and the name: is launchpad-importd ?
[12:17] <cjwatson> there is
[12:17] <cjwatson> aha
[12:17] <cjwatson> 2017-01-17 12:17:01 ERROR juju.worker.dependency engine.go:539 "mgo-txn-resumer" manifold worker returned unexpected error: cannot resume transactions: The dotted field 'launchpad.tar.gz' in 'meta.resources.launchpad.tar.gz' is not valid for storage.
[12:17] <cjwatson> so I need to call the resource something else
[12:17] <cjwatson> I reckon charm proof should complain about that - sound reasonable?
[12:17] <rick_h> cjwatson: +1
[12:22] <cjwatson> https://github.com/juju/charm-tools/issues/300
[12:26] <cjwatson> there, deploy starts doing something once I fix that \o/
[14:56] <Zic> hi here, after rebooting a machine which is part of canonical-kubernetes charms master, "juju status" displayed that its flannel units is in error, I saw that '/run/flannel/subnet.env' is missing if I run a juju debug-log
[14:56] <lazyPower> Zic - Sorry you encountered this, this was a bug in that release of CDK. its been patched in master and is pending a release
[14:57] <Zic> lazyPower: oh, you again, I owe you a pizza :p
[14:57] <lazyPower> Zic - you can resolve the unit in error by juju ssh'ing into that node, and restarting the flannel service manually before executing 'juju resolved kubernetes-worker/#'  where # is the unit # on the node.
[14:58] <lazyPower> simple phaux paus of not enlisting the service as part of the startup chain.
[15:01] <Zic> thanks, I have error in my Grafana endpoints and I presume it was about this Flannel error in my juju status :)
[15:02] <Zic> I confirm it works, and my Grafana is totally repaired :)
[15:03] <Zic> I owe you 2 pizzas lazyPower
[15:04] <lazyPower> Zic Glad we could get it fixed up :)
[15:04] <lazyPower> again, really sorry you were bit by that. regression in our release as of the last CDK publication.
[15:09] <Zic> I was randomly rebooting some etcd & master/nodes to test all was rights since the last time you help me with my etcd quorum which was blocked :) all is working now, restarting Flannel is not a big problem
[15:10] <Prob> juju doesnt work
[15:11] <Prob> says: "Unable to find /home/Peter/.local/share/juju/accounts.yaml"
[15:11] <lazyPower> Prob - Is this a first-run error message? Have you added credentials/bootstrapped a cloud?
[15:12] <Zic> btw, I notice that with the nginx of kube-api-load-balancer, if I poweroff a kubernetes-master, there is no mechanism to pull it down from the upstream at /etc/nginx/sites-enabled/apilb for now?
[15:16] <lazyPower> Zic yeah, we are investigating replacing nginx with haproxy for proper round robin / latency based load balancing
[15:16] <lazyPower> we haven't started the work yet, but there's some early planning docs around it. the current charm in its current form is a stop gap and will be migrated to the haproxy based replacement when its ready.
[15:18] <Zic> lazyPower: oh, it sounds great, I saw an issue in your GitHub which talks about replacing it by HAProxy but I didn't see it was really planned, great!
[15:20] <Zic> when a Juju bundle charms is used, if the mainterner of this bundle switch a technology in one of the charms which compose the bundle, how it works for users?
[15:20] <Zic> maintainer*
[15:21] <lazyPower> Zic well, its up to the charm author to migrate data on charm upgrade. as a user you should only ever have to do juju upgrade-charm $charmname, and potentially at *most* tune some config or run an action to restore services.
[15:21] <Prob> lazyPower, i just followed the steps on the website. i didnt set up anything other than than running the commands here: https://jujucharms.com/canonical-kubernetes/
[15:22] <jcastro> can you do a `juju --version` and tell me what version of juju it is?
[15:22] <lazyPower> Prob ^
[15:22] <Prob> 2.0.2-xenial-amd64
[15:23] <jcastro> does ~/.local/share/juju exist?
[15:23] <Prob> yeah
[15:24] <jcastro> stokachu: any idea here? ^^^
[15:24] <lazyPower> Prob jcastro - one moment, let me spin up a fresh VM and try to reproduce. Our readme instructions changed last week to focus on the conjure-up installation method.  this might be an early-step thing that's unresolved/uncaught.
[15:24] <Prob> just try it yourself, i bet nobody ever tested the localhost deployment
[15:24] <jcastro> https://bugs.launchpad.net/conjure-up/+bug/1656229
[15:24] <mup> Bug #1656229: conjure-up fails on fresh ppc64el xenial machine with 'Exception: Unable to find: /home/ubuntu/.local/share/juju/accounts.yaml <conjure-up:New> <https://launchpad.net/bugs/1656229>
[15:24] <jcastro> I use the localhost one all the time
[15:25] <jcastro> can you check /var/log/conjure-up/combined.log
[15:26] <jcastro> and see if you get that ipv6 error?
[15:26] <Prob> yeah i have the same error as the one in the bug report
[15:26] <jcastro> sudo dpkg-reconfigure -p medium lxd
[15:26] <jcastro> and choose no for ipv6, sec, I have a bug for this
[15:27] <jcastro> https://github.com/conjure-up/conjure-up/issues/495
[15:28] <Prob> i dont understand this weird GUI, what in the world is it doing
[15:28] <Prob> it looks like im supposed to click on configure or deploy
[15:28] <Prob> but i guess not because it says "bootstrapping juju controller in the background ..."
[15:28] <jcastro> yeah use the arrow keys to the deploy buttons
[15:28] <jcastro> and hit enter
[15:29] <jcastro> then when the controller is done deploying it will move on to the kubernetes components
[15:29] <Prob> the blocks keep jumping around
[15:29] <jcastro> yeah that's a progress meter
[15:29] <Prob> no i meant the gui in general
[15:30] <Prob> ok its doing something now
[15:30] <Zic> can I edit /srv/kubernetes/server.[crt|key] to replace it with mine which is signed by a public CA? or juju auto-deploy/replace it?
[15:30] <lazyPower> mbruzek ^
[15:31] <lazyPower> mbruzek - did we add support for self supplied certs? I think we did. but thats been a while ago... and we haven't tested it
[15:31] <lazyPower> since then
[15:31] <mbruzek> Yeah I am not sure that would work, we may have to take a look at the EasyRSA charm.
[15:31] <lazyPower> Zic - i'm going to tentatively say not without some potential issues. we can bug that and get some cycles on testing that path
[15:31] <lazyPower> Zic and if you would bug that, it would be great to get you updates via the bug.
[15:32] <Zic> I can either replace /srv/kubernetes/server.[crt|key] or edit the /etc/nginx/sites-enabled/apilb to an other path which is not managed by EasyRSA, but I imagine this 'apilb' vhost is also "juju-ified"
[15:32] <mbruzek> Zic: https://github.com/juju-solutions/layer-easyrsa/issues please file here.
[15:32] <Zic> thanks
[15:32] <lazyPower> Zic - yeah, all that is under management by the charm.
[15:33] <Prob> how did you make a terminal interface with all these colors and complex gui?
[15:34] <Zic> was just for "fancy", I doesn't really need a publicly-signed cert here, I will just open an issue for wishlisting and see for a next release, thanks anyway :)
[15:34] <Zic> I prefer to not break what Juju do, I am always in PoC-mode and do not want to be too far from upstream configuration
[15:35] <mbruzek> Zic: We modeled the self signed certificate creation and operation of the certificates with the easyrsa charm.
[15:35] <mbruzek> Zic: We may need to modify some of that logic if you bring your own CA
[15:40] <Zic> ok, and last question, I saw that in case I loose the machine or if I destroy (accidentally) the EasyRSA charms, the PKI/CA will be lost, what can I backup to preserve from this?
[15:43] <jcastro> Prob: is it churning through the install now?
[15:46] <mbruzek> Zic: You are correct if you lose the easyrsa charm you do lose pki. You can run some juju ssh or juju scp commands to save the certificates in /var/lib/juju/agents/easyrsa-unit-0/charm/EasyRSA directory. We have not done this work before, so there may be some trial and error needed here
[15:46] <mbruzek> Zic: But I believe it is technically possible.
[15:52] <lazyPower> Zic - can you file a bug rquesting backup capacity for the PKI on easyrsa?
[15:52] <lazyPower> that would be a great contribution
[15:52] <jcastro> I'll file it
[15:56] <mbruzek> Zic: That sounds like a great idea of something to do, we just have not got around to it yet. That could be done with an action. If you file the request here: https://github.com/juju-solutions/layer-easyrsa/issues   We could follow up
[15:58] <Zic> ok, thanks for you answer. jcastro do you file it so? I'm a bit of oldish style and do not have a GitHub account, I maintain my own code on my GitLab locally... but I will be glad to create one just for reporting issue anyway :p
[15:58] <jcastro> filing now!
[15:58] <Zic> thanks
[15:59] <jcastro> but you should make an account so you can follow up, make sure the design of the feature meets your needs, etc.
[15:59] <Zic> I will create an account anyway, as I add #juju as one of my permanent IRC channel :P
[15:59] <Zic> yeah
[16:00] <errr> if I make a charm that uses ansible, can I have the charm run from my ansible master machine that I call my deployment host where all my playbooks are or does it just install a copy of ansible on the host Im trying to provision and run a playbook on what would end up being localhost?
[16:01] <Prob> I HAD TO ABORT THE INSTALLATION BECAUSE MY DISK WAS FULL AND NOW WHEN I TRIED TO RESTART IT NOTHING HAPPENS AND IT'S LOCKED FOREVER
[16:01] <Prob> I CANT DEAL WITH THESE CONSTANT PROBLEMS WITH UBUNTU ANYMORE IM FLIPPING OUT
[16:01] <pmatulis> Prob, ?
[16:02] <errr> http://www.popfi.com/wp-content/uploads/caps-lock-key.jpg
[16:02] <jcastro>  https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/196
[16:02] <lazyPower> errr - we piloted ansible integration in charms, the incantations we came up with are using ansible against localhost
[16:03] <errr> lazyPower: is that the only way or is it possible to have the playbooks run from my ansible master machine?
[16:04] <lazyPower> errr to date, thats the only way. there's an opportunity to hack on  a POC and engage around that style of integration. but i'm not certain what dragons be in that path.
[16:04] <errr> thanks lazyPower :)
[16:05] <lazyPower> errr no problem :)
[16:06] <Prob> for the love of god, throw this weird terminal interface away and make a real one
[16:06] <Prob> a normal one
[16:06] <Prob> the mouse is blinking but nothing happens. how am i supposed to know what is going on?!
[16:08] <Prob> here is a suggestion: write the log in the current directory, so i dont have to search for it. call it installer.log.
[16:11] <pmatulis> Prob, at a very high level, what are you trying to do? are you in the correct IRC channel?
[16:17] <Prob> pmatulis: I'm in the conjure-up installer for kubernetes-canonical
[16:17] <Prob> the installer is REALLY weird
[16:17] <Prob> i click around and then stuff starts to happen
[16:18] <pmatulis> Prob, sounds normal. maybe open a bug with specifics
[16:19] <Prob> my problem is that the installer is horrible in general
[16:19] <Prob> it looks nice on first glance but it's a usability nightmare
[16:19] <pmatulis> again, "horrible" is too vague. just cool down and open a few bugs. do you know where to do that?
[16:24] <Prob> im not gonna open any bugs, you wont fix them anyway
[16:26] <errr> as a software developer who mostly does opensource work I can say thats not how I work and I highly doubt this team does that either
[16:26] <stormmore> How do you know if you don't file the report?
[16:28] <stormmore> (disclaimer: I don't work for Canonical but do believe in speaking up if you have a problem - no IRC isn't ideal for that, just a good place to determine where)
[16:28] <errr> you should take a break, go have a cup of coffee and calm down, then come back and trying asking for some help agian and without bashing the work of the people who will be helping you..
[16:30] <pmatulis> Prob, here you go: https://github.com/conjure-up/conjure-up/issues/new
[16:31] <Prob> what am i supposed to say, "hey, it's not clear whether i should use the mouse or keyboard". The problems start at step -1000
[16:31] <errr> yeah.
[16:32] <errr> dont start with "the installer is horrible"
[16:33] <errr> "Its a bit unclear if this is an interactive installer that requires the use of my keyboard or my mouse.. Does anyone know or should both work?"
[16:34] <errr> try to remember you are talking to other people.. not machines.
[16:35] <errr> there is an old saying my grandpa used to use "you cath more flies with honey than vinegar"
[16:38] <stormmore> shame dear old grandpa didn't learn that you apparently catch more with manure ;-) but otherwise that idiom is awesomely correct
[16:38] <errr> well on the ranch and farm manure is a good thing :)
[16:38] <perrito666> actually you just atract them with manure, honey will make them stick
[16:41] <stormmore> and that is how problems are solved ;-)
[16:43] <perrito666> anyhow, Prob I understan your frustration, software has driven me to the point of wanting to go postal on the devs but please do understand we are doing our best here, which sometimes is not enough, most people here is willing to go along the way to help you solve your problem and perhaps gain some insight on what we did wrong to improve, but please lets not set aside politeness and basic human relation codes
[16:43] <jcastro> I use the arrow keys and enter to navigate the UI
[16:43] <jcastro> it's similar to the curses UI of debconf and other TUI tools
[16:44] <stormmore> but anyway Prob be a detailed as you can, otherwise expect to get questions asking for more detail
[16:53] <Prob> the errors keys dont work properly at all
[16:53] <Prob> i press the sideways error to get to the "deploy" option and it switches to a different screen? what the hell?
[16:53] <Prob> i didnt even press enter yet
[16:53] <jcastro> what terminal are you using?
[16:53] <Prob> bash i guess
[16:54] <jcastro> no like your terminal program, is it the default gnome-terminal or something else?
[16:55] <marcoceppi_> Prob: Since you've had a tough time of it on LXD, we can provide some AWS credentials that allow you to spin up a few machines using conjure-up / juju and offload the stress from your mahcine to the cloud
[16:56] <Prob> i created this issue now: https://github.com/conjure-up/conjure-up/issues/584
[16:56] <marcoceppi_> Prob: we will absolutely be fixing the install experience, but I want you to try kubernetes
[16:58] <Prob> i dont see how that would help a lot, i still have to use this weird installer
[17:00] <marcoceppi_> Prob: well, there are alternatives, as there are to many things
[17:00] <vmorris> does juju-deployer work with juju 2?
[17:00] <marcoceppi_> Prob: if the installer workflow doesn't work for you, you can just use the raw commands underneath that installer
[17:00] <lazyPower> vmorris - its been updated to work with juju-2, yes.
[17:00] <marcoceppi_> Prob: we'd want the installer to make as much sense to everyone, including you, so your feedback thus far has been fantastic
[17:00] <vmorris> lazyPower, thanks
[17:01] <lazyPower> np
[17:02] <Prob> i only made the bug reports because you all seem interested enough to fix it, but i wont be using this juju stuff for now. Doesnt seem to work at all and i dont want to get locked into ubuntu exclusive stuff anyway, i just wanted to quickly try this out and nothing worked, so that's it
[17:02] <tvansteenburgh> vmorris: fyi the latest juju-deployer is in ppa:tvansteenburgh/ppa
[17:06] <jcastro> Prob: do you have a disk with more space on it?
[17:07] <vmorris> tvansteenburgh: oh heh, okay.. going to get this updated soon? https://pypi.python.org/pypi/juju-deployer/
[17:07] <tvansteenburgh> vmorris: yeah, can do
[17:20] <bdx> Prob: you didn't try Juju specifically though, you used a wrapper program that unfortunately gave you grief
[17:20] <bdx> Prob: I suggest just giving juju itself a try, I think you will find it quite pleasant
[17:22] <Prob> i thought this is juju
[17:22] <bdx> Prob: conjure-up is a wrapper around Juju
[17:23] <bdx> Prob: and is a really neat tool, its unfortunate you didn't get the same experience I had with it
[17:23] <Prob> idk it seems like a poorer version of kubernetes, id rather just manage to install kubernetes
[17:24] <errr> the naming in juju and around it remind me of sorcerer linux
[17:24] <bdx> Prob: ehhh ..... if you have your machine provisioned correctly (adequate disk, cpu, and ram), the Juju kub experience is really awesome
[17:24] <errr> was anyone who came up with the name of juju or its other parts like charms and conjure sorcerer linux users?
[17:24] <lazyPower> errr that would be SABDFL himself
[17:24] <bdx> Prob: I can `juju deploy kubernetes-bundle`
[17:24] <lazyPower> i dont think he was.
[17:25] <lazyPower> but who knows, he's had a long walk in life with linux. he potentially did use sorcerer linux.
[17:25] <errr> just the whole casting spells to install stuff is what made me think of that
[17:26] <Prob> bdx, what's the point if i cant run the same thing in production anyway. i just wanted to try kubernetes real quick but in reality i would need to dig deeply into this stuff i think
[17:26] <bdx> Prob: you can though ...
[17:28] <Prob> i typed "juju deploy kubernetes-bundle" and nothing happens for 3 min now
[17:29] <bdx> Prob: my bad ... `juju deploy cs:bundle/canonical-kubernetes-19`
[17:29] <lazyPower> correction on that last one too
[17:29] <lazyPower> juju deploy canonical-kubernetes
[17:29] <lazyPower> is all you need
[17:29] <Prob> that doesnt exactly bode well.. would be nice to know what it is doing
[17:30] <lazyPower> unless you're targeting LXD, in which case that deploy will fail fantasticaly, because there are profile edits on the lxd containers conjure-up is tweaking
[17:31] <mbruzek> Prob: It can do different things on different clouds. I am able to use those commands on AWS and GCE, and Azure, but on your local laptop it requires a few more commands which conjure-up is wrapping. So if you targeted LXD you need to use conjure-up to deploy that bundle. And as you mentioned it does take up disk space to deploy locally.
[17:33]  * rmcadams yawns
[17:34] <stokachu> Prob, im one of the main developers on conjure-up
[17:35] <stokachu> Prob, feel free to file bugs if you have problems and we'll get them addressed
[17:35] <Prob> mbruzek, I'm not targetting anything, im running a console command with no clue what it's doing
[17:35] <stokachu> Prob, i assume https://github.com/conjure-up/conjure-up/issues/584 is you?
[17:35] <stokachu> we've answered those bugs within 3 minutes of you posting it
[17:45] <rmcadams> I <3 Juju
[17:45] <lazyPower> rmcadams we <3 you too
[17:46] <Prob> i still dont get what exactly juju does, is it replacing ansible or kubernetes, is it using docker or lxc? kubernetes only works with docker i think
[17:47] <rick_h> Prob: Juju is doing the work to bring up the components needed to run and operate kubernetes on any substrate that Juju supports, which includes public clouds, private clouds (MAAS/OpenStack), and locally for development/testing (lxd).
[17:48] <rick_h> Prob: if you go through the kubernetes install/setup steps, it's a baked alternative to that how-to basically
[17:48] <rick_h> Prob: it does not replace Kubernetes, it's getting you one that you can use up and running.
[17:48] <rmcadams> +1
[17:49] <Prob> rick_h: that would be good, but it's lock-in right? i can only use this on ubuntu?
[17:50] <rmcadams> CentOS, Windows etc..
[17:50] <rmcadams> RHEL
[17:50] <rick_h> Prob: no, the Juju client works on Ubuntu, OSX, and Windows. You can initiate the deploy from many platforms. If you want another one, it's a compiled Go binary and not too painful to build.
[17:50] <rick_h> Prob: for the deployment end, yes, the current charms (the things doing the deploy/operating of the component) are on Ubuntu.
[17:51] <rick_h> Prob: someone could come along and write those for centos/window if they were inclined
[17:51] <rmcadams> https://cloudbase.it/juju/ is pretty slick
[17:51] <rick_h> Prob: so there's a different between being locked into Juju and locked into Ubuntu. Juju is definitely built and most use is on Ubuntu, but it's far from locked in.
[17:53] <rick_h> Prob: and at the end of the day, you're ending up with VMs on clouds running software. If you decide you want to change/move things it's all there with your ssh access to it.
[17:53] <rick_h> Prob: we think that Juju brings a lot of value in operating things, but it's not like you lose access to your VMs if you decide not to use Juju.
[17:53] <Prob> yeah but now i cant google the kubernetes docs because now it's hidden behind some juju stuff, that's always a danger when you abstract away i guess
[17:56] <jcastro> sure you can, the kubernetes docs include juju
[17:56] <jcastro> once your cluster is up and running you don't really need to use juju unless you're adding hardware or doing backups
[17:58] <rick_h> Prob: yes, abstractions on the one hand help simplify things, and on the other hand feel like they're taking control away. However, Juju as an abstraction hides less than it seems due to the transparency of the charms (open them up and see what they do) and the root level access to the VMs you retain
[17:59] <rick_h> Prob: I mean are you less "locked in" than with kops or puppet, or other install/operation tools
[17:59]  * rick_h runs to a call but let me know if you have any questions or concenrs Prob. Happy to help.
[18:01] <Prob> "juju deploy canonical-kubernetes" does absolutely nothing for me, i added 40gb more to my ubuntu VM now btw
[18:02] <cory_fu> tvansteenburgh: Did you see my update last week on https://github.com/juju/python-libjuju/pull/46 ?
[18:02] <cory_fu> I'd really like to get that simplified example working cleanly
[18:02] <tvansteenburgh> cory_fu: yeah i saw
[18:03] <cory_fu> tvansteenburgh: Any thoughts on why I still get errors?
[18:05] <bdx> Prob: do you have the local lxd provider bootstrapped?
[18:05] <Prob> bdx: no idea
[18:07] <tvansteenburgh> cory_fu: well to put it simply, because it's not a clean shutdown. i was working on it Friday but ran out of time
[18:09] <tvansteenburgh> task.cancel itself is async. it'll return immediately even thought the task may not be stopped yet
[18:09] <tvansteenburgh> so the loop gets stopped while a task is still being exec'd by run_until_complete, which causes a RuntimeError
[18:10] <tvansteenburgh> i have other async programs that shutdown cleanly with Ctrl-C though, so it's not like it can't be done
[18:14] <Prob> juju finally errored out: pc@ubuntu:~$ juju deploy canonical-kubernetes ERROR unable to connect to API: websocket.Dial wss://10.91.98.135:17070/model/4429dd01-9b51-4722-88f6-43dac626c571/api: dial tcp 10.91.98.135:17070: getsockopt: no route to host
[18:14] <Prob> i can reach websites just fine
[18:17] <cory_fu> tvansteenburgh: Yeah, that's kind of what I thought.  I was wondering if you could point me to your other example that works cleanly?
[18:20] <rmcadams> Prob: sounds like your LXD configuration may be not working
[18:21] <Prob> i barely know what LXD is and I certainly didnt configure it to do anything
[18:34] <spaok> anyone know what this error is from? INFO install subprocess.CalledProcessError: Command '['network-get', '--primary-address', 'data']' returned non-zero exit status 1
[18:35] <spaok> I got this when trying to do a juju add-unit
[18:35] <rmcadams> Prob: sorry feeding the kid, be back in a few
[18:38] <lazyPower> spaok what version of juju?
[18:44] <errr> fyi the link in the /topic for the Summit doesnt work http://downforeveryoneorjustme.com/http://summit.jujucharms.com
[18:45] <spaok> lazyPower: 2.0.2-xenial-amd64
[18:45] <ryebot> Is there any means of changing the juju process's runtime environment variables?
[18:46] <ryebot> i.e., I'd like to add LANG=blah to the environment during charm installation
[18:46] <lazyPower> ryebot - only if you add it via config, and set the charm to add that at runtime. Elsewise you'll have to set that ENV var in your layer code.
[18:46] <jrwren> ryebot: afaik, the charm would need to be updated to support this. There are probably some hacks you could do to work around it if you need it only one time.
[18:46] <lazyPower> afaik theres no support for abritrary env vars via model-config or other mechanism than charm hacks.
[18:47] <ryebot> hmm, I -think- I need it outside the charm logic, since I need it set during installation of python packages
[18:47] <ryebot> (from wheelhouse.txt)
[18:55] <lazyPower> ryebot seems like we could add support for that, but it would likely need to be in layer-basic
[18:56] <ryebot> lazyPower: yeah, was just looking in that very direction
[18:56] <ryebot> some kind of "env" config?
[18:56] <lazyPower> yeah, and since thats more than a build time construct, you'd want to add that to config and funnel it down
[18:56] <lazyPower> eg: why do i also get EN if i'm IT
[18:56] <lazyPower> s/also/always/
[19:00] <bdx> lazyPower: can you apply the lxd profile edits needed for kubernetes to deploy to lxd to the lxd profile manually?
[19:01] <bdx> lazyPower: such that you could run `juju deploy canonical-kubernetes` in the model to which you have applied the profile edits needed for kub the kub bundle to deploy to lxd
[19:01] <lazyPower> you can do it manually, but thats a big ask for end users
[19:01] <lazyPower> the template is available in teh spell, 1 sec while i grab you the link
[19:01] <bdx> lazyPower: thx
[19:01] <lazyPower> https://github.com/conjure-up/spells/blob/master/canonical-kubernetes/steps/lxd-profile.yaml
[19:01] <lazyPower> bdx ^
[19:02] <mmcc> bdx: you could probably run the 00_pre-deploy script manually
[19:02] <bdx> mmcc: nice
[19:02] <mmcc> it's in the same dir, that's all conjure-up is doing
[19:02] <bdx> thx
[19:03] <mmcc> it expects an env var "JUJU_PROVIDERTYPE" but just set that to "lxd"
[19:03] <bdx> ok
[19:03] <bdx> mmcc: thx
[19:04] <mmcc> it also expects that the currently selected juju controller/model is the one whose lxd profile you want to change
[19:04] <bdx> entirely
[20:39] <spaok> is there a way to tell juju to add a specific machine as a unit? like if I was doing juju add-unit compute -n1 normally, does juju add-unit --to "MAAS system_id"  work?
[20:40] <mskalka> spaok, the --to works with add unit the same way it does with deploy
[20:49] <spaok> mskalka: I tried to --to against a system_id but it didn't allocate and deploy the node, I've done it before with the hostname but that was 1.25 juju
[20:50] <spaok> I think we found the problem I was having so I'm going to wait for support to call
[20:51] <beisner> spaok, in the past, i've used maas tags and juju constraints to make sure applications landed on the intended metal.
[20:51] <beisner> there may be other ways, but that is an explicit and effective approach
[20:52] <spaok> ya we normally do that, but because I wanted to test not using LACP and only configured one node, and I have a pool of 11 others in ready state, I was trying to target just the one with the active-backup config
[20:52] <spaok> I could mark all the others as broken
[20:52] <beisner> spaok, or add a FOO tag to the 1 machine in maas as a temp/bespoke exercise.
[20:52] <spaok> ya true, good idea
[20:53] <beisner> it seems like there is maas machine uuid 'way' to do this though (i may be dreaming)
[20:53] <spaok> I figured the id was the way since the juju status doesn't show hostnames anymore
[20:53] <spaok> just system ids
[20:54] <jhobbs> the only way to do it on juju2 is with tags
[20:54] <beisner> thx jhobbs :-)
[20:54] <jhobbs> for bootstrap you can use --to <hostname>, but not for anything else
[20:54] <mskalka> huh, TIL
[20:54] <jhobbs> we tag all our machines with their hostnames
[20:55] <beisner> curious jhobbs, do we have a LP bug to track that, even if as a wishlist/feature req?
[20:56] <jhobbs> i don't know beisner, i've been told repeatedly by rick_h it's not a bug
[20:56] <beisner> any time the same object has the same value set to two properties, there be funk.
[20:56] <beisner> i consider that a bug
[20:56] <jhobbs> they don't want to support constraints that only apply to a single provider
[20:57] <beisner> i see
[20:59] <beisner> jhobbs, so i'd ask then:  isn't "tags" a maas-only provider constraint?
[20:59] <jhobbs> https://bugs.launchpad.net/juju/+bug/1554120 we do have a bug for it
[20:59] <mup> Bug #1554120: juju 2.0 bundle support: Missing constraint support for maas names to support bundle placement <juju-release-support> <oil> <juju:Triaged> <https://launchpad.net/bugs/1554120>
[20:59] <jhobbs> feel free to pile on
[21:00] <jhobbs> i agree with you, i think the two fields that need the same value thing is a bigger problem
[21:00] <jhobbs> if they find it's a common enough pattern maybe they'll support it
[21:00] <beisner> right.  i see three power users with the need atm.  thanks jhobbs
[21:01] <stokachu> make that 4
[21:01] <stokachu> but i understand why they dont do it b/c of resuability
[21:01] <stokachu> juju knowing to much about the provider
[21:01] <stokachu> but yes it'd make our use case a lot easier
[21:02] <vmorris> tvansteenburgh, beisner: trying to get juju-deployer to work with a manually bootstrapped cloud, not having much luck getting started. i did add ppa:tvansteenburgh/ppa but juju-deployer doesn't seem to want to update
[21:02] <vmorris> does "apt" not work with ppa's? need to use "apt-get"?
[21:02] <jrwren> no, apt works with PPA
[21:03] <jrwren> vmorris: if you didn't use -u with apt-add-repository, you'll need to apt update to refresh package cache
[21:04] <vmorris> jrwren, okay thanks.. no it's not that, now i'm seeing that the version number is just wrong i think
[21:04] <vmorris> i did install from the ppa after all, but the version number in the command output shows juju-deployer==0.6.4
[21:04] <spaok> beisner: Danny got me fixed, issue was we added new networks to MAAS and the spaces show there but juju didn't know about them
[21:05] <spaok> had to add them
[21:05] <beisner> spaok, sweet.
[21:05] <mmcc> jhobbs: thanks for the pointer to that bug about maas name constraints, I've added the conjure-up perspective (cc stokachu )
[21:06] <jhobbs> cool mmcc
[21:06] <stokachu> mmcc, nice
[21:07] <beisner> jhobbs, ha!  oops, sorry didn't mean to remove your lp tag there
[21:42] <bdx> fyi, you can add network spaces to your controller model, and use them as constraints when you `enable-ha` -> http://paste.ubuntu.com/23818671/
[21:47] <lazyPower> ^
[21:48] <lazyPower> oh, i had a scrollback issue, was ^'ring at beisners comment.
[21:48] <lazyPower> and what bdx said, because thats handy info
[21:52] <rmcadams> is there an easy way to restart a service container with juju?  ie:  RabbitMQ
[21:52] <lazyPower> rmcadams - juju run --application rabbitmq "shutdown -r now"
[21:54] <rmcadams> have I told you that, you are indeed a hero lazyPower ?
[21:54] <lazyPower> rmcadams - dont put me on a pedestal, there's nowhere to go but down from there ;)
[21:54] <lazyPower> but i'm happy i can help.
[21:56] <rmcadams> :)
[21:57] <rmcadams> for some reason, in my lab, this openstack deployment, rabbit is not happy
[22:14] <lazyPower> rabbit does seem to be one of the more finicky charms in the stack. I may be recalling old threads, but it was the one that warranted the most curse words iirc.
[22:26] <Prob> i just dont get it, the juju gui has an option to detroy an application. I click it. It says "this application has been marked to be destroyed on the next deployment". Ok fine, so I write "juju deploy cassandra" butu it says ERROR cannot add application "cassandra": application already exists"
[22:26] <Prob> btw i did not find a cmdline option to destroy an pplication
[22:32] <cmars> Prob, in the CLI, you'd do `juju remove-application cassandra` to get the same effect
[22:32] <Prob> yeah i tried that, didnt do anything
[22:32] <Prob> i also dont understand why it's called "remove" instead of "destroy" like the others
[22:34] <cmars> Prob, when you remove an application, it fires off some events as a result of the change in the model: relations depart and the stop hook eventually fires.
[22:34] <cmars> Prob, sometimes it takes a little bit for the application to completely go away as a result. you can watch `juju status` to see this in action
[22:35] <Prob> why does it say "on next deploy"? what does that even mean?
[22:37] <cmars> Prob, that could be better. it should say "marked to be destroyed when changes are committed" imo
[22:37] <Prob> well i did that already
[22:38] <cmars> Prob, yeah, removing applications in the GUI is awkward, i don't disagree with you. would you be willing to open a bug? https://github.com/juju/juju-gui/issues
[22:39] <cmars> Prob, i see what you mean, you have to deploy to destroy? lol wut? :)
[22:39] <cmars> i think we meant to say "commit" there
[22:40] <Prob> it's not just that either, why is it called "application" but on "juju status" it's called "App"
[22:41] <cmars> well, terminals are only so wide... brevity? :)
[22:41] <Prob> it still isnt deleted, it just wont delete...
[22:41] <Prob> it wont install successfully either, just btw. it stays on "maintenance"
[22:41] <cmars> Prob, any errors when you run `juju status`?
[22:42] <Prob> no
[22:42] <cmars> Prob, that's weird. which charm was this, exactly? do you have the charm URL, or the command you typed to deploy it?
[22:43] <bdx> Prob: sometimes, if applications get "stuck" you might need to result to the command line to resolve removing the application sometimes
[22:44] <bdx> newrelic-sysmond primary I just busted out for controller monitoring https://jujucharms.com/u/creativedrive/newrelic-sysmond-primary/1
[22:44] <bdx> it allows you to do this http://paste.ubuntu.com/23818941/
[22:44] <cmars> Prob, there is a way to hulk-smash an application into oblivion as well. force remove the assigned machine and then the application will surely die
[22:44] <cmars> should be done with care... if you've co-located other applications on that machine, you wouldn't be happy
[22:48] <bdx> Prob: something like `juju run --machine <#> "sudo rm -rf /var/lib/juju/agents/unit-<application-name>-<application-number>"
[22:49] <bdx> Prob: then `juju resolved cassandra/<#> && juju remove-application cassandra`
[22:51] <cmars> Prob, i've opened https://github.com/juju/juju-gui/issues/2331 for ya
[22:54] <Prob> bdx: that worked, but maybe wrap that in a "force remove" button and cli option.. would be good
[22:54] <bdx> true true
[22:58] <rmcadams> yah, Rabbit is really not a happy charm ;)
[23:06] <bdx> rmcadams: it gets so much love though
[23:07] <rmcadams> oh it's not a matter of love, I've deployed the OpenStack bundle now a few times and the only thing I ever have trouble with is Rabbit
[23:07] <bdx> right ...
[23:08] <bdx> rmcadams: are deploying multiple rabbit units on lxd across multiple host using the maas provider?
[23:08] <rmcadams> using maas - it deploys on 4 physicals using lxd for rabbit
[23:09] <bdx> rmcadams: "it" - ?
[23:09] <rmcadams> juju
[23:10] <rmcadams> I'm using Juju against maas.  Juju queues up 4 physical boxes and then slices them up into containers for the services.
[23:10] <bdx> I know there has been alot of work that has gone into making rabbit ha work nicely
[23:11] <bdx> I feel your pain though ... rabbit ha has bitten a few of my deploys ... is it still acting up for you, with the latest version of the charm and maas and juju?
[23:12] <rmcadams> it is
[23:12] <rmcadams> destroying and rebuilding now, to see if I missed something, but I'm 99% sure it's not even HA in the latest version of the bundle
[23:13] <rmcadams> I'm just using the default bundle
[23:13] <rmcadams> for my lab
[23:13] <bdx> rmcadams: my fix, when I was hitting issue with rabbit ha, was to only initially deploy 1 unit of rabittmq, and then add the other units by hand post deploy
[23:13] <bdx> ahh gotcha
[23:13] <bdx> you might want to ensure the bundle you are using has the latest version of rabbit pinned to it
[23:13] <bdx> can you link me to your bundle?
[23:14] <rmcadams> https://jujucharms.com/openstack-telemetry/
[23:14] <rmcadams> sure enough, it's 1 instance of rabbit
[23:15] <bdx> rmcadams: what happens (and what is happening here), is that the bundles aren't always updated with the latest versions of the charms ...
[23:16] <bdx> that bundle might fall behind to some extent bc its not the primary openstack bundle I don't think
[23:16] <bdx> but either way
[23:16] <bdx> pull down the bundle.yaml to your local machine
[23:17] <rmcadams> sure, I've got it
[23:17] <bdx> and make sure the charm versions match the current stable release versions
[23:17] <bdx> e.g.
[23:17] <bdx>     charm: cs:xenial/rabbitmq-server-54
[23:17] <bdx> should be
[23:17] <bdx>     charm: cs:xenial/rabbitmq-server-57
[23:18] <bdx> you should do this for all of the charms
[23:19] <bdx> and we should get bundles added to the openstack charm release cadence
[23:19] <bdx> such that they are kept up to date
[23:20] <bdx> beisner: ^
[23:20] <rmcadams> Yah, that would be nice, for sue.
[23:20] <rmcadams> errr sure
[23:20] <rmcadams> too bad there's not a button on Juju to say "pull latest charm versions"
[23:21] <rmcadams> or "show out of date charms"
[23:21] <bdx> rmcadams: yeah, in the gui there is an 'upgrade-charm' signification of some sort
[23:21] <bdx> but upon initial deploy ... yeah, it would be cool to just say "use latest"
[23:22] <beisner> bdx, we rev up the bundles with each ~quarterly stable release.  beyond that, the "-next" dev bundles will always pull dev/bloody charms
[23:22] <rmcadams> yah, I think you can pick the version after its deployed right?
[23:23] <bdx> beisner: gotcha, nice
[23:23] <bdx> beisner: how does one deploy the -next bundles?
[23:24] <beisner> so that'd be openstack-charmers for stable, and openstack-charmers-next for development (next), in terms of charm store name spaces.  ex:
[23:24] <beisner> https://jujucharms.com/u/openstack-charmers/  vs  https://jujucharms.com/u/openstack-charmers-next/
[23:24] <rmcadams> So here's an interesting question, can you pull down the latest charm on the canvas and just drag/drop it in place and will it assume all of the appropriate relations?
[23:24] <bdx> rmcadams: you can, but you might want the updates from the latest charm for the features that could facilitate the initial deployment of the charm as well
[23:25] <rmcadams> editing the yaml is fine, I'm just curious
[23:25] <rmcadams> also, beisner - how do you pull -dev?
[23:26] <beisner> rmcadams, see bundle yamls @ https://jujucharms.com/u/openstack-charmers-next/
[23:27] <rmcadams> look at that!
[23:27] <beisner> fwiw, we don't leverage charm store channels with the openstack charms yet, as we must continue to test and support production deploys on 1.25.x, and 1.25.x isn't channel-aware.  so, our long-standing use of these two charm store namespaces will be around for some time.
[23:28] <rmcadams> ahhh, that makes sense - we're looking  @ Canonical for one of our next clusters, hence my tinkering... good to know
[23:28] <beisner> keep in mind, -next are dev versions, and as such, are subject to pain.
[23:28] <bdx> beisner: ahh, I see ... so rabbitmq has rev'd 3 times since the last charm release from 54 to 57, thats why 54 is still in the stable bundles?
[23:28] <rmcadams> pain is good ;)
[23:29] <beisner> bdx, correct.   but:  we intend to eventually rev up the stable bundle any time a stable charm revs up.  everything else is auto-magic wrt the charm revs in cs:... just not the stable bundle.
[23:31]  * beisner -> eods
[23:31] <beisner> o/
[23:31] <rmcadams> so given that the bundles are 'zips' which include files other than the yaml, what's the best way to update and import a bundle?
[23:32] <bdx> beisner: nice, thx
[23:32] <bdx> rmcadams: its not a zip ... it just will download as one
[23:32] <rmcadams> ahh, got ya
[23:33] <bdx> the bundle  is basically the bundle.yaml
[23:33] <rmcadams> sure, as an example, openstack has neutron-ext-net neutron-tenant-net and novarc files
[23:34] <rmcadams> so what's the best way to get all that back into a canvas (or command) to make it all work appropriately?
[23:34] <bdx> yea .... those are just helpers for the end user to use to facilitate openstack network provisioniner
[23:34] <bdx> provisioining once openstack is deployed
[23:34] <rmcadams> ahh, roger
[23:35] <rmcadams> just making sure, the whole juju deployment process is new for me, hence my tinkering
[23:35] <rmcadams> I've deployed OpenStack many, many times... just making sure I dont break something by doing it with juju
[23:35] <bdx> I'm not sure how to make the charmstore aware that its a bundle and not a charm ..
[23:36] <rmcadams> I honestly can't imagine how much work has gone into all of this
[23:36] <bdx> rmcadams: possibly the charmstore just recognizes the bundle.yaml and assumes a bundle
[23:36] <rmcadams> having done these deployments myself, as I said yesterday the way it works with juju is mind melting for me
[23:37] <bdx> rmcadams: alot of people care alot about making Juju awesome - thats how
[23:37] <rmcadams> Yah it's pretty obvious, theres a ton of dedication that's gone into this
[23:37] <bdx> rmcadams: yea, I've deployed openstack in a miriad of ways too ... Juju takes the cake hands down
[23:38] <rmcadams> Piston was the easiest openstack I've ever deployed, but it was very "curated"
[23:38] <rmcadams> which made it a pain... but it orchestrated everything for you from the hardware up
[23:40] <bdx> rmcadams: I'm sure you will start to see and appreciate some of the finer aspects of the Juju openstack once you dig in a little deeper
[23:40] <bdx> rmcadams: the openstack charms handle openstack upgrades
[23:41] <rmcadams> oh I'm a big fan of what I've seen so far, the fact that maas handles all that ti does has saved me countless trips up and down the stairs
[23:42] <rmcadams> if only maas could handle multiple hosts running virsh, it would be perfect!
[23:42] <bdx> rmcadams: having the capibility to spin up and down new units of an openstack service at will in lxd containers across your hardware is so key too
[23:42] <bdx> rmcadams: it does
[23:42] <rmcadams> actually it does not, I confirmed it in #maas and we opened a bug
[23:42] <bdx> really?
[23:43] <bdx> link?
[23:43] <rmcadams> https://bugs.launchpad.net/maas/+bug/1656091
[23:43] <mup> Bug #1656091: Power Settings for virsh are not per node, they are global <docteam> <MAAS:New> <https://launchpad.net/bugs/1656091>
[23:45] <bdx> ehh, I don't have a maas at my disposal atm, but this seems strange, I'm sure I've done that before ... possibly it was with an older maas though ....
[23:45] <Prob> why does nothing work out of the box here, i cant even log in: charm push . "test"> ERROR cannot retrieve current username: cannot get discharge from "https://api.jujucharms.com/identity/v1/discharger": cannot start interactive session: cannot get user details for "https://login.ubuntu.com/+id/JTArhzd": not found: not found
[23:45] <bdx> do you use the "add chasis" ?
[23:45] <bdx> Prob: you need a launchpad account
[23:46] <Prob> on the ubuntu page it's telling me that i cant change my username because i have to change it on my launchpad account, so i assume i have one
[23:46] <bdx> Prob: `charm whoami`
[23:47] <Prob> not logged into https://api.jujucharms.com/charmstore
[23:47] <bdx> Prob: `charm login`
[23:47] <Prob> ERROR login failed: cannot get user details for "https://login.ubuntu.com/+id/JTArhzd": not found: not found
[23:47] <bdx> Prob: https://launchpad.net
[23:48] <bdx> create an account there, then use those creds for `charm login`
[23:49] <bdx> rmcadams: from what I remember, if you add a chasis (and specify the virsh creds for that chasis), maas will pick up all the vms provisioned on that chassis* and add them to your nodes
[23:49] <Prob> i did: https://launchpad.net/~trustthesky
[23:49] <Prob> thats me
[23:49] <Prob> created 2011
[23:49] <bdx> rmcadams: I could be entirely off here though too
[23:49] <bdx> Prob: use that username when you `charm login`
[23:50] <Prob> i used the same email
[23:50] <Prob> its not asking for username on 'charm login'
[23:50] <bdx> oooh
[23:50] <Prob> its asking for some kind of 2-factor, i dont have that i think
[23:50] <bdx> Prob: just hit enter to accept the default of 'none'
[23:50] <Prob> yeah i did
[23:51] <Prob> i think it's broken
[23:51] <bdx> Prob: rm ~/.go-cookies
[23:51] <bdx> Prob: ensure you can sign into https://jujucharms.com
[23:52] <bdx> Prob: if you can sign in there^ then the charm command will authenticate
[23:53] <rmcadams> I'll check it in a few, helping son build his trains :)
[23:53] <Prob> bdx: ok suddenly it worked, maybe it was the cookies removal
[23:53] <Prob> and me logging into the launchpad agian
[23:53] <Prob> idk