=== frankban|afk is now known as frankban [11:08] Hi, is there a way to NOT use ncurses based UI during Canonical K8s installation? [11:15] gsimondon: depends where you're trying to install it [11:15] if you're doing a local LXD install you'd have some fun [11:16] if you're doing a cloud based install, you can juju bootstrap and then just juju deploy canonical-kubernetes [11:16] you don't have to use conjure up [11:19] doing bare metal [11:20] @magicaltrout all localhost, bare metal, multi node proof of concept installation [11:20] @magicaltrout but I don't want any UIs and I want to automate the process... without hacking around [11:55] sorry gsimondon yeah in that case you can do what I do all the time [11:56] and do a juju manual cloud bootstrap [11:56] and then just place workloads on units [11:56] https://jujucharms.com/docs/2.2/clouds-manual [11:56] that one [11:57] then i usually just look at https://api.jujucharms.com/charmstore/v5/canonical-kubernetes/archive/bundle.yaml [11:57] and "juju deploy" the required services and then "juju add-relation" to join them up [11:57] so if you have 5 baremetal servers you could do [11:57] juju deploy easyrsa --to 1 [11:58] juju deploy etcd --to 1 [11:58] juju deploy kubernetes-master --to 2 [11:58] and so on [11:58] then [11:58] juju add-relation kubernetes-master etcd [11:58] etc [12:12] magicaltrout: gsimondon to be fair, with the 2.3 work you can update the bundle machine numbers and use --use-existing [12:13] to help automate it into a single bundle deploy now [12:15] ah yeah [12:15] forgot about that [12:17] life-easier++ [15:30] rick_h: magicaltrout, thanks a lot. [16:38] magicaltrout: there's also conjure-up in headless mode.. if you specify the spell plus a cloud, c-u will do it's thing with all defaults selected. it will try to bootstrap, but you can also specify a pre-existing controller... so if i have a bootstrapped controller called aws-e, this will do a headless deploy of k8s-core: conjure-up kubernetes-core aws/us-east-1 aws-e [16:38] i know gsimondon already left, so that's just an fyi for you magicaltrout. you're welcome. [16:39] aaaand docs, https://docs.ubuntu.com/conjure-up/2.2.0/en/usage#running-in-headless-mode [16:39] kwmonroe: oooh, more learnings [16:39] lol [16:40] rick_h: put that in your tips & tricks doc ;) [16:40] jk, i know you don't have one. [16:40] kwmonroe: :P [17:03] hey cory_fu, what's a good convention for setting a new key with data_changed('foo', bar)? i'm trying to avoid name collision. do most people prefix the key with the layer name? or does data_changed jam some uuid in front of the key already? [17:08] nm cory_fu, i see https://github.com/juju-solutions/charms.reactive/blob/master/charms/reactive/helpers.py#L144. i will make my own uniqueness. [17:09] though it's nice to know it's somewhat isolated with reactive.data_chagned. [17:23] kwmonroe: Yeah, there's not really a way it could reliably do a uuid, so it's up to you, but it does have the built-in prefix and I'd recommend using the layer name as well [17:24] +1 cory_fu === frankban is now known as frankban|afk [19:57] manual provider question on juju 2.3 [19:57] I can `ssh ubuntu@` from my client machine [19:58] but from that same machine when I try to bootstrap the manual provider I get "Permissioned denied (publickey)" [19:58] any tips in juju 2.3 and manual provider I should be aware of ? [20:04] o/ arosales [20:04] arosales: I only got the last two lines of your post... what is the issue? [20:11] thumper: hello :-) [20:11] thumper: https://paste.ubuntu.com/26531789/ [20:12] arosales: um... did you try to bootstrap manual from within the machine itself? [20:13] nope, client is on my laptop where ssh keys are [20:13] bootstrap node is on DigitalOcean [20:13] I also sshed into the target node and imported my ssh keys just to be sure [20:13] confusing part is I can ssh just fine from the same node I am trying to bootstrap from [20:15] arosales: when adding the ipaddress for the cloud [20:15] try ubuntu@... [20:15] arosales, maybe its using the keys in ~/.local/share/juju/ssh [20:18] from --debug I saw juju grabbing my key from ~/.ssh/id_rsa.pub [20:18] but perhaps it is not actually using it . . . [20:18] I can try to put my key into ~/.local/share/juju/ssh [20:19] @arosales, I wouldnt suggest overriding the key in ~/.local/share/juju/ssh because juju may have installed it elsewhere. Instead, you can try adding ~/.local/share/juju/ssh public key to the node you are trying to manually bootstrap [20:19] in the remote nodes authorized_keys file [20:20] to confirm my understanding though if I can `ssh ubuntu@` then juju should be able to ssh to the same node, correct? [20:24] @arosales, this part isnt clear to me. But as i said if you are trying to troubleshoot. You can *try* to add the pubkey in ~/.local/share/juju/ssh to the authorized_keys file in the remote node. I honestly dont know why you cant ssh there [20:24] via juju [20:27] i ended up in a weird loop back ssh thing arosales a few months ago [20:27] where the remote machine and my local machine had the same key [20:27] and it got very confused [20:28] yosefrow: the target node will not have ~/.local/share/juju/ssh initially [20:29] it will just be stock ubuntu with my ssh key on it [20:29] magicaltrout: what did you do to resolve? [20:30] arosales: it is probably trying to use your name for the SSH key source on the target machine [20:30] @arosales, i meant to try to put the public key from your juju client machine folder ~/.local/share/juju/ssh into ~/.ssh/authorized_keys on the remote mahcine. but its just a shot in the dark [20:30] you only entered the ip address of the machine (which admittingly is all it asked for) [20:30] but I think you need to say ubuntu@59.65.74.150 [20:31] ubuntu@159.65.74.150 [20:31] thumper: ahhh, let me try that [20:34] @arosales, did you manually edit your target nodes ~/.ssh/authorized_keys? [20:34] nevermind i saw earlier that you did [20:34] just juggled my keys around arosales [20:35] thumper: https://paste.ubuntu.com/26531916/ [20:35] thumper: looking for ubuntu on my laptop now . . . [20:35] magicaltrout: gotcha [20:35] @arosales, the issue was you were trying to use a hostname instead of ip? [20:36] I was only using IP [20:36] aaah [20:36] thumper suggested I append a user name [20:37] @thumper, arosales , this line is misleading: Enter the controller's hostname or IP address: ubuntu@159.65.74.150 [20:37] it should probably say ssh login or user@hostname [20:37] seems like a kind of bug to me [20:38] well putting ubuntu@ didn't work for me [20:38] looks for ubuntu on my local machine, which doesn't exist [20:39] @arosales, it looks like earlier you just tried ip and it failed, then you tried ubuntu@ip and then it worked. Is this correct? [20:39] negative [20:40] did _not_ work with ubuntu@ip [20:43] @arosales, try adding the user ubuntu [20:43] to the machine you are trying to add [20:43] its there on the target controller [20:44] I can `ssh ubuntu@` [20:44] from your current user? [20:44] the same user you are using to bootstrap juju? [20:44] correct [20:47] @arosales, juju bootstrap manual/192.168.1.128 mycloud [20:47] https://jujucharms.com/docs/2.3/clouds-manual [20:47] so in your case juju bootstrap manual/159.65.74.150 po1 [20:49] yosefrow: also tried that, and didn't work --- note you have to drop the last arg. In your command "po1" [20:49] why would you drop the last arg ? [20:49] wiki says to include it, doesnt it? [20:50] yosefrow: https://paste.ubuntu.com/26531977/ [20:50] the docs do, but alas [20:50] what version of juju are you using? [20:50] 2.3.1-xenial-amd64 [20:52] docs for 2.3 say Usage: juju bootstrap [options] [[/region] []] [20:52] , [20:52] looks like a 2.3.2 may be out there. I'll see if that has the same results [20:52] @arosales, not $ juju bootstrap 159.65.74.150 manual/159.65.74.150 do [20:52] instead do $ juju bootstrap manual/159.65.74.150 do [20:53] yosefrow: did you see my pastbin? [20:53] yes you have an extra ip [20:53] in there [20:54] ah yes, my syntax error [20:54] same perm error [20:54] patebin? [20:54] https://paste.ubuntu.com/26532007/ [21:00] same error with 2.3.2 and manually adding ~/.local/share/juju/ssh/juju_id_rsa.pub juju-client-key to the authorized_keys file for both root and ubuntu users on the target bootstrap node [21:01] hmmm [21:01] i did this before [21:02] and it was easy. thats why this is stumping me now [21:02] i wish i had documented it [21:02] let me examine the error again [21:02] no worries, yosefrow I appreciate the help [21:03] I can add anyone's ssh key to this server if they want to try and bootstrap [21:03] right now its just a test machine [21:03] in theory juju just need ssh key access to the machine, which I thought it would get from ~/.ssh/ [21:04] obviously I am missing something [21:04] look in ~/.local/share/juju for a file called environments [21:04] environments.yaml [21:05] yosefrow: doesn't exist on my machine [21:11] @arosales, i see an archaic usage of bootstrap-user probably not used anymore in 2.3 [21:12] * arosales nods [21:12] whats your juju client user? [21:13] arosales? [21:14] I believe so, how do I confirm? [21:15] `whoami` [21:15] oh, just on the local machine -- yes "arosales" [21:17] mv ~/.juju to ~/.juju.bak, remove ubuntu user from the remote machine [21:17] lets start over [21:18] remove ubuntu user in remote machine if not needed [21:19] @arosales, [21:19] let me know when you done it [21:20] done, I'll start with a fresh DO instance as well [21:20] sorry meant to say mv ~/.local/share/juju [21:20] to ~/.local/share/juju.bak [21:21] mv ~/.cache/juju to ~/.cache/juju.bak as well [21:22] k, done [21:24] create a new manual cloud specify just the ip of the node u want to be controller [21:26] k, done [21:26] ssh-copy-id user@host-ip [21:26] to add ur key to new bootstrap node [21:26] then ssh user@host-ip [21:28] @arosales, [21:28] which user are you using to login to the remote node? [21:29] root is the only user on the target controller node [21:29] you added the key for root user? [21:30] no I just added my ssh key when making the instance [21:30] that got added to the authorized_keys on the root account on the target machine [21:30] so I can ssh root@ [21:31] i have an idea [21:31] ; [21:33] @arosales,from root@remote-host$ adduser ubuntu; adduser ubuntu sudo; passwd ubuntu (set some passwd),. Then from juju client create a new cloud, add a new cloud, this time use ubuntu@159.65.74.150 instead of just the ip, ok then finally try to bootstrap to the second cloud [21:34] when it bootstraps it will login, then ask for a passwd [21:34] enter the passwd you created in the previous step [21:34] ill brb to see how it goes [21:34] I did that exact thing with https://paste.ubuntu.com/26531789/ [21:35] I then tried ubuntu@ and that also failed per thumpers suggestion [21:36] ubuntu disables login via username/password by default. It seems juju ends up looking for a Ubuntu user locally to sudo to and fails [21:47] @arosales, show me the output from trying to login to the second cloud i said you should make, where the host is not just the ip but ubuntu@ip [21:48] Enter the controller's hostname or IP address: ubuntu@159.65.74.150 [21:48] instead of Enter the controller's hostname or IP address: 159.65.74.150 [21:50] @arosales, last time your output was https://paste.ubuntu.com/26531916/. this time you should get the same output, except now, you should enter the password for ubuntu that you created in the previous step [21:50] it appears that juju is logging in successfully via ssh and then attempting to run a command on the remote host with sudo [21:51] yosefrow: https://paste.ubuntu.com/26531916/ [21:51] great [21:51] enter the password you created [21:51] on the remote host [21:52] arosales: when you're done messing around, use root@ as your cloud. https://paste.ubuntu.com/26532245/ [21:54] thanks kwmonroe that worked [21:54] yosefrow: complained about Ubuntu not being in sudo file, but I think I could have added it and it may have worked [21:55] root worked with out having to mess with Ubuntu [21:55] i'm sure adding the ubuntu user first and doing key mgmnt will probably get ya there, but since DO gives you root and jams your key in there, i say use it! with metldown and spectre, logging in as root across the internet is the least of your worries ;) now where's my $5 arosales? [21:56] kwmonroe: put it on my tab [21:56] ha! [21:56] @arosales, ubuntu complained because you probably skipped `adduser ubuntu command" [21:57] @kwmonroe, has a good workaround, but personally i prefer not directly ssh as root [21:57] yeah yosefrow, i didn't mean to interrupt there, but you guys were messing up my backscroll. [21:57] i'll mess up your backscroll [21:58] lol [21:58] gah! [21:58] kwmonroe, i thought the point of this chat was to demonstrate how to solve juju related issues and other juju related chat [21:58] yosefrow: thanks or sticking with it for me [21:58] should i have answered him in private messages? [21:58] yosefrow: kwmonroe can tell you I am a tough customer [21:58] next charmer summit I'll buy beers [21:58] @arosales, I still think you should do it the way I suggested if you want to follow best practices [21:58] were you guys at config mgmt camp? [21:59] but if you want a quick solution then kwmonroe is 100% correct [21:59] @arosales, wasnt there [21:59] yosefrow: i was totally messing around -- and no, you don't want to get arosales in a PM. what you guys were walking through was all good. [21:59] kwmonroe, ah ok xD [22:00] @kwmonroe, i might write a blog post about this chat soon [22:00] we'll see [22:00] yosefrow: I can see the light at that tunnel, the odd thing was that I had created the ubuntu user and added "ubuntu" to the sudoers via `usermod -aG sudo ubuntu` [22:00] but not dice [22:00] yosefrow: +1, and if you do, arosales will send you $5 [22:00] but most likely fat fingered it. [22:00] kwmonroe: lol [22:01] in IOUB [22:01] @arosales, thats quite strange [22:02] @arosales, ok well in that case nevermind. the cake goes to @kwmonroe [22:02] i was certain that should work [22:02] @kwmonroe, save me a slice of cake [22:04] so it looks like the manual provisioner really wants to add the ubuntu user: https://github.com/juju/juju/blob/develop/provider/manual/provider.go#L30 [22:04] that calls out to https://github.com/juju/juju/blob/develop/environs/manual/sshprovisioner/sshprovisioner.go#L31, which lists the preconditions which will skip that init [22:05] yosefrow: arosales, so i think the key takeaway might be that arosales' ubuntu user needed to have passwordless sudo access. [22:05] anyway, root4life [22:05] @kwmonroe, rootkits4life :P [22:05] lol [22:05] man kwmonroe reading Go code now ;-) [22:06] nah, i just have thumper in a side channel telling me what to type [22:06] you must have stopped drinking early PM brews [22:06] them's fightin words [22:06] rofl [22:06] @arosales, if you want to do with ubuntu user take kwmonroe suggestion and `visudo`. then https://serverfault.com/questions/160581/how-to-setup-passwordless-sudo-on-linux [22:06] i suspected as much as well [22:07] i honestly forgot how i got it to work last time, but probably like that [22:07] @kwmonroe, did thumper get pissed off with me banging my head against the wall trying to help @arosales ? [22:08] that makes sense since I could sudo apt get with the ubuntu user, but failed when Juju tried [22:09] @kwmonroe, arosales anyway that was interesting. Glad we found a solution. laters [22:11] yosefrow: no, thumper doesn't get pissed. he writes go. good time today, have a good one! [22:13] thanks yosefrow, kwmonroe, magicaltrout, and thumper for the help [22:17] * thumper was on calls, soryr [22:18] wb thumper [22:19] stub: Hey, could you update juju-wait on pypi to match the snap, please, when you get a chance?