| Hetfield | hi there. i'm using juju (2.0.2) to deploy an openstack cloud over maas 2.1. each server has 2 eth ports, one for management/internet the other for openstack internal communication. problem is juju picks the "internal" ip instead of the first one. | 08:00 |
|---|---|---|
| Hetfield | looks something like in https://bugs.launchpad.net/juju/+bug/1616098 | 08:00 |
| mup | Bug #1616098: Juju 2.0 uses random IP for 'PUBLIC-ADDRESS' with MAAS 2.0 <4010> <cpec> <juju:Fix Released by dimitern> <https://launchpad.net/bugs/1616098> | 08:00 |
| kjackal | Good morning Juju world! | 08:38 |
| kjackal | Hetfield: you might want to also report this at #juju-dev and #openstack-charms | 08:39 |
| Hetfield | kjackal: i created a separated network space in maas may fix the issue. how to pass it in a bundle.yaml? | 10:22 |
| kjackal | Hetfield: this is for deploying openstack on top of maas, right? | 11:23 |
| kjackal | Hetfield: Is the extra network suposed to be a configuration parameter on a charm? | 11:24 |
| kjackal | Hetfield: You might need to grab the bundle and edit it to put in the parameter | 11:24 |
| cnf | hmm | 11:45 |
| cnf | i just can't get juju working with sshuttle :( | 11:45 |
| cnf | hmm, not sure how to solve this | 11:48 |
| cnf | and juju doesn't seem to understand ssh jumphosts :P | 11:49 |
| kklimonda | bbcmicrocomputer: do you have a minute to talk about contrail charms? | 11:53 |
| cnf | ugh, why can't juju ssh :( | 12:09 |
| cnf | hmz, i don't understand what juju is doing | 12:18 |
| Hetfield | kjackal: i got the yaml | 13:00 |
| Hetfield | but where to put the contraint? i could not find docs about supported params | 13:01 |
| kjackal | Hetfield: I am not familiar with how openstack is deployed so you will have to help me abit to understand | 13:21 |
| kjackal | Hetfield: Is there a parameter in any of the charms on the bundle you are deploying that should/could consume the extra interface? | 13:21 |
| kjackal | Which bundle are you deploying | 13:22 |
| kjackal | ? | 13:22 |
| Hetfield | it's not important which bundle | 13:23 |
| Hetfield | i mean: you can use contraints: tags=storage arch=amd64 | 13:23 |
| Hetfield | i dunno if i can put space=internal there too | 13:24 |
| cnf | now it's just stuck at "Fetching Juju agent version 2.1.1 for amd64" | 13:25 |
| cnf | --debug should really put all tools it uses into debug | 13:25 |
| kjackal | Hetfield: would that help: https://jujucharms.com/docs/2.1/network-spaces ? | 13:26 |
| cnf | hmz | 13:34 |
| cnf | dammit >,< | 13:34 |
| cnf | ok... i think, maybe. i got it working | 13:43 |
| cnf | maybe | 13:43 |
| cnf | how do I remove a deployed charm? | 13:47 |
| kjackal | cnf: juju remove-application | 13:49 |
| cnf | ok, ty | 13:50 |
| cnf | hmm | 13:51 |
| cnf | now i can't deploy juju-gui to the juju controller | 13:51 |
| cnf | hmm, and I have a machine stuck in "0 pending 172.20.20.14 nx6rta xenial default" | 13:53 |
| cnf | maas shows it as ready | 13:53 |
| kjackal | juju gui should be integrated to the controller in juju 2.0 | 13:56 |
| cnf | yeah, it seems so | 13:56 |
| cnf | i'm new to juju, this is my first setup | 13:56 |
| kjackal | cnf: do a juju gui --show-credentials | 13:57 |
| cnf | yeah, _just_ figured that out :P | 13:57 |
| cnf | thanks :P | 13:57 |
| cnf | ok, only took me 5 days to get maas/juju working :P | 13:58 |
| cnf | now i just need to figure out how I clear this one "0 pending 172.20.20.14 nx6rta xenial default" | 14:00 |
| kjackal | cnf: if this is a provisioning issue you could try juju retry-provision | 14:02 |
| cnf | ERROR unrecognized command: juju retry-provision ? | 14:02 |
| cnf | and the machine is down, i basically want it to release it | 14:02 |
| kjackal | cnf: sorry it is juju retry-provisioning | 14:05 |
| cnf | hmm | 14:44 |
| cnf | no, it's stuck in pending | 14:44 |
| disposable2 | i have tried for days to do a juju bootstrap on a maas kvm node. every single time it ends the same way - "Fetching GUI 2.4.3" followed by "ERROR failed to bootstrap model: bootstrap instance started but did not change to Deployed state". i'm watching the kvm node running through ridiculous amount of cloud-init and useless package installations but always after 20 minutes, it ends at this error. if it's a timeout issue, how do i increase the 20 minute t | 15:14 |
| cnf | disposable2: did you run with --debug? | 15:16 |
| disposable2 | more debug info at pastebin.com/vxQ4WEs7 | 15:16 |
| cnf | there is a command to increase the timeout | 15:16 |
| cnf | but 20 minutes i think should be plenty? | 15:16 |
| disposable2 | cnf: i don't understand why the kvm node on ssd is slower than physical nodes. | 15:17 |
| cnf | are you sure it is? | 15:17 |
| cnf | it might be timeouting on something else | 15:17 |
| cnf | can the node reach the internet? | 15:17 |
| disposable2 | cnf: i don't know how to find out | 15:18 |
| cnf | yeah, i just spent 5 days playing to get the right nobs set :P | 15:18 |
| cnf | disposable2: in maas, you can see the ip of the kvm node | 15:18 |
| cnf | ssh to it, and checlk? | 15:18 |
| disposable2 | cnf: as far as i can tell, every time i switch the node on, it gets pxebooted, runs through cloud init and shuts down. | 15:20 |
| cnf | well, ssh to it as soon sas maas says it's up | 15:20 |
| disposable2 | cnf: since the cloud-inits run through fine and don't give me errors i'd say it's got internet access through the maas server (as a gateway) | 15:21 |
| cnf | well, maybe | 15:24 |
| cnf | but why not test? | 15:24 |
| cnf | disposable2: i had to add --config https-proxy=http://<ip of maas controller>:8000 to my deploy | 15:27 |
| cnf | and --config http-proxy | 15:27 |
| === joedborg_ is now known as joedborg | ||
| disposable2 | cnf: well, i've finally logged in (the kvm machine is ridiculously slow) and no, i can't ping anything. the default route is through the maas server but it hasn't got masquerading/nat-ing so traffic isn't passed further. which is weird bcause at the same time i can see cloud-init installing shitton of stuff | 15:36 |
| cnf | pass the --config http-proxy and --config https-proxy | 15:36 |
| disposable2 | cnf: pass that how to what | 15:37 |
| cnf | juju bootstrap --config http-proxy=http://172.20.20.1:8000 --config https-proxy=http://172.20.20.1:8000 dsmaas dsmaas-controller --debug | 15:37 |
| cnf | is what I ran | 15:37 |
| cnf | 172.20.20.1 beeing the maas controller ip | 15:37 |
| disposable2 | there is a field in x.x.x.x/MAAS/settings -> Network Configuration -> Proxy for APT and HTTP/HTTPS. is that the same thing? | 15:38 |
| disposable2 | cnf: i meant http://x.x.x.x/MAAS/settings | 15:39 |
| cnf | no | 15:39 |
| cnf | not that i know, anyway | 15:39 |
| cnf | mind, i am new at this | 15:40 |
| disposable2 | cnf: ok, ihave to wait for the commissioning to stop before i can try it. | 15:40 |
| cnf | uhu | 15:40 |
| disposable2 | cnf: ok, i'm bootstrapping it with those proxy parameters now. in the meantime, would you happen to remember how to increase the timeout value? | 15:55 |
| cnf | --bootstrap-timeout ? | 15:57 |
| cnf | --config bootstrap-timeout | 15:58 |
| jamespage | rogpeppe: hey can you remind me of the dance to promulgate something in the charm store please | 16:00 |
| rogpeppe | jamespage: one mo, let me remind myself first | 16:01 |
| disposable2 | cnf: where do you get thees config options from? man juju is utterly useless. | 16:01 |
| cnf | haha | 16:01 |
| cnf | 16:01 | |
| cnf | and lots of time | 16:01 |
| disposable2 | cnf: i'm ordering a "marco ceppi voodoo doll". every time there's something missing from the documentation i'll poke it with a fork. that'll teach them... years and years with almost no usable documentation. unless that's the entire point - to have to buy server support from canonical. | 16:04 |
| rogpeppe | jamespage: two possibilities: it seems there's a "charm promulgate" plugin (but i don't know where it is). alternatively this command should work: | 16:06 |
| rogpeppe | jamespage: bhttp -j put https://api.jujucharms.com/charmstore/v5/$charmid/promulgate 'Promulgate:=true' | 16:06 |
| cnf | :P | 16:07 |
| jamespage | rogpeppe: that's the one | 16:07 |
| jamespage | how do I get bhttp? | 16:07 |
| cnf | disposable2: i just want to install openstack, that's all | 16:07 |
| rogpeppe | jamespage: go get github.com/rogpeppe/bhttp | 16:07 |
| disposable2 | cnf: i need ceph + openstack cinder | 16:07 |
| rogpeppe | jamespage: you might need to do: cd $GOPATH/src/github.com/rogpeppe/bhttp; godeps -u dependencies.tsv | 16:08 |
| rogpeppe | jamespage: if you've got some older packages still lying around | 16:08 |
| cnf | gl | 16:08 |
| jamespage | rogpeppe: ok that sorted me out - thanks! | 16:10 |
| * jamespage goes to rewrite promulgate-thing | 16:10 | |
| disposable2 | cnf: ok, same result, i'll try later with increaset timeout value. thanks for now | 16:11 |
| disposable2 | s/easet/eased | 16:11 |
| jamespage | rogpeppe: any idea when that might make it into the charm tool itself? | 16:17 |
| rogpeppe | jamespage: cool | 16:28 |
| rogpeppe | jamespage: there is an external (canonical-only) command to do it | 16:28 |
| jamespage | rogpeppe: ftr its "'Promulgated:=true'" | 16:29 |
| rogpeppe | jamespage: oh drat, thanks | 16:29 |
| jamespage | no worries | 16:29 |
| rogpeppe | jamespage: ftr it's documented here: https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md#put-idpromulgate | 16:29 |
| jamespage | ta | 16:30 |
| rogpeppe | jamespage: go get github.com/CanonicalLtd/charmstore-client/cmd/charm-promulgate | 16:30 |
| rogpeppe | jamespage: actually you'll probably have to manually git clone that | 16:31 |
| rogpeppe | jamespage: as go get doesn't like interactive authentication | 16:31 |
| rogpeppe | jamespage: (there's a useful git config setting to make it work) | 16:31 |
| lutostag | sys.path.insert(0, os.path.join(os.environ['CHARM_DIR'], 'lib')) | 17:08 |
| === hml_ is now known as hml | ||
| Budgie^Smore | o/ juju world | 19:04 |
| Budgie^Smore | hey lazyPower am I reading this right, running juju deploy canonical-kubernetes will update an existing cluster? | 19:08 |
| === Budgie^Smore is now known as stormmore | ||
| stormmore | ok I just ran juju upgrade-juju -m controller and now I keep getting permission denied errors, just how screwed am I? | 19:45 |
| stormmore | apparently not much, updated my client system and reran the upgrade and now it behaving | 19:47 |
| stormmore | well that seems to have worked nicely :) | 19:59 |
| marcoceppi | stormmore: yes, you are correct (re juju deploy) it'll upgrade the charm revisions if that's where you started your k8s deploy | 20:09 |
| stormmore | thanks marcoceppi, I just updated my dev cluster | 20:14 |
| === tbc is now known as Tim | ||
| === Tim is now known as tbc0 | ||
| === tbc0 is now known as tbc | ||
| stormmore | hmmm now I am getting "an error on the server ("unknown") has prevented the request from succeeding " when I try and look at the logs in the k8s dashboard | 21:22 |
| lP|lappy | stormmore: cdk or kube-core? | 22:30 |
| stormmore | lP|lappy, cdk | 22:31 |
| lP|lappy | stormmore: this is a manifestation of the api-load-balancer i suspect. The layer7 proxy causes a lot of not-obvious errors like that. | 22:31 |
| lP|lappy | stormmore: we have a spike coming this week or next to hack on a layer haproxy replacement to bring that down to a layer4 proxy so it can talk SPDY | 22:31 |
| stormmore | :-/ yeah and now after a test reboot of one of the nodes, containerd isn't starting :-/ | 22:31 |
| lP|lappy | stormmore: make sure you're capturing these as issues so we can address them with minimal pain on your side. I've done some reboot tests and the units survived, but that doesn't mean there aren't hidden dragons in there | 22:32 |
| stormmore | lP|lappy, you know me, first "try" and determine if the issue is my side or yours :P | 22:33 |
| lP|lappy | this is true :) | 22:33 |
| lP|lappy | but this is also my mantra | 22:33 |
| lP|lappy | i'd rather close bugs than never know they exist. | 22:33 |
| stormmore | lol metrics at their finest :P | 22:33 |
| === hml_ is now known as hml | ||
| lP|lappy | i honestly wish i could instrument the charms like that | 22:35 |
| lP|lappy | but that also feels like a quick way to lose users | 22:35 |
| stormmore | lP|lappy, this is the error Docker is giving me: FATA[0001] Error starting daemon: error initializing graphdriver: driver not supported | 22:37 |
| stormmore | which seems odd as it was running just "fine" before the reboot | 22:37 |
| lP|lappy | stormmore: is there a "NONE" in /etc/default/docker? | 22:40 |
| lP|lappy | stormmore: i ran into a stray bug i wasn't able ot reproduce last week where a None got rendered in th edefaults file | 22:40 |
| stormmore | Only line worth mention is DOCKER_OPTS=" " | 22:44 |
| stormmore | looks like an issue with the aufs driver | 22:59 |
| stormmore | "modprobe: FATAL: Module aufs not found in directory /lib/modules/4.4.0-65-generic" looks suspiciously like my problem! | 23:01 |
| stormmore | looks like the updates I ran didn't update the linux-image-extras package! | 23:05 |
| Budgie^Smore | love when I am multitasking trying to setup the office network while also troubleshooting cdk! | 23:10 |
| stormmore | OK figured out what the problem was, just needed to install the correct linux-image-extra package | 23:47 |
| stormmore | I am surprised that that doesn't get updated by apt update / upgrade though | 23:48 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!