| jac_cplane | how do i unset no-proxy env variable in juju 2.2? this was not set in 2.1 - juju model-config no-proxy="" give error ERROR no-proxy: expected string, got nothing | 00:28 |
|---|---|---|
| tvansteenburgh1 | jac_cplane: juju model-config --reset no-proxy | 00:33 |
| === tvansteenburgh1 is now known as tvansteenburg | ||
| === tvansteenburg is now known as tvansteenburgh | ||
| jac_cplane | no --reset will set it to the default value - which is set to localhost | 00:33 |
| jac_cplane | there is no --unset command | 00:34 |
| === jac__ is now known as jac_cplane | ||
| === rogpeppe2 is now known as rogpeppe | ||
| Hetfield | good morning. i have a hook failed: "config-changed" error on a unit. someone did something but i don't know what. is there a sort of log of juju action on a units? | 08:36 |
| === frankban|afk is now known as frankban | ||
| Bringe | Hey, i do have a question about juju. | 11:20 |
| zeestrat | Bringe: Fire away. | 11:22 |
| Bringe | What is the best practice if a machine with active services breaks down, how am i supposed to replace that? I'd like to "redeploy" it without having to build a "new" machine and adding every service to that new machine. | 11:23 |
| Bringe | Is there a way to do that? | 11:23 |
| Bringe | In a private cloud with maas. | 11:25 |
| rick_h | Bringe: so you'd add the new machine and then run add-unit to the applications using --to to specify where to place it? | 11:43 |
| rick_h | Bringe: not sure on how it was setup to start with | 11:44 |
| rick_h | Bringe: but that way relations/config/etc are kept and you're just adding a new unit for things | 11:44 |
| Bringe | I agree, i can do that. Is there no more automated way of saying "put services that were on machine X onto machine Y"? :( | 11:46 |
| digv | juju bootstrap fails Error: ld.so:object 'libeatmydata.so' from LD_PRELOAD cannot be preloaded , Running module ntp failed | 11:48 |
| rick_h | Bringe: no, I suppose that's something someone could play with as a plugin using libjuju to script out a search of what's there andrecreate | 11:48 |
| digv | any idea? | 11:48 |
| rick_h | digv: wow, that's a new one for me. Doing a quick look | 11:50 |
| rick_h | digv: what cloud/series is this? | 11:50 |
| digv | MAAS | 11:50 |
| rick_h | digv: do the machines come up cleanly if you just start them from MAAS? | 11:51 |
| rick_h | digv: what OS are they in MAAS? trusty/xenial/? | 11:51 |
| Bringe | Well thanks for the information, rick_h! | 11:51 |
| rick_h | Bringe: sorry, always wish I could say "yea we can do that" but sometimes, not there yet | 11:52 |
| digv | yes.. it is clean setup with ubuntu xenial | 11:52 |
| Bringe | digv: We encountered a similiar problem, try forcing the kernel to be at least at xenial (hwe-16.04), that fixed it for us. | 11:53 |
| Bringe | rick_h: No problem, we will find a solution. | 11:54 |
| digv | Bringe: ok.. Let me try it quickly | 11:54 |
| julen | Hi there! I'm also having some problem bootstrapping juju with MaaS. It stops at: | 12:12 |
| julen | 13:50:14 INFO juju.juju api.go:67 connecting to API addresses: [10.0.100.47:17070] | 12:12 |
| julen | 14:00:12 DEBUG juju.api apiclient.go:837 error dialing websocket: Forbidden | 12:12 |
| digv | Bringe: It's working.. thanks a lot :) | 12:12 |
| julen | Is it possible to define multiple "no-proxy" addresses as: | 12:49 |
| julen | juju bootstrap machine --config http-prox=http... --config no-proxy=192.168.1.2,10.0.0.2 | 12:49 |
| julen | At the node bootstrapped by juju, I can run "nc -vz 10.0.0.51 17070" and it works, but juju still gets stuck on that step | 12:54 |
| julen | If I make curl to that address I get some binary stuff, so the http-proxy stuff seems right, but the API is still not responding | 12:57 |
| cnf | wpk: poke | 12:58 |
| wpk | cnf: pokeback | 12:59 |
| cnf | \o/ | 12:59 |
| cnf | i'm done with meetings for today | 12:59 |
| cnf | back to fixing the juju thingies | 12:59 |
| wpk | yay! | 12:59 |
| wpk | Could you start 'from scratch'? | 12:59 |
| wpk | kill this machine, then add it, and step by step see what happens? | 13:00 |
| cnf | i can juju destroy-model, and juju deploy, if you want | 13:00 |
| cnf | it'll take a while, booting HP machines isn't fast | 13:01 |
| cnf | wpk: will that work, or did you want something more specific? | 13:01 |
| wpk | destroy-model, add-model, model-config logging-config "<root>=DEBUG", deploy | 13:01 |
| wpk | that should be enough | 13:01 |
| cnf | ok | 13:01 |
| cnf | Waiting on model to be removed, 16 machine(s), 16 application(s)... | 13:02 |
| cnf | ERROR expected "key=value", got "logging-config" ? | 13:03 |
| cnf | juju model-config logging-config="<root>=DEBUG" works | 13:04 |
| wpk | Ah, sorry, right | 13:08 |
| cnf | hmz, juju is being weird | 13:15 |
| cnf | wpk: how do i force remove a model from juju? | 13:19 |
| wpk | what do you mean by 'weird'? | 13:19 |
| cnf | MaaS shows all machines as ready | 13:20 |
| cnf | juju says it can't release the machines | 13:20 |
| cnf | i'm stuck at Waiting on model to be removed, 4 machine(s)... | 13:21 |
| cnf | oh, seems it timed out, and did it anyway | 13:22 |
| cnf | o,O | 13:22 |
| cnf | ok, now we wait half an hour | 13:24 |
| cnf | wpk: machines are up, juju is doing its thing | 13:37 |
| wpk | cnf: cool | 13:44 |
| cnf | wpk: it's almost there | 14:13 |
| cnf | wpk: what's the next step? | 14:13 |
| wpk | cnf: what's the current status? | 14:16 |
| wpk | is the machine dead as it was previously? | 14:16 |
| wpk | well, network-wise dead | 14:16 |
| cnf | yep | 14:16 |
| cnf | on that vlan, anyway | 14:16 |
| wpk | could you copy /var/log/juju/* from it somewhere? | 14:16 |
| cnf | https://bpaste.net/show/21ea96526ca7 | 14:17 |
| cnf | do you need unit-neutron-gateway-0.log and unit-ntp-3.log as well? | 14:17 |
| wpk | cnf: could you also provide output of grep "" /etc/network/interfaces/* and ip a ? | 14:20 |
| cnf | you mean /etc/network/interfaces.d/* ? | 14:20 |
| wpk | /etc/network/interfaces* | 14:20 |
| wpk | all files there, new and backups | 14:21 |
| cnf | ok | 14:21 |
| cnf | you had a / there that made me wonder :P | 14:21 |
| wpk | typo :) | 14:21 |
| cnf | wpk: https://bpaste.net/show/c2b08cb540ab | 14:23 |
| cnf | i still don't get where it gets 172.20.19.248 from, it should not have _antyhing_ in that subnet | 14:23 |
| wpk | and ip a? | 14:23 |
| wpk | and ip r | 14:24 |
| cnf | oh, https://bpaste.net/show/55a4b7746004 | 14:24 |
| cnf | i forgot to link it :P | 14:24 |
| wpk | ok, so I don't see bond0.4013 there | 14:25 |
| wpk | I see br-bond0.4013 but no bond0.4013 | 14:25 |
| cnf | wpk: https://bpaste.net/show/4fe735871db0 | 14:25 |
| cnf | wpk: where? | 14:25 |
| cnf | item 30 in ip a | 14:26 |
| wpk | ah, sorry, yues, it's there | 14:26 |
| wpk | but why master ovs-system? | 14:26 |
| wpk | could you also do brctl show ? | 14:27 |
| cnf | brctl show br-bond0.4013 ? | 14:27 |
| cnf | https://bpaste.net/show/ecb674077e42 | 14:28 |
| wpk | brctl show ovs-system ? | 14:29 |
| wpk | where is this getting from, it's not in the e/n/i anywhere | 14:29 |
| cnf | bridge namebridge idSTP enabledinterfaces | 14:30 |
| cnf | ovs-systemcan't get info Operation not supported | 14:30 |
| cnf | wpk: i'm betting that's a neutron thing | 14:31 |
| wpk | hm, it might be. and it's eating bond0.4013 | 14:31 |
| cnf | hmm | 14:32 |
| wpk | cnf: did it worked on 2.1? | 14:32 |
| cnf | yes, but now that you point to that, some other details changed | 14:33 |
| cnf | let me try some shizzle | 14:33 |
| cnf | i need to destroy everything though | 14:33 |
| cnf | anything you'd like me to check before I do? | 14:33 |
| wpk | cnf: copy of syslog and unit-neutron-gateway-0.log | 14:34 |
| wpk | cnf: and then do your thing | 14:34 |
| cnf | k, deploying again | 14:38 |
| cnf | it'll take half an hour :P | 14:38 |
| cnf | ugh, now juju isn't asking for all the machines it needs o,O | 14:59 |
| cnf | why is it not asking for more machines? o,O | 15:01 |
| cnf | wpk: any reason juju isn't asking MaaS for all its machines? | 15:04 |
| wpk | whaa? | 15:05 |
| cnf | i have 4 machines configured in my model | 15:05 |
| cnf | juju is only asking for 1 from maas | 15:05 |
| cnf | i'm watching the maas logs, it's just not getting any other requests | 15:06 |
| cnf | i can't juju retry-provisioning 0 | 15:06 |
| cnf | because machine 0 isn't in an error state | 15:06 |
| cnf | i see in MaaS | 15:07 |
| cnf | Jul 7 16:58:50 MAAS maas.api: [info] Request from user juju to acquire a machine with constraints: [('tags', ['compute', 'openstack']), ('interfaces', ['0:space=5;1:space=1;2:space=2']), ('agent_name', ['a702d342-3c94-4ff5-8548-c296fb5d2dae']), ('zone', ['default'])] | 15:07 |
| cnf | which is right | 15:07 |
| cnf | but no requests for the other 3 machines | 15:08 |
| wpk | So the machine is in proper state in juju but not it's not provisioned? | 15:09 |
| wpk | (brb) | 15:09 |
| cnf | wpk: https://bpaste.net/show/77d220e2a785 | 15:11 |
| cnf | 3 machines in "pending" | 15:11 |
| cnf | nothing happening | 15:11 |
| cnf | now the 1 machine is deployed | 15:13 |
| cnf | the other 3 are not moving | 15:13 |
| cnf | wpk: so i just ran the same juju deploy openstack.yml again | 15:15 |
| cnf | now it's getting the other 3 machines | 15:15 |
| cnf | O,o | 15:15 |
| julen | Hi there! | 15:20 |
| julen | I'm having a weird problem while bootstrapping | 15:20 |
| julen | the process stops waiting for the websocket, at: | 15:20 |
| julen | INFO juju.juju api.go:67 connecting to API addresses: [10.0.0.52:17070] | 15:20 |
| julen | and then... DEBUG juju.api apiclient.go:837 error dialing websocket: Forbidden | 15:21 |
| julen | I am behind a http-proxy, but I the no_proxy and no-proxy variables are defined and from the shell I can access that IP:port with curl | 15:22 |
| julen | how can it be that curl can connect but juju not? | 15:23 |
| cnf | well, curl isn't connecting to the websocket | 15:25 |
| julen | really? ... but after the request, it gets some stuff... binary, but something comes out | 15:25 |
| julen | the error says "error dialing websocket: Forbidden", but I do see the IP and the port, and the proxy is not blocking it... | 15:28 |
| julen | cnf: from my limited knowledge, I got the picture that a websocket is a port listening to http requests, right? | 15:32 |
| cnf | it's listening to websocket requests | 15:32 |
| julen | so, curl http://10.0.100.52:17070 is equivalent to what juju is trying to do | 15:32 |
| cnf | on the same port as your http request, generally | 15:33 |
| cnf | websockets do an upgrade on a http request, and then become a websocket | 15:34 |
| cnf | your curl is just getting the html/css/js files | 15:34 |
| julen | aha! | 15:35 |
| julen | so, my error might have to do with some settings within the API | 15:35 |
| cnf | it might be you not being authenticated | 15:35 |
| cnf | or a proxy not allowing it | 15:35 |
| julen | hmm... that sounds possible... | 15:36 |
| julen | but, I don't think our http-proxy has such a complex settings, to allow traffic but drop authentication | 15:37 |
| julen | is there any way I could check that? | 15:38 |
| cnf | not authentication | 15:39 |
| cnf | just not allowing websockets | 15:39 |
| cnf | you send a websocket upgrade through the http proxy | 15:39 |
| cnf | it responds with "nope!" | 15:39 |
| cnf | that _might_ be one of the reasons | 15:40 |
| cnf | not saying that is the reason | 15:40 |
| julen | ... and I probably can not check that with some curl request or something, right? | 15:40 |
| cnf | well, use your browser, and enable webdev / debug tools | 15:41 |
| cnf | you should be able to see all requests, and the responses | 15:41 |
| cnf | generally under network you can see the headers sent and received | 15:41 |
| julen | well.. the browser seems to work. I get the headers... | 15:46 |
| julen | the response is just a couple of red dots, I don't know what that means | 15:47 |
| julen | cnf: I have an openstack with nova with same settings and it works fully, so I guess I can discard the option of the proxy blocking the websockets, right? | 15:55 |
| cnf | wpk: prod | 16:15 |
| wpk | cnf: ? | 16:16 |
| cnf | wpk: shizzle works, i'm going home :P | 16:16 |
| wpk | cnf: could you explain what have you done? | 16:17 |
| wpk | cnf: for future generations :) | 16:17 |
| cnf | wpk: changed the data-port: in my model | 16:18 |
| cnf | wpk: so i shuffled some machines around | 16:24 |
| cnf | and my data-port was set to the wrong interface | 16:24 |
| cnf | i still don't know where it gets the 172.20.19 network | 16:25 |
| cnf | or why it wasn't aquireing the machines | 16:25 |
| cnf | but it was ovs claiming an interface that was already provisioned (which i think it should not be allowed to, tbh) | 16:25 |
| cnf | wpk: so i guess it was me being an idiot | 16:28 |
| cnf | anyway, i'm going home | 16:28 |
| cnf | wpk: thanks a lot for your help! and have a nice WE! | 16:28 |
| === frankban is now known as frankban|afk | ||
| Budgie^Smore | o/ juju world' | 18:40 |
| === scuttle|afk is now known as scuttlemonkey | ||
| === scuttlemonkey is now known as scuttle|afk | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!