=== tris- is now known as tris | ||
=== stokachu_ is now known as stokachu | ||
=== cargonza_ is now known as cargonza | ||
=== frankban|afk is now known as frankban | ||
cnf | morning | 08:54 |
---|---|---|
cnf | so anyone around that can help me figure out why juju isn't putting ip's in my containers even though i have the correct constraint on the services? | 08:57 |
wpk | What's the setup? | 08:58 |
cnf | i used to do it fine, it stopped doing it when i upgraded to the latest version | 09:00 |
cnf | wpk: running on MaaS | 09:00 |
wgrant | cnf: It's not adding the extra interfaces to the containers? | 09:01 |
cnf | wgrant: right | 09:01 |
wgrant | cnf: Juju 2.2.0? That's fixed in 2.2.1. | 09:02 |
cnf | wgrant: it _is_ getting ip's from maas... | 09:02 |
cnf | oh | 09:02 |
cnf | so i need to upgrade again | 09:02 |
cnf | can you link the bug? | 09:02 |
wgrant | cnf: https://bugs.launchpad.net/juju/+bug/1698443 | 09:02 |
mup | Bug #1698443: Juju-2.2 does not create interfaces in an LXD for all spaces <4010> <canonical-bootstack> <containers> <cpe> <cpe-sa> <eda> <maas-provider> <network> <juju:Fix Committed by wpk> <juju 2.2:Fix Released by wpk> <https://launchpad.net/bugs/1698443> | 09:02 |
cnf | thanks | 09:02 |
cnf | hmz | 11:25 |
cnf | right, so juju messed up my networking, again | 11:28 |
cnf | >,< | 11:28 |
cnf | so when MaaS configures the machine, it works | 11:40 |
cnf | juju takes over, and i can't even make arp work anymore on that vlan | 11:40 |
cnf | damn machine stops replying to arp requests o,O | 11:51 |
wpk | cnf: do you have console access to that machine? e/n/i would be useful | 11:55 |
wpk | or could you set log level to debug, try to add the machine then and look what's in the log | 11:57 |
cnf | i do | 12:01 |
wpk | cnf: Could you post /etc/network/interfaces and /etc/network/interfaces.(backup...) somewhere? | 12:04 |
cnf | wpk: https://bpaste.net/show/59224fa597dc | 12:10 |
cnf | i have 2 backups | 12:11 |
cnf | and a .new | 12:11 |
wpk | cnf: paste everything if you can | 12:17 |
cnf | wpk: https://bpaste.net/show/4873574782b0 | 12:18 |
wpk | and the .new? | 12:19 |
wpk | also, what's happening when you try to do ifup -a ? | 12:21 |
cnf | https://bpaste.net/show/a13728fa885b | 12:29 |
cnf | sorry if i'm a bit slow, helping a coworker install a VM | 12:29 |
cnf | ifup -a just gives a new prompt | 12:29 |
cnf | no messages | 12:29 |
cnf | exit code 0 | 12:30 |
wpk | yet the machine has no network connectivity? | 12:30 |
wpk | I can't see anything wrong with those files, the bridges seem to be configured correctly | 12:31 |
cnf | wpk: only the vlan 4013 | 12:37 |
cnf | the rest works fine | 12:37 |
cnf | i can see arp requests coming in | 12:37 |
cnf | but no replies going back out | 12:37 |
wpk | cnf: could you remove post-up/pre-down lines from bond0.4013 definition (leave them for br-bond0.4013) and then do ifdown -a; ifup -a ? | 12:44 |
cnf | ok, i'm back :P | 12:53 |
cnf | and if i do a ifdown -a, won't i lose connectivity? PP | 12:54 |
cnf | wpk: ^^ | 12:54 |
cnf | i just rebooted | 13:00 |
wpk | I thought you were using a console of some sort, you probably would :) | 13:02 |
cnf | no, juju ssh | 13:03 |
cnf | it's running on MaaS | 13:04 |
cnf | ok, no difference | 13:06 |
cnf | also, all interfaces have MTU 9100 or 9000 | 13:08 |
cnf | except br-bond0.4013 | 13:08 |
cnf | wait, i just noticed the routes in /e/n/i are wrong | 13:13 |
cnf | and also not what is actually set on the system | 13:13 |
cnf | wt... | 13:14 |
wpk | that is a wtf | 13:14 |
wpk | where is it getting them from then? | 13:14 |
cnf | i have no idea | 13:15 |
cnf | it should not even be touching that subnet | 13:15 |
cnf | apart form that, i can't even ping the gateway on the 4013 vlan | 13:16 |
cnf | it works when maas brings it up | 13:16 |
cnf | juju takes over, and it stops working | 13:17 |
cnf | hmm, and it's not outputting icmp packets, either, on that vlan o,O | 13:18 |
cnf | not even when i ping -I | 13:21 |
cnf | i'm flabbergasted o,O | 13:23 |
wpk | The fact that e/n/i is inconsistent with what's really there is weird for me | 13:27 |
cnf | yes | 13:30 |
* cnf sighs | 13:30 | |
cnf | i'm getting a serious juju burnout... | 13:30 |
wpk | hm, one more idea - could you disable jujud on this machine 'permanently', reboot it and see if e/n/i is consistent with configured networking? | 13:37 |
wpk | systemctl disable jujud-machine-0 | 13:38 |
wpk | cnf: juju might be messing with it, but we have to rule out that something is broken in Ubuntu itself | 14:07 |
=== salmankhan1 is now known as salmankhan | ||
julen | Hi there! does someone happen to know the right API endpoint address for MaaS? | 14:30 |
julen | in the docu it says that http://$ip:5240/MAAS | 14:30 |
julen | but my MaaS controller is not listening on port 5240 for IPv4 | 14:31 |
julen | I also tried http://$ip/MAAS/api/2.0/ but it's also not working | 14:31 |
joedborg | hey everyone | 14:36 |
joedborg | is there a "set_staus()" eqv for old bash charms? | 14:37 |
admcleod | joedborg: i think its just status-set ? | 14:40 |
admcleod | joedborg: https://github.com/juju-solutions/charms.reactive/blob/7322a9bc13ffde1960bcfcb9166e3c09341d27bb/docs/index.rst | 14:40 |
joedborg | cheers @admcleod that looks promising | 14:53 |
admcleod | joedborg: ! | 14:56 |
cnf | wpk: rebooting | 14:59 |
cnf | wpk: still the same | 15:04 |
joedborg | admcleod works perfectly! thank you sir | 15:06 |
admcleod | joedborg: ! | 15:06 |
admcleod | joedborg: Success rate is 100 percent (2/2), round-trip min/avg/max = 4/6/8 ms | 15:10 |
wpk | cnf: and jujud is not running? | 15:14 |
cnf | uhm, a lot of jujud things are running | 15:14 |
cnf | https://bpaste.net/show/7f8d48b33757 | 15:15 |
cnf | is that normal? | 15:15 |
wpk | cnf: have you disabled jujud-machine-0 before rebooting? | 15:16 |
cnf | i did sudo systemctl disable jujud-machine-0 | 15:16 |
cnf | hmm | 15:21 |
cnf | wpk: was that not enough? | 15:25 |
wpk | cnf: hm, it should be... | 15:30 |
wpk | cnf: systemctl list-unit-files \*juju\* | 15:30 |
cnf | juju-clean-shutdown.service enabled | 15:30 |
cnf | jujud-unit-neutron-gateway-0.service enabled | 15:30 |
cnf | jujud-unit-ntp-3.service enabled | 15:30 |
wpk | disable all of those, reboot and see | 15:31 |
wpk | hmm | 15:31 |
wpk | but before that | 15:31 |
wpk | set logging to DEBUG, reboot, and copy logs somewhere (or mail them to me) | 15:31 |
cnf | in syslog? | 15:32 |
wpk | in /var/log/juju/ | 15:32 |
cnf | how do i set logging to debug in juju? | 15:32 |
cnf | especially as i just disabled all things juju? | 15:32 |
wpk | reenable jujud-machine-0 | 15:33 |
wpk | something really bad is happening there, I'd guess that multiple units in LXCs are trying to start up container networking all at once | 15:34 |
cnf | ok, how do i systemd jujud-machine-0 ? | 15:34 |
wpk | systemctl start jujud-machine-0 | 15:35 |
cnf | Failed to start jujud-machine-0.service: Unit jujud-machine-0.service not found. | 15:35 |
wpk | juju model-config logging-config="<root>=DEBUG" should work | 15:38 |
cnf | ok | 15:39 |
cnf | but now juju is still disabled, right? | 15:39 |
wpk | But the units should receive this logging | 15:40 |
cnf | but if juju isn't running, how can it log things? | 15:41 |
wpk | jujud-unit-* are running and will be logging | 15:42 |
cnf | $ systemctl list-unit-files \*juju\* | 15:43 |
cnf | UNIT FILE STATE | 15:43 |
cnf | juju-clean-shutdown.service disabled | 15:43 |
cnf | 1 unit files listed. | 15:43 |
cnf | nothing is running? | 15:43 |
wpk | what about the jujud-unit-ntp3.service and jujud-unit-neutron-gateway-0? | 15:44 |
cnf | no, i stopped them all | 15:44 |
wpk | Ah, ok. Reboot is a way to go then probably | 15:46 |
wpk | I have a suspiction of what might be happening, but having a debug log from the units would be really helpful | 15:47 |
wpk | to see -why- it's happening | 15:47 |
cnf | i'm rebooting | 15:48 |
cnf | ok wpk, what log do you want? | 15:51 |
cnf | machine-0 ? | 15:51 |
cnf | wpk: https://bpaste.net/show/c60636a36174 | 15:52 |
cnf | anything in there? | 15:58 |
cnf | k, wpk it's time to go home here, can I poke you tomorrow? | 16:00 |
cnf | I have meetings till 13:00 CET | 16:00 |
wpk | cnf: Please do. | 16:11 |
wpk | cnf: The only unusual thing in the logs is "2017-07-06 13:27:25 INFO juju.api.common network.go:118 no addresses observed on interface "bond0.4013"" | 16:12 |
=== dames is now known as thedac | ||
cnf | hmm | 16:52 |
=== frankban is now known as frankban|afk | ||
=== scuttle|afk is now known as scuttlemonkey | ||
=== scuttlemonkey is now known as scuttle|afk |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!