| Razva | good mornin' folks. why in the world is MAAS running...two DHCP servers...? | 08:15 |
|---|---|---|
| Razva | should I use 14.04 LTS for Ubuntu Cloud Images? or 15.10? | 08:19 |
| haasn | Razva: ps aux | grep dhcp | curl -sF "file=@-" https://0x0.st | 08:35 |
| Razva | mornin' haasn | 08:35 |
| Razva | https://0x0.st/X0i.txt | 08:36 |
| Razva | haasn at this point I've reinstalled Ubuntu | 08:36 |
| Razva | just a sec to setup networking | 08:36 |
| haasn | well that machine is clearly running 0 DHCP servers :p | 08:37 |
| Razva | yup, right now yes | 08:37 |
| Razva | can you please take a look at https://github.com/Razva/newbadmin/blob/master/firewall/nat.sh ? | 08:38 |
| Razva | it's a script one of the good folks here provided for NAT | 08:38 |
| Razva | I've slighly modified it | 08:38 |
| Razva | looks ok? | 08:38 |
| Razva | (for NAT) | 08:38 |
| haasn | Razva: seems reasonable, not sure if the MASQUERADE line would benefit from an extra -i "$INTIF" or not but if it works it works | 08:51 |
| Razva | any idea how to retain this after reboot...? :| | 08:53 |
| haasn | Razva: iptables-save and iptables-restore can be used to save the current settings to a file (and do the opposite). Most distros provide some sort of init script for automatically doing that | 08:54 |
| Razva | got it | 08:55 |
| Razva | haasn: https://0x0.st/X0K.txt < that ps aux you've requested | 08:56 |
| Razva | seems that only MAAS is running, right? | 08:56 |
| haasn | seems to only be running one | 08:56 |
| haasn | on interface enp0s25 (this is your private interface, I take it?) | 08:57 |
| Razva | weird, last time it was i*something-dhcp-server running also... :| | 08:57 |
| Razva | exactly, it's the lan | 08:57 |
| Razva | now, let's try to PXE boot | 08:57 |
| haasn | Razva: when stuff like that happens it's a good idea to look at the -f option, in this case /var/lib/maas/dhcpd.conf | 08:57 |
| haasn | If that's /etc/dhcp/dhcpd.conf or something like that then it's probably some “default” system-wide dhcpd service running accidentally | 08:57 |
| Razva | do you recommend 14 LTS or 15, for Ubuntu Cloud? | 08:57 |
| haasn | I have no idea. I use 14 LTS because I haven't had a reason to change | 08:58 |
| Razva | does Liberty works with 14? | 08:59 |
| Razva | yup, it successfully booted, no idea why last time it basically started two DHCP servers...it was a fresh install... | 09:00 |
| Razva | haasn except "Allow SSH access and prevent machine from powering off", should I check the other two boxes? | 10:14 |
| Razva | Retain network & Retain storage? | 10:14 |
| haasn | What those checkboxes mean is that if you're commissioning a node that you've already comissioned in the past, any storage/network configuration you made manually will be preserved instead of reset back to the default | 10:15 |
| haasn | If this is your first time commissioning that node they do nothing | 10:15 |
| Razva | aaand if I didn't ever commisioned them...I suppose that I should just "SSH and prevent machine from powering off", right? | 10:15 |
| haasn | I don't see any reason to set that unless you're trying to debug something that went wrong | 10:17 |
| Razva | yay, ok | 10:18 |
| Razva | the only "wrong" thing until now is that MAAS is seeing "power on" even if the machine is 100% powered off :| | 10:19 |
| Razva | any idea why> | 10:20 |
| haasn | Razva: the power status is not checked all the time but polled at regular intervals (every few minutes), you can click ‘check now’ to force an update | 10:21 |
| Razva | haasn did it 3 times, they still point as being on | 10:21 |
| Razva | Checking power -> Power on | 10:21 |
| haasn | Razva: What power type? | 10:21 |
| Razva | even tried to "shutdown" them from MAAS | 10:22 |
| Razva | IPMI. they are Dell servers | 10:24 |
| haasn | Razva: try asking ipmipower -D LAN_2_0 -h HOST -u maas -p PASS --stat what it thinks of your host | 10:25 |
| Razva | Power StateOFF | 10:26 |
| Razva | haasn any idea what's the pass? :)) | 10:26 |
| haasn | Razva: you can check in the maas GUI | 10:27 |
| haasn | It should have created a new account/password when enlisting | 10:27 |
| haasn | If ipmipower works but MAAS does not, seems like a bug to me | 10:27 |
| Razva | got it, just a sec to see | 10:27 |
| Razva | HOST = IP or host.maas ? | 10:28 |
| haasn | It should be the same hostname as in the maas power configuration | 10:28 |
| Razva | paltry-cave < who in the world is generating these names? :) | 10:29 |
| Razva | 95.154.222.13: connection timeout | 10:30 |
| Razva | interesting. | 10:31 |
| haasn | Razva: you could also try logging in via SSH | 10:36 |
| Razva | I'm power cycling both machines, see what's the result | 10:38 |
| Razva | long live iDRAC :) | 10:39 |
| Razva | the machines are not even commisioned, so I cannot really SSH in... | 10:39 |
| Razva | oook both are not booting via PXE | 10:40 |
| Razva | ssh key bla bla... | 10:42 |
| Razva | both are waiting here: http://screencast.com/t/LvFwpbHIaBnK | 10:43 |
| Razva | ^ is that "normal"? | 10:43 |
| Razva | aaaand they seem to be stuck at that point :| | 10:44 |
| haasn | Razva: SSH to the service processor (IPMI interface), not the host OS | 10:50 |
| Razva | SSH seems to be working, but I cannot login | 10:54 |
| haasn | Razva: with the maas username/password? | 10:57 |
| haasn | what about root? | 10:58 |
| Razva | haasn yup, with the maas u/p, I cannot login | 11:02 |
| Razva | I'll try to login with root now | 11:02 |
| Razva | 95.154.222.13: connection timeout < from ipmipower | 11:02 |
| Razva | root@95.154.222.13's password: < ssh works | 11:03 |
| Razva | /admin1-> < root worked | 11:03 |
| Razva | any hints now? :D | 11:03 |
| haasn | Razva: set the root un/pw in maas and be done with it? :p | 11:12 |
| haasn | oh, connection timeout from ipmipower with root p/w too? | 11:13 |
| haasn | Razva: try ipmitool -I lanplus -U root -P ... -H ... user list | 11:13 |
| haasn | If ssh works but ipmipower does not, then you might need to specify a firmware-specific workaround flag (via -W) | 11:14 |
| haasn | e.g. I need -W sun20 for some of my old sun servers | 11:14 |
| Razva | haasn exactly | 11:17 |
| Razva | haasn but I cannot even reset IPMI | 11:17 |
| Razva | ipmitool -H 95.154.222.13 mc reset warm | 11:18 |
| Razva | Password: | 11:18 |
| Razva | Error: Unable to establish LAN session | 11:18 |
| haasn | did you try the ipmitool command I suggested? | 11:18 |
| Razva | nope, just a sec | 11:18 |
| Razva | ipmitool -I lanplus -U root -H -H 95.154.222.13 user list | 11:19 |
| Razva | Password: | 11:19 |
| Razva | Address lookup for -H failed | 11:19 |
| Razva | Could not open socket | 11:19 |
| Razva | (same with minus the extra -H) | 11:20 |
| Razva | ipmitool -I lanplus -U root -H 95.154.222.13 user list | 11:20 |
| Razva | Password: | 11:20 |
| Razva | Error: Unable to establish IPMI v2 / RMCP+ session | 11:20 |
| Razva | but I can SSH login... | 11:21 |
| Razva | sh root@95.154.222.13 | 11:21 |
| Razva | root@95.154.222.13's password: | 11:21 |
| Razva | /admin1-> | 11:21 |
| haasn | Razva: maybe something here will help? https://www.gnu.org/software/freeipmi/freeipmi-bugs-issues-and-workarounds.txt | 11:26 |
| haasn | also see man ipmipower (scroll down to workarounds) | 11:27 |
| Razva | I've contacted the data-center...maybe they can do a power-cycle/reset... | 11:31 |
| haasn | doubt it will help | 11:31 |
| Razva | its their server, so I should be able to list users...at least. | 11:36 |
| Razva | hey guys. I have no idea why but maas refuses to interpret the right status from IPMI... | 16:52 |
| Razva | the server is powered down, I can see that via DRAC and console.. | 16:52 |
| Razva | but still maas reports the server as being online. any hints...? | 16:52 |
| redelmann | hi, im having some trubles with unmanaged networks | 17:23 |
| redelmann | im trying to let my local dhcp and dns to manage this network | 17:23 |
| redelmann | but for some reason /etc/resolv.conf is setting maas ip address as DNS | 17:24 |
| redelmann | i double check my local dhcp setting and everything seems fine | 17:24 |
| redelmann | so, why maas is putting herself as DNS of an unmanaged network? | 17:25 |
| Razva | redelmann no idea, but that's what maas does (at least on my setup) | 17:28 |
| redelmann | Razva, ok, thank. It little strange, i was supposing that unmanaged network is UNmanaged :P | 17:30 |
| redelmann | Razva, i think juju bootstrap fail in my setup because of this | 17:30 |
| Razva | redelmann I have a very weird issue. MAAS reports power on, even if power is off (I can see in iDRAC that power is off, and console shows No power) | 17:40 |
| redelmann | Razva, it's that for me? | 17:41 |
| Razva | redelmann yes | 17:42 |
| Razva | maybe you have any ideas... | 17:42 |
| redelmann | Razva, noup. | 17:43 |
| redelmann | Razva, im playing with maas for 2 weeks to now | 17:43 |
| redelmann | Razva, so im not the most indicated person | 17:44 |
| Razva | haha, I'm playing for ehm...2 days :D | 17:44 |
| Razva | I've successfully setup 3 machines, everything PXE boots, but this single darn machine doesn't wants to boot.... | 17:44 |
| Razva | or whatever...boots but doesn't wants to report power as it should | 17:45 |
| redelmann | Razva, maybe hardware problem? | 17:45 |
| Razva | no idea... | 17:46 |
| Razva | do you use IPMI or...? | 17:46 |
| redelmann | IPMI and virsh | 17:46 |
| redelmann | i'am mixing virtual nodes with intel hardware | 17:47 |
| Razva | I have only intel / Dell hardware, but no idea why the heck I cannot control it... | 17:47 |
| redelmann | im just testing maas for a full implementation | 17:47 |
| Razva | well...I didn't had any issues with it until now, only problem was that - at first install - Ubuntu installed a separate DHCP server as well. no idea why, but a clean install solved everything | 17:48 |
| Razva | Power on < I'm gonna kill somebody. | 17:49 |
| redelmann | Razva, you are installing MAAS from apt? | 17:49 |
| Razva | the server is off god damnit... | 17:49 |
| redelmann | Razva, try with "check now" link :P | 17:50 |
| redelmann | Razva, or manually power on the server | 17:50 |
| Razva | yeah if I manually power on it works, but this is not the way to do it... | 17:50 |
| redelmann | Razva, after manually power on, swith to off with maas | 17:50 |
| Razva | how...? | 17:51 |
| Razva | I cannot manually set power off in maas | 17:51 |
| redelmann | Razva, i know but maybe yoy can "sync" what maas think about node and rearl node state | 17:51 |
| Razva | please do me a favor | 17:51 |
| redelmann | Razva, take action -> power off? | 17:51 |
| Razva | ipmipower -D LAN_2_0 -h SOME-IPMI-IP -u root -p THE-ROOT-PASSWORD --stat | 17:52 |
| Razva | does it works for you? | 17:52 |
| redelmann | Razva, im in home rigth know. | 17:55 |
| redelmann | Razva, i dont know why cant ssh to cluster | 17:55 |
| redelmann | Razva, give me a second | 17:56 |
| redelmann | Razva, but usually thats the command i use to check the whole cluster power status without problem | 17:57 |
| redelmann | Razva, i never use -D, only -h ip | 17:58 |
| redelmann | Razva, did you try with --debug? | 18:01 |
| Razva | well...what command do you use, usually? | 18:01 |
| Razva | (ipmipower_powercmd.c, _retry_packets, 1214): host = 95.154.222.13; p = 1; Sending retry, retry count=24 | 18:03 |
| Razva | redelmann are you deploying OpenStack or just MaaS? | 18:09 |
| redelmann | Razva, just MAAS | 18:09 |
| redelmann | Razva, ipmipower -h ip -u root -p THE-ROOT-PASSWORD --stat | 18:09 |
| redelmann | Razva, i let ipmipower to decide ipmi version | 18:09 |
| Razva | 95.154.222.13: connection timeout | 18:10 |
| Razva | but the interesting thing is that I can ssh to it | 18:10 |
| Razva | /admin1-> | 18:11 |
| redelmann | Razva, ipmipower --version? | 18:12 |
| redelmann | Razva, some old ipmipower utilitis had a bug with -p options i think | 18:13 |
| Razva | redelmann via SSH? | 18:14 |
| redelmann | Razva, no, just ipmipower --version in machine you are trying to connect | 18:14 |
| redelmann | Razva, maas server i suppose | 18:15 |
| Razva | ipmipower - 1.1.5 | 18:15 |
| Razva | you? | 18:16 |
| redelmann | Razva, look at this: https://lists.gnu.org/archive/html/freeipmi-users/2005-11/msg00018.html | 18:16 |
| redelmann | Razva, its old | 18:17 |
| redelmann | Razva, forget about it, ipmipower 0.1.3 its the problem | 18:17 |
| redelmann | Razva, ipmipower - 1.1.5 | 18:18 |
| roaksoax | Razva: do you have logs from /var/log/maas/ , it is probably telling why it cannot connect to IPMI | 18:18 |
| roaksoax | if that's the case | 18:18 |
| roaksoax | redelmann: maas doens't really support external DHCP/DNS anymore. But eventhen, MAAS is the defautl DNS server unless you set it differently for each subnet | 18:20 |
| redelmann | Razva, did you try ipmiping <IP> | 18:21 |
| redelmann | roaksoax, http://paste.ubuntu.com/15060537/ | 18:22 |
| redelmann | roaksoax, i set DNS on my maas-eth0 | 18:22 |
| roaksoax | redelmann: what version of MAAs are you using? | 18:22 |
| redelmann | roaksoax, it's that no enough? | 18:22 |
| roaksoax | 1.9 | 18:23 |
| redelmann | roaksoax, 1.9 | 18:23 |
| redelmann | roaksoax, it's testing purpose | 18:23 |
| roaksoax | redelmann: right, so when you deploy, is the interface of the machine configured to do what? DHCP ? | 18:23 |
| redelmann | roaksoax, exactly | 18:24 |
| roaksoax | redelmann: right, so DHCP should be setting the DNS server right? | 18:24 |
| redelmann | roaksoax, rigth | 18:24 |
| roaksoax | redelmann: also, can you get the curtin config ? | 18:24 |
| roaksoax | redelmann: maas <session> node get-curtin-config <system-id> | 18:26 |
| redelmann | roaksoax, ohh THANK, i was losing my head | 18:26 |
| redelmann | roaksoax, http://paste.ubuntu.com/15060721/ | 18:28 |
| redelmann | roaksoax, /etc/resolv.conf contain: 192.168.100.1 (my lan DNS) and 192.168.100.10 (my mass server lan ip) | 18:29 |
| redelmann | roaksoax, i was expecting to have only 192.168.100.1 | 18:30 |
| roaksoax | redelmann: Yeah I see. is the NIC on your system connected to 'maas-eth0' subnet ? | 18:30 |
| roaksoax | redelmann: i think the problem here is that since MAAS is not configure eth0, it is ont telling /etc/network/interfaces to set the DNS of maas-eth0 | 18:31 |
| roaksoax | redelmann: eitherway DHC{P should be setting it | 18:32 |
| roaksoax | and it doesn't seem like it is ? | 18:32 |
| redelmann | roaksoax, maas server and node have a NIC on 'maas-eth0' | 18:33 |
| roaksoax | redelmann: right, but MAAS is not managing DHCP/DNS< and my *guess* is that because of that, DNS server of maas-eth0 is not being set | 18:33 |
| roaksoax | redelmann: on the config | 18:33 |
| redelmann | roaksoax, that's rigth | 18:35 |
| redelmann | right | 18:35 |
| roaksoax | redelmann: so, yif you could file a bug, I'll discuss it with my guys and see if we can fix that | 18:35 |
| roaksoax | cause I think we porbably should | 18:36 |
| roaksoax | redelmann: please attach both, the maas-eth0 subnet and the curtin config | 18:37 |
| roaksoax | to the bug report | 18:37 |
| redelmann | roaksoax, great, between my poor English and my mass-newby it's hard to explain. i will do my best | 18:37 |
| redelmann | roaksoax, ok | 18:37 |
| roaksoax | redelmann: don't worry, just file it, and I'll fix the wording if need by | 18:38 |
| roaksoax | be* | 18:38 |
| redelmann | roaksoax, https://bugs.launchpad.net/maas/+bug/1545500 | 19:23 |
| redelmann | roaksoax, hope you can understand the whole idea/problem :P | 19:24 |
| redelmann | roaksoax, thank for you | 19:26 |
| redelmann | roaksoax, thank for your time | 19:26 |
| mup | Bug #1545500 opened: MAAS unmanaged network set wrong DNS information <MAAS:New> <https://launchpad.net/bugs/1545500> | 19:43 |
| haasn | Hopefully somebody reads this: If you're thinking of managing maas hosts with juju, use ansible instead! My life is so much happier and it's easier to set up, maintain and interact with :) | 21:56 |
| stormmore | somehow I don’t see juju bootstrap being that difficult a command to use | 22:12 |
| === blahdeblah_ is now known as blahdeblah | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!