[08:15] <Razva> good mornin' folks. why in the world is MAAS running...two DHCP servers...?
[08:19] <Razva> should I use 14.04 LTS for Ubuntu Cloud Images? or 15.10?
[08:35] <haasn> Razva: ps aux | grep dhcp | curl -sF "file=@-" https://0x0.st
[08:35] <Razva> mornin' haasn
[08:36] <Razva> https://0x0.st/X0i.txt
[08:36] <Razva> haasn at this point I've reinstalled Ubuntu
[08:36] <Razva> just a sec to setup networking
[08:37] <haasn> well that machine is clearly running 0 DHCP servers :p
[08:37] <Razva> yup, right now yes
[08:38] <Razva> can you please take a look at https://github.com/Razva/newbadmin/blob/master/firewall/nat.sh ?
[08:38] <Razva> it's a script one of the good folks here provided for NAT
[08:38] <Razva> I've slighly modified it
[08:38] <Razva> looks ok?
[08:38] <Razva> (for NAT)
[08:51] <haasn> Razva: seems reasonable, not sure if the MASQUERADE line would benefit from an extra -i "$INTIF" or not but if it works it works
[08:53] <Razva> any idea how to retain this after reboot...? :|
[08:54] <haasn> Razva: iptables-save and iptables-restore can be used to save the current settings to a file (and do the opposite). Most distros provide some sort of init script for automatically doing that
[08:55] <Razva> got it
[08:56] <Razva> haasn: https://0x0.st/X0K.txt < that ps aux you've requested
[08:56] <Razva> seems that only MAAS is running, right?
[08:56] <haasn> seems to only be running one
[08:57] <haasn> on interface enp0s25 (this is your private interface, I take it?)
[08:57] <Razva> weird, last time it was i*something-dhcp-server running also... :|
[08:57] <Razva> exactly, it's the lan
[08:57] <Razva> now, let's try to PXE boot
[08:57] <haasn> Razva: when stuff like that happens it's a good idea to look at the -f option, in this case /var/lib/maas/dhcpd.conf
[08:57] <haasn> If that's /etc/dhcp/dhcpd.conf or something like that then it's probably some “default” system-wide dhcpd service running accidentally
[08:57] <Razva> do you recommend 14 LTS or 15, for Ubuntu Cloud?
[08:58] <haasn> I have no idea. I use 14 LTS because I haven't had a reason to change
[08:59] <Razva> does Liberty works with 14?
[09:00] <Razva> yup, it successfully booted, no idea why last time it basically started two DHCP servers...it was a fresh install...
[10:14] <Razva> haasn except "Allow SSH access and prevent machine from powering off", should I check the other two boxes?
[10:14] <Razva> Retain network & Retain storage?
[10:15] <haasn> What those checkboxes mean is that if you're commissioning a node that you've already comissioned in the past, any storage/network configuration you made manually will be preserved instead of reset back to the default
[10:15] <haasn> If this is your first time commissioning that node they do nothing
[10:15] <Razva> aaand if I didn't ever commisioned them...I suppose that I should just "SSH and prevent machine from powering off", right?
[10:17] <haasn> I don't see any reason to set that unless you're trying to debug something that went wrong
[10:18] <Razva> yay, ok
[10:19] <Razva> the only "wrong" thing until now is that MAAS is seeing "power on" even if the machine is 100% powered off :|
[10:20] <Razva> any idea why>
[10:21] <haasn> Razva: the power status is not checked all the time but polled at regular intervals (every few minutes), you can click ‘check now’ to force an update
[10:21] <Razva> haasn did it 3 times, they still point as being on
[10:21] <Razva>  Checking power -> Power on
[10:21] <haasn> Razva: What power type?
[10:22] <Razva> even tried to "shutdown" them from MAAS
[10:24] <Razva> IPMI. they are Dell servers
[10:25] <haasn> Razva: try asking ipmipower -D LAN_2_0 -h HOST -u maas -p PASS --stat what it thinks of your host
[10:26] <Razva> 	Power State	OFF
[10:26] <Razva> haasn any idea what's the pass? :))
[10:27] <haasn> Razva: you can check in the maas GUI
[10:27] <haasn> It should have created a new account/password when enlisting
[10:27] <haasn> If ipmipower works but MAAS does not, seems like a bug to me
[10:27] <Razva> got it, just a sec to see
[10:28] <Razva> HOST = IP or host.maas ?
[10:28] <haasn> It should be the same hostname as in the maas power configuration
[10:29] <Razva> paltry-cave < who in the world is generating these names? :)
[10:30] <Razva> 95.154.222.13: connection timeout
[10:31] <Razva> interesting.
[10:36] <haasn> Razva: you could also try logging in via SSH
[10:38] <Razva> I'm power cycling both machines, see what's the result
[10:39] <Razva> long live iDRAC :)
[10:39] <Razva> the machines are not even commisioned, so I cannot really SSH in...
[10:40] <Razva> oook both are not booting via PXE
[10:42] <Razva> ssh key bla bla...
[10:43] <Razva> both are waiting here: http://screencast.com/t/LvFwpbHIaBnK
[10:43] <Razva> ^ is that "normal"?
[10:44] <Razva> aaaand they seem to be stuck at that point :|
[10:50] <haasn> Razva: SSH to the service processor (IPMI interface), not the host OS
[10:54] <Razva> SSH seems to be working, but I cannot login
[10:57] <haasn> Razva: with the maas username/password?
[10:58] <haasn> what about root?
[11:02] <Razva> haasn yup, with the maas u/p, I cannot login
[11:02] <Razva> I'll try to login with root now
[11:02] <Razva> 95.154.222.13: connection timeout < from ipmipower
[11:03] <Razva> root@95.154.222.13's password: < ssh works
[11:03] <Razva> /admin1-> < root worked
[11:03] <Razva> any hints now? :D
[11:12] <haasn> Razva: set the root un/pw in maas and be done with it? :p
[11:13] <haasn> oh, connection timeout from ipmipower with root p/w too?
[11:13] <haasn> Razva: try ipmitool -I lanplus -U root -P ... -H ... user list
[11:14] <haasn> If ssh works but ipmipower does not, then you might need to specify a firmware-specific workaround flag (via -W)
[11:14] <haasn> e.g. I need -W sun20 for some of my old sun servers
[11:17] <Razva> haasn exactly
[11:17] <Razva> haasn but I cannot even reset IPMI
[11:18] <Razva> ipmitool -H 95.154.222.13 mc reset warm
[11:18] <Razva> Password:
[11:18] <Razva> Error: Unable to establish LAN session
[11:18] <haasn> did you try the ipmitool command I suggested?
[11:18] <Razva> nope, just a sec
[11:19] <Razva> ipmitool -I lanplus -U root -H -H 95.154.222.13 user list
[11:19] <Razva> Password:
[11:19] <Razva> Address lookup for -H failed
[11:19] <Razva> Could not open socket
[11:20] <Razva> (same with minus the extra -H)
[11:20] <Razva> ipmitool -I lanplus -U root -H 95.154.222.13 user list
[11:20] <Razva> Password:
[11:20] <Razva> Error: Unable to establish IPMI v2 / RMCP+ session
[11:21] <Razva> but I can SSH login...
[11:21] <Razva> sh root@95.154.222.13
[11:21] <Razva> root@95.154.222.13's password:
[11:21] <Razva> /admin1->
[11:26] <haasn> Razva: maybe something here will help? https://www.gnu.org/software/freeipmi/freeipmi-bugs-issues-and-workarounds.txt
[11:27] <haasn> also see man ipmipower (scroll down to workarounds)
[11:31] <Razva> I've contacted the data-center...maybe they can do a power-cycle/reset...
[11:31] <haasn> doubt it will help
[11:36] <Razva> its their server, so I should be able to list users...at least.
[16:52] <Razva> hey guys. I have no idea why but maas refuses to interpret the right status from IPMI...
[16:52] <Razva> the server is powered down, I can see that via DRAC and console..
[16:52] <Razva> but still maas reports the server as being online. any hints...?
[17:23] <redelmann> hi, im having some trubles with unmanaged networks
[17:23] <redelmann> im trying to let my local dhcp and dns to manage this network
[17:24] <redelmann> but for some reason /etc/resolv.conf is setting maas ip address as DNS
[17:24] <redelmann> i double check my local dhcp setting and everything seems fine
[17:25] <redelmann> so, why maas is putting herself as DNS of an unmanaged network?
[17:28] <Razva> redelmann no idea, but that's what maas does (at least on my setup)
[17:30] <redelmann> Razva, ok, thank. It little strange, i was supposing that unmanaged network is UNmanaged :P
[17:30] <redelmann> Razva, i think juju bootstrap fail in my setup because of this
[17:40] <Razva> redelmann I have a very weird issue. MAAS reports power on, even if power is off (I can see in iDRAC that power is off, and console shows No power)
[17:41] <redelmann> Razva, it's that for me?
[17:42] <Razva> redelmann yes
[17:42] <Razva> maybe you have any ideas...
[17:43] <redelmann> Razva, noup.
[17:43] <redelmann> Razva, im playing with maas for 2 weeks to now
[17:44] <redelmann> Razva, so im not the most indicated person
[17:44] <Razva> haha, I'm playing for ehm...2 days :D
[17:44] <Razva> I've successfully setup 3 machines, everything PXE boots, but this single darn machine doesn't wants to boot....
[17:45] <Razva> or whatever...boots but doesn't wants to report power as it should
[17:45] <redelmann> Razva, maybe hardware problem?
[17:46] <Razva> no idea...
[17:46] <Razva> do you use IPMI or...?
[17:46] <redelmann> IPMI and virsh
[17:47] <redelmann> i'am mixing virtual nodes with intel hardware
[17:47] <Razva> I have only intel / Dell hardware, but no idea why the heck I cannot control it...
[17:47] <redelmann> im just testing maas for a full implementation
[17:48] <Razva> well...I didn't had any issues with it until now, only problem was that - at first install - Ubuntu installed a separate DHCP server as well. no idea why, but a clean install solved everything
[17:49] <Razva> Power on < I'm gonna kill somebody.
[17:49] <redelmann> Razva, you are installing MAAS from apt?
[17:49] <Razva> the server is off god damnit...
[17:50] <redelmann> Razva, try with "check now" link :P
[17:50] <redelmann> Razva, or manually power on the server
[17:50] <Razva> yeah if I manually power on it works, but this is not the way to do it...
[17:50] <redelmann> Razva, after manually power on, swith to off with maas
[17:51] <Razva> how...?
[17:51] <Razva> I cannot manually set power off in maas
[17:51] <redelmann> Razva, i know but maybe yoy can "sync" what maas think about node and rearl node state
[17:51] <Razva> please do me a favor
[17:51] <redelmann> Razva, take action -> power off?
[17:52] <Razva> ipmipower -D LAN_2_0 -h SOME-IPMI-IP -u root -p THE-ROOT-PASSWORD --stat
[17:52] <Razva> does it works for you?
[17:55] <redelmann> Razva, im in home rigth know.
[17:55] <redelmann> Razva, i dont know why cant ssh to cluster
[17:56] <redelmann> Razva, give me a second
[17:57] <redelmann> Razva, but usually thats the command i use to check the whole cluster power status without problem
[17:58] <redelmann> Razva, i never use -D, only -h ip
[18:01] <redelmann> Razva, did you try with --debug?
[18:01] <Razva> well...what command do you use, usually?
[18:03] <Razva> (ipmipower_powercmd.c, _retry_packets, 1214): host = 95.154.222.13; p = 1; Sending retry, retry count=24
[18:09] <Razva> redelmann are you deploying OpenStack or just MaaS?
[18:09] <redelmann> Razva, just MAAS
[18:09] <redelmann> Razva, ipmipower -h ip -u root -p THE-ROOT-PASSWORD --stat
[18:09] <redelmann> Razva, i let ipmipower to decide ipmi version
[18:10] <Razva> 95.154.222.13: connection timeout
[18:10] <Razva> but the interesting thing is that I can ssh to it
[18:11] <Razva> /admin1->
[18:12] <redelmann> Razva, ipmipower --version?
[18:13] <redelmann> Razva, some old ipmipower utilitis had a bug with -p options i think
[18:14] <Razva> redelmann via SSH?
[18:14] <redelmann> Razva, no, just ipmipower --version in machine you are trying to connect
[18:15] <redelmann> Razva, maas server i suppose
[18:15] <Razva> ipmipower - 1.1.5
[18:16] <Razva> you?
[18:16] <redelmann> Razva, look at this: https://lists.gnu.org/archive/html/freeipmi-users/2005-11/msg00018.html
[18:17] <redelmann> Razva, its old
[18:17] <redelmann> Razva, forget about it, ipmipower 0.1.3 its the problem
[18:18] <redelmann> Razva, ipmipower - 1.1.5
[18:18] <roaksoax> Razva: do you have logs from /var/log/maas/ , it is probably telling why it cannot connect  to IPMI
[18:18] <roaksoax> if that's the case
[18:20] <roaksoax> redelmann: maas doens't really support external DHCP/DNS anymore. But eventhen, MAAS is the defautl DNS server unless you set it differently for each subnet
[18:21] <redelmann> Razva, did you try ipmiping <IP>
[18:22] <redelmann> roaksoax, http://paste.ubuntu.com/15060537/
[18:22] <redelmann> roaksoax, i set DNS on my maas-eth0
[18:22] <roaksoax> redelmann: what version of MAAs are you using?
[18:22] <redelmann> roaksoax, it's that no enough?
[18:23] <roaksoax> 1.9
[18:23] <redelmann> roaksoax, 1.9
[18:23] <redelmann> roaksoax, it's testing purpose
[18:23] <roaksoax> redelmann: right, so when you deploy, is the interface of the machine configured to do what? DHCP ?
[18:24] <redelmann> roaksoax, exactly
[18:24] <roaksoax> redelmann: right, so DHCP should be setting the DNS server right?
[18:24] <redelmann> roaksoax, rigth
[18:24] <roaksoax> redelmann: also, can you get the curtin config ?
[18:26] <roaksoax> redelmann:  maas <session> node get-curtin-config <system-id>
[18:26] <redelmann> roaksoax, ohh THANK, i was losing my head
[18:28] <redelmann> roaksoax, http://paste.ubuntu.com/15060721/
[18:29] <redelmann> roaksoax, /etc/resolv.conf contain: 192.168.100.1 (my lan DNS) and 192.168.100.10 (my mass server lan ip)
[18:30] <redelmann> roaksoax, i was expecting to have only 192.168.100.1
[18:30] <roaksoax> redelmann: Yeah I see. is the NIC on your system connected to 'maas-eth0' subnet ?
[18:31] <roaksoax> redelmann: i think the problem here is that since MAAS is not configure eth0, it is ont telling /etc/network/interfaces to set the DNS of maas-eth0
[18:32] <roaksoax> redelmann: eitherway DHC{P should be setting it
[18:32] <roaksoax> and it doesn't seem like it is ?
[18:33] <redelmann> roaksoax, maas server and node have a NIC on 'maas-eth0'
[18:33] <roaksoax> redelmann: right, but MAAS is not managing DHCP/DNS< and my *guess* is that because of that, DNS server of maas-eth0 is not being set
[18:33] <roaksoax> redelmann: on the config
[18:35] <redelmann> roaksoax, that's rigth
[18:35] <redelmann> right
[18:35] <roaksoax> redelmann: so, yif you could file a bug, I'll discuss it with my guys and see if we can fix that
[18:36] <roaksoax> cause I think we porbably should
[18:37] <roaksoax> redelmann: please attach both, the maas-eth0 subnet and the curtin config
[18:37] <roaksoax> to the bug report
[18:37] <redelmann> roaksoax, great, between my poor English and my mass-newby it's hard to explain. i will do my best
[18:37] <redelmann> roaksoax, ok
[18:38] <roaksoax> redelmann: don't worry, just file it, and I'll fix the wording if need by
[18:38] <roaksoax> be*
[19:23] <redelmann> roaksoax, https://bugs.launchpad.net/maas/+bug/1545500
[19:24] <redelmann> roaksoax, hope you can understand the whole idea/problem :P
[19:26] <redelmann> roaksoax, thank for you
[19:26] <redelmann> roaksoax, thank for your time
[19:43] <mup> Bug #1545500 opened: MAAS unmanaged network set wrong DNS information <MAAS:New> <https://launchpad.net/bugs/1545500>
[21:56] <haasn> Hopefully somebody reads this: If you're thinking of managing maas hosts with juju, use ansible instead! My life is so much happier and it's easier to set up, maintain and interact with :)
[22:12] <stormmore> somehow I don’t see juju bootstrap being that difficult a command to use