=== CyberJacob is now known as CyberJacob|Away [00:25] in the maas 1.7 changelog it says maas no longer uses squid-deb-proxy but for some reason it's still getting installed... [01:18] does anyone even monitor this channel? [01:51] designate: sudo apt-get dist-upgrade should remove it [01:51] designate: if you did sudo apt-get update it might not have [01:54] roaksoax: thank you, I will try that. [01:56] designate: np! === CyberJacob|Away is now known as CyberJacob === CyberJacob is now known as CyberJacob|Away [13:22] Guys, I need some help. We're trying to comission a Dell R710 server into our newly installed MAAS cluster. However, it fails after turning on and off a couple of times, the GUI only says "Failed to power on node — Timeout after 7 tries ". What could be wrong, where should we look to start fixing this issue? [13:23] The rest of our cluster are only R720 servers. My boss suspects there could be an issue because the R720s have iDRAC 7, while the 710 has iDRAC 6. [14:54] Anyone in here? [14:55] thebozz: what version of MAAS are you using? [14:55] sounds like 1.7 [14:55] there is a button on the node page where you can check the power state of the noe [14:55] *node [14:55] thebozz: can you try that button and see if it works for the 710 [14:56] It ran successfully, and detected the node as off. [14:57] so MAAS can reach the node and has good credentials for it [14:57] when you say it fails after turning on [14:57] and off a couple of times, [14:57] how are you turning it on and off there? [14:58] Clicking on "comission node". [14:58] Let me get the relevant logs, maybe there's some useful info in there. [14:59] is there anything else talking to the BMCs? nagios or something like that [15:00] or serial over lan [15:01] :/ actually, I have no idea what I'm looking at. I haven't been involved in the deployment other than helping here and there. Is there anything I should look at in the logs to help me debug this? [15:01] well /var/log/maas/maas.log and /var/log/maas/maas-django.log might be useful, if you can post them [15:02] if the node is powered off right now, what state is it in in MAAS? Ready? Failed Commissioning? [15:03] Failed Commissioning. Let me do some filtering on those files, I'll try to grab anything that seems relevant. [15:04] can you try commissioning again, since the power check is working? [15:05] if that doesn't work, you should try powering on the node manually via IPMI using MAAS's credentials [15:10] My boss insists he thinks it has to do with every other node having iDRAC 7 while this one has iDRAC 6. Is that relevant at all? [15:12] it could be [15:13] Here are the logs: http://pastebin.com/aTf05ajh => maas.log ; http://pastebin.com/aQEAn3MR => pserv.log ; maas-django.log didn't have any references to the relevant MAC address. Is there anything else I can use to filter? [15:13] About the iDRAC thing... how is it relevant? I don't really understand that. [15:14] there was a bug at one point that affected r710 [15:14] https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1287964 [15:14] Launchpad bug 1287964 in MAAS "MAAS incorrectly detects / sets-up BMC information on Dell PowerEdge servers" [High,Fix released] [15:14] IPMI is quirky - different versions of it react different ways to the same commands sometimes [15:14] just minor differences in how the protocol is implemented [15:15] r710 doesn't look to be ubuntu certified for 14.04 [15:15] r720 is though [15:15] Huh. Then it's worth a shot to do it manually. Will MAAS be able to turn it on and off at will after comissioning? [15:16] well that depends on why it's not working - if it's not working now, and nothing changes, i wouldn't expect it to change after commissioning [15:16] oh, r710 is certified too, so it should be working [15:16] That only makes this even weirder :/ [15:16] http://www.ubuntu.com/certification/hardware/201404-14939/ [15:18] since it works sometimes and sometimes it doesn't, i would suspect either something else is talking to it via IPMI and using its sessions up, or maybe something at the network layer is bad - duplicate IP addresses maybe? [15:18] or maybe the firmware on the bmc is out of date? [15:20] The cluster is on its own subnetwork, pretty much isolated ATM. So yeah, my guess would be network layer issues or firmware. === jfarschman is now known as MilesDenver === roadmr is now known as roadmr_afk === jfarschman is now known as MilesDenver === jfarschman is now known as MilesDenver === roadmr_afk is now known as roadmr [17:48] I am trying to bootstrap an environment using maas/juju (latest stable versions of both) but I'm getting the following error: "401 OK (Authorization Error: 'Expired timestamp: given 1417774874 and now 1417800053 has a greater difference than threshold 300')" despite the fact that I have configured an NTP server in MAAS that is reachable by all servers. === CyberJacob|Away is now known as CyberJacob === roadmr is now known as roadmr_afk === roadmr_afk is now known as roadmr [22:41] since images are now stored in the maas database, can anyone point me in the direction of modifying the ephemeral image? I need to add an NTP server because of clock differences causing oauth errors. === CyberJacob is now known as CyberJacob|Away