[02:41] <mup> Bug #1511589 opened: maas provider, hwclock out of sync means juju will not work <kanban-cross-team> <landscape> <maas-provider> <juju-core:New> <MAAS:New> <https://launchpad.net/bugs/1511589>
[03:46] <digitalsky> hi, my maas server is having trouble detected nodes, am i'm not sure how I can debug this
[03:47] <digitalsky> I have 5 baremetal servers on Softlayer, they all have 4 interfaces.  eth0 / eth2 on each host are on the private network, and eth1 / eth3 are on the public network.  there is another interface for IPMI on the private network.
[03:49] <digitalsky> if i add the hardware manually from the maas gui with the IPMI port's IP / mac, then i seem to be able to control the node.  the node also seem to be able to get dhcp from maas.  but i don't understand why the nodes were not auto detected
[03:49] <digitalsky> the maas i'm running is MAAS Version 1.8.3+bzr4053-0ubuntu1 (trusty1)
[03:49] <roaksoax> digitalsky: the nodes are not autodetect ?
[03:49] <digitalsky> exactly
[03:49] <roaksoax> digitalsky: or IPMI is not autodetected ?
[03:50] <roaksoax> digitalsky: if the nodes are not autodetected, possible issues: 1. The machine never PXE booted from MAAS, which prevented auto-registration to happen
[03:50] <digitalsky> um i'm not aware MAAS would detect the ipmi.  i started maas, configured the networks and expected to see it detect the nodes (which have no OSes installed)
[03:50] <roaksoax> digitalsky: 2. something happen during the autoregistration process that prevented it from registering itself in MAAS, which would require me to look at the console logs
[03:51] <roaksoax> digitalsky: right, so this is what should happen
[03:51] <roaksoax> digitalsky: 1. turn on manually a node
[03:51] <roaksoax> 2. node will dhcp and PXE from MAAS
[03:51] <roaksoax> 3. Node will run the "auto enlistment" process
[03:51] <roaksoax> 4. Node will register itself in MAAS
[03:52] <digitalsky> ah ok!  I thought as soon as the node boot up and get dhcp, then it will already show up as detected.
[03:52] <roaksoax> digitalsky: so first thing to check would be whether the nodes are actually DHCP'ing from MAAs and PXE booting
[03:52] <roaksoax> digitalsky: nope, it will take 5-15 mins (approx) to do so depending on the type of machine you are using
[03:53] <digitalsky> ok i used KVM to see what's going on in one of the nodes, and I see that it is getting an IP from DHCP as well as booting from PXE.  however it got to a point where it tried to contact 169.254.169.254 and failed after timing out.
[03:53] <digitalsky> subsequently, it continues to boot up to a point where there is a login prompt
[03:53] <digitalsky> and then on the MAAS side, it says failed comissioning.  probably because the node failed to get the user-data from MAAS so it didn't get set up properly and only booted a plain ubuntu and nothing else
[03:56] <digitalsky> looks like it's similar to this: http://tinyurl.com/p264fj2
[04:00] <digitalsky> roaksoax: and i don't think there is an /etc/cloud path in my maas host
[04:06] <roaksoax> digitalsky: can you tell me what are the kernel params the system gets hwen PXE booting?
[04:06] <roaksoax> digitalsky: that means that they cannot contact the MAAS server for whatever reason
[04:07] <digitalsky> roaksoax: yea that's what I figured is that they can't get to the maas for the user-data.  let me see if i can get the params
[04:08] <digitalsky> roaksoax: would i be able to see some dhcp logs on the maas to see what params it sent out?
[04:08] <roaksoax> digitalsky: nope, it is either there, or in the kernel while loading the ephemeral image
[04:10] <roaksoax> digitalsky: if not, do: sudo dpkg-reconfigure maas-cluster-controller and make sure it is pointing to the correct IP address (which should be in the same network as the machines that are DHCP'ing)
[04:10] <roaksoax> digitalsky: and try again
[04:13] <digitalsky> roaksoax: ok.
[04:17] <roaksoax> digitalsky: need to run now,past midnight for me, but feel free to leave your findings and will read in the morning
[04:18] <digitalsky> roaksoax: thanks
[04:18] <digitalsky> roaksoax: i'm looking for cloud_init right? it's pointed to the wrong ip
[04:18] <digitalsky> roaksoax: so i'll try to reconfigure it with dpkg and try again
[04:18] <digitalsky> roaksoax: i know you have to run, thanks very much for your help so far! :)
[04:29] <mup> Bug #1511610 opened: TestReleaseAutoIPs.test__calls_update_host_maps_for_next_ip_managed_subnet can fail randomly <unit-tests> <MAAS:Triaged> <https://launchpad.net/bugs/1511610>
[10:54] <mup> Bug #1511689 opened: maasserver.tests.test_forms_nodegroup.TestNodeGroupDefineForm.test_rejects_invalid_interface fails spuriously <tests> <MAAS:Triaged> <https://launchpad.net/bugs/1511689>
[11:09] <mup> Bug #1511689 changed: maasserver.tests.test_forms_nodegroup.TestNodeGroupDefineForm.test_rejects_invalid_interface fails spuriously <tests> <MAAS:Triaged> <https://launchpad.net/bugs/1511689>
[11:15] <mup> Bug #1511689 opened: maasserver.tests.test_forms_nodegroup.TestNodeGroupDefineForm.test_rejects_invalid_interface fails spuriously <tests> <MAAS:Triaged> <https://launchpad.net/bugs/1511689>
[11:24] <mup> Bug #1511690 opened: maasserver.websockets.tests.test_listener.TestNodeFilesystemgroupListener.test__calls_handler_with_update_on_create fails spuriously <tests> <MAAS:Triaged> <https://launchpad.net/bugs/1511690>
[12:24] <mup> Bug #1511713 opened: udev rules not updated to reflect MAC change in node <MAAS:New> <https://launchpad.net/bugs/1511713>
[17:22] <mup> Bug #1511794 opened: node-group probe-and-enlist-hardware model=vmware does not return errors <kanban-cross-team> <landscape> <MAAS:New> <https://launchpad.net/bugs/1511794>
[17:25] <mup> Bug #1511794 changed: node-group probe-and-enlist-hardware model=vmware does not return errors <kanban-cross-team> <landscape> <MAAS:New> <https://launchpad.net/bugs/1511794>
[17:28] <mup> Bug #1511794 opened: node-group probe-and-enlist-hardware model=vmware does not return errors <kanban-cross-team> <landscape> <MAAS:New> <https://launchpad.net/bugs/1511794>
[18:28] <mup> Bug #1511493 opened: Should not allow partitions to be created on bcache device <curtin:Invalid> <MAAS:Triaged> <https://launchpad.net/bugs/1511493>
[21:16] <mup> Bug #1511860 opened: IPMI credentials not set in node power <falkor:New> <MAAS:New> <https://launchpad.net/bugs/1511860>
[21:22] <mup> Bug #1511860 changed: IPMI credentials not set in node power <falkor:New> <MAAS:New> <https://launchpad.net/bugs/1511860>
[21:35] <mup> Bug #1511860 opened: IPMI credentials not set in node power <falkor:New> <MAAS:New> <https://launchpad.net/bugs/1511860>
[22:26] <digitalsky> roaksoax: Hi roaksoax, I updated the MAAS configuration with the internal IP and now all 4 nodes got detected.