[07:18] hey guys does anyone know on maas 1.9.3 can i tag multiple vlans on nodes added in maas itself? i was not able to change the part in the networking config on nodes. === frankban|afk is now known as frankban [07:57] Bug #1604702 opened: Unable to add a 16384-bit ssh key to my account from web UI [08:00] Bug #1604702 changed: Unable to add a 16384-bit ssh key to my account from web UI [08:06] Bug #1604702 opened: Unable to add a 16384-bit ssh key to my account from web UI [09:25] Hi. I have taken 3 physical servers. On one server I installed ubuntu 15.10(Wily). And then installed MAAS. Configured the MAAS Cluster. Now I am trying to register my hardware with MAAS. MAAS done with PXE boot of its nodes and assigned IPs . When i was trying to ssh into the MAAS nodes, it was giving Permission denied (public key issue). In MAAS UI I added public key of MAAS installed server. And finally it was showing commission statu [09:26] Cloud-init v.0.7.5 started running and finished . I am using MAAS Version 1.9.3+bzr4577-0ubuntu1 (wily1). Finally , the MAAS nodes have stuck with the info -> Cloud-init v.0.7.5 finished at wed, 20 jul 2016 13:01:51 +0000 Datasource DataSource ConfigDriveNet [net, ver=2] [Source=/dev/sdb1]. UP 43.10sec. [09:27] I observed the Maas log files under /var/log/maas. I didn’t get any error messages. But I didn’t understand why commission is failing. [09:27] MAAS log info : http://paste.openstack.org/show/537405/ [09:28] Could you please provide me the solution why MAAS commission failed. [10:06] Bug #1604738 opened: Dynamic subnet range disappears after edit and is replaced with "No IP ranges have been reserved for this subnet." === CyberJacob is now known as zz_CyberJacob [12:57] what debugging steps can I take, deployments are failing and I’m not sure why [13:15] acovrig, you'll need to tell us more about the failure first :) [13:16] siva, hey there [13:17] kiko: that’s what I’m asking, all I can tell is it says “Failed deployment”, how can I learn more about the failure to troubleshoot [13:18] acovrig, what are you deploying? [13:23] ubuntu 16.04 [13:23] acovrig: are you using MAAS 2.0 ? [13:23] acovrig: get the installation log from the WebUI (at the bottom of the page) [13:24] acovrig: and if you are using 2.0, look in: /var/log/maas/rsyslog//../messages [13:25] roaksoax: the last thing I get is Node installation - ‘cloudinit’ running modules for final then Node changed status - From ‘Deploying’ to ‘Failed deployment’ [13:27] I see a few “timed out waiting for reply from (ntp)” which would explain it trying to hit a status point of 2012… [13:28] acovrig: my guess is that it deployed successfully, but on the reboot, it either didn't boot from the disk, or it failed to reach maas after the the first reboot [13:28] acovrig: can you pastebin the cloud-init outout + the install log from the wbeui ? [13:32] roaksoax: install log from the webui, where would the cloud-init output be? https://gist.github.com/acovrig/446fc9b616f72979c7995a5132fb607f [13:39] acovrig: yeah, if it is not there, then that means cloud-init/curtin didn't send it back [13:39] acovrig: also, can you pb the rsyslog ? [13:40] roaksoax: sure, currenlty I have the BIOS set to PXE boot first, should I not (I.E. have it boot from the HDD after deployment)? [13:41] acovrig: no you don't, giving it a second look it seems something failed before the machine was actually rebooted [13:41] Node changed status - From 'Deploying' to 'Failed deployment' Tue, 19 Jul. 2016 15:16:52 [13:41] Node installation - 'cloudinit' running modules for final Tue, 19 Jul. 2016 15:16:52 [13:41] acovrig: look at the timestamp [13:41] acovrig: so the rsyslog should show why was that ? [13:42] roaksoax: OK, gist updated (https://gist.github.com/acovrig/446fc9b616f72979c7995a5132fb607f#file-rsyslog) [13:43] roaksoax: interesting: the rsyslog only goes to 13:53, not to 15:16 (time) [13:45] roaksoax: sudo grep 15:16 /var/log/maas/rsyslog/HOST-DELL-*/*/messages shows nothing O.o [13:47] I see a few “timed out waiting for reply from (ntp)” which would explain it trying to hit a status point of 2012… [13:47] roaksoax, acovrig: ^^^ is this not relevant? [13:47] could the times be skewed and failing for that reason? [13:47] yea, I watched the physical console a few times and saw it getting a 401 from my MAAS server because of it; what would cause it to not be able to reach ntp? [13:48] acovrig, IIRC the node is trying to reach ntp.ubuntu.com [13:48] is that reachable? [13:50] kiko: odd, I’m deploying to 2 nodes now to see if it was just a timing issue; I SSH’d into one of them and it can ping the MAAS server (router for the nodes) but not ping google (8.8.8.8), but it can DNS dig google.com... [13:51] acovrig, well, typically dig will work, as MAAS is the DNS server [13:51] acovrig, it looks like routing (or NAT) is broken? [13:52] kiko: I have 3 FW rules on MAAS: -A FORWARD -i eno1.8 -o eno1 -m state --state RELATED,ESTABLISHED -j ACCEPT [13:52] kiko: the NTP shouldn't be an issue release [13:52] -A FORWARD -i eno1 -o eno1.8 -j ACCEPT [13:52] acovrig: what do you have under /var/log/maas/rsyslog/ ? [13:52] and -A POSTROUTING -o eno1.8 -j MASQUERADE [13:52] acovrig: is it just completely empty ? [13:53] acovrig, the MASQUERADE entry looks wrong [13:53] acovrig, shouldn't that be -o eno1? [13:53] oh hmm [13:53] roaksoax: no, I have my 2 hosts/date/messages and maas-enlisting-node/date/{messages,ip} [13:54] roaksoax, right, but if there is no upstream connectivity, could things be failing? [13:54] kiko: the MAAS server is on VLAN 8 (tagged) to get to the nodes (untagged) [13:54] kiko: yeah it could, but NTP shouldn't be a cause of failure [13:55] don’t the nodes use a date to connect back to the MAAS server? [13:55] acovrig, eno1.8 is external, I understand now [13:55] acovrig, do the packets appear correctly masqueraded? [13:55] acovrig, if you plug a laptop into the untagged vlan, can it get out? [13:56] kiko: good question, pending test [13:57] roaksoax, sure, but NTP failing could be an indication that networking in general is failing :) [14:01] kiko, roaksoax: a windows 7 client can’t access icanhazip.com from chrome, so routing must be broken [14:02] acovrig, right. fix the networking first and then come back to me. :) [14:02] I updated the gist (https://gist.github.com/acovrig/446fc9b616f72979c7995a5132fb607f) with my iptables and ifconfig; not sure what’s wrong... [14:03] sysctl net.ipv4.ip_forward is on... [14:03] acovrig, check the input and output chains too, otherwise add logging and use tcpdump/wireshark to see what's happening [14:11] kiko: OK, not sure what was wrong, but fixed iptables and networking works, just started a deployment, we’ll see what happens. [14:12] acovrig, list out the rules you have now (iptables -L FORWARD -n, perhaps INPUT and OUTPUT too) and put it in a paste? [14:12] roaksoax: FYI: I couldn’t get maas to accept an SSH key yesterday and was able to install 2.0.0~beta3+bzr4941-0ubuntu1 and now it accepts my key(s). [14:12] acovrig, cool [14:15] kiko: looking at diff, I got my ifs flipped; I guess the guide I looked at used eth1 as internet and eth0 as lan and I didn’t notice :/ [14:15] aha [14:15] thanks [14:20] * acovrig emits a squeel of excitement: “Deployed” :D [14:22] now to get juju and openstack to work :) [14:42] acovrig, it worked? w00t! [14:44] kiko: yup, now I just need to find a good guide to get juju+openstack to work as a failover cluster for VMs [14:44] acovrig: this should do : https://jujucharms.com/openstack-base/ [14:45] roaksoax: ooh, thanks === frankban is now known as frankban|afk === cmagina is now known as cmagina_lunch [17:11] Bug #1604901 opened: No way to retrieve complete event log through the cli [17:17] acovrig, all a-ok? [17:18] kiko: with MAAS, yes, I’m still working though juju/openstack ‘solutions’ [17:18] okay [17:35] do i need two server to run maas and openstack or can it be don by on mass [17:35] done* [19:59] Bug #1604938 opened: node failed deployment "Exception: I/O operation on closed file." [21:12] Bug #1604962 opened: node set to "failed deployment" for no visible reason [22:06] Bug #1604987 opened: Event log should always include a reason why a node was marked Failed Deployment [22:54] Bug #1546760 changed: Re-attempt power on for initial PXE failure during node deployment to alleviate PXE failures [22:54] Bug #1605008 opened: juju2beta12 and maas2rc2: juju status shows 'failed deployment' for node that was 'deployed' in maas

[23:00] Bug #1605008 changed: juju2beta12 and maas2rc2: juju status shows 'failed deployment' for node that was 'deployed' in maas

[23:00] Bug #1546760 opened: Re-attempt power on for initial PXE failure during node deployment to alleviate PXE failures [23:06] Bug #1546760 changed: Re-attempt power on for initial PXE failure during node deployment to alleviate PXE failures [23:18] Bug #1605008 opened: juju2beta12 and maas2rc2: juju status shows 'failed deployment' for node that was 'deployed' in maas

[23:24] Bug #1605008 changed: juju2beta12 and maas2rc2: juju status shows 'failed deployment' for node that was 'deployed' in maas