[00:36] <mpontillo> shadoxx: I would check that the IP address is correct and reachable, and that tgtd is up and running on the controller (sometimes it can get stuck; for MAAS 2.3 we're moving away from iSCSI)
[00:36] <shadoxx> mpontillo: yeah, have restarted tgtd a few times with sysctl. oddly enough, figured out that i can get it to go if i tcpdump the traffic on the controller
[00:37] <shadoxx> I think these machines have something weird with interrupts happening
[00:37] <shadoxx> They get frozen sometimes up until I start randomly typing on the keyboard. I'm sure it's a quirk with the hardware I'm using (HP), and that there's a kernel parameter, but at this point I just want to get them provisioned
[00:37] <shadoxx> This is the third week trying to get this cluster going
[00:38] <shadoxx> About to just say screw it and go full OpenStack.
[00:38] <shadoxx> also, not sysctl, systemctl
[00:38] <mpontillo> shadoxx: you might check https://certification.ubuntu.com/server/models/?vendors=HPE to see if Canonical has certified the hardware, and if so with what firmware versions, etc
[00:39] <shadoxx> Well, I just did full firmware+bios updates. They were running software from 2013.
[00:39] <mpontillo> heh
[00:39] <shadoxx> Getting that ISO was difficult. But I got it.
[00:40] <shadoxx> Lol, the Gen8 isn't certified, but the Gen7 is.
[00:41] <shadoxx> I'm logged into the first one I tried to provision since the firmware and bios updates
[00:41] <shadoxx> It looks promising.
[00:42] <mpontillo> shadoxx: great! also, if you think the issue is with iSCSI, would you mind giving the latest MAAS 2.3 beta a try? in that case, we'd bypass iSCSI and send the image directly to the booting node via HTTP
[00:43] <mpontillo> (it sounds like it's not iSCSI at this point, probably the firmware...)
[00:46] <shadoxx> No, I think it's iscsi. How do I get the beta? Adding the beta repo?
[00:46] <shadoxx> Mind you, I'm running my own apt mirror because my MaaS cluster is completely segregated
[00:47] <shadoxx> I have one host that has WAN access and is running full repo mirrors for the MaaS Images, Centos 7, and Ubuntu 16.04
[00:48] <mpontillo> shadoxx: what software are you using for apt mirroring? if it's not a full mirror via something like rsync, you may be missing the bootloaders
[00:48] <shadoxx> mpontillo: apt-mirror. i was able to bootstrap all the other machines from it
[00:49] <shadoxx> I've ran full mirrors before, pretty sure that's not the issue. I'd be willing to check though if you'd coach me on what to check. :D
[00:49] <shadoxx> If it's something as simple as missing bootloaders, I'd be in dept
[00:50] <shadoxx> debt*
[00:50] <shadoxx> Right now the cluster is in PoC mode.
[00:50] <shadoxx> So I can run beta if that'll probably fix it
[00:51] <mpontillo> shadoxx: if you've installed MAAS via apt and the PPA servers are reachable, you would do something like "apt-add-repository ppa:maas/next" and then do the normal  "apt-get update && apt-get dist-upgrade -yu"
[00:51] <mpontillo> shadoxx: let me check on the mirror requirements
[00:52] <shadoxx> One issue that I *did* run into was that I wasn't pulling down the source repos with my mirror, which was causing all sorts of issue
[00:52] <shadoxx> Once I pulled in source repos as well, I was able to commission just fine.
[00:53] <mpontillo> shadoxx: I have a post-mirror script that does something like this on my partial mirror http://paste.ubuntu.com/25876414/
[00:54] <mpontillo> shadoxx: basically things broke unless I also grabbed the uefi/ directory for my mirror
[00:54] <shadoxx> Let me check if I have that
[00:58] <shadoxx> I did not have that. Added 'uefi' to the end of my sources list
[00:58] <shadoxx> it's pulling down another 1GB right now. maybe this is it..
[00:59] <mpontillo> shadoxx: ah, I wasn't aware that apt-mirror could treat that like an apt source
[00:59] <shadoxx> apt-mirror is pretty great. :] same syntax are your regular sources.list files
[01:00]  * mpontillo will keep that in mind for the next partial mirror, thanks
[01:00] <shadoxx> MaaS has been the only thing to make me feel stupid about Linux in the past 2 years.
[01:01] <shadoxx> Like, clearly I don't know how this is supposed to work.
[01:04] <mpontillo> shadoxx: well, we want to make it as easy as possible =) .. feel free to file bugs if things don't work or aren't intuitive. for example, I feel like we could do a better job raising the issue if bootloaders are missing, rather than finding out when your machines hang at boot time!
[01:05] <shadoxx> Well, it's not even that it's hanging. Once a machine is deployed, should it still boot from PXE or should it boot from the main drive?
[01:05] <shadoxx> Because, maybe that's why it's failing. I just observed an entire deployment and it seemed fine
[01:06] <shadoxx> But it still booted from PXE
[01:06] <shadoxx> It looked like there was a step where it checks for a local installation, but I could be misinterpreting the scrolling output
[01:07] <mpontillo> shadoxx: that's actually expected, if the machine netboots then MAAS retains some measure of control; the machine might PXE boot and then be instructed to boot local, but if the PXE boot fails the local boot should work fine
[01:08] <shadoxx> Gotcha
[01:08] <shadoxx> Also, hold off on adding uefi to the mirror.list. I'm figuring out the proper syntax.
[01:08] <shadoxx> Should still be able to get it going though...
[01:09] <mpontillo> shadoxx: the shell snippet I sent rsyncs the uefi/ directory from the mirror, and then uses "cp -l" to make hard links to it within the pristine mirror directory, which works for my testbed
[01:10] <mpontillo> shadoxx: I find it easier to just rsync the entire thing if I have a server with enough space, but I like to take a partial mirror with me on a laptop, because I'm crazy like that ;-)
[01:11] <shadoxx> mpontillo: yeah, i found a post that reflects what you're saying too
[01:11] <shadoxx> rsync seems to be the only way to go
[01:11] <shadoxx> I'll fix this in the morning. I'm about to head out of work and I'm already fatigued with this.
[01:11] <shadoxx> But, good news is that UEFI missing might be the ticket. These are Gen8 HP servers. Pretty sure they're UEFI.
[01:11] <shadoxx> Would explain why they boot PXE but always fail to boot from local, even after an attempted install.
[01:21] <shadoxx> mpontillo: actually, i just modified it to work with my setup pretty easily. I'm going to throw this in a post mirror script
[01:27] <shadoxx> boom. done. integrates with my apt -mirror solution nicely. danke
[01:28] <shadoxx> testing everything out tomorrow. afk for now. thanks again for the help!
[01:45] <mpontillo> shadoxx: let us know how it goes!
[06:07] <mup> Bug #1729776 opened: When Maas changes DNS entries, rndc reload fails due to duplicate entries <canonical-bootstack> <MAAS:New> <https://launchpad.net/bugs/1729776>
[06:25] <shrekmaxi> Greetings,every one
[06:33] <shrekmaxi> Can anyone tell me if MAAS does support debian image import? thanks
[12:05] <mup> Bug #1729840 opened: [2.3b3, UI] The SSH keys table in Account is inconsistent with the pattern <2.3qa> <ui> <MAAS:New> <https://launchpad.net/bugs/1729840>
[12:23] <mup> Bug #1729841 opened: [2.3b3] I added a SSH key, but it didn't appear in the list until I clicked somewhere on the screen <2.3qa> <ui> <MAAS:New> <https://launchpad.net/bugs/1729841>
[12:23] <mup> Bug #1729844 opened: [2.3b3, UI] When I add an SSH key the row should be automatically expanded so that the user can see the key <2.3qa> <ui> <MAAS:New> <https://launchpad.net/bugs/1729844>
[13:56] <mup> Bug #1729857 opened: [2.3, UI] Whitespace after checkbox on node listing page <MAAS:Triaged by ack> <https://launchpad.net/bugs/1729857>
[14:02] <mup> Bug #1729857 changed: [2.3, UI] Whitespace after checkbox on node listing page <MAAS:Triaged by ack> <https://launchpad.net/bugs/1729857>
[14:08] <[Kid]> is there a way to clone nodes in maas?
[14:14] <mup> Bug #1729857 opened: [2.3, UI] Whitespace after checkbox on node listing page <MAAS:Triaged by ack> <https://launchpad.net/bugs/1729857>
[14:36] <roaksoax> [Kid]: nope
[14:44] <mup> Bug #1729868 opened: secondary maas-regiond django errors in log <MAAS:New> <https://launchpad.net/bugs/1729868>
[15:32] <mup> Bug #1729694 changed: [2.3, UI, regression] With new node details page, no longer able to see devices (lxc's, kvms) that belong to that machine (e.g. juju cases) <MAAS:Invalid> <https://launchpad.net/bugs/1729694>
[16:11] <mup> Bug #1729868 changed: secondary maas-regiond django errors in log <MAAS:Invalid> <https://launchpad.net/bugs/1729868>
[17:22] <roaksoax> newell:/win 5
[17:27] <catbus> roaksoax: you are not remotely controlling newell's desktop, right?
[17:37] <roaksoax> lol no
[17:56] <mup> Bug #1729902 opened: [2.3] Incorrect warning on summary page when recommissioning node with testing <MAAS:Triaged by ltrager> <https://launchpad.net/bugs/1729902>
[17:56] <mup> Bug #1729904 opened: [2.3, HWTv2] Add warning icon with a tooltip for overriden status in machine details submeny and the header <MAAS:Triaged> <https://launchpad.net/bugs/1729904>
[18:02] <shadoxx> mpontillo: Figured out my issue. I had put my nodes in a different zone than default. It was failing to boot with iscsi for some reason, probably because of the zone
[18:02] <shadoxx> once i pulled them out of the zone, i was able to deploy my first node. :]
[18:02] <shadoxx> Probably my fault and not a bug. I need to comb over the manual and MaaS concepts a bit more.
[18:03] <shadoxx> In any case, the uefi stuff you sent me yesterday probably fixes a problem that I haven't ran into yet, so thanks again
[18:08] <mpontillo> shadoxx: sure, it could be that your MAAS already downloaded the UEFI bits, if it had access to them before you started using your internal mirror
[18:08] <mpontillo> shadoxx: weird that the zone would affect it, I wouldn't expect that. do you mean DNS zone or availability zone?
[18:09] <shadoxx> mpontillo: physical zones. i guess that translates to availability zones?
[18:09] <mpontillo> shadoxx: yes, same concept
[18:09] <shadoxx> Not a huge deal. They're all in the same availability zone right now anyway.
[18:10] <mpontillo> shadoxx: AZs are probably the MAAS feature I've tested the least; probably should step up testing there =)
[18:10] <mpontillo> shadoxx: if you can narrow the cause of the issue, I'd encourage you to file a bug
[18:10] <shadoxx> definitely
[18:11] <shadoxx> I'm going to use one of the nodes as my test node, while leave the other three deployed
[18:11] <shadoxx> Now I just need to figure out how MaaS manages these nodes when they're setup as hypervisors. Through Juju I assume
[18:57] <mup> Bug #1729913 opened: recent update to 1.9 broke dhcpd.conf.template <1.9.5> <maas> <MAAS:Incomplete> <https://launchpad.net/bugs/1729913>
[19:03] <mup> Bug #1729913 changed: recent update to 1.9 broke dhcpd.conf.template <1.9.5> <maas> <MAAS:Incomplete> <https://launchpad.net/bugs/1729913>
[19:09] <mup> Bug #1729913 opened: recent update to 1.9 broke dhcpd.conf.template <1.9.5> <maas> <MAAS:Incomplete> <https://launchpad.net/bugs/1729913>
[21:24] <mup> Bug # changed: 1514045, 1518633, 1527699, 1546235, 1555335, 1581453, 1603156, 1603469, 1603509, 1604375, 1612248, 1615794
[21:33] <mup> Bug #1614387 changed: [2.0rc4] interface name should be version-sorted (ens3 before ens10) in WebUI <MAAS:Invalid> <https://launchpad.net/bugs/1614387>
[21:42] <mup> Bug #1614387 opened: [2.0rc4] interface name should be version-sorted (ens3 before ens10) in WebUI <MAAS:Invalid> <https://launchpad.net/bugs/1614387>
[21:51] <mup> Bug #1614387 changed: [2.0rc4] interface name should be version-sorted (ens3 before ens10) in WebUI <MAAS:Invalid> <https://launchpad.net/bugs/1614387>
[22:03] <mup> Bug # changed: 1467532, 1516393, 1521376, 1534241, 1545500, 1557569, 1571460, 1578333, 1578347, 1588469, 1691778
[22:12] <mup> Bug # changed: 1445073, 1445939, 1450450, 1450719, 1454024, 1459898, 1462299, 1469742, 1470930, 1472338, 1477691, 1479839, 1695290
[22:42] <mup> Bug # changed: 1427573, 1442234, 1442446, 1442575, 1443257, 1444992, 1447709, 1482405, 1490847, 1491742, 1497991, 1499450, 1504863, 1528628, 1529634, 1531493, 1531531, 1531600, 1536320, 1539292, 1541030, 1541298, 1544308, 1548394, 1549174, 1549331, 1549670, 1552882, 1552892, 1555736, 1556158,
[22:42] <mup> 1556963, 1557685, 1559341, 1571101, 1571769, 1619488, 1632638, 1661440, 1664563, 1664618