[09:11] <roaksoax> rvba: ok so upgrades does work, matsubara verified all the bugs, the only thing might be upgrading bind9 itself and having things broken
[09:29] <rvba> roaksoax: ok, did you manage to understand why it's failing to upgrade?  What do the log says?
[09:30] <roaksoax> rvba: no idea, last night after the party I upgraded again and it worked fine.. and continues too.. so maybe it was due to upgrading bind9?
[09:40] <bigjools> roaksoax: where is maas-enlist called from?
[09:41] <roaksoax> bigjools: what do you mean?
[09:42] <jtv> roaksoax: I think he means: when a node enlists, what is it that calls maas-enlist?
[09:42] <bigjools> roaksoax: I want to know where maas-enlist is called so I can see what args it gets passed
[09:42] <roaksoax> bigjools: ah! enlist user data
[09:43] <bigjools> roaksoax: cool thanks
[09:43] <roaksoax> bigjools: /usr/share/maas/preseeds/enlist_userdata
[09:43] <roaksoax> bigjools: is there any issues with enlistment?
[09:43] <jtv> Yes — it uses the wrong hostname.
[09:43] <bigjools> roaksoax: yes, it's setting the hostname to something that fucks up things
[09:43] <roaksoax> bigjools: what hostname?
[09:43] <roaksoax> bigjools: where are you using it?
[09:44] <bigjools> roaksoax: it sets it to an IP address with dashes, right?
[09:44] <roaksoax> bigjools: right
[09:44] <bigjools> roaksoax: that breaks MAAS's DNS
[09:44] <bigjools> because maas generates that style of hostname by default
[09:45] <bigjools> and it really confuses users when the IP address changes
[09:45] <roaksoax> bigjools: right, so then you need to either make DNS static or not pre-assign dns names for IP addresses
[09:45] <bigjools> roaksoax: we already have static DNS entries of the form N-N-N-N
[09:45] <bigjools> which is why this breaks
[09:46] <roaksoax> bigjools: right, but that's not maas-enlist fault
[09:46] <bigjools> we use CNAME for user-amended hostnames
[09:46] <bigjools> roaksoax: what hostname format does maas-enlist send?
[09:46] <roaksoax> bigjools: maas-enlist does not send a hostname, unless it finds a DNS server which is providing a hostname. This is the requirement for having externally managed DNS/DHCP servers
[09:47] <bigjools> roaksoax: ah ok
[09:47]  * jtv facepalms
[09:47] <bigjools> roaksoax: we don't want it to do that
[09:47] <roaksoax> bigjools: but since MAAS is already filling up DNS names for IP address (before even having a system enlistment) the enlistment processs obviously finds a DNS service, with DNS assigned to an IP address
[09:48] <roaksoax> bigjools: right, but we do want it to do that
[09:48] <roaksoax> bigjools: otherwise it breaks things
[09:48] <bigjools> roaksoax: what does it break?
[09:48] <roaksoax> otherwise we wont be able to use it with external DNS
[09:49] <jtv> In what way do we need to use this particular hostname with external DNS?
[09:49] <jtv> Because our DNS entries already point that name to that host.
[09:49] <roaksoax> here's the deal
[09:49] <roaksoax> what if we don't use DNS/DHCP in MAAS
[09:49] <roaksoax> and we have an external DNS/DHCP
[09:50] <roaksoax> if that happens, then the enlistment process needs to detect the DNS name for the particular node
[09:51] <roaksoax> becuase in those case escenarios, you assign a hostname for a particular host
[09:51] <roaksoax> s/hostname/dns-name
[09:51] <roaksoax> so maas-enlist needs to detect that
[09:51] <bigjools> roaksoax: right, that makes sense
[09:52] <roaksoax> right, so now, the problem is when we use MAAS managed DNS/DHCP server, maas pre-fills DNS with hostnames for each IP address regardless of whether we want to assign them or not
[09:52] <roaksoax> but the enlistment process still checks for a DNS name for the IP it's gotten
[09:52] <roaksoax> and since it finds it, it sends it back to maas
[09:52] <roaksoax> becuase it found a dns name for its IP
[09:54] <bigjools> roaksoax: thanks for clarifying, we're talking about how to fix it now
[09:55] <roaksoax> bigjools: I believe that the correct way to do that, is simply make sure that when a node gets an IP from MAAS DHCP server, that IP/DNS name gets "statically" assigned to that server
[09:56] <roaksoax> bigjools: so everytime, the server uses the same IP address and not randomly assigned IP every time it boots
[09:56] <bigjools> roaksoax: that already happens
[09:57] <roaksoax> bigjools: so if that ahppens, then there should not  be anyprobllems
[09:57] <bigjools> roaksoax: the problem is that it passes the hostname from the maas-dns reverse zone, which is also in our forward zone and which we then try and assign to a CNAME
[09:58] <bigjools> which blows up dns
[09:58] <bigjools> we have a workaround
[09:58] <roaksoax> bigjools: right, why does it blow up DNS? is DNS not working anymore because of this?
[09:58] <bigjools> roaksoax: because there's a CNAME which points to itself
[09:59] <roaksoax> bigjools: where's the CNAME stored?
[09:59] <bigjools> in the forward zone
[09:59] <roaksoax> root@ubuntu:/etc/bind# grep -sr 192-168-123-101 *
[09:59] <roaksoax> maas/zone.123.168.192.in-addr.arpa:101 IN PTR 192-168-123-101.master.
[09:59] <roaksoax> maas/zone.master:192-168-123-101 IN A 192.168.123.101
[09:59] <roaksoax> bigjools: ^^
[09:59] <roaksoax> bigjools: that's the only thing I have stored
[09:59] <roaksoax> bigjools: that's with what it is in -proposed
[10:00] <bigjools> roaksoax: because there's a hack in to prevent it getting stored if it matches the existing auto-generated hostname
[10:01] <roaksoax> bigjools: yeah I see now: http://pastebin.ubuntu.com/1302175/
[10:01] <roaksoax> bigjools: so that means we need something more to SRU
[10:01] <bigjools> roaksoax: "maas/zone.master:192-168-123-103 IN CNAME 192-168-123-102"
[10:02] <bigjools> that's massively confusing for people :)
[10:02] <roaksoax> bigjools: indeed
[10:02] <bigjools> we're going to ignore hostnames from maas-enlist if we are managing DNS ourselves
[10:02] <bigjools> roaksoax: we want to generate a special hostname anyway
[10:03] <roaksoax> bigjools: that would make sense
[10:04] <roaksoax> bigjools: so then change would be in maas side rather than enlistment process right?
[10:06] <bigjools> roaksoax: yes
[10:06] <bigjools> roaksoax: FWIW here's our upcoming SRU bug list: https://bugs.launchpad.net/maas/+milestone/12.10-stabilization
[10:06] <ubot5> Ubuntu bug 12 in Launchpad itself ""Next 10 messages" changes Display Settings" [Medium,Fix released]
[10:06] <roaksoax> bigjools: oh wow... so I think we should do those a few at a time
[10:07] <bigjools> roaksoax: we're doing them in the next 2 months and then we're moving on
[10:07] <roaksoax> bigjools: awesome!
[10:07] <roaksoax> there's a couple of them that are probably easy to release now
[10:11] <bigjools> yep
[10:13] <bigjools> roaksoax: your name is on https://bugs.launchpad.net/bugs/1064224 - are you really fixing it?
[10:13] <ubot5> Ubuntu bug 1064224 in MAAS "IPMI detection ends up with power_address of 0.0.0.0" [Critical,Triaged]
[10:14] <roaksoax> bigjools: before I came here i saw exactly the same thing on 1 of the Mini servers I have
[10:14] <roaksoax> bigjools: so I'd like to make somre more investigation
[10:14] <bigjools> roaksoax: AHA!
[10:14] <bigjools> roaksoax: the bMC has no IP address
[10:14] <bigjools> look at dhcp log
[10:14] <bigjools> it doesn't accept the offered IP and continues to request
[10:14] <bigjools> something is screwy in the quantal isc-dhcpd I think
[10:15] <bigjools> I think this happened after I upgraded my maas server to quantal
[10:15] <roaksoax> bigjools: maybe, but in my case I was running dd-wrt
[10:15] <bigjools> oh :/
[10:15] <roaksoax> my dd-wrt router was the one in charge of giving the IP address to the BMC
[10:16] <bigjools> I wonder if the ipmi detection code buggered the bmc then
[10:16] <roaksoax> bigjools: so 2 of the nodes correctly enlisted with the correct IP address
[10:16] <roaksoax> the BMC query to the 3er node returned 0.0.0.0 even though I could reach it
[10:16] <bigjools> doesn't sound like the same problem as mine
[10:16] <roaksoax> bigjools: so I turned the server on with ipmipower command, and that node in particular, returned 0.0.0.0 instead of the IP of the BMC even though it was set
[10:18] <roaksoax> bigjools: i do believe it might be related
[10:18] <jtv1> Does the bmc expose its logs anywhere?
[10:18] <roaksoax> bigjools: maybe your BMC is getting the IP, but the query fails for some reason (or time's out) or something
[10:18] <roaksoax> I will have to test when I get back
[10:18] <roaksoax> jtv1: and not that i know of
[10:19] <bigjools> roaksoax: It is not getting an IP
[10:19] <bigjools> I can confirm this by looking at dhcp logs
[10:22] <roaksoax> bigjools: ok, so your bmc was not accessible then?
[10:22] <bigjools> roaksoax: exactly
[10:23] <roaksoax> bigjools: ok, yeah weird
[10:28] <roaksoax> bigjools: i did have some kind of problems trying to get the  BMC to obtain an IP address
[10:28] <roaksoax> bigjools: but at the end of the day, i just had to disconnect the server from the power
[10:28] <bigjools> roaksoax: can you try to get the bmc to request address from a quantal isc-dhcpd to replicate my scenario
[10:29] <roaksoax> bigjools: i will when i get back home
[10:29] <bigjools> you mean you didn't bring your microservers here? :)
[10:33] <roaksoax> bigjools: lol... i had remote access to my house and the cluster... but it got killed for some reason so I can't access anymore
=== jtv1 is now known as jtv
[10:43] <rvba> bigjools: https://bugs.launchpad.net/maas/+bug/1070774
[10:43] <ubot5> Ubuntu bug 1070774 in MAAS "The hostname of a node can still be changed once the node is in use." [Undecided,New]
[10:45] <rvba> bigjools: https://bugs.launchpad.net/maas/+bug/1070775
[10:45] <ubot5> Ubuntu bug 1070775 in MAAS "The zone name (attached to a cluster controller) can still be changed when it contains in-use nodes. " [Undecided,New]
[12:29] <roaksoax> rvba: yay!! use_squashfs seems broken
[12:29] <rvba> roaksoax: !?
[12:30] <roaksoax> rvba: squashfs image are present, but they are not being used
[12:30] <rvba> roaksoax: I think I already asked you this… but this is fixed in the packaging right? http://paste.ubuntu.com/1302370/
[12:30] <rvba> This is while installing maas from the packaging in the archive.
[12:30] <roaksoax> rvba: dbc_go: not found --> that doesn't really affect anything
[12:30] <roaksoax> rvba: it is present in precise
[12:31] <rvba> That's from the quantal package.
[12:31] <roaksoax> rvba: yeah
[12:31] <roaksoax> rvba: so I fixed it and reverted the change
[12:32] <rvba> It's still a bit ugly to get that error message when installing but the package but ok.
[12:32] <roaksoax> rvba: yeah i need to sru that
[12:32] <roaksoax> rvba: where you at?
[12:35] <rvba> roaksoax: in the breakout room if that's what you're asking.
[12:36] <roaksoax> rvba: bet here in a bit, have a couple problems
[12:37] <rvba> roaksoax: another question for you: debian/maas-region-controller.lintian-overrides contains references to stuff in usr/share/maas/web/static/jslibs/yui/3.4.1/ and /usr/share/maas/web/static/jslibs does not exist (which is normal as we use the packaged version now).
[12:38] <rvba> roaksoax: why do we still have these references in there?
[14:04] <torment> NodesNotAvailable: No matching node is available.
[14:04] <torment> seeing that in maas.log when trying juju bootstrap
[14:04] <torment> all nodes in the webui show 'ready'
[14:13] <torment> do i need to have maas-dns installed?
[14:18] <mgz> torment: what constraints did you use?
[14:18] <mgz> you may need one of arch=(something not amd64) or mem=0
[14:20] <torment> ah, yeah when they provisioned, maas says 0 memory
[14:21] <mgz> right, that's a bug (that should be) fixed in the latest package, but you need to give the mem=0 constraint till then
[14:21] <torment> yaeh mem=0 worked
[14:21] <mgz> ace.
[14:40] <roaksoax> eva/win 14
[15:38] <torment> closer! its not dhcp'ing during bootstrap - is there a way to tell it what interface?
[15:38] <mgz> rvba: ^ any ideas?
[15:39] <torment> actually i see the DHCPOFFER from maas in terminal 4
[15:40] <torment> but it stops saying no DHCPOFFERS received
[15:40] <torment> yeah its trying on eth0
[15:40] <torment> i think, needs to be p3p1
[15:49] <torment> ah i see it hardcoded here in enlist_userdata