/srv/irclogs.ubuntu.com/2012/10/24/#maas.txt

roaksoaxrvba: ok so upgrades does work, matsubara verified all the bugs, the only thing might be upgrading bind9 itself and having things broken09:11
rvbaroaksoax: ok, did you manage to understand why it's failing to upgrade?  What do the log says?09:29
roaksoaxrvba: no idea, last night after the party I upgraded again and it worked fine.. and continues too.. so maybe it was due to upgrading bind9?09:30
bigjoolsroaksoax: where is maas-enlist called from?09:40
roaksoaxbigjools: what do you mean?09:41
jtvroaksoax: I think he means: when a node enlists, what is it that calls maas-enlist?09:42
bigjoolsroaksoax: I want to know where maas-enlist is called so I can see what args it gets passed09:42
roaksoaxbigjools: ah! enlist user data09:42
bigjoolsroaksoax: cool thanks09:43
roaksoaxbigjools: /usr/share/maas/preseeds/enlist_userdata09:43
roaksoaxbigjools: is there any issues with enlistment?09:43
jtvYes — it uses the wrong hostname.09:43
bigjoolsroaksoax: yes, it's setting the hostname to something that fucks up things09:43
roaksoaxbigjools: what hostname?09:43
roaksoaxbigjools: where are you using it?09:43
bigjoolsroaksoax: it sets it to an IP address with dashes, right?09:44
roaksoaxbigjools: right09:44
bigjoolsroaksoax: that breaks MAAS's DNS09:44
bigjoolsbecause maas generates that style of hostname by default09:44
bigjoolsand it really confuses users when the IP address changes09:45
roaksoaxbigjools: right, so then you need to either make DNS static or not pre-assign dns names for IP addresses09:45
bigjoolsroaksoax: we already have static DNS entries of the form N-N-N-N09:45
bigjoolswhich is why this breaks09:45
roaksoaxbigjools: right, but that's not maas-enlist fault09:46
bigjoolswe use CNAME for user-amended hostnames09:46
bigjoolsroaksoax: what hostname format does maas-enlist send?09:46
roaksoaxbigjools: maas-enlist does not send a hostname, unless it finds a DNS server which is providing a hostname. This is the requirement for having externally managed DNS/DHCP servers09:46
bigjoolsroaksoax: ah ok09:47
* jtv facepalms09:47
bigjoolsroaksoax: we don't want it to do that09:47
roaksoaxbigjools: but since MAAS is already filling up DNS names for IP address (before even having a system enlistment) the enlistment processs obviously finds a DNS service, with DNS assigned to an IP address09:47
roaksoaxbigjools: right, but we do want it to do that09:48
roaksoaxbigjools: otherwise it breaks things09:48
bigjoolsroaksoax: what does it break?09:48
roaksoaxotherwise we wont be able to use it with external DNS09:48
jtvIn what way do we need to use this particular hostname with external DNS?09:49
jtvBecause our DNS entries already point that name to that host.09:49
roaksoaxhere's the deal09:49
roaksoaxwhat if we don't use DNS/DHCP in MAAS09:49
roaksoaxand we have an external DNS/DHCP09:49
roaksoaxif that happens, then the enlistment process needs to detect the DNS name for the particular node09:50
roaksoaxbecuase in those case escenarios, you assign a hostname for a particular host09:51
roaksoaxs/hostname/dns-name09:51
roaksoaxso maas-enlist needs to detect that09:51
bigjoolsroaksoax: right, that makes sense09:51
roaksoaxright, so now, the problem is when we use MAAS managed DNS/DHCP server, maas pre-fills DNS with hostnames for each IP address regardless of whether we want to assign them or not09:52
roaksoaxbut the enlistment process still checks for a DNS name for the IP it's gotten09:52
roaksoaxand since it finds it, it sends it back to maas09:52
roaksoaxbecuase it found a dns name for its IP09:52
bigjoolsroaksoax: thanks for clarifying, we're talking about how to fix it now09:54
roaksoaxbigjools: I believe that the correct way to do that, is simply make sure that when a node gets an IP from MAAS DHCP server, that IP/DNS name gets "statically" assigned to that server09:55
roaksoaxbigjools: so everytime, the server uses the same IP address and not randomly assigned IP every time it boots09:56
bigjoolsroaksoax: that already happens09:56
roaksoaxbigjools: so if that ahppens, then there should not  be anyprobllems09:57
bigjoolsroaksoax: the problem is that it passes the hostname from the maas-dns reverse zone, which is also in our forward zone and which we then try and assign to a CNAME09:57
bigjoolswhich blows up dns09:58
bigjoolswe have a workaround09:58
roaksoaxbigjools: right, why does it blow up DNS? is DNS not working anymore because of this?09:58
bigjoolsroaksoax: because there's a CNAME which points to itself09:58
roaksoaxbigjools: where's the CNAME stored?09:59
bigjoolsin the forward zone09:59
roaksoaxroot@ubuntu:/etc/bind# grep -sr 192-168-123-101 *09:59
roaksoaxmaas/zone.123.168.192.in-addr.arpa:101 IN PTR 192-168-123-101.master.09:59
roaksoaxmaas/zone.master:192-168-123-101 IN A 192.168.123.10109:59
roaksoaxbigjools: ^^09:59
roaksoaxbigjools: that's the only thing I have stored09:59
roaksoaxbigjools: that's with what it is in -proposed09:59
bigjoolsroaksoax: because there's a hack in to prevent it getting stored if it matches the existing auto-generated hostname10:00
roaksoaxbigjools: yeah I see now: http://pastebin.ubuntu.com/1302175/10:01
roaksoaxbigjools: so that means we need something more to SRU10:01
bigjoolsroaksoax: "maas/zone.master:192-168-123-103 IN CNAME 192-168-123-102"10:01
bigjoolsthat's massively confusing for people :)10:02
roaksoaxbigjools: indeed10:02
bigjoolswe're going to ignore hostnames from maas-enlist if we are managing DNS ourselves10:02
bigjoolsroaksoax: we want to generate a special hostname anyway10:02
roaksoaxbigjools: that would make sense10:03
roaksoaxbigjools: so then change would be in maas side rather than enlistment process right?10:04
bigjoolsroaksoax: yes10:06
bigjoolsroaksoax: FWIW here's our upcoming SRU bug list: https://bugs.launchpad.net/maas/+milestone/12.10-stabilization10:06
ubot5Ubuntu bug 12 in Launchpad itself ""Next 10 messages" changes Display Settings" [Medium,Fix released]10:06
roaksoaxbigjools: oh wow... so I think we should do those a few at a time10:06
bigjoolsroaksoax: we're doing them in the next 2 months and then we're moving on10:07
roaksoaxbigjools: awesome!10:07
roaksoaxthere's a couple of them that are probably easy to release now10:07
bigjoolsyep10:11
bigjoolsroaksoax: your name is on https://bugs.launchpad.net/bugs/1064224 - are you really fixing it?10:13
ubot5Ubuntu bug 1064224 in MAAS "IPMI detection ends up with power_address of 0.0.0.0" [Critical,Triaged]10:13
roaksoaxbigjools: before I came here i saw exactly the same thing on 1 of the Mini servers I have10:14
roaksoaxbigjools: so I'd like to make somre more investigation10:14
bigjoolsroaksoax: AHA!10:14
bigjoolsroaksoax: the bMC has no IP address10:14
bigjoolslook at dhcp log10:14
bigjoolsit doesn't accept the offered IP and continues to request10:14
bigjoolssomething is screwy in the quantal isc-dhcpd I think10:14
bigjoolsI think this happened after I upgraded my maas server to quantal10:15
roaksoaxbigjools: maybe, but in my case I was running dd-wrt10:15
bigjoolsoh :/10:15
roaksoaxmy dd-wrt router was the one in charge of giving the IP address to the BMC10:15
bigjoolsI wonder if the ipmi detection code buggered the bmc then10:16
roaksoaxbigjools: so 2 of the nodes correctly enlisted with the correct IP address10:16
roaksoaxthe BMC query to the 3er node returned 0.0.0.0 even though I could reach it10:16
bigjoolsdoesn't sound like the same problem as mine10:16
roaksoaxbigjools: so I turned the server on with ipmipower command, and that node in particular, returned 0.0.0.0 instead of the IP of the BMC even though it was set10:16
roaksoaxbigjools: i do believe it might be related10:18
jtv1Does the bmc expose its logs anywhere?10:18
roaksoaxbigjools: maybe your BMC is getting the IP, but the query fails for some reason (or time's out) or something10:18
roaksoaxI will have to test when I get back10:18
roaksoaxjtv1: and not that i know of10:18
bigjoolsroaksoax: It is not getting an IP10:19
bigjoolsI can confirm this by looking at dhcp logs10:19
roaksoaxbigjools: ok, so your bmc was not accessible then?10:22
bigjoolsroaksoax: exactly10:22
roaksoaxbigjools: ok, yeah weird10:23
roaksoaxbigjools: i did have some kind of problems trying to get the  BMC to obtain an IP address10:28
roaksoaxbigjools: but at the end of the day, i just had to disconnect the server from the power10:28
bigjoolsroaksoax: can you try to get the bmc to request address from a quantal isc-dhcpd to replicate my scenario10:28
roaksoaxbigjools: i will when i get back home10:29
bigjoolsyou mean you didn't bring your microservers here? :)10:29
roaksoaxbigjools: lol... i had remote access to my house and the cluster... but it got killed for some reason so I can't access anymore10:33
=== jtv1 is now known as jtv
rvbabigjools: https://bugs.launchpad.net/maas/+bug/107077410:43
ubot5Ubuntu bug 1070774 in MAAS "The hostname of a node can still be changed once the node is in use." [Undecided,New]10:43
rvbabigjools: https://bugs.launchpad.net/maas/+bug/107077510:45
ubot5Ubuntu bug 1070775 in MAAS "The zone name (attached to a cluster controller) can still be changed when it contains in-use nodes. " [Undecided,New]10:45
roaksoaxrvba: yay!! use_squashfs seems broken12:29
rvbaroaksoax: !?12:29
roaksoaxrvba: squashfs image are present, but they are not being used12:30
rvbaroaksoax: I think I already asked you this… but this is fixed in the packaging right? http://paste.ubuntu.com/1302370/12:30
rvbaThis is while installing maas from the packaging in the archive.12:30
roaksoaxrvba: dbc_go: not found --> that doesn't really affect anything12:30
roaksoaxrvba: it is present in precise12:30
rvbaThat's from the quantal package.12:31
roaksoaxrvba: yeah12:31
roaksoaxrvba: so I fixed it and reverted the change12:31
rvbaIt's still a bit ugly to get that error message when installing but the package but ok.12:32
roaksoaxrvba: yeah i need to sru that12:32
roaksoaxrvba: where you at?12:32
rvbaroaksoax: in the breakout room if that's what you're asking.12:35
roaksoaxrvba: bet here in a bit, have a couple problems12:36
rvbaroaksoax: another question for you: debian/maas-region-controller.lintian-overrides contains references to stuff in usr/share/maas/web/static/jslibs/yui/3.4.1/ and /usr/share/maas/web/static/jslibs does not exist (which is normal as we use the packaged version now).12:37
rvbaroaksoax: why do we still have these references in there?12:38
tormentNodesNotAvailable: No matching node is available.14:04
tormentseeing that in maas.log when trying juju bootstrap14:04
tormentall nodes in the webui show 'ready'14:04
tormentdo i need to have maas-dns installed?14:13
mgztorment: what constraints did you use?14:18
mgzyou may need one of arch=(something not amd64) or mem=014:18
tormentah, yeah when they provisioned, maas says 0 memory14:20
mgzright, that's a bug (that should be) fixed in the latest package, but you need to give the mem=0 constraint till then14:21
tormentyaeh mem=0 worked14:21
mgzace.14:21
roaksoaxeva/win 1414:40
tormentcloser! its not dhcp'ing during bootstrap - is there a way to tell it what interface?15:38
mgzrvba: ^ any ideas?15:38
tormentactually i see the DHCPOFFER from maas in terminal 415:39
tormentbut it stops saying no DHCPOFFERS received15:40
tormentyeah its trying on eth015:40
tormenti think, needs to be p3p115:40
tormentah i see it hardcoded here in enlist_userdata15:49

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!