| roaksoax | rvba: ok so upgrades does work, matsubara verified all the bugs, the only thing might be upgrading bind9 itself and having things broken | 09:11 |
|---|---|---|
| rvba | roaksoax: ok, did you manage to understand why it's failing to upgrade? What do the log says? | 09:29 |
| roaksoax | rvba: no idea, last night after the party I upgraded again and it worked fine.. and continues too.. so maybe it was due to upgrading bind9? | 09:30 |
| bigjools | roaksoax: where is maas-enlist called from? | 09:40 |
| roaksoax | bigjools: what do you mean? | 09:41 |
| jtv | roaksoax: I think he means: when a node enlists, what is it that calls maas-enlist? | 09:42 |
| bigjools | roaksoax: I want to know where maas-enlist is called so I can see what args it gets passed | 09:42 |
| roaksoax | bigjools: ah! enlist user data | 09:42 |
| bigjools | roaksoax: cool thanks | 09:43 |
| roaksoax | bigjools: /usr/share/maas/preseeds/enlist_userdata | 09:43 |
| roaksoax | bigjools: is there any issues with enlistment? | 09:43 |
| jtv | Yes — it uses the wrong hostname. | 09:43 |
| bigjools | roaksoax: yes, it's setting the hostname to something that fucks up things | 09:43 |
| roaksoax | bigjools: what hostname? | 09:43 |
| roaksoax | bigjools: where are you using it? | 09:43 |
| bigjools | roaksoax: it sets it to an IP address with dashes, right? | 09:44 |
| roaksoax | bigjools: right | 09:44 |
| bigjools | roaksoax: that breaks MAAS's DNS | 09:44 |
| bigjools | because maas generates that style of hostname by default | 09:44 |
| bigjools | and it really confuses users when the IP address changes | 09:45 |
| roaksoax | bigjools: right, so then you need to either make DNS static or not pre-assign dns names for IP addresses | 09:45 |
| bigjools | roaksoax: we already have static DNS entries of the form N-N-N-N | 09:45 |
| bigjools | which is why this breaks | 09:45 |
| roaksoax | bigjools: right, but that's not maas-enlist fault | 09:46 |
| bigjools | we use CNAME for user-amended hostnames | 09:46 |
| bigjools | roaksoax: what hostname format does maas-enlist send? | 09:46 |
| roaksoax | bigjools: maas-enlist does not send a hostname, unless it finds a DNS server which is providing a hostname. This is the requirement for having externally managed DNS/DHCP servers | 09:46 |
| bigjools | roaksoax: ah ok | 09:47 |
| * jtv facepalms | 09:47 | |
| bigjools | roaksoax: we don't want it to do that | 09:47 |
| roaksoax | bigjools: but since MAAS is already filling up DNS names for IP address (before even having a system enlistment) the enlistment processs obviously finds a DNS service, with DNS assigned to an IP address | 09:47 |
| roaksoax | bigjools: right, but we do want it to do that | 09:48 |
| roaksoax | bigjools: otherwise it breaks things | 09:48 |
| bigjools | roaksoax: what does it break? | 09:48 |
| roaksoax | otherwise we wont be able to use it with external DNS | 09:48 |
| jtv | In what way do we need to use this particular hostname with external DNS? | 09:49 |
| jtv | Because our DNS entries already point that name to that host. | 09:49 |
| roaksoax | here's the deal | 09:49 |
| roaksoax | what if we don't use DNS/DHCP in MAAS | 09:49 |
| roaksoax | and we have an external DNS/DHCP | 09:49 |
| roaksoax | if that happens, then the enlistment process needs to detect the DNS name for the particular node | 09:50 |
| roaksoax | becuase in those case escenarios, you assign a hostname for a particular host | 09:51 |
| roaksoax | s/hostname/dns-name | 09:51 |
| roaksoax | so maas-enlist needs to detect that | 09:51 |
| bigjools | roaksoax: right, that makes sense | 09:51 |
| roaksoax | right, so now, the problem is when we use MAAS managed DNS/DHCP server, maas pre-fills DNS with hostnames for each IP address regardless of whether we want to assign them or not | 09:52 |
| roaksoax | but the enlistment process still checks for a DNS name for the IP it's gotten | 09:52 |
| roaksoax | and since it finds it, it sends it back to maas | 09:52 |
| roaksoax | becuase it found a dns name for its IP | 09:52 |
| bigjools | roaksoax: thanks for clarifying, we're talking about how to fix it now | 09:54 |
| roaksoax | bigjools: I believe that the correct way to do that, is simply make sure that when a node gets an IP from MAAS DHCP server, that IP/DNS name gets "statically" assigned to that server | 09:55 |
| roaksoax | bigjools: so everytime, the server uses the same IP address and not randomly assigned IP every time it boots | 09:56 |
| bigjools | roaksoax: that already happens | 09:56 |
| roaksoax | bigjools: so if that ahppens, then there should not be anyprobllems | 09:57 |
| bigjools | roaksoax: the problem is that it passes the hostname from the maas-dns reverse zone, which is also in our forward zone and which we then try and assign to a CNAME | 09:57 |
| bigjools | which blows up dns | 09:58 |
| bigjools | we have a workaround | 09:58 |
| roaksoax | bigjools: right, why does it blow up DNS? is DNS not working anymore because of this? | 09:58 |
| bigjools | roaksoax: because there's a CNAME which points to itself | 09:58 |
| roaksoax | bigjools: where's the CNAME stored? | 09:59 |
| bigjools | in the forward zone | 09:59 |
| roaksoax | root@ubuntu:/etc/bind# grep -sr 192-168-123-101 * | 09:59 |
| roaksoax | maas/zone.123.168.192.in-addr.arpa:101 IN PTR 192-168-123-101.master. | 09:59 |
| roaksoax | maas/zone.master:192-168-123-101 IN A 192.168.123.101 | 09:59 |
| roaksoax | bigjools: ^^ | 09:59 |
| roaksoax | bigjools: that's the only thing I have stored | 09:59 |
| roaksoax | bigjools: that's with what it is in -proposed | 09:59 |
| bigjools | roaksoax: because there's a hack in to prevent it getting stored if it matches the existing auto-generated hostname | 10:00 |
| roaksoax | bigjools: yeah I see now: http://pastebin.ubuntu.com/1302175/ | 10:01 |
| roaksoax | bigjools: so that means we need something more to SRU | 10:01 |
| bigjools | roaksoax: "maas/zone.master:192-168-123-103 IN CNAME 192-168-123-102" | 10:01 |
| bigjools | that's massively confusing for people :) | 10:02 |
| roaksoax | bigjools: indeed | 10:02 |
| bigjools | we're going to ignore hostnames from maas-enlist if we are managing DNS ourselves | 10:02 |
| bigjools | roaksoax: we want to generate a special hostname anyway | 10:02 |
| roaksoax | bigjools: that would make sense | 10:03 |
| roaksoax | bigjools: so then change would be in maas side rather than enlistment process right? | 10:04 |
| bigjools | roaksoax: yes | 10:06 |
| bigjools | roaksoax: FWIW here's our upcoming SRU bug list: https://bugs.launchpad.net/maas/+milestone/12.10-stabilization | 10:06 |
| ubot5 | Ubuntu bug 12 in Launchpad itself ""Next 10 messages" changes Display Settings" [Medium,Fix released] | 10:06 |
| roaksoax | bigjools: oh wow... so I think we should do those a few at a time | 10:06 |
| bigjools | roaksoax: we're doing them in the next 2 months and then we're moving on | 10:07 |
| roaksoax | bigjools: awesome! | 10:07 |
| roaksoax | there's a couple of them that are probably easy to release now | 10:07 |
| bigjools | yep | 10:11 |
| bigjools | roaksoax: your name is on https://bugs.launchpad.net/bugs/1064224 - are you really fixing it? | 10:13 |
| ubot5 | Ubuntu bug 1064224 in MAAS "IPMI detection ends up with power_address of 0.0.0.0" [Critical,Triaged] | 10:13 |
| roaksoax | bigjools: before I came here i saw exactly the same thing on 1 of the Mini servers I have | 10:14 |
| roaksoax | bigjools: so I'd like to make somre more investigation | 10:14 |
| bigjools | roaksoax: AHA! | 10:14 |
| bigjools | roaksoax: the bMC has no IP address | 10:14 |
| bigjools | look at dhcp log | 10:14 |
| bigjools | it doesn't accept the offered IP and continues to request | 10:14 |
| bigjools | something is screwy in the quantal isc-dhcpd I think | 10:14 |
| bigjools | I think this happened after I upgraded my maas server to quantal | 10:15 |
| roaksoax | bigjools: maybe, but in my case I was running dd-wrt | 10:15 |
| bigjools | oh :/ | 10:15 |
| roaksoax | my dd-wrt router was the one in charge of giving the IP address to the BMC | 10:15 |
| bigjools | I wonder if the ipmi detection code buggered the bmc then | 10:16 |
| roaksoax | bigjools: so 2 of the nodes correctly enlisted with the correct IP address | 10:16 |
| roaksoax | the BMC query to the 3er node returned 0.0.0.0 even though I could reach it | 10:16 |
| bigjools | doesn't sound like the same problem as mine | 10:16 |
| roaksoax | bigjools: so I turned the server on with ipmipower command, and that node in particular, returned 0.0.0.0 instead of the IP of the BMC even though it was set | 10:16 |
| roaksoax | bigjools: i do believe it might be related | 10:18 |
| jtv1 | Does the bmc expose its logs anywhere? | 10:18 |
| roaksoax | bigjools: maybe your BMC is getting the IP, but the query fails for some reason (or time's out) or something | 10:18 |
| roaksoax | I will have to test when I get back | 10:18 |
| roaksoax | jtv1: and not that i know of | 10:18 |
| bigjools | roaksoax: It is not getting an IP | 10:19 |
| bigjools | I can confirm this by looking at dhcp logs | 10:19 |
| roaksoax | bigjools: ok, so your bmc was not accessible then? | 10:22 |
| bigjools | roaksoax: exactly | 10:22 |
| roaksoax | bigjools: ok, yeah weird | 10:23 |
| roaksoax | bigjools: i did have some kind of problems trying to get the BMC to obtain an IP address | 10:28 |
| roaksoax | bigjools: but at the end of the day, i just had to disconnect the server from the power | 10:28 |
| bigjools | roaksoax: can you try to get the bmc to request address from a quantal isc-dhcpd to replicate my scenario | 10:28 |
| roaksoax | bigjools: i will when i get back home | 10:29 |
| bigjools | you mean you didn't bring your microservers here? :) | 10:29 |
| roaksoax | bigjools: lol... i had remote access to my house and the cluster... but it got killed for some reason so I can't access anymore | 10:33 |
| === jtv1 is now known as jtv | ||
| rvba | bigjools: https://bugs.launchpad.net/maas/+bug/1070774 | 10:43 |
| ubot5 | Ubuntu bug 1070774 in MAAS "The hostname of a node can still be changed once the node is in use." [Undecided,New] | 10:43 |
| rvba | bigjools: https://bugs.launchpad.net/maas/+bug/1070775 | 10:45 |
| ubot5 | Ubuntu bug 1070775 in MAAS "The zone name (attached to a cluster controller) can still be changed when it contains in-use nodes. " [Undecided,New] | 10:45 |
| roaksoax | rvba: yay!! use_squashfs seems broken | 12:29 |
| rvba | roaksoax: !? | 12:29 |
| roaksoax | rvba: squashfs image are present, but they are not being used | 12:30 |
| rvba | roaksoax: I think I already asked you this… but this is fixed in the packaging right? http://paste.ubuntu.com/1302370/ | 12:30 |
| rvba | This is while installing maas from the packaging in the archive. | 12:30 |
| roaksoax | rvba: dbc_go: not found --> that doesn't really affect anything | 12:30 |
| roaksoax | rvba: it is present in precise | 12:30 |
| rvba | That's from the quantal package. | 12:31 |
| roaksoax | rvba: yeah | 12:31 |
| roaksoax | rvba: so I fixed it and reverted the change | 12:31 |
| rvba | It's still a bit ugly to get that error message when installing but the package but ok. | 12:32 |
| roaksoax | rvba: yeah i need to sru that | 12:32 |
| roaksoax | rvba: where you at? | 12:32 |
| rvba | roaksoax: in the breakout room if that's what you're asking. | 12:35 |
| roaksoax | rvba: bet here in a bit, have a couple problems | 12:36 |
| rvba | roaksoax: another question for you: debian/maas-region-controller.lintian-overrides contains references to stuff in usr/share/maas/web/static/jslibs/yui/3.4.1/ and /usr/share/maas/web/static/jslibs does not exist (which is normal as we use the packaged version now). | 12:37 |
| rvba | roaksoax: why do we still have these references in there? | 12:38 |
| torment | NodesNotAvailable: No matching node is available. | 14:04 |
| torment | seeing that in maas.log when trying juju bootstrap | 14:04 |
| torment | all nodes in the webui show 'ready' | 14:04 |
| torment | do i need to have maas-dns installed? | 14:13 |
| mgz | torment: what constraints did you use? | 14:18 |
| mgz | you may need one of arch=(something not amd64) or mem=0 | 14:18 |
| torment | ah, yeah when they provisioned, maas says 0 memory | 14:20 |
| mgz | right, that's a bug (that should be) fixed in the latest package, but you need to give the mem=0 constraint till then | 14:21 |
| torment | yaeh mem=0 worked | 14:21 |
| mgz | ace. | 14:21 |
| roaksoax | eva/win 14 | 14:40 |
| torment | closer! its not dhcp'ing during bootstrap - is there a way to tell it what interface? | 15:38 |
| mgz | rvba: ^ any ideas? | 15:38 |
| torment | actually i see the DHCPOFFER from maas in terminal 4 | 15:39 |
| torment | but it stops saying no DHCPOFFERS received | 15:40 |
| torment | yeah its trying on eth0 | 15:40 |
| torment | i think, needs to be p3p1 | 15:40 |
| torment | ah i see it hardcoded here in enlist_userdata | 15:49 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!