=== bigjools changed the topic of #maas to: Discussion of upstream development of Ubuntu's Metal as a Service (MAAS) tool | Mailing list https://launchpad.net/~maas-devel | Please read http://bit.ly/1j716Fc if you're having a problem with a MAAS instance | MAAS documentation: http://bit.ly/1eIPFAg === vladk|offline is now known as vladk === vladk is now known as vladk|offline === CyberJacob|Away is now known as CyberJacob === vladk|offline is now known as vladk === vladk is now known as vladk|offline === vladk|offline is now known as vladk === vladk is now known as vladk|offline === vladk|offline is now known as vladk === vladk is now known as vladk|offline [06:56] jtv: My loop found 2 other spurious failures. I'm trying to recreate them in isolation now… [06:57] What kind of failures? [06:59] http://paste.ubuntu.com/7740610/ [06:59] http://paste.ubuntu.com/7740611/ [06:59] (I've manually merged gmb's fix for bug 1336617) [07:00] bug 1336617 in MAAS "getRandomNetwork() setting upper bound IP lower than lower bound" [Critical,In progress] https://launchpad.net/bugs/1336617 [07:10] bigjools: why did you remove the note in your documentation branch? (I assume because not letting MAAS manage the DHCP server isn't supported) === vladk|offline is now known as vladk [07:15] rvba: because what it said is not true any more [07:15] bigjools: ah, right. [07:17] jtv: did you found a pointer for me where i can start to look at why the ephemeral images don't use a local mirror ? [07:18] i have time to investigate it myself but need a bit guidance [07:18] william_home: the problem I think is mainly one of getting the http_proxy etc. variables set in the "user data." [07:18] That's what goes into cloud-init IIRC. [07:19] If we could inject our own variable definitions into /etc/profile that way, we'd be home safe. [07:20] ok, so if understand it right Maas has al the bits working but the issue is more that cloud init does not take the parameters like a local archive [07:20] Right. Probably, cloud-init would take the parameters if we knew how to pass them. [07:21] We started out with a model where a client (such as Juju) provided the user data, and we just passed it on unchanged. In that model, we have no way that I know of to do this. But I believe now the user data has become more of a pluggable thing. [07:22] src/metadataserver/commissioning/user_data.py might be a good starting point. [07:23] That's where we generate user data for commissioning. [07:23] and how does that info go to the node starting the ephemeral image? [07:24] does it get passed from the kernel cmdline? [07:24] As I recall, cloud-init on the booting image requests it from the maas metadata service. [07:24] The metadata service's purpose is to provide such information to bootstrapping nodes. [07:26] jtv: thanx, I will start my search [07:26] We have a simple framework for sending multiple files to a commissioning node. (It's also in use for auto-enlisting nodes now). The main generate_user_data function composes a big script out of "snippets" that we have in etc/templates/maas/templates [07:26] yes i have seen that, i had to patch the ipmi detect file for doing import glob [07:27] that is still around in precise -updates maas 1.4 [07:27] otherwise the node never gets the ready state [07:27] rvba has worked with userdata more recently... rvba, to get the commissioning scripts working with an http proxy and/or local archive, do you think it would be enough to make the commissioning user-data inject the settings into /etc/profile? [07:29] jtv: let me have a look at the code… [07:32] jtv: yeah, I believe getting cloudinit to set http_proxy in /etc/profile should do the trick. [07:33] why would "make doc" do this: [07:33] bin/sphinx: 1: cd: can't cd to /home/ed/canonical/maas/sandbox/docs/_build [07:33] o_O [07:33] the dir doesn't exist [07:35] bigjools: that's just like a bug which Gav recently fixed... [07:35] what creates the _build dir? [07:35] *cough* *cough* mumble [07:36] buildout... SAY NO MORE [07:36] heh [07:36] * bigjools distcleans in rough hope [07:37] bin/database --preserve run -- bin/maas-region-admin syncdb --noinput [07:37] pg_ctl: could not start server [07:37] lol [07:37] sigh [07:38] distclean has fixed it [07:38] weird === vladk is now known as vladk|offline [07:42] rvba: http://paste.ubuntu.com/7740740/ [07:43] nice [07:44] looks quite nice after rendering it [07:44] * bigjools enlanderates === vladk|offline is now known as vladk [07:47] rvba: urgh, *another* spurious test failure? [07:47] Two of them even. [07:47] jeez, where are they coming from all of a sudden [07:47] (I only filed one bug thus far) [07:48] Well, me running the whole test suite in a loop :) [07:48] * bigjools moves the DHCP feature card to done-done and rejoices [07:50] bigjools: about that; reading your change to the doc I'm thinking about the upgrade path to 1.6: since we don't populate the DNS zone with mappings from the dynamic range it means an existing deployment will be quite broken after you upgrade right? [07:51] I mean, unless you configure the static range *and* re-allocate all your nodes. [07:52] rvba: mmm it won't be broken until the zone is re-written on the first node allocation [07:52] then all the deployed nodes lose their DNS [07:52] oh bugger [07:52] bigjools: the DNS config is rewritten when the package gets re-installed [07:53] s/re-installed/upgraded/ [07:53] then oh bugger without the preamble [07:53] bugger indeed [07:53] We kept the code so it's just a matter of flipping a switch. [07:53] I have to run, but I can talk about this in half an hour [07:54] Okay [07:54] rvba: the solution is to give dynamic nodes DNS entries :/ [07:54] one-line code change [07:54] I know. Like I said, it's easy. [07:54] but that is uuuuugly [07:54] let;'s have a think [07:54] TTYL [07:54] k [07:55] bigjools: we could restrict the dynamic mappings sent to the DNS machinery to the MAC addresses that correspond to allocated nodes. === vladk is now known as vladk|offline === vladk|offline is now known as vladk [09:25] gmb: care to have a look at https://code.launchpad.net/~rvb/maas/bug-1337190/+merge/225444 ? [09:31] rvba: Certainly [09:31] Ta [09:32] rvba: A pithy fix. Me like. [09:33] gmb: as you can imagine, the problem manifests itself very rarely. I had to resort to run-one-until-failure to debug that one :). [09:34] rvba: Yeah, been there, done that :) [09:34] heh [09:45] And another one: https://code.launchpad.net/~rvb/maas/none-interf/+merge/225447 [09:57] gmb: care to have a look? ^ It's tiny. [09:57] rvba: Already reviewed it :) [09:57] Nice; thank you gmb. [11:19] rvba: i don't see any reference that apt_mirror is used by cloud-init anywhere, is that right? [11:20] http_proxy seems to work and otherwise the maas instance gets used with squid-deb-proxy [11:21] when i manually do a wget to the maas instance getting the cloud-cofig-url i see that in the response i get the local mirror setting in apt_mirror: and i see apt_proxy: which is set to the maas instance [11:21] so apt_proxy gets used during init but not the local mirror archive === vladk is now known as vladk|offline === vladk|offline is now known as vladk === liam_ is now known as Guest45753 [12:00] jtv: Istr that you made some changes to the boot resources downloading stuff? [12:42] rvba: when i do a manual cloud-init in a node commisioning then i get only apt_proxy in /var/lib/cloud/instance/cloud-config.txt [12:43] allenap: some changes when? [12:43] when adding manually in that file apt_mirror: http:/xyz/ubuntu and then run cloud-init modules --mode=config then my sources.list gets updated accordingly [12:44] jtv: I dunno… at some time :) Did you add another level to the fs hierarchy? [12:45] william_home: looks like apt_mirror overrides apt_proxy somehow. [12:45] allenap: Not me, but the OS is now an extra layer. === CyberJacob is now known as CyberJacob|Away [12:46] rvba: i think not, because apt_proxy does get set but apt_mirror never gets into the cloud-config.txt file [12:46] william_home: ah, you're right, apt_mirror is only used in the curtin (i.e. fast path installed) user data. [12:46] installer* [12:47] rvba: but is it possible to do this also for a node in commisionng mode? else ipmi packages cannot be installed and never gets in the ready state [12:49] william_home: the configuration you've done (setting apt_mirror in the preseed template) seems like the proper way to do this. [12:49] I'm trying to build a maas/juju/openstack cluster but i have no internet connection whatsoever so i'm left to a local mirror for everything [12:50] also the preseed does not seem to work. [12:50] Right. MAAS assumes the connection to the Ubuntu archive can be done without the global proxy. Which I guess can be considered as a bug. [12:50] Indeed, I’d say that’s a bug. [12:51] that bug is already open but i think the fix is simple [12:52] could maas also for a node in commisioning mode push the apt_mirror variable? [12:54] Was that you rvba or gmb? (Adding an OS directory layer to the boot resources hierarchy) [12:54] I don't recall doing it. [12:55] william_home: that's one way to do it. But now I'm wondering why http_proxy is not considered when retrieving packages. [12:55] I’ll go and dig it out then. I might even have done it, who knows :-/ [12:56] rvba: when using a proxy I still have to use a local mirror [12:56] archive.ubuntu.com is internally not resolveable [12:56] i could add some hooks for that but making maas use local mirror is better i guess [12:57] My environment has an airgap for the internet [12:59] By default, the nodes will use the squid-deb proxy on the region controller. I suggest you configure it to talk to the mirror you want to use. [13:01] rvba: thats what I did already but on the commisioning node the sources.list still holds http://archive.ubuntu.com/ubuntu as the main archive while the apt_proxy is set to the maas node [13:02] the proxy get asked for archive.ubuntu.com and not for my local mirror [13:04] Did you enlist your nodes using the UI/API or did you get them auto-enlisted by booting them up and letting them register themselves with your MAAS server? [13:05] rvba: I preconfigured them in the maas environment [13:05] setup ipmi and added the right mac address [13:08] allenap: I believe it was one of Blake / Jason. [13:08] Or it might have been me. [13:08] william_home: did you change the "Main archive" config option? [13:08] Hehe, welcome to the far side of the hill gmb :) [13:09] :) [13:10] allenap: No, it wasn’t me; that must’ve been part of the CentOS/Windows support work. [13:11] rvba: yes, that one is pointing to the local mirror and when doing a wget to the maas-cloud-config-url......preseed, then I see that the apt_mirror: http://xyz/ubuntu and also the apt_proxy is there [13:11] but it is not there when cloud-init init gets called which writes the cloud-config,txt file [13:13] so if cloud-init calls the cloud-config url for all metadata things it does not get the apt_mirror variable but does get the apt_proxy var [13:16] apt_mirror should clearly be set for the commissioning preseed as well. === jfarschman is now known as MilesDenver === vladk is now known as vladk|offline === vladk|offline is now known as vladk === alexpilotti_ is now known as alexpilotti === vladk is now known as vladk|offline === vladk|offline is now known as vladk === vladk is now known as vladk|offline [15:44] Is there a good place to ask preseed questions? Like #preseed :) I'm trying to set a puppet user with specific uid, but Preseed hangs on me [15:45] http://pastebin.com/eDKH4yvU === vladk|offline is now known as vladk === vladk is now known as vladk|offline === vladk|offline is now known as vladk [15:48] MilesDenver: you should probably try freenode#server. [15:49] rvba: think it is solved now [15:49] ty [15:53] william_home: what did you do? [15:53] I now updated the template file (removed the node from the interface) and readded it [15:53] the template file gets served from the database [15:53] like a cache [15:53] that costed me a lot off time, not a programmer [15:54] :) [15:55] so now i added apt_mirror: http://{{main_archive_hostname}}/{{main_archive_directory}} to user_data_config.template [15:55] readded the node [15:55] and ran cloud-init in it on the rebooted node [15:56] now i got also my local mirror in the cloud-config.txt file [15:57] William: what file are you working in? I'm trying to learn and you seem to be on to something [15:57] the only thing that you have todo manually is adding the local mirror in /etc/squid-deb-proxy/mirror-dstdomain.acl [15:58] gmb: I'm about to sign off for the day but maybe you'll fancy having a look at this spurious failure: http://paste.ubuntu.com/7742482/ [15:58] rvba: Sure. [15:59] restart squid-deb-proxy and add the above oneliner to /etc/maas/templates/commisioning-user-data/user_data_config.template [15:59] Seems related to what you fixed the other day. [16:00] rvba: so adding the oneliner should do the trick and maas should also update the squid-deb-proxy dstdomain acl [16:01] william_home: right, that is bug 1300266. [16:01] bug 1300266 in MAAS "squid-deb-proxy returns 403 when admin configures a custom APT archive" [High,Triaged] https://launchpad.net/bugs/1300266 [16:01] * gmb resorts to run-one-until-failure [16:02] rvba: OH! [16:02] I can see why. [16:02] I’ts because I’m an eejit. [16:02] (which also explains how I use my apostrophes) [16:02] heh [16:02] rvba: yes, and the oneliner should go into the documentation or the template file should be updated, https://bugs.launchpad.net/maas/+bug/1288502 [16:02] Ubuntu bug 1288502 in MAAS "archive and proxy settings not honoured for commissioning" [High,Triaged] [16:03] or maybe not a one liner but something with if statement like the apt_proxy in the same file [16:04] gmb: I also have http://paste.ubuntu.com/7741493/ which seems to be about using make_network*s* instead of make_network. [16:05] Hrm. [16:05] But that's just a guess. [16:06] MilesDenver: did i answer your question in the mean time? [16:06] :) [16:08] rvba: should I do something like adding this info to the lp report 1288502? [16:09] william_home: yes, thanks. I poked around a bit. [16:15] william_home: yes, that would be great. === vladk is now known as vladk|offline === roadmr is now known as roadmr_afk [16:36] I have a DNS question, why is the hostname a cname? [16:36] and not an an A record? [16:37] as this is causing issued when attempting to install CDH5. [16:42] dogfood: That will change in MAAS 1.6, which we’ll be releasing soon (not quite sure when yet) [16:42] Not sure of the historical reasons for having a cname there though. [16:42] when adding a node to CDH5 I use the FQDN from the region controller webUI, say hadoop-1.maas.company.com. However CDH5 does a forward and reverse lookup and uses the 10-122-75-116.maas.company.com FQDN. [16:45] gmb: What will the change be in 1.6? Will the ip based FQDN be removed? === jfarschman is now known as MilesDenver [16:46] dogfood: Sorry, the DNS entry for the hostname will become a real A record rather than a CNAME. [16:46] gmb: but the reverse DNS entry will still point to the ip based FQDN (10-122-75-116.maas.company.com) ? [16:47] dogfood: I’m not sure; let me check and get back to you. [16:47] gbm: thank you [16:53] dogfood: AFAICT from a quick look at the code, the reverse entry will no longer point to the ip-based FQDN; it should work properly using the FQDN from the region. [16:55] gmb: any idea on the 1.6 release? [16:56] not looking for a definite time frame, just a rough idea on when it might be release? [16:57] dogfood: Next couple of weeks, I’d hope. We’re in the process of finalising that. [16:58] gmb: thanks for looking at the code and getting back to me. I really appreciated it. [16:58] dogfood: No problem, glad I could help :). === roadmr_afk is now known as roadmr === jlundy1 is now known as jlundy === CyberJacob|Away is now known as CyberJacob === jfarschman is now known as MilesDenver === vladk|offline is now known as vladk === vladk is now known as vladk|offline === vladk|offline is now known as vladk === vladk is now known as vladk|offline === jfarschman is now known as MilesDenver === jfarschman is now known as MilesDenver [22:31] Hi all, if I want to pass a different DNS server for a managed interface? How I can specific this? When I select dhcp+dns the DNS that it is passed is my maas server which is not running named. I want to pass my main DNS not my maas server === CyberJacob is now known as CyberJacob|Away === jfarschman is now known as MilesDenver