=== cgseller_is_away is now known as imthewherd === zz_CyberJacob is now known as CyberJacob [06:55] Hello, I am newvie for maas, now I got an error "maas boot-resources import". In /var/log/maas/maas.log, I got "[WARNING] Finished importing boot images, the region does not have any boot images available.". The http-proxy exists in my network and thus, it requires user/password authentication. I would like to set HTTP_PROXY to maas. Does anyone know how? === CyberJacob is now known as zz_CyberJacob === wojdev_ is now known as wojdev [07:56] MAAS seems like not accepting "http_proxy=http://:/ipaddr:port" style parameter, as declared . Anyone know the tips? [12:08] Bug #1463378 opened: Downloaded di-initrd on trusty/amd64 overrides /etc/network/interfaces [14:00] Hello!.. [14:00] I'm importing boot image on Maas but the static in web gui still in "Qeued in download" [14:01] vibol, 1.5 or 1.7? [14:01] er.. sorr please wait [14:01] i'm installion from ppa:maas-maintainer/stable [14:04] @@ anyway really sorry how to check maas version @kiko ? [14:05] 1.7.5 @kiko [14:07] hmm [14:07] interesting [14:07] vibol, are you behind a proxy? [14:07] i'm no... [14:07] but i notices [14:07] some repositry of my ubuntu kinda slow [14:07] if you look at *.log in /var/log/maas you'll get an idea of what is happening [14:08] if there are timeouts or similar [14:11] http://paste.ubuntu.com/11672885/ [14:11] [INFO] Started importing boot image [14:11] [WARNING] Finished importing boot images, the region does not have any boot images available. [14:11] WARNING] No boot images have been imported from the region. [14:11] Still have no idea @kiko [14:13] vibol, the region controller downloads the images first, and then the cluster controller downloads them from the region [14:14] vibol, so what's happening is that the region controller is unable to download the images [14:14] vibol, can you confirm you can access the internet from the region controller machine? [14:14] I don't have reqion controllor machine yet maybe @@ [14:15] should i install maas-region-controller etc ? [14:15] vibol, i.e. can you go to streams.ubuntu.com https://cloud-images.ubuntu.com/releases/streams/v1/ ? [14:15] ah [14:15] yeb it accessable [14:15] vibol, you probably do have the region controller package installed in a single machine then [14:15] the region controller hosts the UI [14:16] so if you have web UI the region controller is running [14:16] i'm follow from ubuntu.com/cloud [14:16] I assume you did an apt-get install maas, yes? [14:16] yeb [14:16] !! [14:16] okay [14:17] so for some reason the maas region isn't able to download images [14:17] what does maas.host/images say? [14:18] kiko ... but. [14:18] "maas-region-controller is already the newest version." [14:18] vibol, yes, you're fine [14:18] the region is present [14:18] so the issue is to do with the region not being able to download for whatever reason [14:19] yeb [14:19] vibol, was this a fresh install? [14:19] yeb.. [14:19] first i install ubuntu server and start add ppa [14:19] apt install maas [14:19] i follow on http://www.ubuntu.com/download/cloud/install-ubuntu-openstack [14:20] okay [14:20] in /var/log/maas [14:20] only file apache2 maas-django.log maas.log oops proxy pserv.log rsyslog [14:20] are present [14:21] i'm install ubuntu on vmware to do this... [14:32] @kiko i'm thank for you hlep [14:32] 80% maybe internet connection [14:36] vibol, do you get a spinner on the /images page? [14:36] perhaps share a screenshot? [14:37] i have access http://maas.ubuntu.com/images/ephemeral-v2/releases/ in the log [14:37] but i can't download any file [15:03] vibol, oh, what happens if you download? [15:03] server error [15:03] it from my ips i think [15:05] @kiko [15:06] vibol, so that's your problem right there. I wonder why [15:06] vibol, ah, perhaps you have a transparent proxy affecting your traffic? [15:07] hmm i'm not really sure [15:07] but i don't use any proxy right here [15:07] i try to use another connection to confrim the problem but i need to be at my school lol... === imthewherd is now known as cgseller_is_away [18:36] Hows it going everyone? [18:36] I'm wondering if someone can aid me in identifying the root-path-string of the iscsi service running on maas [18:37] ????^^^^ [18:37] bdx, hmm, maybe we can, but why do you want to know? :) [18:38] kiko: I think I found what may work by running sudo iscsiadm -m discovery -t st -p [18:39] bdx, are you having to gateway the iscsi traffic? or something else? [18:39] kiko: I am trying desperately to get maas to work with our companys existing dns/dhcp configuration [18:39] ah. [18:39] that is indeed a common challenge [18:40] I'm clueless, though -- why do you need the iSCSI root-path? oh, because that's what the DHCP server needs to supply back? [18:40] bdx, have you tried looking at the generated dhcpd.conf? [18:40] and fragments? [18:40] kiko: exactly [18:41] yea....thats what I have based my external dhcp config off of [18:41] but that's missing, is that it? it's in the host maps? [18:41] * kiko can't remember [18:41] I can get nodes to enlist and commission successfully [18:42] https://www.dropbox.com/s/916u59gqoetlt7a/Screenshot%202015-06-09%2011.42.18.png?dl=0 [18:44] kiko: I'm thinking its got to be 10.16.0.2:3260,1 iqn.2004-05.com.ubuntu:maas:ephemeral-ubuntu-amd64-generic-trusty-daily [18:45] bdx, have you tried looking at the console of a node booting? but that does look correct [18:48] kiko: yes.... I can verify that the logs and console show the iscsi url [18:49] I am now thinking because the node enlists and commissions, that iscsi-root-path might not even be my issue [18:49] okay, try kicking off a run and seeing if it works [18:49] my bad [18:49] yea [18:49] I am at a total loss [18:49] I have been trying at this for quite a while...like a few weeks [18:50] wow, you should have come asking for help earlier! [18:50] anyway, tell us where you are stuck next [18:51] kiko: so...I can successfully enlist and commission nodes in maas [18:51] I am using pfsense for dhcp and have it configured to update my dns when it hands out addresses [18:51] so far so good [18:51] what's up with deploying? [18:53] kiko: when I deploy a node that is in "ready" state in maas, the node seems to commission itself, and power off....soon after I get a "deployment failed" status [18:53] bdx, okay, you need to look at the node log or console to see what's up [18:54] kiko: Ok...the console shows that of commissioning a node [18:54] well, you're not commissioning any more [18:54] now you're deploying [18:54] commissioning doesn't touch the drives [18:55] yea....but the node is commissioning when I try to deploy [18:55] ok [18:55] ah [18:55] that is an interesting hint [18:55] blake_r, what do we supply to the node when we are deploying in terms of iscsi path? [18:56] bdx, if you can find that out (do you have a parallel install with managed dhcp/dns) then we don't need to wait for blake_r [18:56] kiko: the DHCP or DNS server does not provide this information, it comes from the PXE request [18:57] kiko: that information is sent to the linux kernel using kernel parameters in the pxelinux.cfg that the server requests once it loads pxelinux.0 [18:58] blake_r, okay, so we basically write out a pxelinux.cfg that is specific to commissioning or deployment? [18:58] kiko: its includes the maas server ip address and the iscsi target to mount [18:58] blake_r, I wonder if bdx is trying to do something he doesn't need to [18:58] kiko: yes it is specific, based on the state of the node and the node [18:58] gotcha [18:58] bdx, ^^ [18:59] kiko, blake_r: what could this indicate in my situation? [18:59] bdx, where are you putting the iscsi target and why? [19:00] kiko: no where yet [19:00] bdx, how do you know it's commissioning instead of deploying? [19:00] kiko, blake_r: because the console shows what it shows when I commission a node [19:01] bdx: commissiong and deploying look the same watching the console of the node, its very similar [19:01] bdx: they both load ubuntu over pxe [19:02] totally....I have been using maas for dhcp/dns in my test lab for a while...I'v watched the commissioning and boot console many a time:-) [19:03] I am now trying to move the setup to our production environment where external dhcp/dns must be used [19:04] bdx, can you take a screenshot of the console in both commissioning and deployment so we can spot the issue? [19:04] ideally including what pxelinux is doing [19:04] perhaps that's more like a movie :) [19:05] kiko, blake_r: I can't say for sure that the node is "commissioning" instead of "deploying".....the console just shows the output of a commissioning ....but it could be a deploy that gets interrupted im thinking [19:05] yes [19:05] though the console will tell you what is going wrong -- i.e. for instance, the node is unable to download packages, etc [19:06] yeah....give me a minute....what is easiest...periscope, meerkat, youknow? [19:06] I guess youtube probably [19:06] I don't actually know, but it needs to be readable 8) [19:06] ok....I'll take a video ...give me 5 [19:11] sure [19:20] kiko, blake_r: here is a video of a node commissioning https://files.darkhorse.com/_RLGdv3A9r2RStR [19:20] kiko: blake_r: here is a deployment https://files.darkhorse.com/_vZGMW6SjX2lSPR [19:25] bdx, hmm, so slow for me to download.. 10 mins! [19:29] iphone -> hdvideo [19:29] ill render them smaller here [19:30] https://files.darkhorse.com/_rWG14-Ti-23SnR [19:31] https://files.darkhorse.com/_GQH8XCCdS2FS5R [19:31] 2/3 gone, now I will wait! [19:31] lol [19:33] bdx: the extract of the image failes [19:33] bdx: check MAAS you should have an installation log [19:34] bdx: on the node details page [19:34] bdx: it will give you the error [19:35] omg I've overlooked the install log button for so long....grrr [19:35] 10.0.3.1 is that your MAAS server address that a node can access? [19:36] bdx: says no route to host [19:36] no...that is a lxcbr0 created by juju....omg the install log shows all [19:36] http://paste.ubuntu.com/11678270/ [19:36] I wonder why it is trying to use that [19:36] sudo dpkg-reconfigure maas-cluster-controller [19:36] check that [19:36] yep [19:37] it is set correctly to "http://10.16.0.2/MAAS" [19:38] bdx, does grep 10.0.3.1 /etc/maas return anything? [19:38] I think maas gets that ip address from the initial PXE request from the client [19:38] possibly I should just delete the unused interface for 10.0.3.0/24 in my cluster interfaces [19:38] it looks at the ip address the request comes in on [19:38] is it routed through the bridge? [19:39] https://www.dropbox.com/s/8re057l9pgw8guj/Screenshot%202015-06-09%2012.38.40.png?dl=0 [19:40] blake_r, that can't make sense, the video shows the node booting on the 10.16 network no? [19:40] 10.16.0.115 is the IP it got, the server is 10.16.0.4 [19:40] bdx, does grep 10.0.3.1 /etc/maas return anything? [19:40] kiko: ah no I am wrong [19:41] kiko: this actually comes from the region [19:41] ah [19:42] kiko: its because he has it them all set to unmanaged [19:42] kiko: the region is then trying to say okay from this node, where should we say to get the images [19:42] kiko, blake_r: I removed lxcbr0 - 10.0.3.0 net from the maas server a while ago.... although it looks like I never removed it from the maas cluster interfaces [19:43] kiko: since noe are managed it just makes the best guess..... [19:43] kiko, blake_r: I should probably remove 10.0.3.0 from my cluster interfaces and retry a deploy yea? [19:43] blake_r, of course [19:43] kiko: which is not the correct guess [19:43] bdx: yes [19:43] bdx, how do we tell maas to give it this interface, specifically :) [19:44] kiko: the region does it best, but since its not managing the DHCP it cannot determine what ip address this node has, since the DHCP is not managed by maas [19:44] kiko, blake_r: seeing as the interface is unmanaged and it never posed an issue while maas was managing dns/dhcp....I guess I never payed it any attention [19:45] bdx: just delete all interfaces, except the one you want the nodes to communicate on [19:45] done. [19:45] bdx: yeah I bet that will work for you, let me know [19:45] will do... omp [19:56] blake_r, kiko: It worked [19:56] bdx: great! [19:56] :-):-) [19:57] I am so releived...this has been plaguing me....and to think the install log was starring me in the face the whole time [19:58] kiko, blake_r: THANK YOU [19:58] x10 [19:58] x1000 [19:58] bdx: your welcome [19:59] kiko, blake_r: It seems the node was given the hostname "ubuntu" [20:00] kiko, blake_r: Is this something I should resolve with a cloud-init post-deploy config script or a modification to the preseed? [20:00] bdx: no cloud-init and maas already do that for you [20:00] bdx: it should get the same name as the name in MAAS [20:00] bdx: are you able to ssh in to the machine? [20:00] bdx: cloud-init might have failed [20:01] yeah...by hostname [20:01] ssh ubuntu@ubuntu.tfawint.com [20:01] on the machine is the hostname "ubuntu" or just in your dns server? [20:01] whats the output of hostname on the server? [20:02] ahhh...when I ssh into the machine it shows its hostname from maas....just dns gets an entry for ubuntu.tfawint.com instead of excited-battle.tfawint.com [20:02] yeah that is something to do with your dns server then [20:02] since your not using maas dns [20:03] I see....the dns server was getting the correct fqdn entries created when a node commissions [20:04] that is probably because the server DHCP before it sets it hostname [20:04] it needs an IP address before it can contact MAAS for cloud-init data [20:04] a reboot of the server might fix your DNS server [20:06] totally....I'll reboot all servers and see what I can't figure out on my own.....I'll let you guys know if I get stuck again.....thanks again for your help [20:10] blake_r: rebooting the deployed node forced the correct entry to be created with the correct fqdn! [20:11] bdx: awesome [20:18] bdx, cool, happy you sorted it out [20:18] blake_r, should I file a bug to cover that use case? it is so obvious in hindsight [20:18] kiko: yeah [20:18] krad [20:20] blake_r, do you know where the code that "guesses" the interface is? [20:20] blake_r, I'm trying to get some context to help me file a bug summary with less than 200 lines [20:21] kiko: yep [20:21] kiko: one second [20:21] kiko: http://bazaar.launchpad.net/~maas-committers/maas/trunk/view/head:/src/maasserver/preseed.py#L588 [20:22] how about [20:22] "Deploying from a multi-homed cluster controller with external DHCP/DNS fails by giving the machine the wrong IP for the cluster controller" [20:22] that comment is interesting [20:23] multi-homes? I would say, "with multiple unmanaged interfaces" [20:27] blake_r, kilo: wow, so the multiple unmanaged interfaces is a documented bug then? [20:27] kiko* [20:27] not documented yet [20:27] but soon :) [20:27] bdx, I'm curious why a machine on 10.16.0.0 couldn't reach the lxcbr0 interface. was it firewalled off? [20:28] oh, it doesn't route though the cluster controller, that's why [20:28] kiko: I had deleted lxcbr0 a while back...it hasn't existed for > 2 months [20:29] at the os level [20:29] I see [20:29] is the cluster controller the router? [20:29] if it isn't, it wouldn't have worked anyway :) [20:30] no....I have vlans trunked in on my maas-controller eth0....from when I was trying to deploy openstack and use multiple nets. [20:32] kiko, blake_r: unfortunately I have not had any luck deploying anything when using > 1 interface managed by maas [20:32] bdx, interesting, that may be yet another bug [20:33] Bug #1463555 opened: Deploying from a multi-homed cluster controller with external DHCP/DNS fails by giving the machine the wrong IP for the cluster controller [20:33] bdx: can you give more detail information on what you mean? [20:33] kiko, blake_r: My workaround was to use external dhcp for all nets that maas was not managing. [20:34] bdx, blake_r: it may stem from the same naive algorithm that code uses [20:35] kiko: maybe' [20:35] it seems to think it's a good idea to search through the IPs on the cluster [20:35] bdx: can you give an example configuration [20:35] instead of trying to match the IP the node is asking on [20:35] that's a recipe for badness [20:36] kiko: yeah we actually know the ip address the node is requesting on the cluster, that just needs to be passed to the region to generate the preseed [20:36] exactly [20:36] blake_r, add a comment to that effect in the bug maybe [20:36] and bdx your comments welcome as well [20:36] kiko, blake_r: Lets say I have storage nodes that need secondary interfaces on a replication net....If I were to add maas managed dhcp on the net which the nodes secondary interface lives...in order to have that interface get an ip when the node boots .... [20:37] kiko, blake_r: enlistment and commissioning completely fail [20:38] kiko: thanks [20:38] bdx: so you have MAAS providing DHCP on two interfaces? in different subnets, and that fails if a node is on both those interfaces? [20:39] bdx: do you have any logs or anything for this error? [20:39] blake_r: true true [20:39] blake_r: I could have them fairly easily [20:40] bdx: that would be helpful, if you could file a bug with that information [20:42] blake_r, kiko: I'll file a bug concerning that issue.....I feel this is one of the biggest disconnects between juju <-> maas. Juju encourages through charm config params the use of multiple networks in many charms [20:42] right, which MAAS doesn't get very right [20:42] blake_r, again, I think it may stem from the same bad code [20:42] true [20:42] if we try and hunt through interfaces without taking into account the requesting IP.. [20:42] okay I need to split [20:43] * kiko waves === jfarschman is now known as MilesDenver === kev is now known as Guest95002 === jfarschman is now known as MilesDenver === jfarschman is now known as MilesDenver