| === cgseller_is_away is now known as imthewherd | ||
| === zz_CyberJacob is now known as CyberJacob | ||
| masumotok | Hello, I am newvie for maas, now I got an error "maas <profie> boot-resources import". In /var/log/maas/maas.log, I got "[WARNING] Finished importing boot images, the region does not have any boot images available.". The http-proxy exists in my network and thus, it requires user/password authentication. I would like to set HTTP_PROXY to maas. Does anyone know how? | 06:55 |
|---|---|---|
| === CyberJacob is now known as zz_CyberJacob | ||
| === wojdev_ is now known as wojdev | ||
| masumotok | MAAS seems like not accepting "http_proxy=http://<username>:<password>/ipaddr:port" style parameter, as declared <http://askubuntu.com/questions/618657/maas-cant-download-boot-images>. Anyone know the tips? | 07:56 |
| mup | Bug #1463378 opened: Downloaded di-initrd on trusty/amd64 overrides /etc/network/interfaces <canonical-bootstack> <MAAS:New> <https://launchpad.net/bugs/1463378> | 12:08 |
| vibol | Hello!.. | 14:00 |
| vibol | I'm importing boot image on Maas but the static in web gui still in "Qeued in download" | 14:00 |
| kiko | vibol, 1.5 or 1.7? | 14:01 |
| vibol | er.. sorr please wait | 14:01 |
| vibol | i'm installion from ppa:maas-maintainer/stable | 14:01 |
| vibol | @@ anyway really sorry how to check maas version @kiko ? | 14:04 |
| vibol | 1.7.5 @kiko | 14:05 |
| kiko | hmm | 14:07 |
| kiko | interesting | 14:07 |
| kiko | vibol, are you behind a proxy? | 14:07 |
| vibol | i'm no... | 14:07 |
| vibol | but i notices | 14:07 |
| vibol | some repositry of my ubuntu kinda slow | 14:07 |
| kiko | if you look at *.log in /var/log/maas you'll get an idea of what is happening | 14:07 |
| kiko | if there are timeouts or similar | 14:08 |
| vibol | http://paste.ubuntu.com/11672885/ | 14:11 |
| vibol | [INFO] Started importing boot image | 14:11 |
| vibol | [WARNING] Finished importing boot images, the region does not have any boot images available. | 14:11 |
| vibol | WARNING] No boot images have been imported from the region. | 14:11 |
| vibol | Still have no idea @kiko | 14:11 |
| kiko | vibol, the region controller downloads the images first, and then the cluster controller downloads them from the region | 14:13 |
| kiko | vibol, so what's happening is that the region controller is unable to download the images | 14:14 |
| kiko | vibol, can you confirm you can access the internet from the region controller machine? | 14:14 |
| vibol | I don't have reqion controllor machine yet maybe @@ | 14:14 |
| vibol | should i install maas-region-controller etc ? | 14:15 |
| kiko | vibol, i.e. can you go to streams.ubuntu.com https://cloud-images.ubuntu.com/releases/streams/v1/ ? | 14:15 |
| kiko | ah | 14:15 |
| vibol | yeb it accessable | 14:15 |
| kiko | vibol, you probably do have the region controller package installed in a single machine then | 14:15 |
| kiko | the region controller hosts the UI | 14:15 |
| kiko | so if you have web UI the region controller is running | 14:16 |
| vibol | i'm follow from ubuntu.com/cloud | 14:16 |
| kiko | I assume you did an apt-get install maas, yes? | 14:16 |
| vibol | yeb | 14:16 |
| vibol | !! | 14:16 |
| kiko | okay | 14:16 |
| kiko | so for some reason the maas region isn't able to download images | 14:17 |
| kiko | what does maas.host/images say? | 14:17 |
| vibol | kiko ... but. | 14:18 |
| vibol | "maas-region-controller is already the newest version." | 14:18 |
| kiko | vibol, yes, you're fine | 14:18 |
| kiko | the region is present | 14:18 |
| kiko | so the issue is to do with the region not being able to download for whatever reason | 14:18 |
| vibol | yeb | 14:19 |
| kiko | vibol, was this a fresh install? | 14:19 |
| vibol | yeb.. | 14:19 |
| vibol | first i install ubuntu server and start add ppa | 14:19 |
| vibol | apt install maas | 14:19 |
| vibol | i follow on http://www.ubuntu.com/download/cloud/install-ubuntu-openstack | 14:19 |
| kiko | okay | 14:20 |
| vibol | in /var/log/maas | 14:20 |
| vibol | only file apache2 maas-django.log maas.log oops proxy pserv.log rsyslog | 14:20 |
| vibol | are present | 14:20 |
| vibol | i'm install ubuntu on vmware to do this... | 14:21 |
| vibol | @kiko i'm thank for you hlep | 14:32 |
| vibol | 80% maybe internet connection | 14:32 |
| kiko | vibol, do you get a spinner on the /images page? | 14:36 |
| kiko | perhaps share a screenshot? | 14:36 |
| vibol | i have access http://maas.ubuntu.com/images/ephemeral-v2/releases/ in the log | 14:37 |
| vibol | but i can't download any file | 14:37 |
| kiko | vibol, oh, what happens if you download? | 15:03 |
| vibol | server error | 15:03 |
| vibol | it from my ips i think | 15:03 |
| vibol | @kiko | 15:05 |
| kiko | vibol, so that's your problem right there. I wonder why | 15:06 |
| kiko | vibol, ah, perhaps you have a transparent proxy affecting your traffic? | 15:06 |
| vibol | hmm i'm not really sure | 15:07 |
| vibol | but i don't use any proxy right here | 15:07 |
| vibol | i try to use another connection to confrim the problem but i need to be at my school lol... | 15:07 |
| === imthewherd is now known as cgseller_is_away | ||
| bdx | Hows it going everyone? | 18:36 |
| bdx | I'm wondering if someone can aid me in identifying the root-path-string of the iscsi service running on maas | 18:36 |
| bdx | ????^^^^ | 18:37 |
| kiko | bdx, hmm, maybe we can, but why do you want to know? :) | 18:37 |
| bdx | kiko: I think I found what may work by running sudo iscsiadm -m discovery -t st -p <maas-server-ip> | 18:38 |
| kiko | bdx, are you having to gateway the iscsi traffic? or something else? | 18:39 |
| bdx | kiko: I am trying desperately to get maas to work with our companys existing dns/dhcp configuration | 18:39 |
| kiko | ah. | 18:39 |
| kiko | that is indeed a common challenge | 18:39 |
| kiko | I'm clueless, though -- why do you need the iSCSI root-path? oh, because that's what the DHCP server needs to supply back? | 18:40 |
| kiko | bdx, have you tried looking at the generated dhcpd.conf? | 18:40 |
| kiko | and fragments? | 18:40 |
| bdx | kiko: exactly | 18:40 |
| bdx | yea....thats what I have based my external dhcp config off of | 18:41 |
| kiko | but that's missing, is that it? it's in the host maps? | 18:41 |
| * kiko can't remember | 18:41 | |
| bdx | I can get nodes to enlist and commission successfully | 18:41 |
| bdx | https://www.dropbox.com/s/916u59gqoetlt7a/Screenshot%202015-06-09%2011.42.18.png?dl=0 | 18:42 |
| bdx | kiko: I'm thinking its got to be 10.16.0.2:3260,1 iqn.2004-05.com.ubuntu:maas:ephemeral-ubuntu-amd64-generic-trusty-daily | 18:44 |
| kiko | bdx, have you tried looking at the console of a node booting? but that does look correct | 18:45 |
| bdx | kiko: yes.... I can verify that the logs and console show the iscsi url | 18:48 |
| bdx | I am now thinking because the node enlists and commissions, that iscsi-root-path might not even be my issue | 18:49 |
| kiko | okay, try kicking off a run and seeing if it works | 18:49 |
| bdx | my bad | 18:49 |
| bdx | yea | 18:49 |
| bdx | I am at a total loss | 18:49 |
| bdx | I have been trying at this for quite a while...like a few weeks | 18:49 |
| kiko | wow, you should have come asking for help earlier! | 18:50 |
| kiko | anyway, tell us where you are stuck next | 18:50 |
| bdx | kiko: so...I can successfully enlist and commission nodes in maas | 18:51 |
| bdx | I am using pfsense for dhcp and have it configured to update my dns when it hands out addresses | 18:51 |
| kiko | so far so good | 18:51 |
| kiko | what's up with deploying? | 18:51 |
| bdx | kiko: when I deploy a node that is in "ready" state in maas, the node seems to commission itself, and power off....soon after I get a "deployment failed" status | 18:53 |
| kiko | bdx, okay, you need to look at the node log or console to see what's up | 18:53 |
| bdx | kiko: Ok...the console shows that of commissioning a node | 18:54 |
| kiko | well, you're not commissioning any more | 18:54 |
| kiko | now you're deploying | 18:54 |
| kiko | commissioning doesn't touch the drives | 18:54 |
| bdx | yea....but the node is commissioning when I try to deploy | 18:55 |
| bdx | ok | 18:55 |
| kiko | ah | 18:55 |
| kiko | that is an interesting hint | 18:55 |
| kiko | blake_r, what do we supply to the node when we are deploying in terms of iscsi path? | 18:55 |
| kiko | bdx, if you can find that out (do you have a parallel install with managed dhcp/dns) then we don't need to wait for blake_r | 18:56 |
| blake_r | kiko: the DHCP or DNS server does not provide this information, it comes from the PXE request | 18:56 |
| blake_r | kiko: that information is sent to the linux kernel using kernel parameters in the pxelinux.cfg that the server requests once it loads pxelinux.0 | 18:57 |
| kiko | blake_r, okay, so we basically write out a pxelinux.cfg that is specific to commissioning or deployment? | 18:58 |
| blake_r | kiko: its includes the maas server ip address and the iscsi target to mount | 18:58 |
| kiko | blake_r, I wonder if bdx is trying to do something he doesn't need to | 18:58 |
| blake_r | kiko: yes it is specific, based on the state of the node and the node | 18:58 |
| kiko | gotcha | 18:58 |
| kiko | bdx, ^^ | 18:58 |
| bdx | kiko, blake_r: what could this indicate in my situation? | 18:59 |
| kiko | bdx, where are you putting the iscsi target and why? | 18:59 |
| bdx | kiko: no where yet | 19:00 |
| kiko | bdx, how do you know it's commissioning instead of deploying? | 19:00 |
| bdx | kiko, blake_r: because the console shows what it shows when I commission a node | 19:00 |
| blake_r | bdx: commissiong and deploying look the same watching the console of the node, its very similar | 19:01 |
| blake_r | bdx: they both load ubuntu over pxe | 19:01 |
| bdx | totally....I have been using maas for dhcp/dns in my test lab for a while...I'v watched the commissioning and boot console many a time:-) | 19:02 |
| bdx | I am now trying to move the setup to our production environment where external dhcp/dns must be used | 19:03 |
| kiko | bdx, can you take a screenshot of the console in both commissioning and deployment so we can spot the issue? | 19:04 |
| kiko | ideally including what pxelinux is doing | 19:04 |
| kiko | perhaps that's more like a movie :) | 19:04 |
| bdx | kiko, blake_r: I can't say for sure that the node is "commissioning" instead of "deploying".....the console just shows the output of a commissioning ....but it could be a deploy that gets interrupted im thinking | 19:05 |
| kiko | yes | 19:05 |
| kiko | though the console will tell you what is going wrong -- i.e. for instance, the node is unable to download packages, etc | 19:05 |
| bdx | yeah....give me a minute....what is easiest...periscope, meerkat, youknow? | 19:06 |
| bdx | I guess youtube probably | 19:06 |
| kiko | I don't actually know, but it needs to be readable 8) | 19:06 |
| bdx | ok....I'll take a video ...give me 5 | 19:06 |
| kiko | sure | 19:11 |
| bdx | kiko, blake_r: here is a video of a node commissioning https://files.darkhorse.com/_RLGdv3A9r2RStR | 19:20 |
| bdx | kiko: blake_r: here is a deployment https://files.darkhorse.com/_vZGMW6SjX2lSPR | 19:20 |
| kiko | bdx, hmm, so slow for me to download.. 10 mins! | 19:25 |
| bdx | iphone -> hdvideo | 19:29 |
| bdx | ill render them smaller here | 19:29 |
| bdx | https://files.darkhorse.com/_rWG14-Ti-23SnR | 19:30 |
| bdx | https://files.darkhorse.com/_GQH8XCCdS2FS5R | 19:31 |
| kiko | 2/3 gone, now I will wait! | 19:31 |
| bdx | lol | 19:31 |
| blake_r | bdx: the extract of the image failes | 19:33 |
| blake_r | bdx: check MAAS you should have an installation log | 19:33 |
| blake_r | bdx: on the node details page | 19:34 |
| blake_r | bdx: it will give you the error | 19:34 |
| bdx | omg I've overlooked the install log button for so long....grrr | 19:35 |
| blake_r | 10.0.3.1 is that your MAAS server address that a node can access? | 19:35 |
| blake_r | bdx: says no route to host | 19:36 |
| bdx | no...that is a lxcbr0 created by juju....omg the install log shows all | 19:36 |
| bdx | http://paste.ubuntu.com/11678270/ | 19:36 |
| bdx | I wonder why it is trying to use that | 19:36 |
| blake_r | sudo dpkg-reconfigure maas-cluster-controller | 19:36 |
| blake_r | check that | 19:36 |
| kiko | yep | 19:36 |
| bdx | it is set correctly to "http://10.16.0.2/MAAS" | 19:37 |
| kiko | bdx, does grep 10.0.3.1 /etc/maas return anything? | 19:38 |
| blake_r | I think maas gets that ip address from the initial PXE request from the client | 19:38 |
| bdx | possibly I should just delete the unused interface for 10.0.3.0/24 in my cluster interfaces | 19:38 |
| blake_r | it looks at the ip address the request comes in on | 19:38 |
| blake_r | is it routed through the bridge? | 19:38 |
| bdx | https://www.dropbox.com/s/8re057l9pgw8guj/Screenshot%202015-06-09%2012.38.40.png?dl=0 | 19:39 |
| kiko | blake_r, that can't make sense, the video shows the node booting on the 10.16 network no? | 19:40 |
| kiko | 10.16.0.115 is the IP it got, the server is 10.16.0.4 | 19:40 |
| kiko | bdx, does grep 10.0.3.1 /etc/maas return anything? | 19:40 |
| blake_r | kiko: ah no I am wrong | 19:40 |
| blake_r | kiko: this actually comes from the region | 19:41 |
| kiko | ah | 19:41 |
| blake_r | kiko: its because he has it them all set to unmanaged | 19:42 |
| blake_r | kiko: the region is then trying to say okay from this node, where should we say to get the images | 19:42 |
| bdx | kiko, blake_r: I removed lxcbr0 - 10.0.3.0 net from the maas server a while ago.... although it looks like I never removed it from the maas cluster interfaces | 19:42 |
| blake_r | kiko: since noe are managed it just makes the best guess..... | 19:43 |
| bdx | kiko, blake_r: I should probably remove 10.0.3.0 from my cluster interfaces and retry a deploy yea? | 19:43 |
| kiko | blake_r, of course | 19:43 |
| blake_r | kiko: which is not the correct guess | 19:43 |
| blake_r | bdx: yes | 19:43 |
| kiko | bdx, how do we tell maas to give it this interface, specifically :) | 19:43 |
| blake_r | kiko: the region does it best, but since its not managing the DHCP it cannot determine what ip address this node has, since the DHCP is not managed by maas | 19:44 |
| bdx | kiko, blake_r: seeing as the interface is unmanaged and it never posed an issue while maas was managing dns/dhcp....I guess I never payed it any attention | 19:44 |
| blake_r | bdx: just delete all interfaces, except the one you want the nodes to communicate on | 19:45 |
| bdx | done. | 19:45 |
| blake_r | bdx: yeah I bet that will work for you, let me know | 19:45 |
| bdx | will do... omp | 19:45 |
| bdx | blake_r, kiko: It worked | 19:56 |
| blake_r | bdx: great! | 19:56 |
| bdx | :-):-) | 19:56 |
| bdx | I am so releived...this has been plaguing me....and to think the install log was starring me in the face the whole time | 19:57 |
| bdx | kiko, blake_r: THANK YOU | 19:58 |
| bdx | x10 | 19:58 |
| bdx | x1000 | 19:58 |
| blake_r | bdx: your welcome | 19:58 |
| bdx | kiko, blake_r: It seems the node was given the hostname "ubuntu" | 19:59 |
| bdx | kiko, blake_r: Is this something I should resolve with a cloud-init post-deploy config script or a modification to the preseed? | 20:00 |
| blake_r | bdx: no cloud-init and maas already do that for you | 20:00 |
| blake_r | bdx: it should get the same name as the name in MAAS | 20:00 |
| blake_r | bdx: are you able to ssh in to the machine? | 20:00 |
| blake_r | bdx: cloud-init might have failed | 20:00 |
| bdx | yeah...by hostname | 20:01 |
| bdx | ssh ubuntu@ubuntu.tfawint.com | 20:01 |
| blake_r | on the machine is the hostname "ubuntu" or just in your dns server? | 20:01 |
| blake_r | whats the output of hostname on the server? | 20:01 |
| bdx | ahhh...when I ssh into the machine it shows its hostname from maas....just dns gets an entry for ubuntu.tfawint.com instead of excited-battle.tfawint.com | 20:02 |
| blake_r | yeah that is something to do with your dns server then | 20:02 |
| blake_r | since your not using maas dns | 20:02 |
| bdx | I see....the dns server was getting the correct fqdn entries created when a node commissions | 20:03 |
| blake_r | that is probably because the server DHCP before it sets it hostname | 20:04 |
| blake_r | it needs an IP address before it can contact MAAS for cloud-init data | 20:04 |
| blake_r | a reboot of the server might fix your DNS server | 20:04 |
| bdx | totally....I'll reboot all servers and see what I can't figure out on my own.....I'll let you guys know if I get stuck again.....thanks again for your help | 20:06 |
| bdx | blake_r: rebooting the deployed node forced the correct entry to be created with the correct fqdn! | 20:10 |
| blake_r | bdx: awesome | 20:11 |
| kiko | bdx, cool, happy you sorted it out | 20:18 |
| kiko | blake_r, should I file a bug to cover that use case? it is so obvious in hindsight | 20:18 |
| blake_r | kiko: yeah | 20:18 |
| kiko | krad | 20:18 |
| kiko | blake_r, do you know where the code that "guesses" the interface is? | 20:20 |
| kiko | blake_r, I'm trying to get some context to help me file a bug summary with less than 200 lines | 20:20 |
| blake_r | kiko: yep | 20:21 |
| blake_r | kiko: one second | 20:21 |
| blake_r | kiko: http://bazaar.launchpad.net/~maas-committers/maas/trunk/view/head:/src/maasserver/preseed.py#L588 | 20:21 |
| kiko | how about | 20:22 |
| kiko | "Deploying from a multi-homed cluster controller with external DHCP/DNS fails by giving the machine the wrong IP for the cluster controller" | 20:22 |
| kiko | that comment is interesting | 20:22 |
| blake_r | multi-homes? I would say, "with multiple unmanaged interfaces" | 20:23 |
| bdx | blake_r, kilo: wow, so the multiple unmanaged interfaces is a documented bug then? | 20:27 |
| bdx | kiko* | 20:27 |
| kiko | not documented yet | 20:27 |
| kiko | but soon :) | 20:27 |
| kiko | bdx, I'm curious why a machine on 10.16.0.0 couldn't reach the lxcbr0 interface. was it firewalled off? | 20:27 |
| kiko | oh, it doesn't route though the cluster controller, that's why | 20:28 |
| bdx | kiko: I had deleted lxcbr0 a while back...it hasn't existed for > 2 months | 20:28 |
| bdx | at the os level | 20:29 |
| kiko | I see | 20:29 |
| kiko | is the cluster controller the router? | 20:29 |
| kiko | if it isn't, it wouldn't have worked anyway :) | 20:29 |
| bdx | no....I have vlans trunked in on my maas-controller eth0....from when I was trying to deploy openstack and use multiple nets. | 20:30 |
| bdx | kiko, blake_r: unfortunately I have not had any luck deploying anything when using > 1 interface managed by maas | 20:32 |
| kiko | bdx, interesting, that may be yet another bug | 20:32 |
| mup | Bug #1463555 opened: Deploying from a multi-homed cluster controller with external DHCP/DNS fails by giving the machine the wrong IP for the cluster controller <MAAS:New> <https://launchpad.net/bugs/1463555> | 20:33 |
| blake_r | bdx: can you give more detail information on what you mean? | 20:33 |
| bdx | kiko, blake_r: My workaround was to use external dhcp for all nets that maas was not managing. | 20:33 |
| kiko | bdx, blake_r: it may stem from the same naive algorithm that code uses | 20:34 |
| blake_r | kiko: maybe' | 20:35 |
| kiko | it seems to think it's a good idea to search through the IPs on the cluster | 20:35 |
| blake_r | bdx: can you give an example configuration | 20:35 |
| kiko | instead of trying to match the IP the node is asking on | 20:35 |
| kiko | that's a recipe for badness | 20:35 |
| blake_r | kiko: yeah we actually know the ip address the node is requesting on the cluster, that just needs to be passed to the region to generate the preseed | 20:36 |
| kiko | exactly | 20:36 |
| kiko | blake_r, add a comment to that effect in the bug maybe | 20:36 |
| kiko | and bdx your comments welcome as well | 20:36 |
| bdx | kiko, blake_r: Lets say I have storage nodes that need secondary interfaces on a replication net....If I were to add maas managed dhcp on the net which the nodes secondary interface lives...in order to have that interface get an ip when the node boots .... | 20:36 |
| bdx | kiko, blake_r: enlistment and commissioning completely fail | 20:37 |
| bdx | kiko: thanks | 20:38 |
| blake_r | bdx: so you have MAAS providing DHCP on two interfaces? in different subnets, and that fails if a node is on both those interfaces? | 20:38 |
| blake_r | bdx: do you have any logs or anything for this error? | 20:39 |
| bdx | blake_r: true true | 20:39 |
| bdx | blake_r: I could have them fairly easily | 20:39 |
| blake_r | bdx: that would be helpful, if you could file a bug with that information | 20:40 |
| bdx | blake_r, kiko: I'll file a bug concerning that issue.....I feel this is one of the biggest disconnects between juju <-> maas. Juju encourages through charm config params the use of multiple networks in many charms | 20:42 |
| kiko | right, which MAAS doesn't get very right | 20:42 |
| kiko | blake_r, again, I think it may stem from the same bad code | 20:42 |
| bdx | true | 20:42 |
| kiko | if we try and hunt through interfaces without taking into account the requesting IP.. | 20:42 |
| kiko | okay I need to split | 20:42 |
| * kiko waves | 20:43 | |
| === jfarschman is now known as MilesDenver | ||
| === kev is now known as Guest95002 | ||
| === jfarschman is now known as MilesDenver | ||
| === jfarschman is now known as MilesDenver | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
