/srv/irclogs.ubuntu.com/2012/07/17/#maas.txt

jtvsmoser, could you help me with this?  I need maas to retrieve some settings that are debconf'ed by the maas-dhcp postinst script.04:35
jtvroaksoax maybe?04:40
jtvrvba: if we have dev_fixture.yamls and initial_data.yamls, a dev setup will load both and a production setup will only load the latter, right?08:22
jtvOh no, please tell me this is not true.08:24
jtvUser.email must be unique, even if it's blank.08:24
jtvWhat #$@% use is that?08:24
rvbajtv: How would dev_fixture.yamls be loaded by the dev service?08:25
jtvI thought a demo run has all the data from dev_fixture.yaml..?08:25
rvbajtv: no, make sampledata loads dev_fixture.yaml08:26
jtvWell that's what you do for a demo run, so a dev setup will load both, right?08:26
rvbaYes.08:27
jtvWell that exactly answers my question.  :)08:27
rvbainitial_data.yamls is loaded automatically but not dev_fixture.08:27
rvbaOk then :).08:27
jtvUnfortunately I'm running up against this weirdness where we can't have two built-in users with email address '', even though we can have one.08:27
rvbaI thought we "fixed" that... let me have a look at the code and refresh my memory.08:28
=== mrevell_ is now known as mrevell
jtvYes, South, I'm making a field nullable.  And, since you seem to insist, specifying a default value of None.  Now, honestly, what do you think I want the default to be?  Do you still need to stop because you don't know?09:28
jtvAh.  Thank you internet.  This is South's way of saying: what if I ever want to _reverse_ this migration?09:41
jtvrvba: I don't think we have the email-may-be-empty-but-must-still-be-unique problem solved.  :(10:24
rvbajtv: really?  IIRC we wanted to use that only to be able to create a system user with an empty email address.10:25
jtvYes10:25
jtvThat's what I'm trying to do.10:25
jtvBut we already have one with an empty email address.10:26
jtvI have a feeling that the solution we found at the time was to use a blank string, and leave the rest of the problem for the future.10:26
jtvProviding a null violates a not-null constraint, providing an empty string violates a unique constraint.10:26
jtvAhhh looks like django 1.4 makes the email address optional.10:27
jtvOr maybe that's only in the check that creates it, rather than in the database.10:28
rvbaBut unfortunately we have to be compatible with Dj 1.3.10:28
jtvI'm not even sure 1.4 solves it anyway.  :(10:28
jtvWhat I found was about create_user:10:28
jtv“The username parameter is now checked for emptiness and raises a ValueError in case of a negative result.”10:28
jtvAnd by a negative result in this case they mean a positive result.10:28
jtvOff-by-True error.10:28
jtvWhat's the system username we create?  Just maas?  I could just use maas@localhost.10:30
jtvAlthough… what do we do for the _next_ system user?  I'll just make it a full description of the user, and a nonexistent address. TFB.10:30
=== matsubara is now known as matsubara-afk
roaksoaxjtv: sure, how do you want to retrieve them?12:17
jtvroaksoax: never mind, I think I'll just leave that as a packaging job, something postinst can do.12:17
roaksoaxjtv: cool12:18
roaksoaxrbasak: howdy! this is what we need to do in setup.py: http://bazaar.launchpad.net/~andreserl/maas/maas_updates_bzr745/revision/5612:19
roaksoaxerrr12:20
roaksoaxrvba: ^^12:20
roaksoaxrbasak: sorry worng person :)12:20
rbasaknp12:20
roaksoaxrvba: debian/rules mainly12:20
rvbaroaksoax: looking.12:21
roaksoaxrvba: override_dh_auto_build12:21
roaksoaxrvba: i just figured it was faster and less painfull for me to build it in package build time rather than copy it manually. But we need to handle it in the setup.py12:22
rvbaroaksoax: looks all right time.12:23
rvbaroaksoax: looks all right to me.12:24
rvba:)12:24
roaksoaxrvba: :)12:24
roaksoaxjtv: so now that maas-import-isos references are being erased, does this mean that we are now using MAAS own PXE server?12:42
jtvNot quite there yet.12:42
jtvFor that we need to assume control of DHCP.12:42
jtvOr the administrator needs to configure their DHCP for MAAS.12:43
jtvCurrently we still use Cobbler.12:43
roaksoaxjtv: right, ok then. So do you have an ETA of when this would happen (getting rid of cobbler)?12:43
jtvWe're working on it… the devil is very much in the details.12:44
jtvTo a large extent the plan unfolds as we execute it.12:44
jtvIt's the price we pay for getting a quick start using Cobbler.12:44
roaksoaxhehe12:45
roaksoaxjtv: alright, I was just wondering so that I can do the appropriate packaging updates then, and not now :)12:46
jtvI'm afraid we're going to go through some stages of brokenness before we come out the other end working but without Cobbler.12:47
roaksoaxi guess that's fine.12:52
roaksoaxI just want to be aware when cobbler is not longer being used so I don't release broken stuff12:53
roaksoaxallenap: how would you describe the fixes for python-tx-tftp?15:13
allenaproaksoax: It adds tsize support; it already has timeout support, so this should make it fully comply with RFC 2349.15:15
allenappxelinux requires tsize support.15:15
roaksoaxallenap: ok cool, just updating patch description :)15:16
allenaproaksoax: Fwiw, here's the pull request: https://github.com/shylent/python-tx-tftp/pull/515:19
roaksoaxthanks15:21
cropalatoHi, is there some documentation about configure maas with a external dhcp?17:06
guimalufcropalato, iis the same thing. the difference is allow the dhcp to redirect the pxe server:17:16
guimalufallow bootp;17:16
guimalufallow booting;17:16
guimalufclass "pxeclients" {17:16
guimaluf        match if substring(option vendor-class-identifier,0,9)="PXEClient";17:16
guimaluf        # Servidor de PXE17:16
guimaluf        next-server 150.164.3.236;17:16
guimaluf        server-name "brontes";17:16
guimaluf        filename "pxelinux.0";17:16
guimaluf}17:16
cropalatoguimaluf, thanks17:23
=== matsubara-afk is now known as matsubara
burnbrighterIt appears in the latest release of maas, the default route needs to be set manually? Is that a bug or by design?19:16
burnbrighterlatest apt-get release rather19:16
roaksoaxsmoser: ping19:48
roaksoaxsmoser: https://bugs.launchpad.net/ubuntu/+source/cobbler/+bug/858867/comments/519:48
ubot5Ubuntu bug 858867 in cobbler (Ubuntu Quantal) "XMLRPC allows unauthed users access to various methods (which it shouldn't) " [Medium,Triaged]19:48
roaksoaxburnbrighter: what's your issue?19:48
roaksoaxburnbrighter: what do you mean by default route?19:49
burnbrighterroaksoax: on enlist, it appears the default route isn't being configured19:56
roaksoaxburnbrighter: you mean it is no enlisting?19:58
burnbrighteron pxe boot, the default route is not being sent19:59
burnbrightermaybe its cobbler not sending the info correctly20:00
burnbrighteror setting up the configuration?20:00
burnbrighterthis worked in my previous maas release20:00
roaksoaxburnbrighter: so if machines are not PXE booting, it is a matter of configuring the DHCP correctly.20:01
roaksoaxburnbrighter: if they are PXE booting but not enlisting, maybe the IP address the MAAS server has is not the one in the configuration20:02
burnbrighterit PXE boots, but then it's not picking up the default route20:02
burnbrighterso as it does its initial configuration and is doing the install20:02
burnbrighterits not getting a default route via dhcp20:02
burnbrighterit gets an IP address, but no default route20:02
roaksoaxburnbrighter: ah so that's a problem of the DHCP server or even of the installer itself20:03
burnbrightercorrect - but that's configured by maas20:03
burnbrighterright?20:03
burnbrighteror maas -> cobbler20:03
roaksoaxburnbrighter: maas attempts to configure it, and tells it to cobbler20:03
burnbrighterok, so something broke there20:04
roaksoaxburnbrighter: so what's the pastebin from /etc/cobbler/dnsmasq.template20:04
roaksoaxq120:04
burnbrightersure, hang on20:05
burnbrightersorry, what is ubuntu's pastebin?20:05
burnbrighterurl20:05
roaksoaxpastebin.ubuntu.com20:06
roaksoaxburnbrighter: you can install pastebinit20:06
roaksoaxand run: pastebinit /etc/cobbler/dnsmasq.template20:06
burnbrighterhttp://paste.ubuntu.com/1097228/20:07
roaksoaxburnbrighter: is this your default gateway? dhcp-option=3,172.16.100.1120:07
burnbrighteryes20:08
roaksoaxburnbrighter: run sudo cobbler sync and pastebin the output please20:08
roaksoaxburnbrighter: so maas has a caching server, would you be hitting it maybe?20:10
burnbrightersorry - need to step away for a bit , will be back20:10
roaksoaxsure20:11
guimalufi give up of this maas stuff!!! past one week I couldnt make it work!20:42
guimalufi've sync the clocks. changed ephemeral images. raw configs. nothing works!20:42
roaksoaxguimaluf: i would say otherwise, I have had various deployments of MAAS working20:47
roaksoaxguimaluf: what are your issues anyway?20:47
guimaluf(0x7f33b5f30700):ZOO_ERROR@handle_socket_error_msg@1579: Socket [127.0.0.1:47231] zk retcode=-4, errno=111(Connection refused): server refused to accept the client20:48
roaksoaxguimaluf: that seems to be juju issue, not maas20:48
guimalufthis time. I've pass so many things... everytime clock problem.20:48
guimalufbut maas cant deploy the ssh-key properly20:49
roaksoaxguimaluf: edit /var/lib/cobbler/kickstarts/maas.preseed and search for the passwrod line and remove the ! and set your password there20:50
roaksoaxguimaluf: then you would be able to ssh in an debug the problem of why the keys are not being installed20:50
guimalufit install the os but I needed to include a default password by myself and even after deploy the sshkey by hand i got this juju problem20:50
guimalufI check everything on web, and the problem is the same. time sync oauth clock problem20:50
guimalufyour maas deployments are working properly? deploy sshkeys and everything else?20:51
roaksoaxif that's the issue then would be matter of making maas' server clock the same as the clients. We are currently working on trying to get that fixed20:51
roaksoaxguimaluf: yes my maas deployments work fine. I've tested the ssh key inyection in various escenarios as well20:52
guimalufroaksoax, I fix it in many ways.20:52
guimalufmaybe the problem occurs because the maas server isnt the dhcp server20:52
roaksoaxsmoser: ^^ oauth issues causing ots of pain, is there a recommended fix?20:52
guimalufnow even with the keys set properly this juju thing dont work :/20:53
guimalufi'm really tired :( whole week just for this.20:53
guimalufi've installed so many times, so many times!20:53
smoserwork around is described in comment 5 at http://paste.ubuntu.com/1097303/20:53
smosererr.20:53
smosercomment 5 at https://bugs.launchpad.net/maas/+bug/97812720:53
ubot5Ubuntu bug 978127 in MAAS "incorrect time on node causes failed oauth" [High,Triaged]20:53
guimalufsorry my whinning20:54
roaksoaxguimaluf: https://bugs.launchpad.net/maas/+bug/978127/comments/520:54
roaksoaxthat's the work around for the issue you are having20:54
roaksoaxsmoser: btw... did you read the cobbler stuff?20:54
smoserroaksoax, i did.20:54
roaksoaxsmoser: any thoughts?20:54
guimalufroaksoax, I allready did that. :/20:55
roaksoaxguimaluf: is the juju client accessible to the nodes via DNS ?20:55
guimalufyes, I put the node address in /etc/hosts20:56
smoserroaksoax, just commented in bug.20:57
roaksoaxguimaluf: well I think we'd need logs in order to determine what is wrong20:58
guimalufroaksoax, which ones?20:58
roaksoaxsmoser: ok ;)20:58
roaksoaxguimaluf: in the bootstrap node, we'd need cloud-init logs, juju logs20:58
smoserguimaluf, roaksoax is right. if you can get to the bootstrap node (via ssh), then we need to see what went wrong.20:59
smoseri suspect there are errors in /var/log/cloud-init-output.log if you pastebin that20:59
guimalufsmoser, I can ssh the node just because I've put the keys by hand. :/ anyway. i'm pasting, just a sec21:00
smoser(just in case you were unaware, guimaluf there is a tool called 'pastebinit' that pastes to paste.ubuntu.com from file or stdin)21:01
guimalufsmoser, yes, ofcourse :)21:01
guimalufcloud-init: http://pastebin.ubuntu.com/1097316/21:05
guimalufjuju -v status: http://pastebin.ubuntu.com/1097319/21:05
smoserguimaluf, /var/log/cloud-init-output.log ?21:05
smoseryour maas server seems to have given empty user-data to cloud-init.21:06
guimalufsmoser, there's no file with this name21:06
smoser(as evidence by 'Unhandled non-mlutipart userdata '')21:06
smoserright.21:06
smoserlok.21:06
smoser(duh, it wouldn't be there... you didn't get any cloud-config user data to tell it to write it :)21:06
smoserso now, your issue is that that node is not geting to the maas server.21:06
guimalufbut this user is set in the maas.preseed?21:07
smoseruser?21:07
smoserok. i have to run21:07
guimalufwhich user are you talking about?21:07
smoseryou said "this user"21:08
smoseri didn't know what that meant.21:08
smoserah. user-data.21:08
guimalufsorry, poor english :/21:08
smoserso, to debug from here.21:08
smoserthe issue is that cloud-init does not get data from the maas server21:08
roaksoaxguimaluf: can you pastebin /etc/maas/maas_local_settings.py?21:08
smoserso you need to debug that21:08
smoserroaksoax, i have to run21:08
smoserbut there should be files in /etc/cloud.cfg.d21:08
roaksoaxsmoser: alright thanks for the input21:08
smoserthat have Maas config info21:08
smoserand /usr/share/pyshared/cloudinit/DataSourceMAAS.py is a maas user-data client of sorts21:09
guimalufsmoser, thanks alot man! :) I will bother roaksoax a little more!21:09
smoserpython /usr/share/pyshared/cloudinit/DataSourceMAAS.py --config /etc/cloud/cloud.cfg.d/91*.cfg crawl <htat-url>21:09
smoserfind which file has "maas" in it in /etc/cloud/cloud.cfg.d (grep would work)21:10
smoseropen it.21:10
smoserit has a url in it21:10
smoserand do like the above21:10
guimalufroaksoax, http://pastebin.ubuntu.com/1097339/21:10
smoserthat will show you an error, or at least trigger one hopefully21:10
smoserlater.21:10
roaksoaxguimaluf: I think I found the issue21:11
roaksoaxguimaluf: is this DEFAULT_MAAS_URL = "http://150.164.3.236/" the IP address for the local network?21:11
roaksoaxMAAS's IP address for the local network?21:11
guimalufyes21:11
roaksoaxguimaluf: can you pastebin apache's log? (/var/log/apache2/error.log and /var/log/apache2/access.log)21:13
burnbrighterroaksoax: sorry, now back - here is the pastebin you requested: http://paste.ubuntu.com/1097349/21:13
roaksoaxburnbrighter: are the client machines precise or quantal?21:15
guimalufroaksoax, http://pastebin.ubuntu.com/1097352/21:15
guimalufinvalid consumer :/21:16
roaksoaxguimaluf:21:17
roaksoax150.164.3.240 - - [17/Jul/2012:17:37:22 -0300] "GET /MAAS/metadata//2012-03-01/meta-data/public-keys HTTP/1.1" 404 230 "-" "Python-urllib/2.7"21:17
roaksoax150.164.3.240 - - [17/Jul/2012:17:37:22 -0300] "GET /MAAS/metadata//2012-03-01/user-data HTTP/1.1" 404 242 "-" "Python-urllib/2.7"21:17
roaksoaxthat seems to be an issue21:17
roaksoaxguimaluf: So, nodes have enlisted, they have commissioned, and they were ready to deploy ebfore using juju?21:18
guimalufroaksoax, but my ssh is there in the root preferences21:18
roaksoaxguimaluf: what it seems is that no cloud-init meta-data or user-data have been generated, as if the node hasn't commissioned21:19
guimalufbut every step occurs properly.21:19
guimalufthe node is enlisted, comissioned and ready21:20
guimalufnow is Allocated to root21:20
burnbrighterroaksoax: precise21:20
roaksoaxburnbrighter: that's strange then, it might be an issue with dnsmasq21:20
burnbrighterit appears something broke between the latest release and the last because this worked with out problems before21:21
roaksoaxburnbrighter: when was the last time you upgraded quantal's maas version?21:22
burnbrightertoday - I had been not working on things for about two weeks21:23
roaksoaxburnbrighter: can you sudo apt-get update && sudo apt-get install python-txtftp and see if an update gets installed21:24
roaksoaxguimaluf: uhmm so the problme seems to be the fact that the client is not getting the meta-data / user-data from the MAAS server. Config file seems just fine21:25
guimalufroaksoax, is not a problem of time syncronization. what else can be?21:25
roaksoaxguimaluf: would be helpful if you juju bootstrap and pastebin the error/access log from apache while the node is PXE booting and installing, and what happens after install21:25
roaksoaxguimaluf: and check whether the node is complaining about not being able to reach the meta-data server (client node)21:26
guimalufroaksoax, i think the Invalid Consumer in the error.log happen when I was installing the nodfe21:27
roaksoaxguimaluf: ok, so do you have physical access to the client maas node?21:28
guimalufyes21:28
roaksoaxguimaluf: do you have a screen connected to it?21:28
guimalufyep21:28
guimalufshould I flush maas db again and reinstall the node?21:29
guimalufI did that 1208397198237 times xD21:29
roaksoaxguimaluf: ok, so, juju bootstrap, after installation finishes it will reboot. Once it boots into logging prompt, it should show that 1. either is failing to get meta-data or 2. it is being able to obtain meta-data. If yo could see what shows there and let me know, that would be very helpful21:30
roaksoaxguimaluf: or 3. when you fluished the db, you might have flashed node information that has the cloud-init meta-data21:30
guimalufroaksoax, I didnt flushed the db this time21:31
guimalufI start from scratch again21:31
guimalufwas my last try before "give up"21:31
roaksoaxguimaluf: so the screen output should tell if something wrong is going on21:32
roaksoaxit will 1. either obtain the user-data and do stuff with it (cloud-init). 2. or cloud-init will complain about not being able to reach the maas-server21:32
guimalufroaksoax, so I should reboot my node right now and check what happens before the login prompt?21:32
roaksoaxguimaluf: reinstall the node please21:32
guimalufok21:32
guimalufthere's a secure way for removing the node?21:33
guimalufI always do cobbler system delete and maas shell Node.objects.all().delete()21:33
roaksoaxguimaluf: yeah I guess that would work. however, before you do that21:33
roaksoaxguimaluf: could you pastebin: sudo cobbler system getks --name node-XYZ21:34
roaksoaxwhere node-XYZ is the node-XYZ from sudo cobbler system list21:34
roaksoaxthat is your bootstrap node21:34
guimalufsure :)21:34
guimalufroaksoax, http://pastebin.ubuntu.com/1097392/21:35
roaksoaxcloud-init   cloud-init/maas-metadata-url  string http://150.164.3.236/MAAS/metadata/ -->that seems right, its telling that it should contact the MAAS server for meta-data21:36
roaksoaxguimaluf: please proceed with the reinstall21:37
guimalufok21:37
guimalufremoving nodes...!21:37
guimalufwow, I can delete the node from the UI it isnt allocated to root!21:38
roaksoaxyeah21:38
guimaluf:)21:38
guimalufgood sign!21:38
guimalufroaksoax, I should keep eye on "meta-data" information right?21:39
roaksoaxguimaluf: yes, it should appear right on the screen after the installation proices, after the reboot (before/after showing the logging prompt)21:40
guimalufafter comissioning right?21:40
roaksoaxguimaluf: 1. enlist. 2. accept & commission. (turn on the node and allow it to commission) 3. juju bootstrap. 4. wait installation to finish and reboot. 5. watch :)21:42
guimalufjuju bootstrap?21:42
guimalufjuju bootstrap runs on the node?21:43
roaksoaxguimaluf: yes21:43
roaksoaxguimaluf: it runs the bootstrap node which is juju21:43
roaksoaxguimaluf: it runs the bootstrap node which is juju's zookeeper21:43
roaksoaxguimaluf: you always need that node to deploy with juju21:43
guimalufbut I dont run manually right?21:44
roaksoaxif juju bootstrap doesn't start a node automatically, then you need to start it manually21:44
=== matsubara is now known as matsubara-afk
guimalufjuju bootstrap is for the local machine?21:44
roaksoaxguimaluf: make sure your .juju/environments.yalm is also correctly configured21:45
guimalufroaksoax, in my maas server right?21:45
roaksoaxguimaluf: if you are running juju ocmmands from your maas server, then yes21:45
roaksoaxjuju bootstrap will take 1 node from the maas pool of servers21:45
guimalufthere is a error21:46
guimalufutil.py21:46
roaksoaxhuh?21:46
roaksoaxpastebin21:47
guimalufutil.py[WARNING] 'addres.../metadata/DATE/meta-data/instance-id' failed http error [403]21:47
guimalufI cant pastebin this one21:47
roaksoaxthat;s commissioning21:47
guimalufyes21:47
roaksoaxguimaluf:  that's oauth issue then i thjink21:47
roaksoaxplease apply the patch as above21:48
guimalufwhich patch?21:48
guimalufstill commissioning. if I reboot the node it will commit21:49
roaksoaxerr work around for the ephemeral iomage21:49
roaksoaxhttps://bugs.launchpad.net/maas/+bug/978127/comments/521:49
ubot5Ubuntu bug 978127 in MAAS "incorrect time on node causes failed oauth" [High,Triaged]21:49
guimalufits done already21:49
roaksoaxmaybe the image iupdateed by the cron job that's why it failed21:49
guimalufI rebooted the node, it will be commissioned now.21:51
guimalufroaksoax, after the commissioning many py errors showed up on the screen, but I couldnt get it21:53
roaksoaxguimaluf: can you show the log from apache? error and access please21:54
guimaluferror.log is the same21:54
guimalufroaksoax, http://pastebin.ubuntu.com/1097427/21:56
roaksoaxguimaluf: so does the MAAS Web UI say the node is ready?21:57
guimalufroaksoax, strangely now it is allocated to root21:57
roaksoaxthere's an error there: 150.164.3.240 - - [17/Jul/2012:18:55:55 -0300] "GET /MAAS/metadata//2012-03-01/meta-data/instance-id HTTP/1.1" 403 239 "-" "Python-urllib/21:57
guimalufyes, the same error.21:57
guimalufreboot and goes normal21:57
roaksoaxso it seems ot be failing commissioning21:57
guimalufnow it is installing21:58
roaksoaxok so lets wait and see :)21:59
guimalufthis python-urllib is happening everytime, then I reboot and everything runs as expected21:59
roaksoaxbut after it installed ubuntu22:03
roaksoaxbefore/after logging pronmt onreboot after installatio n22:04
roaksoaxit should show cloud-init trying to do some work22:04
guimalufno instance data found in start-init22:06
roaksoaxyeah that's what's failing22:07
roaksoaxi need to run for a bit now22:07
guimalufok22:07
guimalufany glue or tip?22:07
guimalufany glue or hint?22:07
guimalufi'll search for that22:08
guimalufthank you very very much for everything22:08
guimalufworked!22:08
guimaluf:D22:08
guimalufAEUhaUEhaUEHA22:08
guimalufLOL22:08
guimalufit works!22:08
guimalufbut maybe it is because I put the authorized_keys inside the /home/ubuntu/.ssh/ of the node22:09
guimalufand i think the juju bootstrap before the installation process helped alot!22:09
guimalufroaksoax, thanks man! really thanks! :D22:09
burnbrighterroaksoax: http://paste.ubuntu.com/1097463/22:26

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!