| roaksoax | bigjools: howdy!! | 02:18 |
|---|---|---|
| roaksoax | kurt___: sorry if you are still around, maybe bigjools can help you. otherwise I can help you tomorrow | 02:18 |
| roaksoax | bigjools: so I uploaded the SRU bugs to -proposed so need verification | 02:18 |
| roaksoax | as soon as they get approved | 02:19 |
| kurt___ | hi bigjools - are you around? | 02:31 |
| kurt___ | bigjools: I have a question when you are around | 03:49 |
| jtv | kurt___: what log file was that traceback in? | 04:34 |
| jtv | And any idea what was going on at the time? | 04:35 |
| kurt___ | jtv: that was in the maas.log | 04:36 |
| kurt___ | I was trying to install the juju-gui | 04:36 |
| jtv | Ah yes, stupid of me. | 04:36 |
| kurt___ | and it doesn't matter which version of juju-gui. I think the problems are lower than that. | 04:36 |
| jtv | I don't suppose you saw an error from juju, juju-gui, the installation process..? | 04:36 |
| jtv | It could mean that juju simply isn't authorized to access the MAAS. | 04:37 |
| kurt___ | the app installs on the root node | 04:37 |
| jtv | It needs credentials to talk to the MAAS API over http. | 04:38 |
| kurt___ | but it is never able to spin up the additional nodes | 04:38 |
| jtv | Was juju ever able to do anything on this maas? | 04:38 |
| kurt___ | I can bootstrap with no problem | 04:38 |
| kurt___ | that creates the root node | 04:38 |
| jtv | Oh, and is this juju-core? Or the older python-based juju? | 04:39 |
| kurt___ | eh… :) | 04:39 |
| kurt___ | I'm installing from apt-get - or the results were the same when I was doing it from the iso | 04:40 |
| jtv | I don't know actually know which releases come with which Juju... I guess you're working with 12.04? | 04:40 |
| jtv | Ubuntu 12.04, I mean? | 04:42 |
| kurt___ | I've tried both 12.04 + 12.10 | 04:42 |
| jtv | I wonder if juju-gui is doing something that requires admin access to the maas... | 04:44 |
| jtv | *If* you've got Juju authenticating as a regular user, you could try making that user an admin through the MAAS web UI and seeing if that fixes the problem. | 04:45 |
| kurt___ | how is that done? | 04:45 |
| kurt___ | my particular user is an admin I believe | 04:46 |
| * jtv is a bit rusty on all this | 04:46 | |
| jtv | Have you logged in on the MAAS web UI before? | 04:46 |
| jtv | You should be able to access it on http://<your-region-controller>/MAAS | 04:48 |
| jtv | Should be fairly self-explanatory from there. | 04:49 |
| kurt___ | yeah for sure | 04:50 |
| kurt___ | I have to work with it for the other nodes | 04:51 |
| kurt___ | Ok, I added that user - the only other user was root. I will try destroying the environment and trying again | 04:53 |
| kurt___ | jtv: latest full trace from debug-log is here http://pastebin.ubuntu.com/5979922/ | 05:17 |
| jtv | Looking... | 05:17 |
| jtv | Ah, so that's the Python-based Juju. | 05:18 |
| jtv | It may become useful at some point to know which codebase we're dealing with. :) | 05:19 |
| kurt___ | quantal | 05:19 |
| kurt___ | juju .7 | 05:19 |
| jtv | One obvious error in there (not necessarily *the* error) seems to be about clock skew. So it may help to make sure that all machines NTP to the same server. | 05:20 |
| jtv | Or maybe it just means that a response is too old by the time it is received, which could happen if things were very very slow. | 05:20 |
| kurt___ | right, but I'm not sure how to deal with that from standard install | 05:20 |
| jtv | I *think* a standard install sets up NTP uniformly. | 05:20 |
| kurt___ | here is the time from node 0: | 05:21 |
| kurt___ | ubuntu@nf4xd:~$ date | 05:21 |
| kurt___ | Mon Aug 12 18:20:54 EDT 2013 | 05:21 |
| jtv | (I think given the "300" we're looking for a difference of at least 5 minutes) | 05:21 |
| kurt___ | here is the time from the controller | 05:21 |
| kurt___ | kurt@maas-ctrl02:~/.juju$ date | 05:21 |
| kurt___ | Mon Aug 12 22:21:22 PDT 2013 | 05:21 |
| jtv | I'm guessing there's probably a 4-hour difference between EDT and PDT. :) | 05:21 |
| jtv | Oh, there *is* a known problem with installing nodes whose clocks are far off. | 05:22 |
| jtv | Any way you can check that? What's particularly insidious is that the Windows world ships machines with hardware clocks set to the local timezone, not UTC. | 05:23 |
| jtv | Maybe the node you're trying to allocate just has a weird clock setting. | 05:24 |
| kurt___ | its a VM | 05:24 |
| jtv | Ugh. | 05:24 |
| kurt___ | I need proof of concept before I go spend $15k in machines :) | 05:25 |
| jtv | Ah-hah! That clock lag amounts to exactly 7 hours. | 05:26 |
| kurt___ | ok? :) | 05:26 |
| jtv | So that does sound as if the VM might be on a local clock instead of UTC. | 05:26 |
| jtv | Maybe it has a setting for that? | 05:26 |
| kurt___ | there is a synchronize time setting | 05:27 |
| jtv | (Not saying "ugh" about you using a VM, just about my not knowing how each virtualisation system manages its clocks :) | 05:27 |
| kurt___ | ah gotcha | 05:27 |
| jtv | I don't think this is a matter of synchronizing the time, but IIRC some virtualization systems have an option for "make the guest's hardware clock look like it's set to local time." | 05:28 |
| jtv | It may even be implied by an "I'm going to run Windows on this" setting. | 05:28 |
| kurt___ | that's not totally obvious in Fusion - I may need to investigate | 05:28 |
| jtv | If it's the bug I was thinking of, then a fix should already be available: https://bugs.launchpad.net/maas/+bug/978127 | 05:36 |
| ubot5 | Launchpad bug 978127 in MAAS "incorrect time on node causes failed oauth" [Critical,Fix released] | 05:36 |
| kurt___ | it's mind numbing trying to figure this out from vmware | 05:40 |
| jtv | I'm searching the general internet for this one. | 05:43 |
| kurt___ | there is this, but … layers and layers too deep | 05:43 |
| kurt___ | http://www.vmware.com/files/pdf/techpaper/Timekeeping-In-VirtualMachines.pdf | 05:43 |
| jtv | If the bootstrap node and the one you're trying to start up are near-identical guests on the same VMWare instance, then how in blazes can one have clock skew but not the other!? | 05:48 |
| kurt___ | great question | 05:48 |
| kurt___ | isn't it cloud-init that's taking care of setting the time? | 05:49 |
| jtv | I think that was part of the solution to the bug I quoted, yes. | 05:49 |
| jtv | Are you saying that maybe you already have that fix, and it may be going wrong somehow? | 05:50 |
| kurt___ | I think that fix was rolled in to quantal already | 05:50 |
| jtv | smoser might knoww. | 05:50 |
| kurt___ | *I think* | 05:50 |
| jtv | *know. | 05:50 |
| kurt___ | bigjools would know for sure | 05:50 |
| kurt___ | alas he is elusive today :D | 05:51 |
| jtv | I don't think he'll be available today. But smoser is the oracle on all matters cloudinit. | 05:53 |
| kurt___ | ok | 05:53 |
| jtv | That bug I linked to did mention vmware as a bit of a problem case IIRC. | 05:54 |
| jtv | Hmm... no, not vmware by name. | 05:54 |
| kurt___ | yeah that bug is old and it appears the fix went in to precise | 05:57 |
| jtv | Yes, but long after Precise was released IIRC. | 05:58 |
| kurt___ | right | 05:58 |
| kurt___ | I've been seeing this for a few days and trying to figure out how to solve that part | 05:59 |
| kurt___ | its a real head scratcher | 05:59 |
| kurt___ | the clocks are never in sync | 05:59 |
| jtv | Hmm... but it only happens when you try to play with juju-gui, doesn't it? | 05:59 |
| === CyberJacob|Away is now known as CyberJacob | ||
| jtv | Maybe those requests are coming from somewhere unexpected..? | 06:00 |
| jtv | roaksoax: are you familiar with the clock-skew problem by any chance? kurt___ seems to be running into it in vmware. | 06:00 |
| kurt___ | so the OAuthUnauthorized happens during juju-gui | 06:00 |
| jtv | (Timezones are thwarting us in more ways than one at the moment) | 06:01 |
| kurt___ | but the clock is skewed no matter what | 06:01 |
| jtv | Well... I've been assuming that the clock skew is the cause of the authorization errors. | 06:01 |
| jtv | Something to do with oauth. | 06:01 |
| kurt___ | yes, I agree wholeheartedly | 06:01 |
| kurt___ | but oauth has nothing to do with creating the skew though I think | 06:02 |
| jtv | True. | 06:02 |
| jtv | It shouldn't, unless it's a side effect of that bug workaround. | 06:02 |
| kurt___ | my worry is that this is a non-issue for real hardware and is not a priority because it is a VM | 06:03 |
| jtv | Might be, yes... | 06:04 |
| kurt___ | there must be loads of people like myself who *need* this to work via VM | 06:04 |
| === jam1 is now known as jam | ||
| jtv | And we've done some testing with VMs as well, but not vmware I think. | 06:05 |
| jtv | All qemu/kvm IIRC. | 06:05 |
| kurt___ | well, I think we are stuck until we can hear from jools or smoser | 06:06 |
| jtv | 'fraid so. :( The VM test setups I know of are kvm-based. | 06:07 |
| jtv | And I'm going to have to run an errand. | 06:12 |
| kurt___ | ok, thanks for your help | 06:13 |
| === CyberJacob is now known as CyberJacob|Away | ||
| disposable | if i use the maas-dhcp package, will that interfere with my existing dhcp server or will it only reply to requests from MAC addresses it knows about? (i can set my other dhcp server to ignore certain mac addresses) | 10:14 |
| melmoth_ | disposable, it will interfere | 10:16 |
| disposable | melmoth_: thank you | 10:16 |
| melmoth_ | disposable, i think there is a way to use a third party dhcp, but you have to tell it (your alreadt existing one) that the next server is maas | 10:16 |
| melmoth_ | and where to find the pxe boot image. | 10:16 |
| melmoth_ | unfortuniately, i dont know the specific details on how to do that. | 10:17 |
| disposable | melmoth_: i am already using a different dhcp server successfully, i just wanted dhcp+dns integration | 10:17 |
| disposable | dhcp+dns+maas integration | 10:17 |
| smoser | i'm not in till tomororw really, but kurt___ and jtv, the issue with oatuh shoudl be fixed. | 13:53 |
| smoser | cloud-init does complain that the clock is very far off, but adjusts its timestamp to that of the server when it tries again. | 13:53 |
| smoser | you're just seeing warnings of it saying "your clock is messed up, you shoudl fix it". | 13:53 |
| smoser | i'm not saying that htere are notstill bugs, but the issue is generally fixed in any up to date ephemeral image. | 13:54 |
| === CyberJacob|Away is now known as CyberJacob | ||
| === lifeless_ is now known as lifeless | ||
| === CyberJacob is now known as CyberJacob|Away | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!