bigjoolsiunruh: I'd start with double checking the ipmi parameters and run the commands manually to see if they work00:05
bigjoolsiunruh: then when you say " couldn't get anything from PXE", exactly what is on the console?00:05
iunruhbigjools: I'll try some things out with IPMI.. it seems to work sporadically00:08
iunruhbigjools: the message I get from PXE is "No DHCP or DHCP Proxy Offers received"00:08
iunruhboth happen sporadically, I'm wondering if its network related or an issue on my MAAS controller00:09
bigjoolssounds like networking problems00:09
bigjoolswe have a CI suite that tests this stuff using quite a few nodes every day and it's pretty reliable (apart from the bugs we root out!)00:09
bigjoolsare you able to sniff packets on the network using another machine?00:10
iunruhyeah, I can sit in the middle00:22
=== CyberJacob is now known as CyberJacob|Away
=== lifeless_ is now known as lifeless
adam_ghas anyone else encountered sporatic but frequent OAuthUnauthorized errorsfrom MAAS using the juju provider? seeing it across two different clusters03:13
jtvadam_g: clutching at straws but does it go away if you give admin privileges to the maas account which your juju environment uses?03:32
bigjoolsprobably re-using a nonce03:55
=== Tm_K is now known as Tm_T
=== rvba` is now known as rvba
=== CyberJacob|Away is now known as CyberJacob
jtvgmb, rvba: no changes to the Zones requirements yesterday?  Julian says no need for a config item for the default zone name — we can just fix a name in the code.07:56
jtvI have a branch here that tests just about all the changes we need.07:56
rvbaI haven't heard from gmb yet.07:56
rvbaHe'll probably fill us in during the standup.07:57
jtvI was hoping for a few hours earlier!  gmb, any news on that?07:59
adam_gjtv, i've found it happening on both clusters using admin account07:59
gmbrvba, itv: Dean and I haven't had chance to speak yet; bigjools doesn't want us to block on this, so we're going to go with the non-renamable default zone.07:59
jtvadam_g: then bigjools's guess sounds better than mine.08:00
jtvgmb: Great, thanks.  I've been working on that assumption.08:00
adam_gjtv, bigjools is there any way to furhter debug / fix the Nonce issue? fwiw, i am not seeing the 'Nonce already used' or whatever error i used to see often. only the OAuth error in the maas logs and the gomaasapi error on the juju side08:01
jtvadam_g: one important thing to check is that the machines' clocks are in sync.08:04
jtvIf they drift too far apart, oauth becomes a problem.08:04
adam_gjtv, hmm. i'll check that they are.  on one cluster that may be an issue (juju client running in an instance on a cloud talking to a maas in another DC), but the other is juju client running on the machine hosting the MAAS  API endpoint08:07
=== wgrant_ is now known as wgrant
adam_gill check tomorrow. thanks, jtv08:09
=== gnuoy` is now known as gnuoy
rvbajtv: could you share the code you're working on right now?  I'm afraid we might be both working on exactly the same thing.08:14
jtvrvba: just have a look at my code page... I've been pushing updates there.08:22
rvbaOkay, ta.08:23
jtvrvba: I assumed that you are creating the default zone, at which point I will have a bunch of "factory.make_zone(name=DEFAULT_ZONE)" statements that will start failing, at which point they can be deleted.08:23
jtv(I deliberately kept them that way — it may seem inefficient sometimes, but this makes the update simple and mechanical)08:24
rvbajtv: yep, that's what I'm working on indeed.  Plus preventing the deleting or the renaming of that zone.08:24
jtvAh cool, then I don't have to do that — but I do have the tests for it.08:24
rvbaNice :)08:25
jtvBy the way, if we create the default zone in a migration, will it be preserved between tests..?08:25
jtvAnd you've got dozens of branches still in Development status...  Better clean that up from time to time!08:26
rvbajtv: it should be preserved between tests yes (or rather, between each test, the db is reverted to the state it was in after all the migrations ran)08:27
rvbajtv: yeah, I know.  We only have a proper landing bot for maas.  All the other projects leave branches in Dev status, even after they've been merged.08:31
rvbajtv: WIP - https://code.launchpad.net/~rvb/maas/default-zone/+merge/20063508:31
jtvAh thanks — in a moment I'll try my tests against that.08:32
=== jtv1 is now known as jtv
bigjoolsrvba: maas-test has a lander09:51
jtvrvba: we have some amusingly small differences between our branches — I have Zone.objects.get_default() where you have Zone.objects.get_default_zone(), and I have Zone.can_delete() where you have Zone.is_default().  I'll change mine to fit yours, and see what tests need porting over.09:52
jtvOh, and I have DEFAULT_ZONE where you have DEFAULT_ZONE_NAME.09:53
rvbabigjools: well, for some reason, I've got merged maas-test branch which are still "in Development."09:55
bigjoolsbranch scanner is broken then09:55
rvbabranches* even09:56
bigjoolsnot a lander job09:56
melmothhola ! i cannot comission node anymore (used to work all right "before", that was..last year). now when i comissiona node it end up with  failed [2/5] ( 00-maas-01-lshw 00-maas-02-virtuality)10:30
melmothany idea what to do to ?10:30
jtv1melmoth: any chance you can see the consoles of the failed nodes?10:41
jtv1You should also have the output of the commissioning scripts in the database, but I don't recall whether we show it in the UI.10:42
melmothi do, but it s going to fast for me to spot any error10:42
=== jtv1 is now known as jtv
melmothAHhhhh, seems to work better after i restarted a squid proxy i changed some config in it.10:44
melmothprobably test were failing because it needed to download pakcaged that were denied by the proxy10:44
jtvYes, that'd do it.10:44
jtvThe nodes don't talk to the internet themselves; they all go through the proxy.10:45
jtvYup, the next commissioning script after 00-maas-02-virtuality would be 00-maas-03-install-lldp.10:45
jtvWhich is where it tries to install a package...10:45
rvbajtv: I don't understand why (in your hardcoded-default-zone branch), you had to cope with the possibility for the zone to be None.  Wouldn't it be better to simply change the dropdown that contains the list of zone so that it won't show that option?10:50
rvbalist of zones*10:50
jtv"The" zone to be None?10:51
jtvAre you talking about the bulk action?10:52
rvbaSorry if that wasn't clear.10:52
jtvI'm not sure about that — I kept it in for the time being as protection against accidents.10:52
jtvFor example, if you run a JS blocker and forget to create an exception for your maas, you'd get:10:53
jtv"Oh, I can set the zone for these nodes.  That's what I want to do.  Hit the Go button."10:53
jtvAnd then you think it'll ask for a zone, but actually it just set your nodes to the default zone.10:53
rvbaWell, maybe that protection can be left.  But I still think the dropdown should be fixed.10:54
jtvCan we do that without also accidentally setting a default for the dropdown?10:55
rvbaYes, the "----" is there because we said the field wasn't required.10:56
jtvOK, then I can remove the empty string from the dropdown.10:57
rvbajtv: I'll merge your branch now… unless there are other things you want to do with it before I merge it.11:06
jtvrvba: better wait a bit more.11:06
jtvThere are various conflicts, and of course the expected failures I mentioned.11:07
jtvI'm currently doing an experimental merge to see what else needs fixing, and a few things have come up.11:07
rvbaOkay.  Our branches are already conflicting quite badly.11:07
jtvIt's not that bad.11:07
rvbaRats, I was doing the same.11:07
jtvI've already resolved the conflicts.11:07
jtvIt's OK — this needs a few trial runs.11:07
jtvAlso, don't forget to check for lint because we have some duplicated definitions and such.11:08
rvbaRight.  Well, just give me a go when I can merge your branch.11:09
jtvrvba: one thing that breaks when I merge your branch is test_AdminForm_sets_zone_initial_value — there no longer is a self.initial['zone'].11:24
rvbajtv: yeah, the test can be dropped now.11:25
jtvAlso, your validation on the ZoneForm raises an error about renaming the default zone when you try to change the *description* on the default zone.11:26
rvbaYeah, I'm not sure what to do about this.  Maybe we should not allow anything to be changed on the default zone.11:28
jtvIt seems counterintuitive.  Why not let people describe what the default zone means to them?11:32
rvbahum, good point.11:33
gmbjtv, rvba: Remember, the default zone is there just to make the cloud installer's job a bit easier; We're doing a bit of ZFDD here — if they want to describe what a zone means to them they should add one.11:33
* gmb stops parroting Jools.11:34
jtvEither works, but then we should change the error message.11:35
jtvOh, and also hide the edit button.11:35
jtvSo actually, disallowing updates to the description is more work.11:35
jtvIt's easier just to add one condition to the "if."11:36
jtvrvba: also getting unhelpful errors in the Selenium tests...  “Zone matching query does not exist.”11:39
rvbaI saw that, it's the get_default_zone method failing… not sure why.11:40
jtvI guess some kind of surprise in how the database gets restored...  :/11:45
jtvLooking again at the zones dropdown on the bulk "set zone" action on the nodes listing, I don't see how to remove the "----" entry...11:46
jtvWe can't make that field required.11:46
rvbajtv: http://paste.ubuntu.com/6708671/11:53
* gmb lunches12:08
=== rbasak_ is now known as rbasak
jtvrvba: your answer to "we can't make that field required" (I hope the reasons are obvious) is a pastebin link to a diff that makes the field required.  What's the context?  Are you saying yes we can?  I'm getting tons of broken tests, as I would expect.12:16
jtvOr are you telling me _how_ to change the boolean but you didn't try it?12:17
rvbajtv: I tried and it seemed to work okay.12:19
jtvI'm getting dozens of failures...12:19
jtvAnyway, it's not very helpful pasting me a diff for a single True/False change — I know how to do that, but it's the explanation that matters!12:20
rvbaSorry :).12:21
rvbaThat's very strange that this is causing a lot of test failures.12:21
rvbajtv: sorry, I'm just trying to help while having lunch at the same time :)12:31
jtvSeriously, don't let helping me drag you away from lunch — you'll burn yourself out!12:39
jtv(Well not from doing it once, of course, but...  :)12:39
jtvI was expecting the failures, because we don't get the zone field if we submit a different bulk action.12:40
jtvI guess to work around that we'd have to specify the default as well, but at that point we do lose the "accident insurance" we talked about earlier.12:40
jtvOh wait, you do have the default!12:40
jtvI missed that because in your diff it wasn't marked as diff.12:41
tych0hey, suppose i run a power up command and it fails16:48
tych0is that log stored anywhere?16:48
bjornewhy this happend for me? the client get user-data file when it starts up for the first time... only for comis/installing and not after that like this in the log:17:55
bjorne172.16.1.114 - - [06/Jan/2014:19:34:25 +0100] "GET /MAAS/metadata/enlist/2012-03-01/user-data HTTP/1.1" 200 15192 "-" "Cloud-Init/0.7"17:55
bjorne172.16.1.114 - - [06/Jan/2014:19:36:15 +0100] "GET /MAAS/metadata//2012-03-01/user-data HTTP/1.1" 200 28222 "-" "Python-urllib/2.7"17:55
bjorne172.16.1.114 - - [06/Jan/2014:19:39:14 +0100] "GET /MAAS/metadata/curtin/2012-03-01/user-data HTTP/1.1" 200 33546 "-" "Python-urllib/2.7"17:55
bjorne172.16.1.114 - - [06/Jan/2014:19:41:35 +0100] "GET /MAAS/metadata//2012-03-01/user-data HTTP/1.1" 404 200 "-" "Python-urllib/2.7"17:55
=== _bjf is now known as bjf
bjornehave no one seen that problem before i have?19:11
=== CyberJacob is now known as CyberJacob|Away
=== CyberJacob|Away is now known as CyberJacob
=== CyberJacob is now known as CyberJacob|Away
bigjoolstych0: no, that's on the large list of things for which we need better debugging22:07
tych0bigjools: ok, another thing i noticed is that maas ignores the result of the celery job and just assume the machine started/stopped successfully22:14
tych0is there a bug for that, or should i file one?22:14
bigjoolstych0: yep :(22:14
bigjoolsthere are bugs and blueprints22:15
tych0ok, cool22:15
bigjoolsit needs an overhaul22:15

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!