=== CyberJacob is now known as CyberJacob|Away
roaksoaxbigjools: howdy! when did the change to update-dns stuff landed and when did the integration tests start failing?02:25
bigjoolsroaksoax: not sure, raphael did the investigation work02:25
bigjoolsbut before xmas at least02:25
roaksoaxbigjools: err upstream_dns stuff02:27
bigjoolsI thought it was your changes for the charm?02:27
roaksoaxbigjools: nope, I never made that change02:28
roaksoaxbigjools: oh it is the DNS forwarders stuff02:28
bigjoolsdon't think so, that's not changed in packaging02:28
bigjoolsroaksoax: revno 215 on 2013-12-17 is the problem IIRC02:29
bigjoolsyour packaging change02:29
bigjools"Split maas-region-controller into maas-region-controller-min"02:29
roaksoaxbigjools: i was talking about the DNS forwarders changes02:29
bigjoolsroaksoax: oh sorry didn't understand what you were talking about02:30
bigjoolsno, those are not even started yet02:30
roaksoaxbigjools: http://pastebin.ubuntu.com/6742345/02:32
bigjoolsroaksoax: that's the upstream change - I thought you were talking about packaging changes?02:33
roaksoaxbigjools: so the issue hasn't really been a packaging change... thanks to the packaging change is that the issue was discovered02:33
bigjoolsanyway that change didn't make the integration tests start failing AFAIK02:33
bigjoolsah cool, you found out what's up?02:34
roaksoaxbigjools: the issue is that 'maas set_up_dns' is accessing the DB, when before it didn't use to do that02:34
bigjoolsmakes sense02:34
roaksoaxbigjools: so, http://paste.ubuntu.com/6742346/ -> So this is a hack I use to "fix" this02:34
bigjoolsdo you have a recommendation?02:34
roaksoaxbigjools: combined with http://paste.ubuntu.com/6742350/ I think we can solve the problem02:35
bigjoolsyeah we don't need it in postinst for sure, the job will deal with it02:35
roaksoaxbigjools: obviously for the upstream change, instead of a try/except, we can simply add a parameter to specify /'maas set_up_dns --upstream-dns=None02:35
bigjoolsyeah I'd prefer that over a try/except02:36
bigjoolsparam to override it fetching from config02:36
bigjoolsit's amazing what innocuous changes will break elsewhere ...02:36
bigjoolsthanks for figuring it out dude02:37
roaksoaxbigjools: yeah! But anyways, adding the parameter (which should be fairly simply), and removing the line in maas-dns.postinst, we have things back to normal (without having to revert changes back)02:37
bigjoolsif that's definitely the cause, I'm good with it02:37
roaksoaxbigjools: yeah I installed maas-cluster-controller-min , and then started running DNS commands manually (those in the psotinst) and that's how I found out what was wrong02:38
bigjoolsand bazinga02:38
roaksoaxbigjools: so do we want maas set_up_dns to receieve a boolean parameter to not try to obtain the value from the DB, or do we want a parameter where we can actually set the forwarder?02:52
bigjoolsroaksoax: set value is better02:53
roaksoaxbigjools: as in. 1. set_up_dns --no-upstream-dns or 2. set_up_dns --upstream-dns={None,}02:53
bigjoolsroaksoax: in fact do we need the code to fetch it from config at all?02:53
bigjoolsthe postinst is the only place that calls it IIRC02:53
roaksoaxbigjools: uhmmm I don't think so, cause celery tasks would also configure that, wouldn't they?02:54
bigjoolscelery uses a different code path02:54
bigjoolsthis stuff is only a "maas XXX" command02:55
roaksoaxbigjools: right, but besides from maas set_up_dns, who sets the upstream_dns when found in the DB?02:56
bigjoolsroaksoax: celery jobs02:56
roaksoaxbigjools: ok, so in reality, we don't really need it in set_up_dns, because the celery job will update that once it is manually added on maas then02:57
bigjoolsroaksoax: indeed02:57
bigjoolsthat's what I was starting to think02:57
roaksoaxbigjools: ok, so this should do it: http://paste.ubuntu.com/6742417/02:59
roaksoaxbigjools: I'll get that tested alongside the packaging change02:59
roaksoaxbigjools: ok it works : https://code.launchpad.net/~andreserl/maas/no_upstream_dns_on_setup/+merge/20133203:44
bigjoolsjtv: super quick pre-imp?05:49
jtvbigjools: sure...  firing up laptop.05:55
bigjoolsjtv: plug it in this time? :)05:56
jtvbigjools: any chance you could give this version of second DHCP detection a whirl?  https://code.launchpad.net/~jtv/maas-test/second-dhcp-check/+merge/20123207:02
jtvThis is meant to solve the problem you had detecting the DHCP server you were still running on the same machine you ran maas-test on.07:02
jtvIt should _also_ help with the case where the test interface has no IP address.07:02
=== CyberJacob|Away is now known as CyberJacob
rvbajtv: I get a failure when I run maas-test (trunk)…10:05
rvbait seems related to the DHCP stuff:10:05
rvbaIt looks (again, I might be wrong) as if we're running 'uvt-kvm ip <>' before the vm is even created…10:06
rvbajtv: does that sound possible?10:06
jtvIt seems unlikely — AFAIK we do a "uvt-kvm wait" first.10:07
rvbaOk, I'll investigate…10:08
jtvLooks like the problem you pasted is in get_maas_version, no?10:08
jtvI think I know what it is.10:08
jtvMore or less.10:09
jtvI made the install_maas() happen earlier than it used to.10:09
jtvShouldn't be before the "uvt-kvm wait" though.10:09
rvbaLooks like the MAAS fixture is setup before the KVM fixture :/10:09
jtvNo, but maas gets installed before the maas fixture is set up.10:10
jtvCrossing changes.  :(10:10
rvba(Revision 218 works)10:10
jtvThe exception seems to happen the second time install_maas runs.10:11
jtvDuring MAASFixture.setUp().10:12
jtvAt any rate, if the traceback is correct, everything looks wrong.  Why would "uvt-kvm ip" fail that late in the game!?10:13
rvbaWhat second time?  install_maas should only be run once.10:14
jtvAh.  That is a change I made, but I guess it's still in the branch I'm working on.10:14
jtvNote that we seem to be running "uvt-kvm ip" for every single command we run on the VM.  So you'd expect it to have succeeded at least once before this point.10:17
rvbaThe vm doesn't seem to exist at all when we run this command.10:17
jtvMissing sudo somewhere maybe?10:18
jtv← desperate guess10:18
rvbaFrankly, I'm really confused by the reorganisation revision 119 introduced.10:18
jtvIt just removes some class fixtures and makes them variables in main().10:19
rvbaNo, I put a debugging statement in run_command() and we simply don't create the VM at all.10:19
rvbaIt's a bit more involved than that :).10:19
rvbaThe result is that KVMFixture.setUp() is not called (or rather, we blow up in install_maas before it's even called).10:20
jtvNow I remember.  This was a fix I had wanted to have landed, but it got held up by the current packaging trouble.10:22
jtvIt's a missing setUp call.10:22
rvbagmb: btw, what I had in mind when I spoke about using run_command to include hardware information in the report is something along the lines of http://paste.ubuntu.com/6744069/11:22
rvbagmb: of course, we'd want to format the output a bit better.11:22
gmbrvba: Ah, thanks for the tip! I was looking for a way to do that as an API call in the mass-test code, but that seems straightforward enough (formatting notwithstanding).11:23
rvbaAn API call will also work, but since we have everything setup to shell out on the VM, the solution above might be a bit more easier.11:24
gmbYep, that seems perfectly sane to me.11:24
rvbagmb: for your viewing pleasure (I've always been very found of that expression), here is the result of that script on my test machine: http://paste.ubuntu.com/6744089/11:26
gmbrvba: Lovely, thank you. Although when I say lovely I may actually mean "gaaaagh." But I'm not to worried about making it nice as I am about making it functional right now.11:27
rvbaGetting it to work sounds like a good first step to me :).11:28
rvbagmb: actually, now that I think about it, it might be ever more simple to just do a "maas dumpdata."11:31
gmbrvba: Does that basically give us the whole DB?11:31
gmbMore data == more better.11:31
rvbaBut the DB won't contain anything big so it's safe.11:31
rvbaPlus we're sure we're not missing anything.11:31
rvbaself.maas.kvm_fixture.run_command(['sudo', 'maas', 'dumpdata'])11:32
* gmb does that.11:32
rvbaI'll give you an example of what you get back in a minute…11:34
rvba(I killed my vm :/)11:34
rvbagmb: http://paste.ubuntu.com/6744188/11:52
gmbrvba: Why all the leading whitespace?11:53
rvbagmb: don't know… looks like it's an artifact created by paste of something, here is the file verbatim: http://people.canonical.com/~rvb/dump11:54
gmbrvba: My feeling is we should just dump it, attach it, and care about cleaning it up later. Thoughts?11:55
rvbagmb: I agree.  It's pure JSON so it's easy to parse.11:56
rvbagmb: as a second step, if we still want to do that, it's easy to exact identifiers from the lshw output.11:57
gmbrvba: Okay. I'll finish this branch for gathering it, then I'll look at extracting stuff that's useful.11:58
rvbaSounds good to me.11:58
=== CyberJacob is now known as CyberJacob|Away
* gmb -> lunch12:28
=== matsubara_ is now known as matsubara
tomixxxi have the problem that my nodes stay in state "commissioning"17:10
tomixxxthey never get "ready"17:10
=== CyberJacob|Away is now known as CyberJacob
=== CyberJacob is now known as CyberJacob|Away

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!