[07:41] <rvba> bigjools: meanwhile, would you have time for a review (the diff looks big because there is a migration in it but the changes are pretty mechanical): https://code.launchpad.net/~rvb/maas/improve-broken-state/+merge/228132 ?
[07:43] <bigjools> rvba: I saw it don't worry, just clearing my inbox and then I'll get to it
[07:44] <rvba> bigjools: cool, thank you
[09:32] <allenap> Does anyone have a moment for a review? https://code.launchpad.net/~allenap/maas/rpc-call-get-preseed-data/+merge/228179
[09:33] <rvba> allenap: sure, I'll take it
[10:02] <rvba> allenap: at the bottom of the diff, why are you changing metadata_url to be of type `urlparse.ParseResult`?  Doesn't that parameter come from `absolute_reverse` which returns a string?
[10:05] <allenap> rvba: It actually gets passed via RPC as a tuple; see the ParsedURL type. I had previously stubbed it out using a string instead of a urlparse.ParseResult.
[10:05] <allenap> Which was a mistake.
[10:10] <rvba> allenap: oh, I see; the conversion is actually done in src/maasserver/clusterrpc/osystems.py:get_preseed_data;  that's the part I was missing.
[11:50] <bigjools> allenap: can you help me set up python's logger in pserv please?  There's currently zero feedback from the downloader service which is a little jarring :)
[11:52] <allenap> bigjools: Quick and dirty is to call logging.basicConfig(level=logging.DEBUG) in p.plugin…makeService().
[11:53] <bigjools> allenap: yesterday you said the loggingobserver needs setting up
[11:54] <bigjools> I detest logging in twisted
[11:54] <allenap> bigjools: That’s to get Twisted’s logs to go via the stdlib’s logging module.
[11:54] <allenap> So you can configure both in there.
[11:54] <allenap> bigjools: Fwiw, so do most people who use Twisted.
[11:55] <allenap> Where “configure both in there” means you can select levels and whatnot using the familiar stdlib’s logging stuff.
[11:55] <allenap> For Twisted logs.
[12:02] <bigjools> allenap: I am slightly confused by it all.  The log there now is direct from twisted, right?  If Python's log is configured will it just pick it up and show it?
[12:04] <allenap> bigjools: They’re separate at the mo. If you call twisted.python.log.err or .msg it’ll print something to can’t remember where (that’s configured in the plugin). If you use the stdlib’s logging module then it must be configured to show your messages; by default the level is set to WARN.
[12:05] <allenap> bigjools: PythonLoggingObserver can be used to bridge the Twisted logs over to logging, but that’s not in there yet.
[12:05] <allenap> not in there = not being used yet
[12:06] <bigjools> allenap: yeah I think the pserv service is started with --logfile=/dev/null
[12:06] <bigjools> stdout is redirected instead
[12:09] <allenap> bigjools: p.services.LogService gets all Twisted logs to go to the filename specified, which is defined in pserv.yaml iirc.
[12:10] <bigjools> yeah
[12:10] <bigjools> so is that going to pick up python's logging?
[12:10] <bigjools> Guess I'll try it out
[12:19] <allenap> bigjools: Nope… but you could add a call to logging.basicConfig into LogService.startService.
[12:19] <bigjools> allenap: that's what I mean, if it's configured, where will it go?
[12:20] <allenap> bigjools: basicConfig send the logs to stdout or stderr iirc.
[12:21] <bigjools> allenap: yeah I think so.  might interleave with twisted...
[12:21] <allenap> bigjools: The —logfile=… argument to twistd only affects Twisted’s logs I think. It doesn’t redirect stdio or configure the logging module.
[12:21] <allenap> bigjools: As long as you can see the logs for development that’s a start.
[12:22] <bigjools> I think we added the loggingservice to make the log location sane
[12:22] <bigjools> I am past development, thinking of production/QA now
[12:29] <allenap> bigjools: I would think about changing LogService to create a new PythonLoggingObserver to bridge Twisted logs over, and to configure the logging module using the log file defined in pserv.yaml.
[12:30] <allenap> The latter can be done in p.plugin…makeService().
[12:30] <bigjools> allenap: yeah, sounds reasonable
[12:30] <allenap> bigjools: The current functionality of LogService can be removed.
[12:30] <bigjools> right
[13:10] <rvba> allenap: I'd like to create a RegionRPCFixture similar to the CluserRPCFixture.  I'm thinking about doing something like: http://paste.ubuntu.com/7855652/ but it requires Region (from maasserver) to be import from provisioningserver which is obviously wrong… Any idea what I can do instead?
[13:11] <allenap> rvba: Yeah :) Import Region from p.rpc.region instead; that’s where it’s defined.
[13:12] <allenap> rvba: Btw, self.patch() doesn’t work on Fixture.
[13:12] <rvba> allenap: oh, right, I'll MonkeyPatcher then
[13:12] <rvba> allenap: thanks!
[13:13] <rvba> I'll use* MonkeyPatcher
[13:13] <allenap> rvba: And I’m wrong about Region().
[13:13] <rvba> allenap: yeah, that's what I just found out
[13:14] <allenap> rvba: I think you’ll have to make it easy to create stubs.
[13:16] <rvba> allenap: okay, I'll look into it… but I'd like a fixture that wires things up magically (i.e. a fixture that removes all the AMP transport and calls the 'responder' directly).
[13:16] <bigjools> easy karma for someone: https://code.launchpad.net/~julian-edwards/maas/touch-maasmeta/+merge/228301
[13:17] <bigjools> rvba, allenap: we really need a full end-to-end fixture...
[13:17] <allenap> rvba: That’s not going to be easy at all; you’ll need to get Django configured and all that jazz.
[13:19] <allenap> bigjools: I don’t think we do. I’ve cheated with ClusterRPCFixture and I’m having second thoughts. We should develop the region and cluster against each other’s RPC schema. We ought to get into the habit of writing QA tests for the whole-system side of things.
[13:20] <bigjools> allenap: but I want integration tests that talk each way
[13:20] <allenap> bigjools: You may think you want that ;)
[13:21]  * bigjools dodges allenap's meta-argument :)
[13:23] <bigjools> can someone review my simple branch please!
[13:30] <rvba> bigjools: looking at your branch now
[13:31] <bigjools> thank you rvba!
[13:42] <jseutter> bigjools: I have a workaround for that grub issue where maas couldn't install on some nodes from earlier this week.  I dropped several drives from the raid array (down to 1.1TB array size) and the maas install worked.  What is weird is that the ubuntu cd installer works with all drives in the array.
[13:49] <bigjools> jseutter: I've seen the installer fail for me when I've got raid partitions present (outside of maas)
[13:50] <jseutter> bigjools: ok, good to know :)
[16:05] <bigjools> 600-line upper decker for your Friday afternoon enjoyment. https://code.launchpad.net/~julian-edwards/maas/replace-celery-download-job-with-pserv/+merge/228168
[16:05] <bigjools> no takers?  Beuller? :)
[16:10] <bigjools> allenap: the logging is doing my head in, let's talk about it next week
[16:11] <allenap> bigjools: Yeah, let’s :)
[16:11] <allenap> bigjools: I haven’t looked at it in a long long time and it does need some attention.
[16:11] <bigjools> allenap: if we send to stdout it makes things somewhat awkward
[16:11] <bigjools> we have our own custom logging service
[16:11] <bigjools> that takes a filename
[16:11] <bigjools> not sure we need a filename, but at the same time we do need to write a log to the right place