[08:21] <jtv> gmb: that file was part of the simplestreams code drop, which lacked tests.
[08:21] <jtv> We did some of the most basic cleanup, and there's an end-to-end test, but more unit tests are still needed.
[08:32] <bigjools> gmb: loads of that import code is untested, we're adding it as we go
[09:02] <bigjools> jtv: perhaps if you have some time you might want to take a look
[09:02] <bigjools> https://code.launchpad.net/~julian-edwards/maas/report-subarches-in-bootimage/+merge/219987
[09:02] <bigjools> at least someone please give me something I can work on tomorrow :)
[09:03] <jtv> bigjools: is this just because when I commented on it earlier it was in the wrong channel, which _you_ picked?
[09:03] <jtv> Or is this a different one?
[09:03] <rvba> gmb: btw, I just noticed that you used django's BinaryField in BootSource.  It's all right but it means we now have 2 different BinaryFields in use: the one we created (back when the other one did not exist) and the one provided by Django.
[09:04] <bigjools> jtv: same one
[09:04] <jtv> If we had time, I'd love to look into abolition of our home-grown BinaryField...
[09:04] <bigjools> jtv: doesn't need to be you, just *someone*, please!
[09:04] <rvba> Yeah, that would be great.
[09:04] <rvba> bigjools: I'll have a look later today.
[09:05] <jtv> bigjools: as I said, needs documentation.  :)
[09:05] <bigjools> jtv: next branch
[09:06] <jtv> OK.  One thing that's still sadly lacking is a mention of what that meta file is _for_.
[09:06] <jtv> (And if you're going to chide me for the terminal preposition, up shut)
[09:08] <bigjools> jtv: an abomination up with which I shall not put
[09:08] <jtv> On which, spot.
[09:08] <jtv> It's awkward and off it pisses me.
[09:09] <bigjools> I shall pop in some comments to explain for what the file is used
[09:09] <jtv> Huh what?
[09:09] <jtv> Oh, I see what you did there...
[09:30] <jtv> bigjools: I think extract_image_params suffers a bit from the vagueness of that term "params."  It makes you call individual elements of params "param" despite it containing multiple parameters.
[09:30] <jtv> Is there no more specific name you can give each of those dicts?
[09:30] <bigjools> jtv: yes, like I said, the branch needs some tidying
[09:30] <bigjools> it has evolved somewhat
[09:31] <jtv> Just adding specifics, because you needed things to work on...
[09:31] <bigjools> just bear in mind this is nothing I've added :)
[09:32] <jtv> AFAICS "param" is new.
[09:32] <bigjools> oh that bit yeah
[09:33] <bigjools> well the function name started it I supposed
[09:33] <bigjools> extract_image_params is returning a variable called... params.  Fancy :)
[09:35] <bigjools> I need to EOD now, rather tired
[09:35] <bigjools> all that coding has left me exhausted!
[09:35] <jtv> Good night!
[09:35] <bigjools> thanks for looking at the branch
[09:35] <bigjools> I appreciate your comments
[09:36] <jtv> Apart from the one about your face, obviously.
[09:36] <jtv> Good night.  :)
[09:36] <bigjools> you said nice things about it once
[09:37] <jtv> You mean the one where it was too high for me to see?  :)
[09:37] <bigjools> or maybe that was in one of your dreams
[09:37]  * jtv wonders how bigjools would hear him talk in his sleep...
[09:37] <bigjools> I am turning this PC off now before this gets too exciting
[09:37] <jtv> *chuckle*
[09:37] <bigjools> cheerio :)
[09:38] <jtv> nn
[10:14] <rvba> gmb: arg, it's getting worse: a BinaryField is non-editable.  Which means we can't use forms to accept base64-encoded input to change the value of that field.
[10:16] <gmb> rvba: Damn.
[10:17] <rvba> gmb: I'll figure a way… but it will require changes to the model…
[10:17] <gmb> Yeah.
[10:17] <gmb> rvba: So, what about (this is horrible) storing the keyring data as a (base64-encoded) string, and using a file upload field to get the keyring data in there (and then dealing with the file behind the scenes).
[10:17] <gmb> Could that work with the API/CLI?
[10:17]  * gmb doesn’t know whether we do file-stuff already within the CLI.
[10:18] <rvba> I seem to remember the CLI supports file upload now…
[10:18] <jtv> It does
[10:18] <rvba> gmb: it would be better to figure out a way that work with Django's binary field though.
[10:19] <rvba> works*
[10:19] <gmb> rvba: Definitely. I’ll happily defer to your Django experience rather than my hacktastic approach :).
[11:01] <jtv> gmb: BootSource.objects.get_boot_sources_for_cluster() takes more typing than doing the query yourself.  Maybe we should just have a 1:n attribute on NodeGroup.
[11:02] <gmb> jtv: Totally WFM.
[11:02] <gmb> jtv: I’ll stick a card on the board.
[11:02] <jtv> Low priority though.  :)  This is tech debt.
[11:02] <jtv> Fairly easy to pay off.
[11:03] <gmb> Yeah.
[11:03] <gmb> jtv: Actually, IIUI, it should be there already as NodeGroup.bootsources_set.
[11:03] <gmb> But I’d forgotten that when I wrote the function :)
[11:04] <jtv> Might be, actually.  I forget whether that needed any work.
[11:04] <gmb> I think it’s automatic.
[11:21]  * gmb -> Dinner (by which I mean Northern ‘dinner,’ which Southerners call “Lunch”).
[11:22]  * jtv very confuse
[11:32] <rvba> gmb: I think I spotted another problem.  We use a FilePathField for BootSource.keyring_filename.  If you have a look at the doc you'll see that this will validate against the *region*'s file system.  Where in fact this file only has meaning in the context of the cluster.
[11:39] <jtv> I thought it was intended for use to refer to a region-side file..?
[11:39] <jtv> IIRC the region then reads that file and sends the _contents_ to the import code.
[11:44] <rvba> Then I don't understand why we have bot a keyring_filename and a keyring_data field.
[11:44] <rvba> both*
[11:46] <jtv> My understanding was: if you set a filename, it will get read from the filesystem every time the imports are run, which means the keyring could come with a package, and get automatic updates.
[11:46] <jtv> Whereas if you have a custom keyring, you can pass the keyring contents centrally and have the import code use that instead, and you manage updates yourself.
[11:46] <jtv> But I'm only guessing, really.
[11:47] <rvba> That would make sense.  I'll let gmb confirm that it's what he had in mind.
[11:47] <jtv> And document.  :)
[11:48] <rvba> jtv: then I guess we need to think about migrating the config from the cluster's config file to the region's DB.  Because the meaning of the keyring_filename is changing (from a file on the cluster to a file on the region).
[11:49] <jtv> I thought the plan was not to migrate those files.
[11:49] <rvba> So, use the default?
[11:49] <jtv> Yes, download the world.
[11:50] <jtv> (For Precise/Trusty).
[11:50] <jtv> With a UI for those who want to pare it down.
[11:50] <rvba> I see.
[12:01] <jtv> gmb: could you also confirm to us that there is no plan to migrate existing bootresources.yaml files?  I've been working on the assumption that we always start off with The World.
[12:01] <jtv> (Things get a lot more complicated otherwise.)
[13:07] <jtv> gmb: I see there's a card for removing parts of bootresources.yaml.  I'd just leave the file alone until we delete it.  Changing it further only invites yet more conflicts during upgrade.
[13:09] <gmb> jtv: +1; that card’s from before I knew what we needed to do :)
[13:09] <jtv> OK
[13:09] <jtv> And can you confirm that we have no plans to migrate existing bootresources.yaml settings?  It wasn't meant to be very end-user-ish anyway, so I'm not sorry.
[13:10] <jtv> The initialisation code I have up for review gives a user a clean slate, even after upgrading.
[13:21]  * jtv steps out for real
[14:03]  * gmb -> switching locations
[14:57] <jtv> allenap, gmb, rvba: I get the impression that "./bin/test.pserv provisioningserver.import_images.tests.test_boot_resources" actually downloads images...
[14:57] <allenap> Belt, braces, and belly.
[14:57] <jtv> It goes into the ppc64el boot method, whose install_bootloader starts downloading.
[15:11] <jtv> allenap, gmb, rvba: fix is included in https://code.launchpad.net/~jtv/maas/no-bootresources-rewrite/+merge/220046 — who will review it for me?
[15:13] <allenap> jtv: +1d
[15:14] <jtv> Thanks!
[15:39] <jtv> gmb: does the plan for ditching bootresources.yaml involve migrating existing config?
[15:54] <gmb> jtv: It doesn’t ATM. It definitely *could*, whether it’s an absolute necessity I’m not sure.
[15:54] <gmb> (I wonder how many people will have touched their bootresources.yaml)
[15:55] <jtv> I see some complications, e.g. "should we also try to migrate from pre-existing images" and "must we go through an API exchange right before importing, just for this upgrade?"
[16:02] <rvba> gmb: the API work for boot source is up for review.
[16:02] <gmb> rvba: I saw. Thanks :). That’s going to be a nice task to take me to EoD :)
[16:02] <rvba> gmb: good luck :)
[16:02] <rvba> gmb: I'm sure you can leave some of it for Julian.
[16:03] <gmb> Now there’s a thought…
[16:06] <jtv> gmb: NodeGroup.bootsource_set.  :)
[19:56] <AskUbuntu> not getting impi after installing 14.4 maas, and commisioning servers | http://askubuntu.com/q/469306