[00:03] <cyphermox> slangasek: so I think it should still work even if it allocates the free space
[00:04] <cyphermox> it certainly does run grub-installer properly on x86 :)
[00:04] <slangasek> who here is familiar with how partman-md is /supposed/ to work?
[00:05] <slangasek> cyphermox: the fundamental issue here is that we are *not* wanting to create a RAID over two partitions, we're wanting to create a RAID over two disks and then create partitions on the RAID
[00:05] <cyphermox> sure
[00:05] <cyphermox> but why is that an issue?
[00:05] <cyphermox> if it seems to work properly...
[00:06] <slangasek> cyphermox: how are you figuring that it works properly?  we're having this conversation precisely because a bug has been reported
[00:06] <cyphermox> slangasek: a bug specifically on ppc64el, which uses very different code paths than x86
[00:07] <cyphermox> and I seem to be able to reduce the difference there too (pending testing)
[00:07] <cyphermox> part of the problem on ppc64el is that we also depend on PReP partitions which are not the typical grub /boot partitions and get detected via prep-bootdev, an "external" program grub-installer also ships
[00:08] <slangasek> cyphermox: the firmware needs to see a PReP partition.  You can accomplish that one of two ways.  1) Combine two disks into a RAID1 array, partition it, make one of those partitions a PReP partition; or 2) create partitions on the underlying disks, combine those individual pairs of partitions into RAID1 arrays, and mark one of those as a PReP partition
[00:08] <cyphermox> so my analysis as to "it seems to work" is based on the fact that on amd64, it seems to work, I don't deny we still need to figure out how well that scales to ppc64el/opal booting
[00:08] <slangasek> cyphermox: from what you've told me, you've tried to do exactly what I also expected to work - that you could select two disks for RAIDing and create partitions on top of them and have it work
[00:09] <slangasek> but that is *wrong* and doesn't work *anywhere* except MBR
[00:09] <cyphermox> slangasek: it's also why it's explicitly marked as not supported in anything but x86 :/
[00:09] <slangasek> that is wrong> the implementation is wrong and does not match the intent
[00:09] <slangasek> cyphermox: it's not even supported on x86 with UEFI, it only works for MBR
[00:10] <slangasek> do we automatically add the PReP partition to fstab?
[00:10] <cyphermox> no
[00:10] <slangasek> ok, so it's a magical thing that's handled via grub directly
[00:10] <slangasek> do you have the bug number for this handy? :)
[00:10] <cyphermox> yeah
[00:12] <cyphermox> so I'm already working on a fix, but there's multiple moving parts and I'm new at the whole thing, so trying to understand things as I change them :)
[00:12] <cyphermox> https://bugs.launchpad.net/ubuntu/+source/debian-installer/+bug/1487365
[00:13] <cyphermox> it clearly currently doesn't work on ppc64el; we've established that
[00:13] <slangasek> cyphermox: how are you intending to solve this?  The install method presented in the bug, that of /dev/md0p1 being used as the PReP partition, will never work with the current state of partman-md
[00:15] <cyphermox> there's two pieces; teach grub-installer to correctly pick the members of the array when installing grub; and fix partman-md to use the underlying disk when we can somehow detect that it's otherwise unpartitioned
[00:15] <slangasek> ok, so you are planning to change partman-md to use whole disk instead of creating a partition
[00:15] <cyphermox> so presumably in the case where either it's just free space, or there is no partition table
[00:15] <slangasek> sorry, I didn't understand that was your aim
[00:15] <cyphermox> it wasn't until we discussed this
[00:15] <slangasek> if you do that, then yes, this all works
[00:15] <slangasek> ah ;)
[00:16] <cyphermox> I don't pretend to know all the intricacies of RAID or booting, so we refine as we go ;)
[00:16] <cyphermox> I'm starting to think I'll have to discuss this with KiBi too before I get too deep in the code
[00:19] <cyphermox> my question for you though would be, should this really be a priority for 15.10, or should we focus on more broken things since there is a workaround and people need to write their own partitioning recipe anyway?
[00:19] <slangasek> cyphermox: basically, it's like this.  software RAID is invisible to the firmware (whichever firmware it is).  The only RAID that's compatible with being booted from is RAID1, because it's a straight mirror, so the firmware just sees it as two disks with matching content.  Any other RAID scheme requires that the boot logic sit *outside* the software RAID array
[00:20] <slangasek> on MBR you dodge this because MBR doesn't boot from a partition
[00:20] <cyphermox> yeah
[00:20] <slangasek> on UEFI or opal, you need a partition to boot from; which means either full-disk software RAID + partitions inside it, or selective RAID of individual partitions
[00:22] <cyphermox> yup, got that
[00:25] <slangasek> cyphermox: so instead of committing to redesign partman-md, I think we should instead look at better enforcement of partitioning policies
[00:25] <slangasek> because you shouldn't have to wait for grub to try to install before you find out there's no prep partition
[00:25] <slangasek> historically there have been architecture-specific partman modules to enforce this
[00:26] <cyphermox> there alreday is a partman-prep IIRC, I could enforce things there
[00:26] <slangasek> IOW I think option 1) is what people generally *want* to have working, but until that works we can get by with directing people to option 2)
[00:29] <cyphermox> enforcing PReP not to be on RAID for now should be simple
[00:30] <slangasek> it doesn't require that it not be on RAID, it just requires that it not be a /dev/mdNpM partition
[00:30] <slangasek> well
[00:31] <slangasek> the use case here is that you want both disks to be equally bootable
[00:31] <slangasek> if grub can handle that without having to be pointed at /dev/md0 == (/dev/sda1+/dev/sdb1), that's fine
[00:32] <slangasek> but having a RAID1 over the individual PReP partitions may be preferable
[00:33] <cyphermox> well, that would get complicated to do without an early_script, but I'll give it a test too
[00:40] <slangasek> cyphermox: so I was mistaken, the partitioning enforcement is apparently usually done by the bootloader installer packages rather than partman modules per se.  e.g. silo-installer (available in Debian only)
[00:41] <slangasek> aboot-installer had one, but good luck finding that in any Debian release...
[00:41] <cyphermox> there is still some level of enforcement in partman- packages, for instance in the size/location for the ESP
[00:41] <slangasek> ok
[00:41] <cyphermox> and partman-prep already checks for location
[00:42] <cyphermox> so it should be fine to check there that PReP isn't over /dev/md0p[0-9]+
[00:42] <cyphermox> that's the easy part though, I still need to get grub-installer right for when it's partitioned right :)
[00:43]  * slangasek nods
[00:43]  * cyphermox uploads a new grub-installer revision to his installer-dev PPA
[00:45] <cyphermox> this is going to be pretty much all for today I think, I'll start an install with it Monday so I get the results on Tuesday morning.
[00:45] <slangasek> alright
[00:46] <slangasek> I'm going to braindump to the bug
[00:46] <cyphermox> cool.
[01:23] <cyphermox> slangasek: you mentioned dvorak too?
[01:24] <slangasek> I did
[01:24] <slangasek> let me double-check it
[01:26] <slangasek> cyphermox: possibly specific to UEFI boot
[01:31] <cyphermox> wat
[01:31] <cyphermox> I don't see how it would change anything
[01:31] <slangasek> cyphermox: well when you're using syslinux you get the option to select the keyboard layout from the bootloader
[01:31] <slangasek> so that's what most people use?
[01:31] <cyphermox> sure
[01:31] <slangasek> if you're using booting under UEFI you don't have that
[01:32] <cyphermox> well, perhaps, and that will change bits in the command-line
[01:32] <cyphermox> but when you detect the keyboard and select dvorak it should still get you the right thing
[01:32] <slangasek> so it's possible the keyboard selector within d-i is broken for everything, but not usually noticed on !UEFI
[01:32] <slangasek> well yes, it should ;)
[01:32] <cyphermox> it's possible, but I thought I had fixed it when you last reported this months ago
[01:32] <cyphermox> it was missing kbd.
[01:33] <cyphermox> I'll try the server image with and without EFI in a bit, when I'm done with laundry
[01:33] <slangasek> was my last report before or after 15.04 release?
[01:34] <cyphermox> after
[01:34] <slangasek> ah
[01:34] <cyphermox> wasn't it this summer?
[01:34] <slangasek> quite possibly
[01:34] <slangasek> but as I said, I was booting a 15.04 image
[01:34] <cyphermox> oh
[01:34] <cyphermox> well, then maybe I should SRU this fix ;)
[01:34] <slangasek> SRUing it won't get it into the already-produced image
[01:34] <cyphermox> no
[01:35] <cyphermox> ugh, and console-setup and kbd need to be on the d-i image itself
[01:35] <cyphermox> try a 15.10 image, so at least I'm reassured it works?
[01:36] <cyphermox> I could sprint with a dvorak keyboard, too :)
[01:37] <slangasek> yeah just finished downloading that
[01:39] <slangasek> cyphermox: bad news; same problem on 15.10 beta
[01:43] <slangasek> Oct 10 01:38:13 main-menu[235]: (process:1141): /etc/console-setup is not writable. No files will be saved there.
[01:43] <slangasek> Oct 10 01:38:13 main-menu[235]: (process:1141): gzip is not accessible. Will not save cached keyboard map.
[02:02] <cyphermox> looks like the same kbd issue :(
[02:04] <cyphermox> so, first, bios boot and I pick dvorak at the keymap panel in gfxboot.
[02:05] <cyphermox> I do get dvorak
[02:05] <cyphermox> now, uefi
[02:06] <cyphermox> so, English (US), then English (US) - Dvorak
[02:06] <cyphermox> still got dvorak?
[02:08] <cyphermox> that was with ubuntu-server from 20151009
[02:08] <cyphermox> what image did you use?
[02:08] <cyphermox> slangasek: ^
[02:09] <cyphermox> or are you using the keyboard-detection?
[02:11] <cyphermox> hum, if you go back and re-do the keyboard selection it looks like maybe it won't update the keymap properly
[02:12] <cyphermox> neh, the keyboard-detection is broken altogether
[02:14] <cyphermox> yup, the detection algo is broken, but the selection from lists appears to work, at least as far as picking between us and us:dvorak and having the right setting applied following that is concerned
[02:16] <cyphermox> I bet it's broken in Debian too
[02:19] <slangasek> cyphermox: ah; I didn't yet test a manual keyboard selection with 15.10, only the keyboard detection method, and that failed
[02:20] <cyphermox> yeah I confirm it's really broken
[02:20] <slangasek> ok, but manual selection works
[02:20] <slangasek> on 15.10
[02:20] <cyphermox> ah, no detection in debian?
[02:20] <slangasek> it didn't work for me in 15.04
[02:20] <slangasek> correct, the detection is an Ubuntuism
[02:20] <cyphermox> slangasek: yeah, the manual selection works
[02:22] <cyphermox> how would you go about picking dvorak on Debian?
[02:22] <cyphermox> doesn't look like I have the choice at all after American English
[02:24] <cyphermox> oh, so that's the 'plugin-detect-keyboard'
[02:24] <cyphermox> shiny.
[02:27] <cyphermox> slangasek: so where does this plugin come from?
[02:34] <cyphermox> so after detection it looks like you got setupcon to run, as it should, but still not setup the keymap properly
[02:38] <cyphermox> ah, oops. looks like we mangle XKBVARIANT along the way
[02:41]  * cyphermox adds that to the list for Tuesday
[03:45] <slangasek> cyphermox: ok; so did you file a bug to track this?
[08:48] <infinity> slangasek: FWIW, OPAL doesn't require a PReP partition, since it doesn't actually use grub.  OpenFirmware does, however, and we install the same way under OF/SLOF/OPAL to preserve sanity (and let disk images migrate).
[08:49] <infinity> slangasek: And, not, it's not added to fstab, because it's a raw partition with bootloader of choice (yaboot or grub, generally) blatted directly to it, is contains no filesystem.
[08:50] <infinity> s/is contains/it contains/
=== psivaa_ is now known as psivaa