[01:12] <lotuspsychje> good morning
[01:22] <daftykins> \o
[01:22] <lotuspsychje> hey daftykins
[10:08] <lotuspsychje> http://ubuntu-news.org/2020/10/09/ubuntu-community-council-election-2020-underway/
[10:08] <lotuspsychje> good luck to all!
[13:10] <tomreyn> hehe, i didn't know there's this https://launchpad.net/~not-canonical
[13:18] <jeremy31> I know of a few team members
[13:20] <oerheks> i only met Jan Claeys i think
[13:21] <oerheks> oh, and samba-jelmer
[13:26] <jeremy31> Didn't the group owner do UWN for a while?
[13:27] <tomreyn> didn't he invent the phone?
[18:02] <daftykins> TJ-: just the man! can i pick your brains about something? :)
[18:02] <TJ-> Sure
[18:04] <daftykins> TJ-: i've acquired some old servers for cheap and intend to put one in at my friend's house in England running XCP-ng and some ubuntu 20.04 VMs atop it for running his medical data analysis thing, i was going to follow your 'almost' FDE guide for setting up the VMs but subiquity doesn't seem to lend itself to working with that right now?
[18:05] <daftykins> i know the guide is really for use with the desktop image, but i'd like to stick to server
[18:06] <TJ-> daftykins: I've never touched subiquity at all; my opinion about is isn't printable
[18:06] <daftykins> nor mine :D
[18:07] <TJ-> daftykins: but the principle is the same ; use commands to pre-prepare the storage system in the cofiguration you want THEN run the installer so it just has to select the LVM/partition for each file-system, rather than create them.
[18:07] <daftykins> TJ-: the server live image lets you get quite far into your guide, it was only at the install partitioner the wheels came off - i selected the vg root to install to and the /boot partition, but it still asks you to pick a disk to boot from - and none of the entries let you do that upon them
[18:07] <daftykins> in my case it was /dev/xvda due to being a Xen VM
[18:09] <TJ-> daftykins: In your scenario I'd simply do a manual install - get the storage volumes as I want then I'd use debootstrap to install the minimal system them chroot into the /target/ to complete the install, starting with "apt install ubuntu-server" and then adding user accounts, openssh-server, and setting the default locale, tz, etc. Then reboot it to ensure it works and use SSH from there
[18:09] <TJ-> on
[18:09] <daftykins> hmm that sounds good, time for some learning then :D
[18:10] <daftykins> i wonder if the live server image is still capable
[18:12] <TJ-> If it has the 'try ubuntu' option to run a regular (not castrated) shell then it may be possible
[18:12] <TJ-> as in not just a 'drop to root shell' like the debian-installer has that only has busybox and a minimal user-space and no option to install packages into the in-memory live system
[18:13] <daftykins> *nod* pretty much, you let it start up into subiquity then drop out to a TTY which has the full shell
[18:13] <daftykins> i still don't know what i'm doing with LVM ;) always avoided it in the past
[18:13] <daftykins> this seems like a nice way to get stronger encryption whilst still keeping the virtual disk images (VDIs) thin provisioned so i can keep VM backup speedy
[18:14] <daftykins> alternatively i could wimp out and use server's built-in encryption, but it's considered as old as the Ark now isn't it?
[18:14] <TJ-> sounds like a good plan to me
[18:17] <daftykins> thanks :)
[18:19] <daftykins> not a bad machine i got for £150, R820 with 4 x Xeon E5-4620s, 128GB RAM, iDRAC 7 enterprise, over a dozen 15K SAS 300GB disks
[18:23] <TJ-> Amazing what you can pick up now that power consumption is the prime concern
[18:24] <daftykins> i made an excellent contact at a big bank :D
[18:24] <daftykins> i may even end up taking 2 of those CPUs out, because i think dividing the RAM between them is suboptimal for my use-case
[18:24] <daftykins> trouble is i'll need to buy the plastic baffle to redirect the airflow then
[18:26] <TJ-> which'll cost £250
[18:26] <daftykins> :D
[22:15] <daftykins> TJ-: ok i'm doing a trial run, i've created the disk as per the guide, but then i've formatted /dev/mapper/ubuntu--vg-root as ext4 and mounted it to /mnt, then created /boot upon it and mounted /dev/mapper/LUKS_BOOT to /mnt/boot - does that make sense or am i losing my way?
[22:15] <daftykins> my VM is BIOS boot only so i'm not sure if i need to do anything with the EFI-SP there before i try to debootstrap install to /mnt
[22:18] <TJ-> daftykins: that looks fine. I usually make the mountpoint /target/ since that is what debian-installer and thus ubiquity etc usually use
[22:18] <TJ-> daftykins: no EFI-SP required then - I do that for maximum flexibility - doesn't cost much to reserve the space for one (256MB is all that is needed)
[22:19] <daftykins> very true
[22:19] <TJ-> I do use GPT and create a bios boot partition of 2MB for GRUB's core image
[22:19] <daftykins> that's a point, where should i mount that?
[22:20] <TJ-> it isn't a mount point
[22:20] <daftykins> ah ok
[22:20] <TJ-> it's simply a raw partition that GRUB writes its boot-loader code into
[22:20] <daftykins> does it autodetect it's there and do that based on the label?
[22:20] <TJ-> the boot sector (MBR) boot-strap code reads and executes that
[22:21] <TJ-> it needs the bios boot code which when using gdisk et al is EF02
[22:21] <TJ-> that's the friendly version of the full GPT GUID for BIOS boot partition type
[22:22] <daftykins> oh yep, i recall that step
[22:23]  * daftykins runs debootstrap
[22:30] <daftykins> is "#14 Latin1 and Latin5 - western Europe" the sensible console font choice?
[22:30] <daftykins> seems like the best fit xD
[22:31] <jeremy31> Is the bios boot partition used for something other than a legacy/bios boot with GPT?
[22:31] <TJ-> so long since I did that, I cannot recall
[22:32] <TJ-> jeremy31: it's just a marker, depends on the software.
[22:32] <TJ-> jeremy31: you could use it as a file-system or anything else as long as that use doesn't conflict with something else
[22:33] <TJ-> there are a huge number of reserved GUIDs in GPT to avoid those kind of conflicts though
[22:34] <daftykins> ok got all that config done from installing "ubuntu-server", added a user and added it to the 'sudo' group, not sure if i need to do some funky grub-install fu?
[22:35] <daftykins> also seem to be lacking netplan so presumably i'll have no network at boot
[22:36] <TJ-> daftykins: netplan doesn't do networking :D
[22:36] <daftykins> hehe you know what i mean!
[22:36] <TJ-> it tells other network management tools to do it
[22:36] <TJ-> it's a server; just configure systemd-networkd via /etc/systemd/network/ with a .network file
[22:37] <TJ-> netplan only generates run-time configs for systemd-networkd, not permanent configs (puts them under /run/systemd/network/ which is a tmpfs)
[22:37] <daftykins> hmm will have to look that one up, how about GRUB / otherwise ensuring boot will work now?
[22:38] <TJ-> chroot /target grub-install /dev/sda; chroot /target update-grub
[22:38] <TJ-> ensure you've installed kernels first!
[22:38] <TJ-> with debootstrap it is easy to forget to do that
[22:38] <daftykins> ooh yeah nothing there
[22:38] <TJ-> chroot /target apt install linux-generic
[22:39] <daftykins> yep was just running "apt install linux-generic"
[22:39] <TJ-> also, you may want to generate the locales if you haven't already
[22:39] <daftykins> dpkg-recnfigure locales ?
[22:39] <daftykins> without the typo
[22:39] <TJ-> chroot /target apt install language-pack-en
[22:40] <TJ-> and chroot /target dpkg-reconfigure locales
[22:40] <daftykins> oh hello, adding linux-generic has triggered grub-pc
[22:41] <TJ-> set /etc/hostname and /etc/hosts correctly
[22:41] <TJ-> I hope you've got the /dev/ file-system etc, mounted inside the /target/
[22:41] <daftykins> yep /proc /sys /dev
[22:41] <daftykins> grub-install: error: attempt to install to encrypted disk without cryptodisk enabled. Set `GRUB_ENABLE_CRYPTODISK=y' in file `/etc/default/grub'.
[22:41] <daftykins> fun one :D
[22:41] <TJ-> ^^^^ vey much needed
[22:42] <TJ-> I put that in /etc/default/grub.d/local.cfg
[22:42] <TJ-> that way if grub-pc package gets updated you don't suffer a 'diff' prompt for /etc/default/grub
[22:43] <daftykins> ah yes
[22:43] <daftykins> hopped to a second term and sorted that, installed smoothly to /dev/xvda now
[22:44] <daftykins> cryptsetup: WARNING: target 'xvda5_crypt' not found in /etc/crypttab
[22:44] <TJ-> yup, you need an entry there
[22:44] <TJ-> e.g. I have:
[22:45] <TJ-> LUKS_VG UUID=3cd375aa-2b43-4760-9881-7330f7646ca4 /etc/luks/iSSD.keyfile luks,discard
[22:45] <TJ-> LUKS_BOOT UUID=db9cd343-9ca9-4115-845f-d27943c14437 /etc/luks/iSSD.keyfile luks,discard
[22:45] <TJ-> and of course create a small random data file as the key
[22:46] <TJ-> and use 'cryptsetup' to add it to a slot of the LUKS volumes
[22:46] <daftykins> hmm for locale do i want en_GB.UTF-8
[22:46] <TJ-> AND make sure, if you are encrypting /boot/ that it was created using LUKS version 1 NOT version 2 else it'll fail because GRUB doesn't understand version 2
[22:47] <TJ-> daftykins: yes unless you want to talk gibberish!
[22:47] <daftykins> ;D
[22:47] <TJ-> en_US == gibberish :D
[22:47] <daftykins> xD
[22:48] <daftykins> other choice was some ISO-8859-1
[22:49] <TJ-> don't get smilies with that; that's the old default MS Windows
[22:50] <daftykins> the horror!
[22:50] <daftykins> hmm keyfile eh, what's an easy way to knock one of those up?
[22:50] <daftykins> bit of /dev/urandom to a file of x length?
[22:51] <TJ-> chroot /target dd if=/dev/random of=/etc/luks/file.key bs=64 count=1
[22:52] <TJ-> make sure to make the /etc/cryptsetup-initramfs/conf-hook with KEYFILE_PATTERN="/etc/luks/*.key" in that case
[22:53] <TJ-> and set the /etc/initrams-tools/initramfs.conf with # for cryptsetup-initramfs
[22:53] <TJ-> UMASK=0077
[22:53] <TJ-> I think that lot was in the tutorial
[22:53] <TJ-> but things will break if those are forgotten
[22:54] <daftykins> oh that's a very good point, debootstrap may have bypassed the install but i forgot there was more to go back and look at
[22:55] <TJ-> ✓✓✓
[22:55] <TJ-> can't do that in ISO-8859 !
[22:56] <daftykins> hopping back to your crypttab example, does the "LUKS_VG" part matter that i don't appear to have one called that from my "blkid" output?
[22:56] <daftykins> i'm feeding it the UUID of my root file system but i feel like that's a mistaken guess
[22:57] <daftykins> because it's the volume group i want to unlock there and not the partition inside it eh?
[22:57] <TJ-> LUKS_VG is the LUKS container that has the LVM inside that has the root-fs etc
[22:58] <TJ-> so you might have /dev/sda2 = LUKS > LUKS_VG = LVM > root-fs = ext4
[22:59] <daftykins> xvda5 is crypto_LUKS
[23:00] <TJ-> created via 'cryptsetup luksFormat --type luks1 /dev/sda2 ...; cryptsetup open /dev/sda2 LUKS_VG; pvcreate /dev/mapper/LUKS_VG; vgcreate MYVG /dev/mapper/LUKS_VG; lvcreate -L 8G -n rootfs MYVG ...
[23:01] <TJ-> errr, typo, --type luks2  !
[23:01] <TJ-> --type luks1 is for the encrypted /boot/
[23:02] <daftykins> mm, xvda2 in my case was the 2MB GRUB, so i think 5 is right
[23:03] <TJ-> if '5' is where the OS is installed to, yes
[23:06] <daftykins> hmm your guide had a bs=4096 for the keyfile generation
[23:07] <TJ-> daftykins: yes, a big mistake that!
[23:07] <daftykins> too strong? :)
[23:07] <TJ-> when I write it I got confused as to how many bits were in a byte and used the number of bits instead of dividing by 8!
[23:08] <TJ-> waste of space basically, beyond a certain point. LUKS truncates reading so the rest is not used
[23:08] <daftykins> ah ha
[23:10] <daftykins> hmm and i need that to read *.key instead / or rename my .key to .keyfile
[23:10] <TJ-> yes
[23:10] <TJ-> I didn't realise my example there wsa different to what I did here
[23:11] <daftykins> i also bypassed an 'error' earlier on where you used p1 or p'n' due to working with a PCIe SSD whereas i am using a conventional virtual disk so i needed to drop the p
[23:12] <daftykins> you might say i was taking the p...
[23:14] <daftykins> WARNING: Locking directory /run/cryptsetup is missing!
[23:14] <daftykins> oh that was in your example, nm
[23:14] <daftykins> going cross-eyed now ;)
[23:21] <TJ-> glad you were keeping your eyes open on that one; I had to amend the orinal article to cope with NVME names and that was a challenge. Don't ask me to do it for Xen too!
[23:21] <daftykins> haha
[23:21] <TJ-> yeah - you didn't bind-mount /run/ - always a good thing in my opinion else services in the chroot start and then get confused and won't be stopped
[23:22] <daftykins> hrmm update-initramfs spitting another: "cryptsetup: WARNING: target 'xvda5_crypt' not found in /etc/crypttab" must have chosen the wrong one
[23:24] <daftykins> or maybe i need to name it that instead of LUKS_VG
[23:24] <TJ-> did you do it with UUID?
[23:24] <TJ-> well yes, you need to give the correct reference :D
[23:24] <daftykins> yeah i popped in the UUID but i wasn't feeling 100% on my choice of which drive that is
[23:24] <daftykins> hey i'm just bashing the keyboard with patterns 'til i get the right answer ;)
[23:25] <TJ-> Monkeys ✓
[23:25] <daftykins> can confirm
[23:26] <daftykins> https://termbin.com/ayoo - so i was picking the UUID of xvda1 earlier, hmm that's definitely wrong
[23:26] <daftykins> no i wasn't! xvda5
[23:27] <TJ-> so UUID="2c17c32e-8d32-4947-9347-e49d4d39c342"
[23:28] <daftykins> yep - and i deleted "LUKS_VG" and put "xvda5_crypt" instead
[23:28] <daftykins> completed without error this time \o/
[23:28] <daftykins> ok so i still haven't got networking but i feel brave enough to consider a boot
[23:29] <daftykins> lol i made my passphrase 30 characters of gibberish, so i'm going to have to type that in via the command line... nice
[23:29] <daftykins> via the XCP-ng VM console window
[23:29] <TJ-> make sure to finish off with the grub-install/update-grub just in case something changed... ensure update-grub DOES generate a line.. it won't if /boot/initrd.img-$VERSION is missing for example, which can happen if update-initramfs failed
[23:30] <TJ-> for testing add a simple passphrase to another slot before you restart it
[23:30] <daftykins> that executed cleanly
[23:30] <TJ-> "letmein"
[23:31] <TJ-> as in cryptsetup luksAddKey /dev/xvda2 or whatever your /boot/ FS is
[23:31] <TJ-> ahh, xvda1 I think
[23:32] <daftykins> both 1 and 5 surely
[23:32] <daftykins> oh boot only
[23:33] <daftykins> ok let's unmount all and test boot :O
[23:34] <daftykins> hrmm how do you deal with sys and dev claiming to be busy o0
[23:35] <TJ-> first ensure you've not got a terminal with its PWD inside the chroot
[23:36] <TJ-> then try "umount --lazy ..."
[23:36] <daftykins> ah ha \o/
[23:38] <daftykins> Boot Device: Hard Disk - success.
[23:38] <daftykins> Attempting to decrypt master key...
[23:38] <daftykins> Enter passphrase blah blah :)
[23:38] <daftykins> ooh it's not receiving keyboard input
[23:38] <TJ-> is that the GRUB prompt? because the LVM should be auto-unlocked via keyfile
[23:39] <daftykins> yeah says "Enter passphrase for hd0,gpt1 (<string here>):
[23:39] <TJ-> what kind of keyboard connection does the VM have? if it is via USB HID rather than a PS/2 serial GRUB will need additional modules including in its core image
[23:40] <daftykins> what'd be the easy way to pull that from an existing VM i have?
[23:40] <TJ-> ideally it should be an i8042 PS/2 serial keyboard interface
[23:40] <TJ-> not sure how Xen handles such things !
[23:41] <daftykins> so close :D
[23:42] <daftykins> oh hello, maybe it take take input - or it times out? it just dropped to GRUB rescue
[23:42] <TJ-> right, it doesn't echo anything to screen
[23:42] <TJ-> you just type the passphrase and hit enter
[23:42] <daftykins> no carat was present either though so i wasn't even sure what was going on
[23:43] <TJ-> if you're at the rescue prompt then do...
[23:43] <TJ-> ... cryptomount hd0,gpt1
[23:43] <daftykins> i think its' error might be relevant, it says there's no such cryptodisk found
[23:43] <TJ-> then enter passphrase... once you get it correct you'll see "Slot X open" then do "insmod normal" followed by "normal" and it'll load the grub.cfg and show the menu
[23:44] <TJ-> I suspect you did NOT create the /boot/ LUKS container using cryptsetup luksFormat --type luks1 /dev/xvda1
[23:45] <TJ-> so it is likely LUKS2 (which is the default in cryptsetup nowadays)
[23:45] <daftykins> ooh "slot 2 opened"
[23:45] <TJ-> ha no, you're good!
[23:45] <daftykins> i think maybe the quirk with the virtual console just took my passphrase incorrectly the first time
[23:45] <daftykins> on that grub rescue attempt, i got a proper carat cursor etc
[23:45] <TJ-> OK so the insmod normal + normal and you're good to go
[23:47] <daftykins> booting... black screen so far
[23:47] <daftykins> Ubuntu 20.04 plymouth-text type interface :O
[23:48] <TJ-> looking good
[23:49] <TJ-> tap Esc to see the kernel console messages
[23:49] <daftykins> have i mentioned it's on a 6 x 15K SAS RAID10? :) 6Gb
[23:49] <daftykins> chug chug
[23:50]  * TJ- winds the handle
[23:51] <daftykins> still got the purple background with a blinking cursor top left
[23:52] <daftykins> hmm the hypervisor suggests there's no disk activity
[23:53] <Deyaa> Hello
[23:54] <daftykins> o/
[23:54] <Deyaa> Does Facebook block reverse proxy?
[23:55] <TJ-> hmmm, if you saw the plymouth splash then the initrd was working
[23:55] <TJ-> try tapping a key... might be its a just-in-time login prompt as systemd often does
[23:55] <daftykins> omg
[23:55] <daftykins> i hit alt+F1 and see the prompt xD
[23:56] <TJ-> haha!
[23:56] <daftykins> got a bunch of read only file system errors, hrmm i never did write an fstab o0
[23:57] <TJ-> so maybe it started on the wrong VT? does /proc/cmdline have a vt.handoff= set?
[23:57] <TJ-> uhoh
[23:57] <daftykins> yep vt7
[23:57] <daftykins> hah i'm missing sudo also
[23:57] <daftykins> but i'm in!
[23:58] <daftykins> i can probably fix those up with another live session boot and chroot
[23:59] <TJ-> not even that; get into the GRUB menu, advanced, and choose a recovery option and you'll be in as root
[23:59] <daftykins> ah that's true
[23:59] <TJ-> tap Esc as soon as Slot 2 open is seen and GRUB should stop at the menu