=== himcesjf_ is now known as him-cesjf
[06:14] <arighi> morning
[14:14] <TJ-> just a heads-up the recent Intel updates may be preventing the kernel from starting at boot: Bug #1829620
[14:14] <ubot5> bug 1829620 in linux-hwe-edge (Ubuntu) "cryptsetup stuck at loading initramfs" [Undecided,Confirmed] https://launchpad.net/bugs/1829620
=== cshep is now known as platonical
[14:15] <alkisg> Thankfully cryptsetup isn't preinstalled
[14:16] <TJ-> the report has nothing to do with cryptsetup, I think, just the user reports that since they're expecting to be prompted for LUKS pass-phrase and aren't
[14:38] <alkisg> AFAIK cryptsetup only gets installed if the users selects "encrypt disk" at ubiquity
[14:38] <alkisg> Which isn't the default
[14:38] <alkisg> So if it only affects encrypted setups, we're fine here :P
[14:43] <TJ-> Nope, this affects unencrypted systems too
[14:44] <alkisg> Ouch. I'll keep an eye open.
[14:50] <apw> TJ-, that people seem to be unbroken by removing and reinstalling cryptsetup makes me wonder if it could be an initramfs contents issue
[14:51] <TJ-> apw: yes, that was my other thought... my next Q in that bug will be "has /boot/ run out of space"
[14:51] <apw> if it was disco i also had a bug where grub didn't initialise the terminal at all; which was nasty
[14:51] <TJ-> apw: my reason for thinking Intel microcode is it is added to the early_initramfs isn't it? therefore it could be the cause
[14:52] <apw> TJ-, entirely possible too yes
[14:53] <TJ-> We're watching #ubuntu for similar reports, we've had 1 so far
[14:53] <apw> TJ-, the 'how you get it working again' is going to be key to working out what
[14:55] <apw> TJ-, or perhaps there is a bug in rebuilding initramfs's when microcode changes; which would account for 'rebuilding my initramfs for another package' fixing it
[14:55] <TJ-> apw: indeed :) you noticed the additional reporter and colleague in that bug says they aren't using LUKs either, so I guess they got to the bug due to the last messages on-screen being from GRUB
[14:55] <apw> TJ-, or an ordering bug when getting a new kernel and a package which needs to rebuild initramfs'
[14:55] <apw> TJ-, which is entirely possible
[14:56] <TJ-> apw: what is weird is every ubuntu kernel broke
[14:56] <apw> they all share microcode, they mostly all share 'update ordering issues'
[14:56] <TJ-> a new kernel would only create the initrd.img for itself, whereas I *guess* a microcode update might apply itself to all initrd.imgs ?
[14:57] <apw> TJ-, a microcode update likely triggers all initramfs' for all installed kernels to be rebuilt
[14:57] <apw> to get that microcode included 'in' the initramfs image
[14:58] <TJ-> indeed, which would explain all Ubuntu kernels having the same problem
[16:57] <TJ-> apw: confirmed it is the ucode updates 
[16:58] <apw> TJ-, ugg
[16:59] <TJ-> disabling loading of the ucode at boot-time solved it, and it appears to be model specific, not packaging
[16:59] <apw> sigh
[17:01] <tyhicks> TJ-, apw: unless something has recently changed, an update to intel-microcode only gets injected into the newest initramfs (as well as any future initramfs's)
[17:05] <tyhicks> sbeattie: ^ note bug #1829620 (you'll likely see the bug mail now that you've been assigned to the bug)
[17:05] <ubot5> bug 1829620 in linux-hwe-edge (Ubuntu) "intel-microcode on ASUS makes kernel stuck during loading initramfs on bionic-updates, bionic-security" [Undecided,Confirmed] https://launchpad.net/bugs/1829620
[17:07] <tyhicks> sbeattie: I did test the whole stack of updates, with an encrypted root partition, on an old laptop running Cosmic and Disco (I upgraded the system while we were testing) but didn't see an issue
[17:07] <tyhicks> that machine is very old but did a corresponding microcode update
[17:11] <sbeattie> tyhicks: ugh, okay, thanks.
[17:18] <tyhicks> sbeattie: I don't have a good feeling for what's going on there
[18:12] <sbeattie> tyhicks: me either, I'm confused by the report that things work with the linux-hwe-edge kernel
[18:13] <sbeattie> tyhicks: I *think* if I have mapped things properly, the cpuid is 0x000806ea (so microcode 06-8e-0a)
[18:13] <sbeattie> (but waiting on confirmation on both)
[18:14] <tyhicks> sbeattie: it would be interesting to know if the updated microcode and kernel (the combo that doesn't boot) can boot with mds=no
[18:15] <sbeattie> oh yes, that's a good question, too.
[18:18] <tyhicks> sbeattie: I think that would help us understand if it is a kernel issue (seems unlikely since this isn't a widely reported issue) or something specific to the VERW related changes in the microcode update
[18:20] <tyhicks> sbeattie: I wasn't very clear... if it boots with mds=no, then it could mean that it is a kernel issue OR an issue with the VERW microcode changes
[18:20] <tyhicks> (rather than a general microcode issue...)
=== ben_r_ is now known as ben_r
=== himcesjf_ is now known as him-cesjf