[10:22] realtime-neil: linux-image-generic is invalid selections. [10:22] realtime-neil: you get the kernel that's available on the iso..... the isos are built with hwe kernels. [10:23] realtime-neil: if you want 4.15 kernel; you do need to rebuild your iso to be with the GA kernel available. And/or have it in the pool. [10:23] realtime-neil: if we can't find a kernel asked for, we install a kernel we have. [10:24] realtime-neil: 5.4 is the same kernel as in focal btw. and will be supported for the rest of the bionic lifetime. [12:38] xnox: understood. Could you explain how one "asks for" a different kernel? I was under the impression that this was impossible with ubiquity. [12:57] xnox: Yesterday, cjwatson explained that, with the introduction of ubiquity, the generally accepted means to install a desired kernel image is to include it in the filesystem.squashfs. I have been able to demonstrate that this works --- but the problem is that the hwe kernel (the one with which the isos are built) is installed _also_. I could tolerate the presence of two kernels if I could force grub to default to the one I put in the fil [12:57] esystem.squashfs. [13:28] realtime-neil: in focal, we added support to correctly select hwe or oem kernel; and uninstall hwe kernel, when a different one is installed. [13:28] realtime-neil: that was not backported to bionic ubiquity i don't think. [13:30] xnox: How was this implemented? [13:35] realtime-neil: i think it's these 2020 commits https://git.launchpad.net/ubiquity/log/scripts/check-kernels [13:37] xnox: I saw that script in the Bionic ubiquity source package --- I figured I could just overwrite it with logic of my own choosing. The problem I'm having is that I don't know where it "lands" inside the iso. [13:38] it's weird. it's in the ubiquity.deb itself in the filesystem.squashfs [13:38] i think [13:39] realtime-neil: but there are matching things in python code too! [13:40] there is this https://git.launchpad.net/ubiquity/tree/ubiquity/components/check_kernels.py [13:40] but also [13:40] xnox: do I understand correctly that you're suggesting I override the _python_ script that calls the shell script `check-kernels`? [13:40] https://git.launchpad.net/ubiquity/log/scripts/plugininstall.py [13:40] which also has matching 2020 changes for the kernels replacement (install new one, remove old one) [13:41] and be careful there are multiple plugininstall.py files [13:42] realtime-neil: not sure. but when i looked at it in april. it was not possible to correctly install a different kernel flavour from the iso pool and/or the internet, instead of the one that the image itself is built with (i.e. the hwe one by default) [13:42] realtime-neil: and i had to fix all of those pieces to make it work correctly. Especially with/without nvidia; with/without secureboot. [13:46] xnox: Ick. Might I be able to stub it out altogether? Effect some kind of `d-i base-installer/kernel/image string none`? I already have the kernel I want in the filesystem.squashfs. [14:18] realtime-neil: trace things with print() and/or sh -x, and yeah find things you don't want, and skip them. [14:18] realtime-neil: but be careful, because you want linux-generic installed (not just linux-image-generic) [14:19] realtime-neil: and at the end of things, on the installed system it must be marked as installed manually (with like apt-mark manual) because otherwise ubiquity might try to remove it at the end. [14:19] or like user could run $ apt autoremove and get it auto removed too, if it's isntalled auto. [14:20] realtime-neil: why are you using 4.15 kernel, which has less support, for new laptops/desktops/VM/hypervisor when compared to the v5.4 kernel? [14:20] xnox: understood ... Can you give me some guidance as to where the "action" starts? I have a unpacked iso in front of me, but I don't know what (or where) the "entrypoint" to the ubiquity things is. [14:20] xnox: I'm using the 4.15 kernel because $DAYJOB says $VENDOR only supports LTS kernel [14:21] xnox: point taken regarding linux-generic vs. linux-image-generic [14:21] realtime-neil: v5.4 is the LTS kernel! [14:22] I mean for Bionic, not Forcal [14:22] both everyone, canonical, ubuntu, and kernel.org [14:22] I mean for Bionic, not *focal* [14:22] v5.4, is LTS on both *focal* _and_ *bionic* =) [14:22] we always offer next lts kernel, on lts-1. [14:22] with the same lifecycle support. [14:23] so $VENDOR is telling lies =) [14:23] okay, maybe I'm mixing terminology --- what's the "common ubuntu" name for the thing pointed to by the package `linux-image-generic`? [14:24] usually we refer to it as GA kernel, meaning it's the same major series at release time, as it will be for the lifetime of the series. [14:24] which minority of our installs use [14:24] "General Availability"? [14:25] yeah [14:25] Okay. And it's unpopular because why? [14:25] ubuntu desktop defaults to hwe kernel; oem certified laptops use rolling linux-oem kernel; Public Clouds, use cloud speicific rolling kernel i.e. linux-aws [14:26] it gets old, doesn't support newer hardware, and is slower on newer hardware => because newer kernel series squeeze more performance out of existing hw because they are coded to do better. [14:26] thus 5.4 vs 4.15 on the same hardware is faster, uses less power, supports more hw. [14:27] thus you installing 4.15 on ubuntu desktop, deviates from what Ubuntu Desktop installs do, by default, in 2020. of the bionic release. [14:27] because it's not April 2018 anymore. [14:28] also for example as virtualbox / vmware / public-cloud /hyperv host platforms improve they typically need newer kernels to work properly. [14:28] point taken --- your advice is to just take the hwe kernel that the Bionic Desktop installer wants to give me? [14:29] I also note that this looks promising: [14:29] ```$ apt-cache policy linux-oem [14:29] linux-oem: [14:29] Installed: (none) [14:29] Candidate: 4.15.0.1097.101 [14:29] Version table: [14:29] 4.15.0.1097.101 500 [14:29] 500 http://us.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [14:29] 500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [14:30] 500 http://archive.ubuntu.com/ubuntu bionic-security/main amd64 Packages [14:30] 500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [14:30] 4.15.0.1004.2 500 [14:30] 500 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 Packages [14:30] 500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [14:30] realtime-neil: do not use linux-oem => that's kernel tuned to work on certified laptops, with specific quirks for particular laptop models. [14:31] xnox: understood ... I figure it's not by accident that it's based on the GA kernel. [14:31] realtime-neil: it's not for general consumption, and may be better or worse. we have explicit metadata to only install it when needed; and roll-off to the hwe-kernel as soon as all the fixes are srued. [14:31] there is linux-oem-5.6 => based on 5.6 which matches none of the GA/hwe kernels from any series. [14:31] etc. [14:32] there is linux-oem-osp1 too [14:32] there are many different oem kernels for specific SKUs. [14:32] only one of them is v4.15 [14:32] is there any documentation or catalog with these listed? [14:33] no, because that's private with the hw vendors. as we cannot referenced unreleased SKU names until they are public and for sale, yet we enable them before hand. [14:33] got it [14:33] ubiquity in focal; automatically picks and installs the right thing. [14:33] which for ubuntu desktop, the default is the linux-hwe-MM.YY metapackage. eventually all oem kernels roll onto that. [14:34] (but on per sku basis, because raisins) [14:34] and we've established that this behavior --- at least in bionic --- is not easy to override [14:34] back to my question about the ubiquity "entrypoint" in the installer iso .... where does it start in earnest? [14:36] I've looked at the init scripts within the casper/initrd, but I got lost trying to follow the invocations. [14:37] ubiquity.service no? [14:38] but you can also boot to live session, and launch the ubiquity.desktop file from the desktop if one is interactive [14:38] so the answer is, it has many =) [14:48] xnox: Oh, boy! This whole "multiple entrypoints", "infinite wrappers" seems to be a recurring theme in this software game. :D [15:04] only slightly better than d-i with 110 source packages. [15:04] i guess it's best to clone the git repo and/or bzr branch [15:04] and run update, because then you know this is the installer bits all on this directly [15:05] because image has a lot of ubuntu-desktop stuff which is not quite always related [15:05] (casper is though) [15:32] xnox: I want to revisit this "run update" of which you speak, but what I really want to know is how the ubiquity artifacts map to iso contents. Is that detailed in the source I just cloned out (https://git.launchpad.net/ubuntu/+source/ubiquity) ? [15:35] realtime-neil: for bionic, focal, or groovy? did you get the right branch & a matching repo? [15:35] it's years of development between them. most things don't stay the same in terms of repos/layout/etc. [15:36] I've checked out ubuntu/bionic [15:38] ``` [15:38] $ git remote --verbose [15:38] origin https://git.launchpad.net/ubuntu/+source/ubiquity (fetch) [15:38] origin https://git.launchpad.net/ubuntu/+source/ubiquity (push) [15:38] $ git status --short --branch [15:38] ## ubuntu/bionic...origin/ubuntu/bionic [15:38] $ git grep check_kernels.py [15:38] debian/ubiquity.install-any:ubiquity/components/check_kernels.py usr/lib/ubiquity/ubiquity/components [15:40] xnox: And yet: [15:40] $ find ubuntu-18.04.5-desktop-amd64.d/ -name '*ubiquity*' [15:40] ubuntu-18.04.5-desktop-amd64.d/pool/main/u/ubiquity [15:40] ubuntu-18.04.5-desktop-amd64.d/pool/main/u/ubiquity-slideshow-ubuntu [15:40] xnox: I'm clearly missing something obvious, but I have no idea what that is. [15:41] realtime-neil: i don't know waht ubuntu-18.04.5-desktop-amd64.d is [15:42] it's the unpacked contents of http://cdimage.ubuntu.com/releases/bionic/release/ubuntu-18.04.5-server-amd64.iso [15:42] realtime-neil: ubiquity is a source package. it builds many .debs. Some of them are optional. Some of them are installed inside filesystem.squashfs [15:42] ubiquity installer is not used on server.iso at all [15:42] it only is used on the desktop iso [15:42] whoops, my bad, I meant the desktop iso [15:43] (some of them are in the pool, and some of them are not anywhere at all! or like are flavour specific, i.e. the kde/qt frontend is shipped on kubuntu iso) [15:43] inside that bionic ubiquity tree you should be able to do [15:43] ./debian/rules update [15:43] xnox: you're saying the ubiquity things are already in the filesystem.squashfs ? That's where I should be making changes (if any)? [15:43] to download vendored pieces [15:43] realtime-neil: that's where the installer runs from, yes. it's preinstalled in the filesystem.squashfs as it must start and run on boot, and it does. [15:44] cause that's how it works. [15:44] if you are patching ubiquity you might want to patch source, and build new debs, chroot into filesystem.squashfs and install updated debs there. [15:44] or just patch/hack files inline up to you. [15:44] i'm busy. [15:45] xnox: let me see if I understand --- ubiquity runs from the flilesystem.squashfs, but the filesystem.squashfs is also installed to the disk partition that the ubiquity installer prompts the user to choose? [15:45] sorry [15:45] I'll bother you some other day [15:46] realtime-neil: yes! [15:46] initrd on the iso, mounts the iso, then mounts filesystem.squashfs as /cdrom then also mounts it with overlayfs tmpfs. [15:46] I have much to study; thank you very much [15:46] during install it copies /rofs contents to target disk; and cleans up things, like self-removing ubiquity [15:47] because you don't want installed system, to try to boot into isntaller again (doh) [15:47] hence it's a very much 2-in-1 thing, both the installer and what gets installed into target [15:47] that's either brilliant or crazy [15:47] hence you can find in ubiquity sourcecode a bit of a harakiri, where it supposedly removes itself, but it does it in the chroot /target => (aka in-target) instead of the running system. [15:48] think of it as a virus, that infects all the things with itself =) [15:48] I wrote a self-destructing systemd unit like that once; I think I felt similarly about it :D [15:48] and it tries to be smart about things, i.e. it tries not to copy things over which will be deleted later anyway. [22:56] Why is it adding `linux-image-generic-hwe-18.04` to the `casper/filesystem.manifest-remove` in the Ubuntu Bionic Desktop installer isoaccomplishes nothing; i.e., the installed system still has the unwanted `linux-image-generic-hwe-18.04`?