| Eickmeyer | bdmurray: Personally, on my laptop, I like to have the lowlatency kernel when I'm doing hardware-intensive work (e.g. audio) and the generic for general purpose/power saving. Add that to the oem kernel, and things get funny. Additionally, a bug in packagekit is marking auto-installed as manual. | 00:03 |
|---|---|---|
| Eickmeyer | Therefore, we end up with overflows of /boot very easily. | 00:03 |
| Eickmeyer | bdmurray: Check the details provided by Mike Mikowski in the bug report I linked above. | 00:05 |
| Eickmeyer | It explains everything. | 00:05 |
| bdmurray | Eickmeyer: neither you nor Mike explained how you installed or what flavor of Ubuntu you installed. | 00:14 |
| Eickmeyer | bdmurray: It's Kubuntu. | 00:15 |
| Eickmeyer | The issue is usually a bug in packagekit which Discover uses as a backend when upgrading packages. | 00:15 |
| Eickmeyer | Hopefully that's been rectified in a recent commit to Packagekit, but it doesn't completely mitigate the problem. Scientists that use our laptops tend to install more than one kernel flavor. | 00:16 |
| Eickmeyer | And Mike doesn't use IRC, so I recommend asking him questions via the bug report. | 00:17 |
| Eickmeyer | And installed via oem installer. | 00:17 |
| rbasak | bdmurray: I just happened to glance at the SRU queues, and python-apt is flagged as missing bug references. Did you miss that? I don't see a relevant bug though. | 00:44 |
| rbasak | Oh, sorry. | 00:45 |
| rbasak | It is an exception. | 00:45 |
| rbasak | https://wiki.ubuntu.com/StableReleaseUpdates#ubuntu-release-upgrader_and_python-apt | 00:45 |
| === genii-core is now known as genii-meeting | ||
| === genii-meeting is now known as genii | ||
| === genii is now known as genii-core | ||
| === not_phunyguy is now known as phunyguy | ||
| === didrocks999 is now known as didrocks | ||
| ahasenack | indeed | 15:33 |
| vorlon | ahasenack: thanks for your posts pointing out classes of python3 transition failures, I think that's very useful | 16:26 |
| ahasenack | thanks! | 16:26 |
| === genii-core is now known as genii | ||
| enr0n | vorlon: I have been looking at an ubuntu-release-upgrader lock screen issue (https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1959458) and have tried using the org.freedesktop.ScreenSaver DBus interface to solve it. Since the upgrade script rus as root, and the DBus interface is on the session bus, there have been several hiccups in this approach. | 18:38 |
| ubottu | Launchpad bug 1959458 in ubuntu-release-upgrader (Ubuntu Jammy) "Lock screen disabling needs updating" [High, Confirmed] | 18:38 |
| enr0n | bdmurray said the two of you have discussed whether screen locking is still a concern during upgrade. We are wondering if you think this issue is still worth pursuing given the DBus troubles. | 18:39 |
| jbicha | enr0n: hi! I think screen locking can be a major issue during upgrade. I believe there had been problems with people being unable to unlock their screen when parts of the login system are updated | 19:30 |
| sarnold | hah yeah, that can happen.. | 19:30 |
| jbicha | I presume that's only with major upgrades. I don't think that it's been a concern for Stable Release Updates | 19:32 |
| jbicha | I don't even remember seeing bugs recently for people who run the development release so maybe things do work better now. But it feels risky updating 2 years at one time from one LTS to the next | 19:34 |
| sarnold | yeah, I think I've only seen that bug filed during release upgrades, but I can't recall if it was lts -> lts or interim -> interim etc | 19:36 |
| bdmurray | Let's take a step back and look at how long inhibiting the screen lock has been broken. Then check to see if there have been complaints during that time frame. | 19:37 |
| sarnold | of course, bug reports is only going to represent a portion of peoplke who have the problem | 19:37 |
| sarnold | certainly *something* around screen locking breaks every release, in at least one of the flavours | 19:37 |
| vorlon | jbicha, sarnold, enr0n: however I don't think this has been an issue with *GNOME* in particular any time in the past decade, because authentication happens in a subprocess, precisely because they saw and solved this issue | 19:43 |
| vorlon | this is why I have my doubts that it's worth extraordinary efforts to inhibit the GNOME screensaver | 19:44 |
| bdmurray | IIRC enr0n thinks the Kubuntu screensaver quirk is broken too | 19:45 |
| vorlon | I can't speak to the importance of fixing that one, not having used KDE in quite a while | 19:47 |
| vorlon | though in any event I will say that bug reports for either of these environments against the Debian PAM package have basically been nonexistent, and Debian doesn't use u-r-u | 19:48 |
| enr0n | Yeah, it tries to use org.kde.ScreenSaver (probably wants to be org.freedesktop.ScreenSaver), and can't connect to the session bus because $UID == 0 etc. | 19:48 |
| vorlon | well $UID==0 means you also have the privileges to change UID... it probably makes most sense to do this in a subprocess | 19:49 |
| bdmurray | vorlon: but I gather your first inclination is not to do continue inhibiting it at all. | 20:01 |
| vorlon | bdmurray: I think it is acceptable to not do so; but this should be cross-checked with the Desktop Team (and, it seems, the Kubuntu devs) to be sure | 20:02 |
| vorlon | personally, given how long an upgrade may take, I would prefer if my desktop could be locked for the duration | 20:03 |
| Eickmeyer | bdmurray: Hi! ^ mmikowski just joined and wants to chat with you about the /boot partition size issue as reported in bug 1959971 | 21:40 |
| ubottu | Bug 1959971 in partman-auto (Ubuntu Jammy) "increase /boot partition size" [High, Triaged] https://launchpad.net/bugs/1959971 | 21:40 |
| mmikowski | bdmurray: Hi Brian. Thanks for the intro Erich. | 21:42 |
| bdmurray | mmikowski: Hi there, so you are suggesting that we adjust the size of /boot so that it can accomodate 3 kernel flavors w/ two kernels per flavor is that correct? | 21:49 |
| mmikowski | Yes, the latest and penultimate version of each kernel. | 21:50 |
| mmikowski | Many people maintain at least two kernel flavors - lowlatency and hwe. If they are testing or have oem, then the have 3. | 21:51 |
| bdmurray | Your bug report mention advanced users, it would seem to me that advanced users can increase their /boot partition size to accomodate 3 kernels. | 21:52 |
| mmikowski | Of course advanced users can do this, but it is very time consuming. | 21:52 |
| Eickmeyer | These aren't end users doing the installation though, these are end users of an OEM machine. | 21:53 |
| mmikowski | Also, even with the default size with a single kernel flavor we are seeing many people overfill their disks. | 21:54 |
| bdmurray | Sure I think we should adjust the sizing for OEM installs which sounds like two kernels. | 21:54 |
| mmikowski | We frequently see generic-hwe and oem. Then a testing kernel or lowlatency-hwe. | 21:55 |
| mmikowski | These file sets take 180MB each. So the default size of 705M only allows one copy each before the disk is over-full. | 21:55 |
| mmikowski | And recovering from that situation is very difficult for all but the most advanced users. | 21:56 |
| bdmurray | Can you show me an install method that puts all those kernels on disk? | 21:56 |
| bdmurray | Yeah, it was my seeing bug reports that lead me to start looking at the /boot partition sizing. | 21:57 |
| mmikowski | Certainly the installer only provides a single kernel. It's when customers add a kernel later. | 21:57 |
| mmikowski | The reason we advocate a larger size is two fold: first, we have spent perhaps 100 hours supporting people who overfill their disks. | 21:58 |
| mmikowski | This is because when it happens, it is catestrophic. | 21:58 |
| mmikowski | And require enormous effort and time. | 21:58 |
| mmikowski | Many, many users do not know how to recover and lack the skills to do so. | 21:59 |
| mmikowski | The second reason is even with just one kernel, this situation can arrise. | 21:59 |
| bdmurray | I understand the consequences of when things go wrong | 21:59 |
| mmikowski | And we are seeing things go wrong far more frequently than is good for anyone. | 22:00 |
| mmikowski | So when one considers the expense when things go wrong, the likelyhood of things going wrongt, and how inexpensive 2G is for most users these days... | 22:01 |
| mmikowski | It seems that playing it safe is better. | 22:02 |
| bdmurray | I agree that playing it safe is better but your premise is based on a "best practice to manage 3 kernel packages" which I don't understand. | 22:03 |
| bdmurray | When I say premise I mean size estimate. | 22:03 |
| mmikowski | By best practice, I'm referring to the practice of always retaining the latest and penultimate version of each kernel flavor. | 22:04 |
| mmikowski | So that gets us 2 x 3 = 6 file sets. | 22:04 |
| bdmurray | Right why 3? | 22:04 |
| mmikowski | Three flavors concurrently installed. | 22:05 |
| mmikowski | oem, generic-hwe, lowlatency-hwe | 22:05 |
| mmikowski | or similar | 22:05 |
| bdmurray | Yes, but why would you have 3 kernel flavors installed? I don't understand that as a best practice. | 22:05 |
| mmikowski | customers can and do install other kernels why installing. | 22:05 |
| mmikowski | Oh, *that's* not a best practice. It's just what people do for any number of reasons. | 22:06 |
| bdmurray | We can't support everything people decide to do. Having 2 kernel flavors seems reasonable to me and we need to draw the line somewhere. | 22:06 |
| mmikowski | I do that for development purposes. | 22:06 |
| mmikowski | ok, two flavors, but then if they are considering or testing a third, we are back to 3. | 22:07 |
| mmikowski | Of course we can't support everything. I'm reporting what we are seeing in the field and causing users lots of pain and discouragement. | 22:08 |
| bdmurray | Could you tell me a bit more about who "we" is in this case? | 22:08 |
| mmikowski | Again, I wouldn't be such a strong advocate for caution if the results of overfilling weren't so severe. | 22:09 |
| mmikowski | I lead the Kubuntu Focus project. We ship Kubuntu preinstalled world-wide. | 22:09 |
| mmikowski | The referenced bug 1960089 lists all the calcs and reasons. | 22:11 |
| ubottu | Bug 1960089 in ubiquity (Ubuntu Jammy) "Ubiquity Boot Partition for LVM needs to be 2.0 GB for 22.04LTS" [High, Confirmed] https://launchpad.net/bugs/1960089 | 22:11 |
| blahdeblah | Good old /boot sizing. The gift that keeps on giving. I've been using 2 GB as standard ever since 250 GB drives became common. I'll probably up it to 4 GB one of these years. | 22:11 |
| mmikowski | bdmurray: I certainly understand the confusion over "best practice". Hopefully what I meant is now clear. | 22:13 |
| bdmurray | mmikowski: It might help if you redid your math according to the calculations in bug 1716999 instead of just 10 * 200 | 22:14 |
| ubottu | Bug 1716999 in partman-auto (Ubuntu) "installer creates rather small /boot partition" [High, Fix Released] https://launchpad.net/bugs/1716999 | 22:14 |
| mmikowski | bdmurray: The math is based on the estimate that kernel file sets will be around 200 soon (they are 180M now). | 22:16 |
| mmikowski | I also explain the number 10 as well. Unattended upgrades (when it works) can leave extra versions around during upgrades. | 22:17 |
| mmikowski | If one has generic-hwe and lowlatency-hwe, that's two additional file sets. | 22:17 |
| mmikowski | As those are usually released together. | 22:18 |
| mmikowski | So 8. Add 2 for safety, and we are at 10. | 22:18 |
| sarnold | blahdeblah: yeah, I'm a big fan of overshooting /boot sizes, too. burning another gig on never needing to truncate files by hand again... | 22:20 |
| mmikowski | bdmurray: Also, consider that we have recently seen spates where multiple kernel versions are released within a day or so of one another. Thus the safety factor. | 22:21 |
| bdmurray | I understand what you are saying. | 22:22 |
| mmikowski | bdmurray: We had to write a kernel cleaner tool to prevent over-filling, which works quite well and we are looking to upstream. | 22:24 |
| mmikowski | The fact we had to create to ensure customer (and slash support cost) I think is indicative of the magnitude of this problem. | 22:25 |
| bdmurray | Where is this unattended-upgrades work you allude to taking place? | 22:25 |
| mmikowski | bdmurray: The unattended-upgrades is supposed to clean up unused kernels. However, it doesn't always work for a few reasons. Those are enumerated in bug 1960089. | 22:27 |
| ubottu | Bug 1960089 in ubiquity (Ubuntu Jammy) "Ubiquity Boot Partition for LVM needs to be 2.0 GB for 22.04LTS" [High, Confirmed] https://launchpad.net/bugs/1960089 | 22:27 |
| mmikowski | Even when it *does* work, the algo for unattended upgrades can result in overfull /boot partitions. | 22:27 |
| mmikowski | The current /boot partition only supports 3 kernels. Thus when a new kernel comes out, there are 3 kernels installed until the next clean. | 22:30 |
| mmikowski | If another kernel comes out before reboot, that overfills the disk. Again, that's just one kernel flavor. | 22:31 |
| bdmurray | I read the bug report I'm asking were are you "currently working with others to improve the unattended-upgrades algorithm"? | 22:32 |
| mmikowski | bdmurray: https://github.com/Bleuzen/ubuntu-kernel-autoremove | 22:32 |
| mmikowski | That's a start. | 22:33 |
| mmikowski | bdmurray: See https://github.com/Bleuzen/ubuntu-kernel-autoremove/issues/1 | 22:33 |
| ubottu | Issue 1 in Bleuzen/ubuntu-kernel-autoremove "General discussion" [Open] | 22:33 |
| mmikowski | See the analysis. I don't want to go too far down the rabbit hole, but in summary, even when unattended-upgrades works as designed, a 2GB partition is recommended based on the analysis. | 22:35 |
| bdmurray | Okay, I thought you were actually working on fixing an issue with unattended-upgrades. | 22:36 |
| mmikowski | Package kit could allow for up to 8 concurrent kernels. | 22:36 |
| mmikowski | Oh, yes! | 22:36 |
| mmikowski | unattended-upgrades has big issues :) | 22:36 |
| mmikowski | Actually, just for kernel cleaning. The first isn't unattended-upgrades fault, but it gets broken because packagekit marks auto-installed packages as manual. | 22:37 |
| Eickmeyer | ^ That's a bugfix that has been committed upstream in Packagekit that we're testing and I'll report to juliank about. Hopefully packagekit can do a release before FF. | 22:39 |
| mmikowski | The second issue is it uses a heuristic that doesn't consider the system intent but instead purges mostly everything but the last 3-4 kernels. | 22:39 |
| mmikowski | What we envision is specifying the kernels to keep, and then the system applying the algo where last and penultimate versions are retained. | 22:40 |
| mmikowski | The third issue is that unattended-upgrades doesn't consider disk space. So it never gets triggered if /boot is too full. | 22:41 |
| mmikowski | All of those we want to address. | 22:41 |
| mmikowski | bdmurray: most of those issues are already addressed in our kernel clean tool, so this is just an issue of upsteaming and integrating the algo. | 22:46 |
| mmikowski | @Eickmeyer does this work? | 22:46 |
| Eickmeyer | mmikowski: I read you loud and clear. | 22:47 |
| mmikowski | bdmurray: I'll be around tooling with the Jammy code. If you need anything else from me, just give me a bump. Thanks for your interest and questions. | 22:48 |
| mmikowski | bdmurray: Just reviewing our conversation, I want to clarify that a 2GB disk is needed for any improved unattended upgrades. I can see how one might misconstrue that the improvements would reduce the need for /boot size. | 23:31 |
| mmikowski | However, that is not the case. Any improvements would work in conjunction with a larger /boot partition. | 23:32 |
| mmikowski | In particular, it would 1) Consider available space, 2) Honor user intend by keeping the kernel flavors they specify; and 3) Keep the latest and penultimate version of each flavor if space allows. | 23:34 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!