[02:54] <handsome_feng> Hi, kc2bez, dbungert, bdmurray, franksmcb, thanks a lot for your testing  on Ubuntu Kylin. 💐
[06:49] <ginggs> mwhudson: i suspect some of them were already FTBFS due to gcc 11.  I'm going to compare those to the FTBFS report
[07:27] <LocutusOfBorg> vorlon, doko trying to enable flang and upload in ppa
[07:32] <LocutusOfBorg> vorlon, lack of time to enable and make it build lol :D
[08:15] <kc2bez> Of course handsome_feng happy to help out.
[08:26] <handsome_feng> :)
[09:40] <xnox> Eickmeyer:  i'm not sure how it could be a race condition; because nvidia driver is linked in a postinst; and initramfs is rebuilt as a trigger at the end of transaction. note that during apt run all calls to update-initramfs are diverted to a trigger.
[09:40] <xnox> Eickmeyer:  nvidia driver postinst should interest the initramfs trigger it is harmless / correct thing to do.
[09:40] <xnox> especially since we have done nvidia-only respins before
[13:43] <rbasak> slyon: success. Thank you for trying it! https://git.launchpad.net/ubuntu/+source/cdbs/log/
[13:45] <slyon> rbasak: uhh, nice! Kudos to you for putting it all together :)
[15:07] <Eickmeyer> xnox: I agree, but the question is why we're not rebuilding initramfs for *all* installed kernels ala "update-initramfs -u -k all".
[15:10] <Eickmeyer> xnox: bug 1947043
[15:11] <Eickmeyer> xnox: This is something the company I work for and I have identified, not just me as an individual. :)
[15:12] <Eickmeyer> xnox: Long-story short, a recent kernel upgrade coupled with nvidia driver upgrade at the same time in people's systems created this condition where their nvidia drivers failed to function following a reboot, and this was identified as the culpret.
[15:12] <Eickmeyer> s/people's/customer's
[15:17] <xnox> Eickmeyer:  the choice to rebuild current initrd only, or all initrds is scary.
[15:17] <xnox> Eickmeyer:  we have had regression in e.g. busybox; and then you don't want to rebuild all initrds, cause then none of them will work.
[15:18] <xnox> Eickmeyer:  and also we have not been rebuilding many initrds due to changes of things that are vendored into it. I.e. i think there have been things that are included into initrd (without their knowledge) but do not update the initrd
[15:18] <xnox> i.e. security upgrades of shared libraries that are linked to binaries in the initrd i don't think currently universally trigger initramfs rebuilds.
[15:24] <Eickmeyer> xnox: True, but it's really disheartening when we're receiving support emails that their system has broke because the nvidia driver and kernel updated in the same "apt update". We'd rather err on the side of usability.
[15:24] <Eickmeyer> The fix has 90% been "update-initramfs -u -k all".
[15:28] <xnox> Eickmeyer:  how did the machines ended up with nvidia dkms, instead of pre-signed lrm modules?
[15:28] <xnox> Eickmeyer:  did you run ubuntu-drivers to install matching pre-built and pre-signed nvidia modules _instead of_ having dkms instaled?
[15:32] <Eickmeyer> xnox: They install nvidia-drivers-470. They're running focal, and we preinstall for machines that ship.
[15:34] <Eickmeyer> xnox: The flavor is Kubuntu in this case.
[15:47] <Eickmeyer> xnox: nvidia-drivers-470 pulls-in the dkms modules.
[15:48] <xnox> Eickmeyer:  that's the wrong way to install nvidia-drivers which breaks things =/
[15:49] <Eickmeyer> xnox: Isn't that what ubunutu-drivers does?
[15:49] <xnox> Eickmeyer:  one must use ubuntu-drivers to install, cause then it will not install dkms; and install the prebuilt & presigned kernel modules that are updated in lock step
[15:49] <xnox> Eickmeyer:  no it finds the correct meta to install prebuilt drivers; and it configures nvidia prime; and configures support for power management of nvidia gpus; and makes things work on boot
[15:50] <Eickmeyer> xnox: Noted, I'll figure out how to implement this for our customers.
[15:50] <xnox> i.e. even installing things that $ ubuntu-drivers list => is not enough
[15:50] <xnox> Eickmeyer:  i.e. $ ubuntu-drivers list
[15:50] <xnox>  for me offers
[15:50] <xnox> nvidia-driver-470, (kernel modules provided by linux-modules-nvidia-470-generic-hwe-20.04)
[15:50] <xnox> without having dkms, becuase prebuilt modules linux-modules-nvidia-470-generic-hwe-20.04 will be used
[15:51] <xnox> Eickmeyer:  and ubuntu-drivers install also configures nvidia-prime and power management
[15:51] <xnox> Eickmeyer:  thus one should always do ubuntu-drivers install to get the right set of nvidia for the right gpu and the right config that affects initramfs.
[15:52] <xnox> Eickmeyer:  note this is supported by cloud-init too
[15:52] <xnox> can be preseeded in ubiquity and maas
[15:52] <xnox> and used in the clouds too
[15:53] <Eickmeyer> xnox: I'll relay this to the CTO, he has historically not been keen to trust processes he hasn't developed.
[15:57] <vorlon> LocutusOfBorg: no worries, just wanted to draw attention to it to someone who might be able to answer authoritatively :)
[16:17] <xnox> Eickmeyer:  using ubuntu-drivers is the only way to have nvidia drivers, signed with Canonical keys, possible to use with SecureBoot on, and with support for nvidia-prime, improved power management, and ability to dynamically switch from/to nvidia drivers, and is the correct thing to do on laptops, desktops, servers, cloud.
[16:18] <xnox> Eickmeyer:  and you still have ability to choose which series you are after, if one doesn't want to use the latest recommended one. But note that you will remain supported as things are rolled over.
[16:19] <Eickmeyer> xnox: I'm noticing that it sets to on-demand mode, and historically that mode has been problematic for our customers, we've recommended nvidia mode since most of our customers are data scientists for AI and deep learning.
[16:22] <xnox> Eickmeyer:  we enable the modes that we have worked with nvidia to provide. Note if you want drivers not for UI/Graphics, but for AI/ML only, you can opt into -server variant of the drivers. Those are only menat to be used as gpgpu accelerator without using nvidia for graphics.
[16:22] <xnox> Eickmeyer:  this improves the ML/AI performance without affecting (wasting) gpu for Ubuntu Desktop Gui
[16:23] <Eickmeyer> xnox: The reason we haven't gone with the -server variant is that we want people to be able to do both, so that wouldn't work for our use case.
[16:23] <xnox> -server has a lot less depenedencies, does not install many utils, and is what we use in our k8s deployments on MAAS and Public Cloud k8s offerings.
[16:24] <xnox> Eickmeyer:  if you need/want/may have both, please use ubuntu-drivers with on-demand, as that's the only way Ubuntu and nVidia supports things for recent cards.
[16:24] <Eickmeyer> The computers we sell are laptops.
[16:25] <xnox> all Canonical Certified machines are configured with nvidia graphics drivers as installed with ubuntu-drivers in ondemand mode; and it is also the default way things are configured when installing nvidia drivers using nvidia.com scripts.
[16:25] <Eickmeyer> xnox: Good to know, we're working on getting Canonical certification.
[18:12] <jawn-smith> bdmurray, dbungert: I'll take a look at LP: #1873565 since I've worked with gnutls before
[18:12] <bdmurray> jawn-smith: okay, I'll finish the netkit-telnet one
[18:17] <jawn-smith> bdmurray: It looks like this bug (LP: #1873565) needs an SRU to bionic, but no patch has been uploaded so I think the sponsors team could be unsubscribed
[18:18] <bdmurray> jawn-smith: what's in comment #2?
[18:18] <dbungert> bdmurray: jawn-smith: looking at LP: #1840348
[18:18] <jawn-smith> Sorry I'm mixing terms. It's a patch, not a debdiff. I meant that no Bionic debdiff has been uploaded
[18:19] <jawn-smith> I'd be happy to work on the SRU (filling out the template and creating a debdiff) if that's in scope for this "trim the sponsoring queue" exercise
[18:20] <bdmurray> Well we could create a debdiff, so the next question is does it have enough information that you feel comfortable SRU'ing it? e.g. test case
[18:20] <jawn-smith> It seems to include enough detail to recreate it, though I'd of course need to fill out the SRU template still. Shall I proceed with that?
[18:22] <bdmurray> Yeah, let's try and do the right thing here
[18:23] <jawn-smith> on it, thanks!
[18:24] <bdmurray> FYI there's a tool which can produce this comment https://bugs.launchpad.net/ubuntu/+source/netkit-telnet/+bug/1891021/comments/2
[18:29] <bdmurray> looking at bug 1581594
[18:31] <dbungert> looking at bug 1794194
[18:34] <dbungert> bdmurray: would you unsubscribe sponsors from bug 1794194?  We can pick this up in the next package merge.
[18:35] <dbungert> looking at bug 1886809
[18:59] <bdmurray> dbungert: done, thanks!
[19:08] <bdmurray> I've commented on bug 1891110
[19:20] <dbungert> bdmurray: we should probably remove sponsors from bug 1886809, I've added a comment with some suggestions.
[19:20] <dbungert> looking at bug 1890572
[19:30] <bdmurray> looking at bug 1871306
[19:37] <dbungert> bdmurray: for bug 1890572, upstream declined it and we've already asked the submitter to work with new upstream.  I think this is another one to drop sponsors from.
[19:37] <jawn-smith> bdmurray FWIW the patch that was uploaded doesn't apply cleanly in bionic at all, and needs a bit of work to get working in bionic. I'm working on it now
[19:39] <bdmurray> jawn-smith: well in that case it'd probably be a better use of time to ask for them to redo the patch
[19:39] <jawn-smith> okay! I'll add a comment to the bug
[19:41] <jawn-smith> bdmurray: done, but now sponsors should be unsubscribed right?
[19:46] <jawn-smith> Looking at LP: #1903548 now. Looks like a community member uploaded a debdiff so I'll give it a test
[19:47] <bdmurray> jawn-smith: yes, I'll unsubscribe them
[19:48] <jawn-smith> thanks
[19:53] <bdmurray> dbungert: Did you look at the cups source wrt bug 1890572?
[19:54] <dbungert> bdmurray: no, I was relying on the comment at https://github.com/apple/cups/pull/5818#issuecomment-796685862
[19:55] <bdmurray> dbungert: well it looks fixed to me, I'll follow up
[19:59] <dbungert> looking at bug 1892825
[20:02] <dbungert> that one looks fine, looking at 1890491
[20:02] <bdmurray> looking at bug 1820083
[20:03] <dbungert> bdmurray: jawn-smith had that one I think
[20:04] <jawn-smith> hmm nope I havne't looked at that one
[20:05] <jawn-smith> different tls related bug :)
[20:05] <bdmurray> ddstreet: is there a reson bug 1820083 hasn't been sponsored?
[20:05] <bdmurray> oh maybe because sts-sponser is spelled wrong
[20:06] <sarnold> hmm, I've seen two bugs recently in the installer that had problems resolving archive hosts moments after downloading packages from those archive hosts: https://bugs.launchpad.net/ubuntu/+source/grub-installer/+bug/1946784 https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1947061
[20:07] <bdmurray> maybe there are too many clients?
[20:11] <jawn-smith> bdmurray: I've asked the bug reporter on LP: 1903548 for more information, if you feel like unsubscribing sponsors
[20:15] <jawn-smith> Looking at LP: 1913656 I think it should also have the sponsors team unsubscribed. It's an SRU assigned to a Canonical employee. There is a link to an upstream commit that fixes the issue
[20:15] <bdmurray> I'm looking at bug 1898005
[20:15] <bdmurray> I wonder if we should send a report to ubuntu-devel whenever we "finish"
[20:17] <bdmurray> jawn-smith: do you want to subscribe to the bug which asked for more information in? ;-)
[20:18] <bdmurray> there's no patch in that ghostscript bug so unsubscribing
[20:18] <jawn-smith> bdmurray: seeing as how it's been assigned to a member of the Desktop team since Feb, would it be appropriate for me to prepare the SRU for LP: 1913656 ?
[20:19] <jawn-smith> I'd ask the desktop team member but it's quite late in central Europe and they are therefore not online at the moment
[20:19] <seb128> jawn-smith, I think if you want to go ahead with the SRU that would be welcome
[20:27] <bdmurray> I'll be sponsoring the patch in the bug I'm working on
[20:31] <bdmurray> I unsubscribed sponsors from bug 1754069
[20:34] <bdmurray> looking at bug 1916250
[20:44] <dbungert> looking at bug 1882998
[20:45] <bdmurray> mwhudson: could you have another look at the MP associated with bug 885086?
[20:46] <mwhudson> oh yeah i have scripts to make testing initramfs changes much easier now!
[20:48] <bdmurray> mwhudson: thanks! we are trying to have a look at the sponsorship queue and cleaning it up a bit
[21:30] <jawn-smith> seb128: I prepared that SRU. If you have time would you be able to review it?
[21:42] <vorlon> rbasak: /snap/bin/git-ubuntu.merge-changeloges  - oops
[21:50] <jawn-smith> bdmurray: (or any other member of ubuntu-sponsors) it appears racb requested more information in LP: #1472288 a few months ago and hasn't heard anything, so it seems that the sponsors team could be unsubscribed for now.
[21:52] <rbasak> vorlon: thanks. That shows how many users are actually using that entry point then? Or maybe they all autocomplete?
[21:55] <rbasak> https://code.launchpad.net/~racb/usd-importer/+git/usd-importer/+merge/410255 - I'll land as soon as CI passes.
[21:58] <rbasak> jawn-smith: thanks! I've commented and unsubscribed sponsors.
[22:01] <jawn-smith> rbasak: thanks! We're trying to help clean up the sponsoring queue
[22:08] <rbasak> Appreciated!
[22:22] <vorlon> bdmurray noticed that sway was accepted into impish-proposed despite not having been signed off by a member of the SRU team; https://launchpad.net/ubuntu/impish/+queue?queue_state=3&queue_text=sway shows that the Ubuntu Archive Robot accepted it.  I've disabled the auto-accept cronjob on snakefruit, looking to see where that is missing from our process
[22:22] <vorlon> also, removing sway from impish-proposed
[22:30] <vorlon> impish-changes shows other packages were wrongly accepted the same way; gsmlib ( slyon ), checkinstall, cdbs, osinfo-db ( jbicha ), I've removed each of these now.  They can potentially be copied into JJ-proposed when it opens, please let me know if you want that to happen
[23:34] <vorlon> bdmurray: to prevent this in the future: https://bazaar.launchpad.net/~ubuntu-archive/ubuntu-archive-scripts/trunk/revision/313
[23:35] <ddstreet> bdmurray re: lp:1820083 it's because the customer asking us for it frequently gets distracted by shiny things right after asking for something, so they've been unresponsive when we asked for testing; usually, they get back around to things they ask for after a few months or so
[23:36] <ddstreet> i'll check back with the customer contact to see if they're ready to test before we upload it