=== chris14_ is now known as chris14
=== JanC_ is now known as JanC
[10:52] <arighi> ricotz, I think it's a mistake, I don't even see nvidia-530 in lunar
[11:10] <arighi> I'll double check
[11:11] <ricotz> arighi, https://launchpad.net/ubuntu/+source/nvidia-graphics-drivers-530
[11:12] <ricotz> I commented on https://bugs.launchpad.net/ubuntu/+source/nvidia-graphics-drivers-525/+bug/2015491
[11:12] <arighi> ricotz, yep, I see it in proposed for all the releases, but not in lunar (not yet at least)
[11:12] -ubottu:#ubuntu-kernel- Launchpad bug 2015491 in nvidia-graphics-drivers-525 (Ubuntu) "Introduce the 530 (UDA) NVIDIA driver series in Bionic, Focal, Jammy, Kinetic, Lunar" [High, In Progress]
[11:12] <arighi> oh ok sorry, you meant in general, I was only checking in lunar
[11:13] <ricotz> yeah, those packages are broken
[11:13] <arighi> ricotz, let me ask around :)
[11:13] <ricotz> arighi, thank you
[13:32] <tseliot> ricotz, hey, what's up with the driver?
[13:33] <ricotz> tseliot, hi, see https://bugs.launchpad.net/ubuntu/+source/nvidia-graphics-drivers-525/+bug/2015491
[13:33] -ubottu:#ubuntu-kernel- Launchpad bug 2015491 in nvidia-graphics-drivers-525 (Ubuntu) "Introduce the 530 (UDA) NVIDIA driver series in Bionic, Focal, Jammy, Kinetic, Lunar" [High, In Progress]
[13:34] <ricotz> tseliot, it seems bad to copy packages which ftbfs already in a PPA to the archive
[13:36] <tseliot> Yes, I must've missed that. Only Lunar and bionic, apparently
[13:37] <ricotz> tseliot, all of them, since i386 isn't picked up on the other series yet
[13:38] <tseliot> Oh, I see. I thought you already had it in the PPA, and that 530 would be already whitelisted
[13:38] <ricotz> I have, but it seems it only enabled lunar :\
[13:40] <tseliot> oh
[13:40] <tseliot> Well, let me fix that build dep here
[13:41] <ricotz> tseliot, there is another change for the PPA build, which seems to got fixed in debhelper
[13:41] <ricotz> tseliot, https://paste.debian.net/plainh/b8c2b72c
[13:43] <tseliot> So that it doesn't try it on all the packages?
[13:44] <ricotz> yes, it failed a couple of days ago, apparently not on your side, so seems to got fixed
[14:05] <ahasenack> rbasak: hi, is there a way to somehow highlight the "main" branch of development in a package? This list is very confusing to newcomers: https://code.launchpad.net/ubuntu/+source/nss
[14:05] <ahasenack> 320 results!
[14:06] <ahasenack> I suppose this needs some new launchpad feature?
[14:07] <ahasenack> ideally we would have "main" (or "devel"), then one branch per release, and the rest is just details (foo-security, foo-proposed, applied and not applied, etc)
[14:23] <ahasenack> rbasak: do you think we can let thunderbird be imported again? It's currently in the denylist, under "# uses up too much disk"
[14:24] <ahasenack> and, hm, openssh is in the denylist too? Just saw it, a few lines below. But we are importing it. Are these lists still being used?
[14:24] <ahasenack> but thunderbird is indeed not imported (or the default branch wasn't flipped): https://code.launchpad.net/ubuntu/+source/thunderbird
[14:25] <tseliot> ricotz, err... zstd is not in main, at least in bionic and focal. xnox , is there anything we can do about this?
[14:25] <ricotz> tseliot, it is a build-dep which fine
[14:26] <tseliot> Yes, I know it's the actual dependencies that are not allowed, but still
[14:48] <rbasak> ahasenack: interesting choice of channel?
[14:48] <ahasenack> rbasak: bah
[14:48] <ahasenack> off-by-one error
[14:48] <ahasenack> ubuntu-devel is just above
[14:53] <ghavil> Howdy, I opened https://bugs.launchpad.net/ubuntu/+source/linux-aws/+bug/2016186 yesterday and was wondering if anyone here had advice as to how I could debug further. I'm not sure how actual disk I/O would translate into these cgroup files, maybe I just need to go understand that side in more depth first. Curious if folks have ideas 
[14:53] -ubottu:#ubuntu-kernel- Launchpad bug 2016186 in linux-aws (Ubuntu) "5.19 not reporting cgroups v1 blkio.throttle.io_serviced" [Undecided, New]
[15:23] <arighi> ghavil, hi, it really seems to be a cgroupv1 vs cgroupv2 issue, I think in kinetic with 5.19 we completely moved to cgroupv2, so you are not supposed to be able to use cgroupv1 anymore unless you tweak the kernel boot options/systems to enable the hybrid cgroupv1+cgroupv2 way
[15:24] <ghavil> Hmmm oh boy that's what I'm worried about
[15:24] <arighi> you can `grep cgroup /proc/mounts` to see the version of cgroup that is mounted
[15:24] <arighi> if it says cgroup2, well, you're using version 2
[15:25] <ghavil> These are for sure cgroups v1 configured, I was trying to get us to v2 but it's breaking a lot of stuff along the way 
[15:25] <arighi> have you seen this? https://docs.kernel.org/admin-guide/cgroup-v2.html#io-interface-files 
[15:26] <arighi> v2 has a slightly different ABI for blkio, but you should be able to enforce the same IO limitations and get the same stats using cgroupv2
[15:26] <ghavil> Yep, our issue w/ cgroups v2 is much more nuanced at the moment. Moving to it isn't an option for now 
[15:27] <ghavil> Our kernels are configured with 'systemd.unified_cgroup_hierarchy=0 systemd.legacy_systemd_cgroup_controller=true' for reference 
[15:28] <ghavil> If 5.19 removes cgroups v1 support, I can work on pinning all of our stuff to 5.15 for now. Bummer though as this upgrade seems to be automatically happening for linux-aws packages (I don't think we opted into anything) and this is a Jammy node not Kinetic 
[15:28] <ghavil> i.e. https://packages.ubuntu.com/jammy-updates/linux-aws is showing the bump to 5.19.0.1022.23~22.04.6 and my expectation was that cgroups v1 would continue to work for 22.04 at least 
[15:33] <arighi> ghavil, if you boot with unified_cgroup_hierarchy you should still be able to use cgroupv1, the kernel still supports it
[15:33] <arighi> I'm not sure if systemd is handling it properly.. I haven't done any test recently with cgroup1
[15:35] <arighi> let me do a quick test...
[15:36] <ghavil> Okay sounds good, I'm been testing as well, I cannot get anything but '0's to appear in the /sys/fs/cgroup/blkio/blkio.throttle.io_serviced files on nodes w/ the cgroups v1 configuration & the 5.19.0.1022.23~22.04.6 kernel running. Downgrading to the 5.15 works and I unfortunately cannot change to cgroups v2 at this time. 
[16:04] <arighi> ghavil, which IO scheduler are you using? (grep . /sys/block/*/queue/scheduler)
[16:05] <ghavil> '[none] mq-deadline' it looks like
[16:06] <ghavil> Same for both of my testing 5.15 and 5.19 instances too 
[16:07] <arighi> ghavil, it looks like 'none' in 5.19 doesn't support cgroup blkio anymore (I'm trying to find the commit that changed this behavior)
[16:07] <arighi> ghavil, if you switch to bfq for example (apt install linux-modules-extra and modprobe bfq) then you can see IO stats in the blkio.* in cgroupfs
[16:08] <ghavil> Oh interesting
[16:08] <ghavil> Lemme give that a whirl 
[16:10] <arighi> ghavil, BTW you also need to enable the bfq scheduler for your block devices, like `echo bfq > /sys/block/<dev>/queue/scheduler`
[16:10] <ghavil> ack
[16:26] <ghavil> arighi: Dumb question, should I be installing 'linux-modules-extra-aws' or is 'linux-modules-extra' good enough? 
[16:29] <ghavil> AH HA that does work!
[16:29] <ghavil> I installed linux-modules-extra-aws for the record 
[16:32] <arighi> ghavil, yep modules-extra-aws is fine
[16:33] <xnox> tseliot: i'm not sure what you are askinga bout zstd. what needs it in main? and if the request is legit we can promote it.
[16:34] <tseliot> xnox, apparently, nvidia-530 needs it as a build-dep, only on i386. I suppose it should be fine for now.
[16:34] <arighi> ghavil, however... I see the IO stats only in the blkio.bfq.* files, not in blkio.throttle.*, that means we are likely hitting a behavior change upstream (probably not many people noticed that, because it's cgroup1)
[16:40] <ghavil> arighi: Ah good catch, yeah I see that too, bummer but this is better than nothing 
[17:04] <xnox> tseliot83:  packages in main and restricted, can build-depend on packages in universe/multiverse, if they don't produce runtime Depends on them.
[17:04] <xnox> tseliot83:  i believe this has been true for long enough for you to use this fact everywhere.
[17:04] <xnox> tseliot83:  https://lists.ubuntu.com/archives/ubuntu-devel/2016-March/039274.html
[17:04] <xnox> i love the title of this email, as it is so easy to find
[17:05] <tseliot83> xnox, right, it's good enough for me. Thanks!
[17:05] <xnox> i think we did it 2016, so all releases you can add Build-Depends: zstd and it will work
[19:42] <jchittum> ghavil: for pinning to the 5.15 kernel, you can use `linux-aws-lts-22.04` package: https://packages.ubuntu.com/jammy/linux-aws-lts-22.04
[19:45] <ghavil> Ah thanks for the tip! 
[19:46] <ghavil> jchittum: Is there a linux-gcp equivalent? I found https://packages.ubuntu.com/jammy/linux-azure-lts-22.04 for Azure 
[21:35] <jchittum> ghavil: based on history there will be. it looks like the gcp kernel hasn't rolled to 5.19 yet. I believe that the Kernel team makes the -lts-$VERSION around rolls. I see linux-gcp-lts-20.04 and linux-gcp-lts-18.04
=== halvors1 is now known as halvors