=== paride3 is now known as paride
=== cpaelzer_ is now known as cpaelzer
[06:38] -queuebot:#ubuntu-release- Unapproved: linux-firmware (jammy-proposed/main) [20220314.gitcd01f857-0ubuntu2 => 20220329.git681281e4-0ubuntu1] (core, kernel)
[07:37] -queuebot:#ubuntu-release- Unapproved: gnome-initial-setup (jammy-proposed/main) [42.0.1-1ubuntu1 => 42.0.1-1ubuntu2] (ubuntu-desktop)
[08:13] -queuebot:#ubuntu-release- Unapproved: screen-resolution-extra (jammy-proposed/main) [0.18.1build1 => 0.18.2] (ubuntu-desktop)
[08:50] <juliank> if anything else is stuck for beta in proposed due to waiting for tests please let me know
[08:50] <juliank> I can move stuff in the normal queue to the top
[08:50] <juliank> and I can move stuff from huge to normal
[08:50] <juliank> I already moved up grub2 and u-boot
[08:51] <juliank> oh sil2100 ubiquity probably should move up too
[08:52] <sil2100> I skiptested ubiquity as we have no arm* consumers right now
[08:52] <sil2100> So it's not so critical
[08:53] <juliank> Ah
[08:53] <juliank> So I'm also running around killing tests that have not produced output for an hour
[08:54] <juliank> as we still get significant hangs (we had 8 hour+) on armhf
[09:13] <Wimpy> Trevinho: Do you want to take up juliank on his offer to speed up the migration of yaru-theme?
[09:13] <juliank> yaru-theme has no pending tests, Wimpy
[09:14] <sil2100> I can unblock it in that case
[09:14] <Wimpy> Please unblock yaru-theme and ubuntu-mate-artwork (which depends on yaru-theme)
[09:14] <sil2100> Please remember that we're in beta freeze so uploads need to be manually unblocked!
[09:15] <juliank> yep
[09:15] <sil2100> Ok, unblocked
[09:15] <Wimpy> Yeah, those uploads were just moments after the  freeze :-)
[09:15] <Wimpy> sil2100: Thank you!
[09:15] <sil2100> Wimpy: just yaru-theme or also some other package is part of that, do you know?
[09:16] <juliank> basically stuff that will be unblocked can be moved up the queue, decision is up to the release team
[09:16] <Wimpy> yaru-theme and ubuntu-mate-artwork.
[09:16] <juliank> final grub2 tests running
[09:17] <sil2100> Wimpy: ok, the other I already unblocked, so you should be good!
[09:17] <Wimpy> Thanks very much!
[09:17] <sil2100> Ok, I need to go AFK now for a while, but I'll be back later and working into laaate evening
[09:17] <Wimpy> ETA for iso creation?
[09:17] <seb128> juliank, sil2100, we would like to get gnome-shell if possible but that's behind atm due to armhf autopkgtest queue
[09:18] <sil2100> I would say later afternoon when things finally migrate and stabilize
[09:18] <Wimpy> Thanks.
[09:18] <juliank> that needs to migrate together with mutter it seems
[09:18] <juliank> sil2100: shall I bump gnome-shell and mutter up?
[09:19] <juliank> oh seb128 migrating gnome-shell/42.0-1ubuntu1/amd64 to testing makes gnome-shell-extension-easyscreencast/1.5.0-1/amd64 uninstallable
[09:19] <seb128> juliank, I just deleted that one
[09:20] <juliank> hmm
[09:21] <seb128> juliank, it's an extension which wasn't updated for the new shell, we decided to delete those, that will be sorted out in the next refresh but we still need to clear the tests
[09:21] <seb128> sil2100, ideally we would have like the new xorg stack in, which was blocked on azure kernel, do you know what's the status there?
[09:22] <sil2100> juliank: I guess you could. When I'm back we can see how things are and unblock things before the RCs
[09:22] <sil2100> seb128: sadly, I don't know much. But I'd say this is more unlikely for beta now
[09:22] <seb128> :-(
[09:22] <juliank> xorg stack is too big to migrate now
[09:23] <seb128> it's in proposed for 27 days
[09:23] <seb128> it's annoying how we don't get transitions moving
[09:24] <seb128> it sort of defeat the point of having a beta if we don't test what is current and going to be in the release
[09:25] <juliank> seb128: It can't have been in proposed for 27 days or it would have its tests run
[09:25] <juliank> queues were virtually empty 5 days ago
[09:27] <juliank> xorg-server was uploaded a while ago, sure
[09:27] <juliank> but this depends on much newer uplaods
[09:27] <juliank> eek it depends on my newer uploads
[09:27] <juliank> don't ask me why
[09:28] <juliank> why would xorg-server have implicit dependency on xorg, xserver-xorg-video-dummy no-change rebuilds
[09:29] <juliank> also it has a regression in firejail on armhf
[09:32] <seb128> juliank, I'm going to retry a migration-reference/0 there
[09:33] <juliank> ugh
[09:33] <juliank> I'd rather not
[09:33] <seb128> why not?
[09:33] <juliank> it has 167 items in the queue ahead of it
[09:34] <tjaalton> seb128: the blocker is aws kernel now
[09:34] <tjaalton> for nvidia-390, which has blocked xserver for two weeks
[09:34] <tjaalton> and weeks before that
[09:35] <juliank> firejail never succeeded?
[09:35] <juliank> well 0.9.56-2 once did
[09:35] <juliank> but there's no migration-reference/0 run which is _odd_
[09:36] <juliank> Also I believe we need to adjust release opening procedure for autopkgtests to copy over the latest migration-reference/0 instead of last successful run
[09:36] <tjaalton> firejail? that's news to me
[09:37] <juliank> tjaalton: bryce retried firejail in proposed with a hole lot of packages causing it to become a failure instead of neutral for xorg-server and others
[09:37] <juliank> despite the version in release pocket being neutral
[09:37] <juliank> britney is not looking at the right test result
[09:38] <tjaalton> heh, ok
[09:38] <tjaalton> I've been chasing down kernel failures for the past two weeks trying to get nvidia-390 to migrate
[09:38] <juliank> tjaalton: but now xorg-server seems to have gained implicit dependencies on all sorts of later uploads and I'm not sure wwhat's up with that
[09:39] <juliank> or does "Implicit dependency" mean "reverse dependency"?
[09:40] <juliank> tjaalton: like I rebuild bunch of -video- packages for ppc64el baseline rebuild, and xorg-server now has "Implicit dependency: xorg-server xserver-xorg-video-amdgpu (not considered)" and similar
[09:42] <juliank> ah so it did really need newer drivers, they were synced same day as the no-change rebuilds, so they essentially got built twice
[09:45] <juliank> so it also waits for xserver-xorg-video-dummy to finish running tests
[09:45] <juliank> which ended up in the huge queue as it was part of the ppc64el rebuilds
[09:51] <juliank> why don't we have a transition tracker for xorg-video-abi-25?
[09:52] <tjaalton> we have trackers?
[09:53] <seb128> tjaalton, https://people.canonical.com/~ubuntu-archive/transitions/
[09:54] <juliank> ben needs to learn to track those provides automatically and auto-create transitions
[09:55] <juliank> sorry about the video nochange uploads those made things a bit nastier
[09:56] <seb128> anyway we would really benefit to have the xorg update in betéa
[09:56] <Trevinho> how transitions can be added there?
[09:57] <seb128> because without the current nvidia 510 we can't really test wayland sessions
[09:59] <tjaalton> seb128: heh, it's blocked by many kernels
[09:59] <tjaalton> not xserver
[09:59] <seb128> xnox, ^ do you know what's the status for kernel migrations?
[10:00] <tjaalton> tseliot should herd his uploads through..
[10:01] <juliank> Trevinho: https://bazaar.launchpad.net/~ubuntu-transition-trackers/ubuntu-transition-tracker/configs/changes
[10:03] <seb128> tjaalton, also yes
[10:42] -queuebot:#ubuntu-release- Unapproved: gemmi (jammy-proposed/universe) [0.5.3+ds-1ubuntu1 => 0.5.3+ds-2] (no packageset) (sync)
[10:44] -queuebot:#ubuntu-release- Unapproved: cd-boot-images-amd64 (jammy-proposed/main) [18 => 19] (desktop-core)
[10:44] -queuebot:#ubuntu-release- Unapproved: notmuch (jammy-proposed/universe) [0.35-1ubuntu2 => 0.35-2ubuntu1] (no packageset)
[10:49] -queuebot:#ubuntu-release- Unapproved: cd-boot-images-arm64 (jammy-proposed/main) [14 => 15] (desktop-core)
[11:04] -queuebot:#ubuntu-release- Unapproved: cd-boot-images-ppc64el (jammy-proposed/main) [11 => 12] (desktop-core)
[11:44] <xnox> seb128:  all our kernels are a mess at the moment =)
[11:44] -queuebot:#ubuntu-release- Unapproved: cd-boot-images-riscv64 (jammy-proposed/main) [1 => 2] (no packageset)
[11:45] <xnox> seb128:  i'll chat to the team today, to see if we can push out LRM respin, such that all nvidia stuff migrates.
[11:46] <xnox> vorlon:  cd-boot-images-riscv64 says no-packageset but it is seeded in supported-installer-desktop.... does that not get used for riscv64 at all
[11:46] <xnox> ?
[11:46] <seb128> xnox, thanks
[11:48] <xnox> juliank:  uploaded cd-boot-images-* respins ^^^^
[11:48] <juliank> xnox: oops forgot those once again
[11:49] <juliank> xnox: I wish we'd not have them and builds just used grub etc directly
[11:50] <juliank> xnox: so just to make sure: you say you uploaded them?
[11:50] <juliank> ah I see them at the top
[11:50] <juliank> launchpad confused me
[11:54] <xnox> juliank:  held up in the queue.... because freeze
[11:55] <xnox> juliank:  or like maybe cd-boot-images-* source packages could be generated at the same time as grub2-unsigned.
[11:56] <juliank> xnox: why do we have them?
[12:03] <xnox> juliank:  because debian-cd runs on amd64; cannot easily pull packages for $foreign-arches or execute commands to manipulate them; and we must have one package per arch; if we want to build things natively and publish as arch:all. They are replacement for d-i's tarballs that used to be produced by d-i and then included onto isos by it.
[12:03] <juliank> ah
[12:04] <juliank> but if it used tarballs before it could still publish tarballs automatically as grub2-signed build artefacts or so
[12:04] <xnox> it would be nice if like it was possible to produce arch-all packages from multiple binary builds.
[12:04] <juliank> yes
[12:05] <juliank> You want like
[12:05] <juliank> Package: grub-efi-arm64-bin
[12:05] <juliank> Architecture: all
[12:05] <juliank> Build-Architecture: arm64
[12:05] <xnox> cause ideally i'd want like grub-efi-amd64-signed to be arch:all and grub-efi-arm64-signed arch:all and the bins
[12:05] <juliank> yes, that
[12:05] <xnox> and the grub-pc thing should _prebuild_ and _preship_ the three image files we use from it.
[12:05] <xnox> (cd, hybrid, and normal one)
[12:06] <juliank> grub-pc-signed basically
[12:06] <juliank> just um not signed
[12:06] <juliank> grub-pc-prebuilt
[12:06] <juliank> grub-pc-images
[12:06] <xnox> at the moment we do have cd-boot-images-$(arch) that does that for a given $(arch) and publishes as arch:all package
[12:07] <juliank> still couldn't we upload those as tarballs and then fetch them from ftpmaster?
[12:07] <xnox> but also vendors other things too (like shim) and is very static contents without any maintainer scripts etc, i.e. more or less a tarball that one can just unpack
[12:08] <xnox> apt download / pull-lp-deb is easier than custom uploads =)
[12:08] <juliank> well
[12:08] <xnox> plus the theory was that ubuntu-image will be able to use those too to make images
[12:09] <juliank> I wish launchpad would just build the -signed and cd-boot-images source packages itself
[12:09] <juliank> from um some templates
[12:09] <xnox> just don't go there =)
[12:09] <ricotz> hello ubuntu-archive, please let libreoffice/impish-proposed 1:7.3.2~rc2-0ubuntu1 transition to release
[12:10] <xnox> if we had Architecutre:all/build-architecture:$blah that would be good enough.
[12:10] <xnox> but we only have XS-Build-Indep-Architecture:$arch
[12:11] <xnox> juliank:  we could merge cd-boot-images-* into grub packaging, to make it more insane (as in have the cd-boot-images-* templates inside it) such that generate-grub2-unsigned would generated 5 source packages to upload
[12:12] <juliank> arm64 is somewhat did
[12:12] <juliank> * dead
[12:13] <juliank> 30%-50% of tests fail abnormally
[12:13] <juliank>                                                                                                              gpg: requesting key 622CF918 from hkp server keyserver.ubuntu.com
[12:13] <juliank>                                                                                                              gpg: no valid OpenPGP data found.
[12:13] <juliank> lots of those
[12:14] <ricotz> oops, of course I meant liberoffice/jammy-proposed :\ , please let 1:7.3.2~rc2-0ubuntu1 transition to release
[12:14] <juliank> PPA tests fail Cannot initiate the connection to ppa.launchpad.net:80 (2001:67c:1560:8008::19). - connect (101: Network is unreachable) Could not connect to ppa.launchpad.net:80 (91.189.95.85), connection timed out
[12:15] <juliank> salsa doesn't work fatal: unable to access 'https://salsa.debian.org/systemd-team/systemd.git/': Received HTTP code 403 from proxy after CONNECT
[12:16] <juliank> I don't know what to do really
[12:26] <juliank> arm64 and armhf queues seemingly don't move
[12:26] <juliank> I know that armhf is running flawlessly
[12:26] <coreycb> cpaelzer: I missed that new upstream release todo for xmlschema. do you want that done in jammy?
[12:27] <juliank> well it runs, but clearly not fast enough
[12:28] <cpaelzer> coreycb: it was james who brought it up, and the reason is sounds as the current version is rather outdated and thereby odd to start in main
[12:28] <cpaelzer> coreycb: I know we are behind most freezes, but OTOH this has no other deps but yours AFAICS
[12:28] <cpaelzer> coreycb: so talk to the release team and then yes if possible update it
[12:33] <juliank> xnox: we really need to fix this mess where --apt-pocket=proposed=src:linux-meta-riscv triggers dkms modules on arm64 and then everything faisl because it's not there
[12:34] <juliank> xnox: and it fails with "no versioN", and then retries and retries and retries
[12:36] <coreycb> cpaelzer: 'reverse-depends python3-xmlschema' is showing nothing so I'm confused
[12:37] <juliank> E: Unable to locate package ^linux-image-generic-riscv$
[12:37] <juliank> no shit sherlock
[12:37] <cpaelzer> it shows nothing because one removed the dep as it couldn#t be in main before
[12:39] <coreycb> cpaelzer: python3-pysaml2 still depends on it though
[12:45] <coreycb> probably because it's only an rdepends in jammy-proposed
[12:48] <xnox> juliank:  i am confused why anything triggers for linux-meta-riscv at all.... it doesn't exist on anything but riscv64 and there are no riscv64 autopkgtests......
[12:48] <xnox> juliank:  is it britney or kernel's teams adt matrix scheduling that?
[12:48] <juliank> xnox: I don't know, I'd think the latter?
[12:49] <xnox> it has automated-testing invalid.... so it shouldn't do that.
[12:55] <juliank> {'submit-time': '2022-03-25 23:36:10', 'triggers': ['linux-meta-riscv/5.15.0.1005.5']}
[13:22] <seb128> sil2100, could you unblock nautilus? it's ready to migrate, only an application, and there is an annoying segfault in the current version which is fixed by the update
[13:29] <ricotz> sil2100, hi, please do the same for libreoffice 7.3.2~rc2 :)
[13:48] -queuebot:#ubuntu-release- Unapproved: alsa-ucm-conf (jammy-proposed/main) [1.2.6.3-1 => 1.2.6.3-1ubuntu1] (core)
[14:03] <xnox> seb128:  trying to prioritise migrating all the kernels & lrms to jammy-release, before entagling more stuff into it.
[14:04] <xnox> is the conclusion from our standup.
[14:04] <seb128> xnox, thanks
[14:07] <xnox> apw ubuntu-release - can you please unblock migration of linux-restricted-modules linux-restricted-signatures nvidia-graphics-drivers-390
[14:07] <xnox> and all other related linux-restricted-* packages
[14:07] <xnox> linux-restricted-* basically and nvidia-graphics-drivers-*
[14:09] <xnox> as being blocked by ubuntu-release block all
[14:12] <apw> xnox, looking into it.
[14:16] -queuebot:#ubuntu-release- Unapproved: accepted simplestreams [source] (bionic-backports) [0.1.0-48-gb936edd4-0ubuntu1~bpo18.04.1]
[14:17] <juliank> ** arm64 error rate is down from 40% to 0% **
[14:17] <juliank> this should speed things up
[14:18] <xnox> nicd
[14:18] <juliank> I still don't know why queue shrinks by 40 packages in 6 hours and ppc64el by 1000, when ppc64el has double the test throughput
[14:18] <juliank> Realistically 4 times the pass rate
[14:18] -queuebot:#ubuntu-release- New binary: simplestreams [amd64] (bionic-backports/main) [0.1.0-48-gb936edd4-0ubuntu1~bpo18.04.1] (ubuntu-cloud, ubuntu-server)
[14:19] <juliank> well 3 times maybe
[14:19] -queuebot:#ubuntu-release- New: accepted simplestreams [amd64] (bionic-backports) [0.1.0-48-gb936edd4-0ubuntu1~bpo18.04.1]
[14:19] <juliank> arm64 queue went 8993 -> 8945
[14:20] <juliank> ppc64el queue went 1241 -> 185
[14:20] <juliank> you can see how this does not line up with the throughput factor of 2-3x, unless someone pushes new arm64 (and armhf) tests all the time, but no ppc64el/s390x
[14:22] <juliank> xnox: how can I find arigi on irc? Queuing a ton of canonical-kernel-team/bootstrap tests *now* is not productive
[14:22] <juliank> or constructive
[14:22] <juliank> we should really reject any PPA or upstream tests right now
[14:22] <juliank> because if only to figure out why those queues are not moving
[14:27] <juliank> xnox: so i need to figure out if I can cancel them now and then we can redo them later
[14:29] <juliank> there have been 200 PPA test requests on arm64 today
[14:29] <juliank> compared to 130 on amd64
[14:29] <juliank> or about 65 on ppc64el and s390x
[14:29] <juliank> this is a problem.
[14:30] <juliank> (requests accepted for processing)
[14:30] <coreycb> ubuntu-release: I'm seeking a +1 to sync elementpath and python-xmlschema from unstable to jammy. they build successfully on jammy and will be tested with openstack regression. this is a TODO that came out of MIR 1953363. openstack has a feature freeze exception and these are mostly but not specifically openstack dependencies.
[14:30] <juliank> or better, requests acted on
[14:32] <xnox> juliank:  do not reject PPA requests
[14:32] <juliank> things are *not* working
[14:33] <xnox> juliank:  the ones comming from kernel team are required to pass for us to release kernels. cause we do tests (kernel) X (dkms) X (arch) and they do point to PPAs
[14:33] <xnox> juliank:  ditto security/embargo
[14:33] <xnox> juliank:  i think you can reject ci-train requests; but not kernel/security team ppas.
[14:33] <juliank> we shrink the arm queues by 50 packages in 6 hours, this is not working
[14:33] <juliank> and we have no idea why
[14:34] -queuebot:#ubuntu-release- Unapproved: update-notifier (jammy-proposed/main) [3.192.52 => 3.192.53] (ubuntu-desktop, ubuntu-server)
[14:34] <xnox> juliank:  ..... wow.....
[14:34] <xnox> sounds like fun.
[14:34] <athos> Hi, could anyone from the SRU team take a look at https://bugs.launchpad.net/ubuntu/+source/postgresql-12/+bug/1961127 ?
[14:34] <ubottu> Launchpad bug 1961127 in postgresql-12 (Ubuntu Focal) "New upstream microreleases 10.20, 12.10, 13.6 and 14.2" [Undecided, In Progress]
[14:35] <juliank> xnox: ppc64el moves 20 times faster, but it only has 3 times as many tests exiting
[14:35] <juliank> xnox: hence the idea to kill ppa and upstream arm* tests to figure out what goes on
[14:35] <juliank> or really any requests
[14:35] <juliank> see if that makes things go down
[14:38] <sil2100> o/
[14:39] <sil2100> Let me unblock nautilus and I want to start looking at kicking the images
[14:40] <seb128> sil2100, thanks
[14:40] <seb128> sil2100, rico also wanted to see libreoffice in, seems like little risk
[14:41] <xnox> juliank:  but power is faster arch than our arm nodes.
[14:41] <xnox> ditto s390x
[14:41] <sil2100> seb128: I think I'll unblock that one as well, since we're anyway waiting
[14:41] <juliank> xnox: please read again. they are 3 times slower, but the queues shrink 20 times slower
[14:41] <sil2100> uh oh, grub2 still didn't migrate?!
[14:42] <juliank> xnox: so something must be adding hundreds of new arm64 autopkgtests every hour
[14:42] <juliank> (same for armhf)
[14:42] <seb128> sil2100, thanks
[14:43] <bdmurray> juliank: is there no log of test request submissions?
[14:43] <sil2100> Argh, the promotion
[14:43] <sil2100> Let me promote grub-efi-riscv64
[14:43] <juliank> bdmurray: not _really_
[14:43] <juliank> bdmurray: there's the apache access log
[14:43] <bdmurray> sil2100: You might look at the update-notifier I just uploaded
[14:44] <sil2100> Looking!
[14:44] <juliank> So update-github-jobs.service on the web workers is broken
[14:46] -queuebot:#ubuntu-release- Unapproved: intel-media-driver (jammy-proposed/universe) [22.2.1+dfsg1-1 => 22.2.1+dfsg1-1build1] (i386-whitelist, kubuntu)
[14:55] <seb128> could someone accept ^
[14:55] <seb128> the current binary is just not working, that's a no change rebuild
[14:55] <juliank> Furhter analys
[14:55] <apw> xnox, it looks like we are too close to the wire to achieve that to go into the beta; i've asked ppisati to get all the ducks fed and in a row just in case we need a respin on it (cc: sil2100)
[14:56] <juliank> 290 arm64 requests,  189 ppc64el requests
[14:56] <seb128> it seems the new libva is abi incompatible or something
[14:56] <juliank> this does not account for the difference in tests completed vs queue reduction
[14:57] -queuebot:#ubuntu-release- Unapproved: rejected yt-dlp [source] (focal-backports) [2022.03.08.1-1~bpo20.04.1]
[14:57] -queuebot:#ubuntu-release- Unapproved: accepted yt-dlp [source] (focal-backports) [2022.03.08.1-1~bpo20.04.1]
[14:58] -queuebot:#ubuntu-release- Unapproved: rejected gallery-dl [source] (focal-backports) [1.21.0-1~bpo20.04.1]
[14:58] <juliank> if ppc64el is 3 times faster, and gets the same amount of new tests as arm64; then arm64 queues should shrink 3 times slower than ppc64el, not 20 times
[15:00] -queuebot:#ubuntu-release- Unapproved: accepted gallery-dl [source] (focal-backports) [1.21.0-1~bpo20.04.1]
[15:02] <xnox> apw:  but with some of those packages blocked with freeze, i'm not sure it is easy to tell which ones need lrm respin (if any)
[15:03] <xnox> apw:  as far as i can tell, it's the ubuntu-release block that prevents migration; rather than missing lrm respins
[15:07] -queuebot:#ubuntu-release- Unapproved: r-cran-openssl (jammy-proposed/universe) [1.4.6+dfsg-1 => 1.4.6+dfsg-1ubuntu1] (no packageset)
[15:07] <sil2100> bdmurray: hm, so LP: #1966811 is basically crashing for people for no reason? Or only when they have oem packages installed?
[15:07] <ubottu> Launchpad bug 1966811 in update-notifier (Ubuntu Jammy) "list-oem-metapackages crashed with TypeError in any_oem_metapackages_installed(): 'apt_pkg.Cache' object is not iterable" [High, In Progress] https://launchpad.net/bugs/1966811
[15:07] <sil2100> bdmurray: I'll accept it into proposed, but not sure if it's something we'll be able to pull into the Beta. I guess it doesn't break the installation, right?
[15:08] -queuebot:#ubuntu-release- Unapproved: accepted update-notifier [source] (jammy-proposed) [3.192.53]
[15:09] -queuebot:#ubuntu-release- Unapproved: accepted intel-media-driver [source] (jammy-proposed) [22.2.1+dfsg1-1build1]
[15:10] <seb128> sil2100, wasn't that bug fixed in the update-notifier updated that migrated yesterday?
[15:10] <bdmurray> seb128: not completely
[15:10] <bdmurray> sil2100: It looks like list-oem-metapackages is called by the update-notifier applet
[15:11] <seb128> sil2100, also unsure if you saw the ping a bit early but could you accept intel-media-driver from the unapproved queue? seems like libav got an ABI change which broken the current binary, that's a no change rebuild and the package is in universe
[15:11] <sil2100> seb128: I did! As seen above!
[15:12] <seb128> sil2100, ah, lol, as I was typing... thanks!
[15:12] <sil2100> Ok, let me handle cd-boot-images-riscv64 as we have some changes needed on top
[15:13] <xnox> sil2100:  i just uploaded one to the queue, with just a rebuild. also pushed to repo. not sure what other things you need for it.
[15:13] <xnox> sil2100:  or do you just need the rebuild?
[15:17] -queuebot:#ubuntu-release- Unapproved: cd-boot-images-riscv64 (jammy-proposed/main) [1 => 3] (no packageset)
[15:18] <sil2100> xnox: all good!
[15:19] -queuebot:#ubuntu-release- Unapproved: rejected cd-boot-images-riscv64 [source] (jammy-proposed) [2]
[15:19] -queuebot:#ubuntu-release- Unapproved: accepted cd-boot-images-riscv64 [source] (jammy-proposed) [3]
[15:19] <sil2100> Basically xypron had some changes made that we need but used the debdiff approach, but I just put that on top of your changes
[15:21] <xnox> sil2100:  cool, commit it to repo as well
[15:21] <xnox> sil2100:  i pushed my changes there.
[15:21] <sil2100> xnox: all done!
[15:22] <sil2100> bdmurray: so you had concerns regarding the auto-accept script being outdated, yes?
[15:22] <sil2100> You think it might be risky to turn it on in the state it is now?
[15:23] <bdmurray> sil2100: No, I just wanted to make sure it really was the right script.
[15:25] <sil2100> I'll double-check just in case - I'm sure it's the right one, although I never really switched it on. But I think I've been seeing some packages people would normally think should get auto-accepted not being auto-accepted
[15:25] <sil2100> So maybe the list was a bit outdated since a while
[15:27] <laney> You just uncomment the auto-accept line in ubuntu-archive@snakefruit's crontab
[15:27] <laney> hi :-)
[15:28] <bdmurray> laney: I was put off by this https://pastebin.ubuntu.com/p/rR8w6RHFMC/
[15:28] <laney> bdmurray: what about it?
[15:29] <laney> apart from being a bit out of date, shouldn't be harmful
[15:29] <laney> I think you could clear out SEED_WHITELIST and PACKAGE_WHITELIST
[15:29] <bdmurray> laney: okay, the out of dateness made me wonder if it was the right thing
[15:31] <laney> those were old exemptions. seeded packages are not auto-accepted with the exception of things mentioned there
[15:31] <laney> but all seem obsolete so it'd probably be clearer to future travellers like yourself to remove them
=== mfo_ is now known as mfo
[15:53] <sil2100> bdmurray: can you uncomment the auto-accept line in the crontab then ;) ?
[15:55] -queuebot:#ubuntu-release- Unapproved: cinder (impish-proposed/main) [2:19.0.0-0ubuntu3 => 2:19.0.0-0ubuntu4] (openstack)
[16:01] <bdmurray> sil2100: will do
[16:01] <sil2100> Thanks o/
[16:01] <bdmurray> it also seemed weird that the mail from it goes to stgraber
[16:02] <sil2100> It's an old line
[16:02] <bdmurray> s/old/vintage/
[16:03] <coreycb> hello SRU team, I've just uploaded a new version of cinder to the impish unapproved queue that fixes a bug that will help unblock release of another project that depends on it. would love to get a review, thanks in advance.
[16:04] -queuebot:#ubuntu-release- Unapproved: ppa-purge (jammy-proposed/universe) [0.2.8+bzr63 => 0.2.8+bzr63-0ubuntu1] (ubuntu-budgie)
[16:08] -queuebot:#ubuntu-release- Unapproved: mesa (jammy-proposed/main) [22.0.0-0ubuntu2 => 22.0.0-0ubuntu3] (core, i386-whitelist, xorg)
[16:09] <tjaalton> ^ this fixes some apps crashing
[16:18] -queuebot:#ubuntu-release- Unapproved: elementpath (jammy-proposed/main) [2.3.0-1 => 2.5.0-1] (no packageset) (sync)
[16:18] -queuebot:#ubuntu-release- Unapproved: python-xmlschema (jammy-proposed/universe) [1.4.2-1 => 1.10.0-1] (no packageset) (sync)
[16:56] <bdmurray> sil2100: yes list-oem-metapackages is called by update-notifier regularly
[17:04] -queuebot:#ubuntu-release- Unapproved: edk2 (jammy-proposed/main) [2022.02-2 => 2022.02-3] (ubuntu-server) (sync)
[17:40] <sil2100> bdmurray: so basically will this error pop up for all users constantly after install?
[17:40] <sil2100> bdmurray: if that's the case, we might want to include that into the Beta as well...
[17:40] <sil2100> grub2, nautilus, cd-boot-images-riscv64 and libreoffice will go in in the nearest britney tick
[17:41] <sil2100> But if it's indeed as serious, we might want to get update-notifier in as well
[17:41] <sil2100> bdmurray: can you make sure it gets in ASAP? We can try pinging juliank for prioritizing its tests if needed
[17:41] <sil2100> Might need an unblock hint
[17:43] <Eickmeyer> sil2100: +1. I see that error *every time* I boot into Ubuntu Desktop.
[17:44] <sil2100> Then this is what one can call a 'release-blocking bug'!
[17:44] <sil2100> I'll commit the unblock hint
[17:45] <sil2100> Actually
[17:45] <sil2100> I know I'm probably drunk on power and overusing this right now, but I'll skiptest it!
[17:45] <Eickmeyer> hehehe
[17:46] <bdmurray> I did test the fix and also ran the amd64 autopkgtest
[17:46] <sil2100> Because this is literally a one-line change, and I know one-line changes can break as well - but we'll know instantly during beta testing if it regressed anything. If it regressed some tests, this is highly unlikely
[17:46] <sil2100> And then we'll just have to live with it
[17:47] <sil2100> Although, I need to re-run the ppc64el build as it FTBFSed
[17:47] <sil2100> uh, on some weird errors in the unit tests
[17:47] <sil2100> https://launchpadlibrarian.net/593747202/buildlog_ubuntu-jammy-ppc64el.update-notifier_3.192.53_BUILDING.txt.gz
[17:47] <sil2100> rerunning
[17:48] <Eickmeyer> Yeah, that doesn't look right. Temporary failure, it seems.
[17:49] <sil2100> Ok, hinted. Let's make sure the ppc64el build succeeds and then keep an eye out for it to migrate
[17:50] <sil2100> With this, image candidates should be building in uh, like 1-2 hours
[17:50] <sil2100> We're late, but still not terribly late
[17:50] <Eickmeyer> I've seen worse.
[17:51] <bdmurray> The ppc64el build finished already
[17:51] -queuebot:#ubuntu-release- Unapproved: cctbx (jammy-proposed/universe) [2021.12+ds1-2ubuntu1 => 2021.12+ds1-4] (no packageset) (sync)
[18:10] -queuebot:#ubuntu-release- Unapproved: softhsm2 (jammy-proposed/universe) [2.6.1-2build1 => 2.6.1-2ubuntu1] (i386-whitelist)
[19:05] -queuebot:#ubuntu-release- Unapproved: accepted cinder [source] (impish-proposed) [2:19.0.0-0ubuntu4]
[19:55] <juliank> To share this publicly
[19:55] <juliank> arm64 diff between 1737 and 2142 CEST: https://paste.ubuntu.com/p/WtkpqqjmyQ/
[19:55] <juliank> grafana shows arm64 queue change from 8.94K to 8.92K
[19:56] <juliank> aka 30 tests passed
[19:56] <juliank> diffing queues.json we get 9242-900=242 tests less in the queues
[20:01] <juliank> ok and ubuntu metrics dashboard: https://ubuntu-release.kpi.ubuntu.com/d/yIC34LpGk/ubuntu-metrics?viewPanel=4&orgId=1&from=1648568220000&to=1648582920000 has the right queue sizes
[20:03] <juliank> sorry people
[20:03] <ahasenack> is this good news or bad news?
[20:03] <ahasenack> I mean, I can tell the queue is huge
[20:03] <juliank> this is um good news, the queue size is actually going down at a decent
[20:03] <ahasenack> but did it get better, even though slowly?
[20:03] <juliank> rate
[20:03] <ahasenack> ok
[20:04] <juliank> 10 times better than the graph showed
[20:04] <ahasenack> sounds great
[20:05] <sil2100> hmmmm, still the britney run didn't finish
[20:06] <sil2100> Guess britney just has a LOT to do
[20:07] <juliank> so what happened is I changed the query to use max("count") instead of sum("count"), as sum("count") produces wrong results if you group by an interval larger than the reporting period
[20:07] <juliank> e.g. you have 10k tests pending at 10 minutes, and 10k tests pending at 20 minutes, and your reporting period is 20 minutes, you get 20k tests reported for that time.
[20:07] <juliank> oh I think I see what's going on
[20:07] <juliank> It was reporting progress for the *largest* queue, that is, the huge queue only
[20:08] <juliank> wonder how one would fix that
[20:08] <juliank> essentially we need to sum up the queues per arch, but then pick the maximum (or median) in the report period
[20:09] <juliank> but this does mean the huge queue is going slowly
[20:09] <juliank> but that's to be expected as arm* have normal queues still
[20:09] <juliank> or had
[20:14] <sil2100> grrr, update-notifier will only migrate with the next britney run
[20:14] <sil2100> I was too late with the hint
[20:14] <sil2100> Oh well
[20:14] <ahasenack> I uploaded samba on friday, and it's ready to migrate, will the release team go over each package in such state and make a decision, or only if I ping explicitly about this, or something else?
[20:16] <juliank> ahasenack: if it is seeded, you need the new version on the install media, ping
[20:16] <ahasenack> postgresql-14 ftbfs fix was also uploaded on friday, but still has dep8 tests to run in non-amd64 arches
[20:16] <juliank> ahasenack: I'm not sure samba or postgresql matter much for install media testing in the beta
[20:17] <bdmurray> sil2100: I think update-notifier just migrated
[20:29] <sil2100> Oh shit
[20:29] <sil2100> Was probably the first thing on the new britney run
[20:30] <sil2100> bdmurray: let's wait for rmadison to say it's fully published and I'll be spinning the candidate images
[20:31] <ahasenack> juliank: right, doesn't matter for media testing
[20:31] <ahasenack> I was considering a wider audience using them, that's all, but apt-get post-install can do that
[20:33] -queuebot:#ubuntu-release- Unapproved: accepted horizon [source] (focal-proposed) [3:18.3.5-0ubuntu2]
[20:35] <bdmurray> coreycb: It'd be good to bug 1962582 verified
[20:35] <ubottu> Bug 1962582 in horizon (Ubuntu Impish) "[SRU] Xena stable releases" [Undecided, Fix Committed] https://launchpad.net/bugs/1962582
[20:41] <coreycb> bdmurray: yes, we need to get that done
[20:45] <sil2100> Ok, kicking first beta candidates!
[20:49] <juliank> armhf now has 33% more workers again, they accidentally got stuck stopped
[20:49] <juliank> will try to improve resilience tomorrow
[20:49] <juliank> but there's a kernel bug on armhf where lxd containers leave around unkillable [defer] processes
[20:50] <juliank> basically the hosts running lxd then need to be hard-reset
[20:50] <juliank> it's a bit annoying
[20:53] <sil2100> bdmurray, vorlon: triggered most of the Beta RC builds via tracker, all besides the server images - I'll do those from ancientminister as I want to build the riscv64 installer image as part of the set (but separately, so that all the other arches don't have to wait for the riscv64 installer livefs to finish)
[20:55] -queuebot:#ubuntu-release- Unapproved: rejected ceph [source] (focal-proposed) [15.2.16-0ubuntu0.20.04.1]
[20:56] <bdmurray> sil2100: ack
[20:59] -queuebot:#ubuntu-release- Unapproved: rejected iio-sensor-proxy [source] (focal-proposed) [2.8-1ubuntu2]
[21:02] <bdmurray> sil2100: Are you retrying the failures?
[21:08] -queuebot:#ubuntu-release- Unapproved: accepted lxd [source] (bionic-proposed) [3.0.3-0ubuntu1~18.04.2]
[21:27] <sil2100> Argh
[21:27] <sil2100> I will now, but the fudge
[21:28] <bdmurray> I can do it too if you want to bail
[21:39] <sil2100> bdmurray: did they basically all fail?
[21:41] <bdmurray> sil2100: yeah
[21:46] <sil2100> bdmurray: hm, I see that possibly most of the amd64 ones have failed to build. Do you know if there's an easy way to only get rid of those from the build queue?
[21:46] <sil2100> Since `rebuild-requests -c jammy iso` would cancel all the requests
[21:47] <sil2100> And I guess we're not yet sure if the other arches have failed, right?
[21:49] <bdmurray> sil2100: No, I don't know of an easy way. Could we check the livefs builds for the other arches?
[21:50] <sil2100> bdmurray: I see that all ubuntu-base tarballs have succeeded (riscv64 still building though)
[21:51] <sil2100> Xubuntu amd64 succeeded as well, Lubuntu as well
[21:51] <sil2100> ARGH
[21:52] <bdmurray> So many flavors it's like Baskin Robbins
[21:52] <sil2100> We have a problem...
[21:53] <sil2100> I suspect that vorlon's recent change to debian-cd to rip-out unused series has broken debian-cd :|
[21:53] <sil2100> This is why we got no indication of success of those builds like Lubuntu and Xubuntu
[21:53] <sil2100> ===== Building Lubuntu daily CDs =====
[21:53] <sil2100> Tue Mar 29 21:11:07 UTC 2022
[21:53] <sil2100> ./build_all.sh: 95: ./CONF.sh: Syntax error: redirection unexpected (expecting ")")
[21:54] <sil2100> And I see that CONF.sh has been touched recently by removing a lot of the old series from the case statement
[21:54] -queuebot:#ubuntu-release- Builds: Ubuntu Desktop arm64+raspi [Jammy Beta] (20220329) has been added
[21:54] <sil2100> ^ preinstalled work as they do not invoke debian-cd
[21:55] -queuebot:#ubuntu-release- Unapproved: ubuntu-mate-artwork (jammy-proposed/universe) [22.04.11 => 22.04.12] (ubuntu-mate)
[21:57] <Wimpy> sil2100: If you could accept and unblock ubuntu-mate-artwork 22.04.12 I'd appreciate it
[21:58] <Wimpy> Oh. Sorry. Just read the back scroll.
[21:59] <sil2100> bdmurray: aaaargh, so it seems vorlon's changes caused a conflict in the debian-cd branch on ancientminister and there's basically conflict tags in the CONF.sh now
[22:00] <sil2100> bdmurray: ok, let me cancel all the jobs then and start fresh
[22:08] <sil2100> bdmurray: ok, cancelled, will now re-run those
[22:08] <sil2100> Conflict fixed
[22:09] <sil2100> bdmurray: so the situation is as follows:
[22:11] <sil2100> Basically due to the debian-cd conflict, none of the installer images could succeed. So now I re-triggered all of those via tracker. ubuntu-base needs no re-triggering as those have passed and are just waiting for riscv64. Server images are still building via ancientminister cmdline, and they should pass as they build sequentially: first preinstalled, then subiquity normal arches and then
[22:11] <sil2100> subiquity riscv64
[22:11] <sil2100> So the subiquity ones should just finish when the time is right
[22:12] <sil2100> The riscv64 image might still fail, as I was testing some stuff on those, but I'll take care of it tomorrow
[22:12] <bdmurray> ack and if things fail (due to the networking issues) what is the right way to retry the build?
[22:14] <sil2100> hm, I think basically what you can do is just run `rebuild-requests -c jammy iso` on ancientiminster, clear out all of the pending rebuilds and trigger those that need re-triggering again via the tracker
[22:15] <sil2100> Since the rebuild requests only matter at the beginning. If a build started and the rebuild request that triggered it gets cancelled, it changes nothing as the build is already started
[22:15] <sil2100> So once you're sure all of those actually started, you can clear them all out and just retrigger the ones that failed
[22:15] <bdmurray> what do you mean by "clear them out"?
[22:39] -queuebot:#ubuntu-release- Builds: Kubuntu Desktop amd64 [Jammy Beta] (20220329.1) has been added
[22:39] -queuebot:#ubuntu-release- Builds: Ubuntu Kylin Desktop amd64 [Jammy Beta] (20220329.1) has been added
[22:39] -queuebot:#ubuntu-release- Builds: Xubuntu Desktop amd64 [Jammy Beta] (20220329.1) has been added
[22:39] -queuebot:#ubuntu-release- Builds: Lubuntu Desktop amd64 [Jammy Beta] has been updated (20220329.1)
[22:42] -queuebot:#ubuntu-release- Unapproved: golang-github-hhatto-gorst (jammy-proposed/universe) [0.0~git20181029.ca9f730-2 => 0.0~git20181029.ca9f730-2ubuntu1] (no packageset)
[22:44] -queuebot:#ubuntu-release- Builds: Ubuntu Desktop amd64 [Jammy Beta] (20220329.1) has been added
[22:44] -queuebot:#ubuntu-release- Builds: Ubuntu MATE Desktop amd64 [Jammy Beta] (20220329.1) has been added
[22:47] -queuebot:#ubuntu-release- Builds: Ubuntu Studio DVD amd64 [Jammy Beta] (20220329.1) has been added
[22:59] <Wimpy> bdmurray: Please can you accept and unblock ubuntu-mate-artwork 22.04.12
[23:05] <bdmurray> Wimpy: okay
[23:07] -queuebot:#ubuntu-release- Unapproved: accepted ubuntu-mate-artwork [source] (jammy-proposed) [22.04.12]
[23:09] <Wimpy> Thanks bdmurray
[23:38] -queuebot:#ubuntu-release- Builds: Ubuntu Base amd64 [Jammy Beta] (20220329) has been added
[23:38] -queuebot:#ubuntu-release- Builds: Ubuntu Base arm64 [Jammy Beta] (20220329) has been added
[23:38] -queuebot:#ubuntu-release- Builds: Ubuntu Base armhf [Jammy Beta] (20220329) has been added
[23:38] -queuebot:#ubuntu-release- Builds: Ubuntu Base ppc64el [Jammy Beta] (20220329) has been added
[23:38] -queuebot:#ubuntu-release- Builds: Ubuntu Base riscv64 [Jammy Beta] (20220329) has been added
[23:38] -queuebot:#ubuntu-release- Builds: Ubuntu Base s390x [Jammy Beta] (20220329) has been added