[00:09] oh heh i wrote this too oops [00:19] vorlon: is there a more up to date version of https://people.canonical.com/~vorlon/armhf-time_t/ftp-override-map.txt for ubuntu somewhere? [00:37] mwhudson: no [00:38] sorry, that was being maintained as input to the ftp team specifically, and any variations from that would basically need to be looked at on a case-by-case basis at this point (e.g. is this a completely new soname with no revdeps yet in Ubuntu so we don't need to rename? is it an Ubuntu-specific package? etc) [00:39] nuts, notest showing i386 again [00:39] afk again [00:39] vorlon: yes fair enough [00:54] uh why is there no entry in Sources for sagemath?? [00:57] uh i should probably try not to recreate britney here [01:34] did you mean to ask "Why are we still shipping sagemath"? :) [01:35] mwhudson: race condition vs a deletion by doko https://launchpad.net/ubuntu/+source/sagemath/+publishinghistory [01:41] a little NBS cleanup for libcanberra, hopefully cinnamon will be a candidate now [01:45] vorlon: ah hah [01:45] ok i think i have uploaded all the packages with a build-dep that has been replaced in the t64 transition now [01:46] probably not maximally efficiently [01:49] adsys/armhf deleted and it doesn't have a build-dep on sssd only a runtime dep and I'm not worrying about whether it comes back [01:49] libsss-nss-idmap-dev staaaaaahp [01:50] ok sssd armhf binaries on their way out [02:38] ok well I did a no-change rebuild of cups-browsed locally and it fixed the segfault. But the no-change rebuild in launchpad still segfaults... [02:51] ah it's apparmor [02:51] LP: #2058866 [02:51] -ubottu:#ubuntu-release- Launchpad bug 2058866 in cups-browsed (Ubuntu) "proposed-migration for cups-browsed 2.0.0-0ubuntu8" [Undecided, New] https://launchpad.net/bugs/2058866 [02:52] checking now if it's also reproducible in an amd64 container (since that's also an armhf difference) === chris14_ is now known as chris14 [04:31] something we can just remove https://bugs.launchpad.net/ubuntu/+source/threadscope/+bug/2058878 [04:31] -ubottu:#ubuntu-release- Launchpad bug 2058878 in threadscope (Ubuntu) "please remove threadscope from ubuntu" [Undecided, New] [04:31] ubuntu-archive: something we can just remove https://bugs.launchpad.net/ubuntu/+source/threadscope/+bug/2058878 [05:12] mwhudson: I know no-one seems to be caring much about it, but if you _do_ just relax the build depends, does it build? [05:12] RAOF: no idea [05:15] oh it only takes 2 mins to build, let's find out [05:18] RAOF: no [05:18] Bam. [05:18] rmadison knows ssreflect bins are removed on riscv64, why doesn't britney >:| [05:18] is there some kind of fashion for putting upper bounds on your deps in haskell? [05:19] haskell generally uses ABI hashes in virtual packages, or do you mean build-deps? [05:19] build deps [05:19] no idea [05:19] RAOF: if i remove the other build dep constraint it appears to build... [05:21] uploading [05:21] Huh. I ask because it seems to have been in and out of testing quite a bit, but it _looks_ like the sort of package that might have a quiet userbase. [05:23] tpm2-abrmd fails because of open/open64 mocking games on armhf, probably sensible to skip tests there [05:24] (i'm about to have dinner though) [05:25] skip tests, rather than remove armhf binaries from release pocket and fix later? [07:19] vorlon: well i don't know. strongswan is in the rdep chain [07:25] uploaded a version with tests skipped [07:59] mwhudson: only as a test revdep [08:03] britney seems stuck [08:10] VM is being squirrely, rebooting [08:16] back up, britney will start again [08:26] I ... should only show builds that actually affect the architecture, lots of arch: all only builds [08:26] For the new reverse-build-depends feature [09:31] sil2100: t64-related RM: LP: #2058912 [09:31] -ubottu:#ubuntu-release- Launchpad bug 2058912 in xwiimote (Ubuntu) "t64: please remove xwiimote and rdeps armhf binaries from noble" [Undecided, New] https://launchpad.net/bugs/2058912 [09:33] sil2100: erf, nevermind, it seems they're already removed. juliank I pulled this from one of your report ( https://magenta.jak-linux.org/ubuntu-archive/distcheck/noble.armhf/global-ben.reverse.txt ) would it be possible to amend it not to consider these cases? [09:39] ...! [09:40] Sorry about that :) Live and learn [09:46] schopin: hmm [09:48] schopin: it's still there? xwiimote | 2-4 | noble/universe | source, amd64, arm64, armhf, ppc64el, riscv64, s390x [09:50] juliank: I didn't see the file in the "Package Files" section of LP so assumed it was gone, but now I see it? [09:51] Doing too many things at once :/ [10:11] schopin: just use rmadison :) [10:29] ftr: amd64 queue is being cleaned up [10:58] ubuntu-archive: remove-package -a armhf -b osmo-bts osmo-pcu # leaf; ftbfs in armhf; blocking libosmoabis10 libosmocodec0 libosmocoding0 libosmocore19 libosmoctrl0 libosmogsm18 libosmotrau2 libosmovty9 [10:58] Well I guess osmo-bts is currently in progress building but still we can drop it for now [10:59] global-ben.reversed.txt says: src:osmo-bts: libosmocoding0 - needs a sourceful upload anyhow [11:00] Removing it first makes the transition view easier though [11:04] after the amd64 queue being cleaned up, is there script to requeue all the missing tests? i see packages being blocked from migration due to pending results on amd64, but no job is queued. [11:05] queue is being cleaned up be patient [11:05] zhsj: andersson1234 is requeuing the tests that were there [11:05] ack [11:05] zhsj: some tests were purposefully dropped, so rerunning all would be bad [11:14] requeuing will take about 4 hours due to some process issues [11:19] -queuebot:#ubuntu-release- New sync: piuparts (noble-proposed/primary) [1.4.1] [11:19] -queuebot:#ubuntu-release- New: accepted piuparts [sync] (noble-proposed) [1.4.1] [11:21] -queuebot:#ubuntu-release- New binary: piuparts [amd64] (noble-proposed/none) [1.4.1] (no packageset) [11:22] -queuebot:#ubuntu-release- New binary: piuparts [arm64] (noble-proposed/none) [1.4.1] (no packageset) [11:23] -queuebot:#ubuntu-release- New binary: piuparts [ppc64el] (noble-proposed/none) [1.4.1] (no packageset) [11:23] -queuebot:#ubuntu-release- New binary: piuparts [armhf] (noble-proposed/none) [1.4.1] (no packageset) [11:23] -queuebot:#ubuntu-release- New binary: piuparts [s390x] (noble-proposed/none) [1.4.1] (no packageset) [11:25] I got +q on #ubuntu-devel sigh [11:29] -queuebot:#ubuntu-release- New: accepted piuparts [amd64] (noble-proposed) [1.4.1] [11:29] -queuebot:#ubuntu-release- New: accepted piuparts [armhf] (noble-proposed) [1.4.1] [11:30] -queuebot:#ubuntu-release- New: accepted piuparts [s390x] (noble-proposed) [1.4.1] [11:30] -queuebot:#ubuntu-release- New: accepted piuparts [arm64] (noble-proposed) [1.4.1] [11:30] -queuebot:#ubuntu-release- New: accepted piuparts [ppc64el] (noble-proposed) [1.4.1] [11:39] juliank, removed [11:40] -queuebot:#ubuntu-release- New binary: piuparts [riscv64] (noble-proposed/none) [1.4.1] (no packageset) [11:42] seb128: ta [12:14] -queuebot:#ubuntu-release- New: accepted piuparts [riscv64] (noble-proposed) [1.4.1] [12:35] mwhudson: left a question in the threadscope removal bug [13:26] juliank: your rustc uploads failed to build. do you know if we can remove some old versions? [13:36] doko: I do not know, we should ask our Rust person I suppose [13:37] * rustc-1.74 build-depends on conflicting packages: libllvm17t64=1:17.0.6-9build2 vs libllvm17=1:17.0.6-5build1 [13:37] - chain 1: libllvm17t64:armhf (>= 1:17.0.2) [13:38] - chain 2: rustc-1.73:armhf -> libllvm17:armhf [13:38] Oh god [13:38] So the rustc compilers build-depend on the older ones so they go bootstrap recursively [13:39] What about Ada? [13:39] 3 packages have unsat BDs [13:39] frog, libgnatcoll-bindings [13:39] oh that's just two [13:39] also we have nfs-ganesha build-depends on conflicting packages: liburcu8t64=0.14.0-3.1 vs liburcu8=0.14.0-3 [13:39] I'll go look a bit [13:40] libucto5t64 needs rebuild against libticcutils8t64, curious [13:41] Somehow did not see that [13:42] global-ben doesn't see it [13:43] but the Ada ones, weird [13:44] Oh I guess those are provides [13:46] ok so libxmlada-unicode-dev needs a rebuild [13:47] ah wait wrong arch I look at [13:47] gnat-13-cab06c16 is old [13:47] gnat-13-2eaa09e6 is new [13:52] juliank: yes, that should only be a armhf change [13:52] libgnatcoll-bindings removed on armhf [13:58] ubuntu-archive: removal for pyfltk https://bugs.launchpad.net/ubuntu/+source/gl-image-display/+bug/2058932 [13:58] -ubottu:#ubuntu-release- Launchpad bug 2058932 in pyfltk (Ubuntu) "RM: pyfltk and rdeps, FTBFS" [Undecided, New] [13:59] doko: I did just upload no-change rebuild for libgnatcoll I believe it though [14:00] which promptly failed with gprbuild: raised CONSTRAINT_ERROR : a-calend.adb:371 overflow check failed [14:00] on armhf [14:02] I thought ada had a different time type with more constraints and range :/ [14:03] heh [14:10] -queuebot:#ubuntu-release- New source: nemos-dev-key (noble-proposed/primary) [1.7] [14:10] -queuebot:#ubuntu-release- Unapproved: octavia (jammy-proposed/universe) [1:10.1.0-0ubuntu1 => 1:10.1.1-0ubuntu1] (openstack) [14:14] juliank: on it [14:15] juliank: I guess removing mrcal as well, right? Since it's a recommends [14:16] And I suppose only the armhf binaries, right? [14:16] Or do you want the whoel thing gone? [14:18] -queuebot:#ubuntu-release- New binary: dante [amd64] (noble-proposed/universe) [1.4.2+dfsg-7.1~willsync1] (no packageset) [14:19] -queuebot:#ubuntu-release- New binary: dante [arm64] (noble-proposed/universe) [1.4.2+dfsg-7.1~willsync1] (no packageset) [14:19] -queuebot:#ubuntu-release- New binary: dante [s390x] (noble-proposed/universe) [1.4.2+dfsg-7.1~willsync1] (no packageset) [14:19] -queuebot:#ubuntu-release- New binary: dante [ppc64el] (noble-proposed/universe) [1.4.2+dfsg-7.1~willsync1] (no packageset) [14:20] -queuebot:#ubuntu-release- New: rejected nemos-dev-key [source] (noble-proposed) [1.7] [14:21] -queuebot:#ubuntu-release- Unapproved: motion (mantic-proposed/universe) [4.5.1-3build1 => 4.5.1-3ubuntu0.1] (no packageset) [14:22] pyxplot is missing builds: they failed on all arches, with "*** buffer overflow detected ***: terminated"; I'd like to retry the builds but I don't see a way to do it (now?), is that something that can be triggered? [14:22] I saw that error a few days ago already but I have no idea if it was for the same package so I'd like an up-to-date one (I'm also going to build it locally) [14:23] pyxplot is one of the tests blocking readline, specifically on armhf [14:23] -queuebot:#ubuntu-release- Unapproved: motion (jammy-proposed/universe) [4.3.2-1 => 4.3.2-1ubuntu0.1] (no packageset) [14:34] -queuebot:#ubuntu-release- Unapproved: livecd-rootfs (jammy-proposed/main) [2.765.40 => 2.765.41] (desktop-core, i386-whitelist) [14:36] dbungert: mini-iso-tools now depends on dhcpd (universe). don't we have alternatives already in main? [14:39] doko: I typoed the depends then. I'll fix it. [14:48] So there is a bug in autopkgtest causing it not to pick the right version of the package when package is the trigger [14:48] which causes britney to loop and reschedule tests all the time [14:48] for example, https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/i386/n/neutron-vpnaas/20240325_110427_a8513@/log.gz [14:55] -queuebot:#ubuntu-release- New binary: dante [riscv64] (noble-proposed/universe) [1.4.2+dfsg-7.1~willsync1] (no packageset) [15:03] / [15:03] l/ [15:03] * apw orders a new keyboard [15:08] juliank, doko: remove old rusts: we did talk with the kernel team and they are meant to go away [15:09] sil2100: "since it's a recommends" we don't remove packages for unsatisfied recommends... and are you documenting removal of the reverse-depends in the sync-blocklist repo (extra-removals) or is this not needed? [15:12] ok, removing everything before 1.74 [15:13] getting armhf on https://ubuntu-archive-team.ubuntu.com/proposed-migration/update_output_notest.txt, that's convenient [15:13] doko: uh this only applied to 1.62 and 1.68 which were there for kernels, I don't know if there's another reason to keep 1.73, please double-check this with Zixing first [15:14] vorlon: merge my critical britney fix before doing other stuff [15:14] :D [15:14] this one https://code.launchpad.net/~juliank/britney/+git/britney2-ubuntu/+merge/463029 [15:14] I haven't told people here but I saw 16 retries for the same package issued by britney [15:15] tl;dr: autopkgtest does not correctly fetch source packages from proposed when triggered with proposed version; paride knows about the bug [15:17] OOC is this expected to retry all non-armhf autopkgtest with all-proposed=1 (iff one has not already been triggered)? [15:18] * tsimonq2 crosses his fingers that it'll help with qtbase-opensource-src :) [15:19] -queuebot:#ubuntu-release- Unapproved: xmms2 (mantic-proposed/universe) [0.8+dfsg-23build1 => 0.8+dfsg-23ubuntu0.1] (no packageset) [15:21] -queuebot:#ubuntu-release- Unapproved: xmms2 (jammy-proposed/universe) [0.8+dfsg-22build3 => 0.8+dfsg-22ubuntu0.1] (no packageset) [15:27] vorlon: added `"migration-reference/0" not in triggers and` line to not add all-proposed for migration-reference/0 ones :) [15:29] juliank: thanks [15:39] vorlon: So we know know that it's the combination of throughput and britney requeuing the same I don't know, 100 tests every run that's making amd64 go sideways [15:40] And QA team is cowboying some stuff I think to make sure that workers can't loop anymore [15:42] We have run my script today on all queues so they are all 2-3 times smaller :) [15:42] I believe all those "rust-*" packages failed on any architecture could be removed. Most Ubuntu packages use vendored source anyways [15:43] $ LANG=C join -j1 <(LANG=C sort /tmp/armhf-new-uninstallables.txt) <(LANG=C sort /tmp/julian-recommends-removing.txt) | wc -l [15:43] 1334 [15:43] $ [15:43] kill them [15:43] juliank: this doesn't look at all armhf binaries from a given source package, AIUI [15:43] correct [15:43] so that won't unblock migration for all of them [15:43] but that is an acceptable tradeoff at this point IMO [15:44] but I don't want to just take all of the binaries in the list, map to source packages, and remove all binaries [15:44] Just drop the binaries and see what happens? [15:44] I will [15:44] I'm just pointing out the weakness [15:45] TIL join, I always use comm -12 [15:45] Also if you want to see uninstallabilities on armhf in britney, just patch britney "random" architecture to armhf ;) [15:46] of course [15:46] but as mentioned, the !armhf ones also matter [15:47] particularly because those don't get fixed by armhf binary removals [15:47] true [15:48] ok churning through these binary removals now [15:48] I have a list of uninstallabilities on amd64 by dose-distcheck in https://magenta.jak-linux.org/ubuntu-archive/distcheck/noble/proposed.txt [15:48] no wait this is build-depends [15:48] I can create binary depends though [15:49] yeah I don't care about build-depends today :) [15:49] It's interesting britney findsd 1334 uninstallabilites on armhf [15:49] dose-distcheck has like 20 [15:49] But it does proposed + release [15:50] I'd have to construct a Packages file after migration to make it see a more realistic post-migration scenario [15:50] no, britney finds *3600* uninstallablilities on armhf, when attempting the migration [15:50] 1334 is the intersection of these with the things you've identified as leaves [15:51] well yes [15:51] construct a Packages file> so you mean the old implementation of update-output-helper that I rewrote, heh [15:51] hhe [15:53] alright, so churning through those removals now; gotten to 'c' [15:53] (calling for removals one at a time because of the O(n^2) issue I mentioned to you before) [15:57] -queuebot:#ubuntu-release- Unapproved: ngspice (jammy-proposed/universe) [36+ds-1 => 36+ds-1ubuntu0.1] (no packageset) [16:16] juliank: and your identification of leaf packages on https://magenta.jak-linux.org/ubuntu-archive/distcheck/noble.armhf/removal-candidates.with-proposed.txt excluded seeded packages, right? we might want to revisit that [16:17] vorlon: that is correct, yes [16:17] mostly to avoid ubuntu-meta getting deleted [16:17] heh [16:17] if anything, desktop flavors might appreciate not having their beta at risk because of their packages being out of date on armhf [16:17] I can make it check if ubuntu-${task} == name or something [16:17] * LocutusOfBorg brings a sledgehammer to vorlon for britney [16:18] I just want the key packages of the task really [16:18] right [16:18] i.e. the meta packages [16:18] but for all flavors not just ubuntu-* [16:19] should I just ignore section: metapackages maybe? [16:19] hmm let's see [16:22] vorlon: I now check "ubuntu-"+ task or task, which I think catches all of them [16:22] vorlon: I think it's only the mimimal task that becomes ubuntu-minimal? [16:22] For the others, task == name [16:23] vorlon: please do have a look at the file now and see if it has anything bad [16:24] probably should have diffed it [16:25] but I assume you have a local copy :) [16:25] I guess if you join it with brintey, you can use https://magenta.jak-linux.org/ubuntu-archive/distcheck/noble.armhf/removal-candidates.txt [16:26] otherwise you will not see a package foo in the release pocket that gets broken by migrating bar if there is no foo in proposed? [16:28] what do you prefer for source removals? ask here directly or batch them? [16:29] adrien: source removals should have bug reports, unless it's a transient removal because the fix is already in -proposed [16:29] adrien: it can be an existing Debian bug report or a new one in Launchpad [16:30] sorry, I mean, after having create bug reports [16:30] (although I haven't checked yet on debian) [16:31] nothing reported debian-side; removal requests: https://bugs.launchpad.net/ubuntu/+source/pyxplot/+bug/2058940 and https://bugs.launchpad.net/ubuntu/+source/eso-midas/+bug/2058947 [16:31] -ubottu:#ubuntu-release- Launchpad bug 2058940 in pyxplot (Ubuntu) "pyxplot: Please RM due to FTBFS" [Undecided, New] [16:31] -ubottu:#ubuntu-release- Launchpad bug 2058947 in eso-midas (Ubuntu) "eso-midas: Please RM due to FTBFS" [Undecided, New] [16:31] adrien: ah - throw us the bug numbers here directly, the "normal" ubuntu-archive queue is not being looked at right now AFAIK [16:32] I'm not entirely happy as I'd prefer to investigate these because they seem similar but too little time and I guess it's the time of the year when we look at even more failures than usual [16:34] adrien: we all have the impulse to investigate and fix, but now is not the time for this :) [16:34] ah it looks like riscv64 has caught up on coq, only waiting on mathcomp-analysis and coq-unimath builds now AFAICS [16:34] :) [16:37] adrien: in general I prefer output from `reverse-depends src:$src` and `reverse-depends src:$src -a source` as part of the removal bug to clearly document what revdeps there are if any [16:38] ok, I can add that a bit later today (I need to afk a bit) [16:39] don't worry about it for these first 2, I've already confirmed locally and am removing [16:40] juliank: so confirming the semantics of https://magenta.jak-linux.org/ubuntu-archive/distcheck/noble.armhf/removal-candidates.with-proposed.txt it appears that packages are in this list if -proposed has a new version of the source package for the binary, but it's not guaranteed the binary exists yet on armhf. so some of these might be ftbfs in -proposed, the armhf binary removal will unblock them [16:40] for migration, and we'll lose track of the armhf build failures? [16:41] juliank: sorry, I meant ftbfs in -proposed on armhf only [16:41] vorlon: that's correct [16:41] well [16:41] They could ftbfs on other arches too, I don't check [16:41] juliank: ok. I'm currently going through and getting a list of binaries for which this is the case, and will file bugs [16:41] In this report I only check there is a newer source package [16:41] right [16:42] the first one on the list, 0ad, is in -proposed and ftbfs on all archs [16:42] There is one where I also check that were are missing binaries [16:42] so that's fine :) [16:42] But I don't have one that checks we *have* binaries, because those are not unblockable by removal [16:42] Well ok they might unblock *other* packages [16:44] -queuebot:#ubuntu-release- Unapproved: lcov (mantic-proposed/universe) [2.0-1ubuntu0.1 => 2.0-1ubuntu0.2] (i386-whitelist) [16:45] vorlon: Do you want that report too? [16:50] vorlon: thanks! [16:50] vorlon: Can we lock the archive now so we don't get new transitions uploaded? [16:51] sil2100: I did not see your pings but the removal requests were for source packages, and ignoring mrcal [16:51] juliank: ACK [16:52] I'll keep that in mind for next ones ;) [16:52] sil2100: If it's binary only I send remove-package -b commands with comments here [16:53] I feel like someone else asked for locking the archive already but can't find where :D [16:55] I think I'll ask QA to do queue cleanup tomorrow again [16:55] the amd64 queues are at 7683 items, but cleaned up we are at 7482 [16:55] this gets worse with every britney run so a daily purge and requeue sorts that out [16:58] -queuebot:#ubuntu-release- New binary: mathgl [amd64] (noble-proposed/universe) [8.0.1-7.1] (no packageset) [16:58] -queuebot:#ubuntu-release- New binary: mathgl [armhf] (noble-proposed/universe) [8.0.1-7.1] (no packageset) [17:00] -queuebot:#ubuntu-release- New binary: mathgl [ppc64el] (noble-proposed/universe) [8.0.1-7.1] (no packageset) [17:02] -queuebot:#ubuntu-release- New binary: mathgl [s390x] (noble-proposed/universe) [8.0.1-7.1] (no packageset) [17:11] -queuebot:#ubuntu-release- New binary: mathgl [arm64] (noble-proposed/universe) [8.0.1-7.1] (no packageset) [17:19] vorlon: sil2100 I propose we also merge https://code.launchpad.net/~juliank/britney/+git/britney2-ubuntu/+merge/463045 to britney for the next run(s) such that it stops queueing the same tests each run [17:19] juliank: ok, filing 124 update-excuse/time-t bugs for packages whose armhf binaries are being removed, and have built in -proposed but not on armhf [17:19] juliank: that report too> which report exactly? [17:20] This was the culprit here really I think, we trigger foo with foo/2 and get foo/1 result, result is silently ignored; but pending test was deleted so it gets requeued. [17:20] We have sicne requeued all tests with all-proposed but it takes ages for the queue to be done to the point that it did all the tests that britney is waiting on [17:20] vorlon: leaf packages that have a build in proposes [17:20] *d [17:21] juliank: lock the archive> at the moment I'm not aware of new transitions coming in that are a problem for us, and managing an unapproved queue is also overhead, for the moment I prefer to leave it as-is [17:21] There are two queues in NEW [17:21] * two transitions in NEW [17:21] juliank: I don't think that's useful as a separate report, really, since I'm post-processing the existing report to decide what to remove and then further post-processing that to see if I should file bugs [17:22] ok [17:22] I'm scared someone approves libevent or mathgl NEW binaries, it's fine they are there but we should only approve them after beta [17:22] then we can handle a handful extra transitions [17:23] juliank: I saw your other MP and I haven't worked out for myself if this will actually fix it. We're seeing cases of britney repeatedly queuing a test when there has been *no* new result since the previous request [17:24] vorlon: Yes I analysed the log and it refetches the same "new" results, because it doesn't "store" the result because it's for an old version [17:24] so if this is because we're seeing a result but it's invalid and then requeuing, shouldn't we filter them out earlier at the point where it detects these as "new" results? [17:25] vorlon: This is during the import stage from the db [17:25] before it analyzes them [17:25] ok [17:25] -queuebot:#ubuntu-release- Unapproved: rejected landscape-client [source] (focal-proposed) [23.02-0ubuntu1~20.04.2] [17:25] It's literally in fetch_sqlite_results helper functions :) [17:25] -queuebot:#ubuntu-release- Unapproved: rejected landscape-client [source] (jammy-proposed) [23.02-0ubuntu1~22.04.2] [17:27] I think I'm going to have to look at the code in context to understand [17:27] before committing it [17:28] as a general principle, "you got a result from the db but it's not what you wanted" tells me either the schema is wrong or the query is wrong, so I have to assess if I want to go down that rabbit hole right now ;) [17:28] that's true to some extent, we did not install version comparison function inside the table to put them in the query [17:29] and triggers are a string rather than their own table that you join [17:29] so it's hard to query `if any(trigger == name) and trigger_version == version` [17:29] can't even extract the trigger, let alone the version from it [17:30] or rather you'd want WHERE trigger != name OR trigger_version == version [17:30] but you get the idea [17:31] So the query only filters on series, architecture, package being tested, and last run id we imported (which we never update when we ignore new test result, hence you see same tests appear "new" each time) [17:31] It's only add_trigger_to_results() that actually validates the trigger [17:32] and checks that if trigger == package name that the package version is not older (I'd have tested for == but meh) [17:32] -queuebot:#ubuntu-release- Unapproved: landscape-client (jammy-proposed/main) [23.02-0ubuntu1~22.04.1 => 23.02-0ubuntu1~22.04.2] (ubuntu-server) [17:34] normally you do actually want it to retry rather than lose results so this is not a perfect patch [17:34] the perfect patch would fetch queued.json from autopgktest.u.c and check the queues whether the test is already queued [17:34] -queuebot:#ubuntu-release- Unapproved: landscape-client (focal-proposed/main) [23.02-0ubuntu1~20.04.1 => 23.02-0ubuntu1~20.04.2] (ubuntu-server) [17:35] juliank: don't try to "fox" rustc-* needs a build with a working rust version [17:35] but that is harder to write? [17:35] doko: not fixing any further [17:36] so we want to keep britney's own pending tests but additionally also check queued.json to avoid requeueing tests scheduled otherwise [17:36] but that's too hard this week [17:37] (we want to keep britney's own notion of what is pending in addition such that we can skip tests we don't want to run ;) [17:38] juliank: ok got it, yes will do a quick pass on the code and merge [17:41] vorlon: So I need to setup britney locally to run rather than just do pyflakes but then I can submit the full blown patch to consult queued.json before sending new tests [17:41] which is nicer than this, but this is minimal and easy to read [17:41] hmmm we don't already consult queued.json? terrible, terrible [17:42] also see internal side channel discussion about the fact that queued.json should be a sqlite db itself [17:43] vorlon: uh, I'd rather get rid of the sqlite db we have rather than add more [17:43] in favor of what [17:44] I think queued.json being a sqlite db is feasible but introduces more complexity where it may not be necessary [17:44] it should be a json dump of a postgres db I suppsose [17:45] what's the rationale behind making it a db? Where's this side channel discussion also, i missed it [17:45] I haven't fully thought about it :) [17:46] andersson1234: well queued.json is garbage [17:46] like literally garbage, because it doesn't get garbage collected [17:46] and the whole thing gets written out by britney after every single autopkgtest query or result [17:47] I don't follow, it is rendered by cache-amqp [17:48] and cache-amqp needs to visit the entire queue anyway [17:48] ah wait sorry [17:48] I was talking about a different .json without checking the name [17:48] You mean the pending test one from britney itself? [17:48] that one is eww [17:48] yes [17:49] state/autopkgtest-pending.jso [17:49] state/autopkgtest-pending.json [17:49] Still need it, because otherwise you can't drop tests from the queues, if britney only consults queued [17:49] yes, we need it but the fact that it's json is awful [17:49] Or I need to find force-skiptest hints before looking at scheduling tests [17:50] i.e. when scheduling a test, search force-skiptest hint (and cache that) [17:50] ah, okay, gotcha, thought u were talking about a-u-c, ignore me :) [17:50] every time britney schedules a test and there's an upload of a new version before the test result comes in, you get an orphaned entry [17:50] then you can do force-skiptest followed by dequeuing [17:50] and britney sees force-skiptest and doesn't queue any tests despite having no state of what it has queued itself [17:50] so autopkgtest-pending.json grows without bounds, except for when I once a cycle delete it while britney is quiesced [17:51] Does that sound reasonable? force-skiptest not queueing any tests? [17:51] It can be a bit expensive [17:51] eh I'm not going to spend the thought on that question right now [17:51] there's armhf to squash [17:51] jbicha: gst-python1.0 ftbfs, test failures. Debian added an extra b-d on pygobject >= 3.48.1-1.1. that's a version we don't have [17:51] I can make britney garbage collect the pending tests too :) [17:52] doko: there's an open update-excuse bug about this [17:52] https://bugs.launchpad.net/ubuntu/+source/gst-python1.0/+bug/2057836 [17:52] -ubottu:#ubuntu-release- Launchpad bug 2057836 in pygobject (Ubuntu) "gst-python1.0 fails to build: test failures" [High, Triaged] [17:52] I know because gst-python is in the list of packages I just removed armhf binaries for, and is built in -proposed on at least one arch (riscv64 because nocheck) [17:52] juliank: I don't want britney to garbage collect, I want a GC to be able to run in parallel [17:53] by, you know, having a database allowing multiple writers [17:53] instead of a json file that's a dump of britney's brain every minute [17:54] anyway, the round of armhf binary removals from release pocket is done, and 12 bugs left to file against the missing armhf binaries in -proposed [17:54] CD image lubuntu/noble/daily-live failed to build on 20240325 - FileNotFoundError: [Errno 2] No such file or directory: '/srv/cdimage.ubuntu.com/www/full/mantic/source/20231011.1' [17:54] uh wat === tsimonq2_ is now known as tsimonq2 [17:56] good question [17:57] I did some nuking of old source builds over the weekend [17:58] that particular path was a symlink I used to try to hack up building source images for mantic after they were oopsed out of existence [17:58] so that I could republish them to full/releases/mantic/release/source [17:58] have dropped the symlink now [17:59] Ah damn I think I wrote the britney <-> queued integration now, but I should ook at it when actually awake and filled with carbs: https://code.launchpad.net/~juliank/britney/+git/britney2-ubuntu/+merge/463049 [17:59] (hence still WIP) [18:00] vorlon, juliank, doko: please let me know where i should focus my attention today [18:00] yes queued.json containing nested json in a string is funny [18:00] I'll defer to vorlon [18:00] mwhudson: I'll defer to vorlon [18:00] (once i've had coffee and breakfast and done the school run) [18:01] I didn't have breakfast and I worked for 9 hours [18:01] :D [18:01] or lunch [18:01] ok I had a banana [18:01] mwhudson: general guidance I gave this morning, in the absence of any specific urgent delegations from me, is to treat it as one big +1 maintenance exercise now for packages uploaded < 25 days ago [18:02] I am hoping to have the underbrush cleared within a couple of hours on glib2.0 specifically [18:02] to be able to recommend specifics [18:02] * juliank is mostly doing sidequests of anaylsing and fixing infrastructure [18:03] why does update_output_notest now list ada crap that wasn't there yesterday [18:03] mwhudson: ^^ maybe you can answer me that [18:03] all the ada stuff is broken on armhf but i thought doko had removed [18:03] it [18:03] vorlon: thanks :) kicking off a Lubuntu retry to confirm [18:03] well the list of new uninstallables on arm64 is now: [18:03] * arm64: edubuntu-desktop, edubuntu-desktop-minimal, edubuntu-desktop-raspi, gdm3, gnome, gnome-core, gnome-remote-desktop, gnome-shell, gnome-shell-extension-alphabetical-grid, gnome-shell-extension-manager, gnome-shell-extension-prefs, gnome-shell-pomodoro, ibus-tests, libadasockets12-dev, libcoq-iris, libgmpada12-dev, libgnatcoll-gmp21-dev, libgnatcoll-iconv21-dev, libgnatcoll-lzma4-dev, [18:03] and none of the ada stuff was there yesterday [18:03] libgnatcoll-omp3-dev, libgnatcoll-postgres3-dev, libgnatcoll-python3-2-dev, libgnatcoll-readline21-dev, libgnatcoll-sql5-dev, libgnatcoll-sqlite21-dev, libgnatcoll-syslog5-dev, libgnatcoll-xref22-dev, libgnatcoll-zlib4-dev, libgnatcoll21-dev, libgnatprj10-dev, libmutter-13-0, libplplotada5-dev, librust-libslirp-dev, librust-ripcalc-dev, librust-transmission-client-dev, librust-zbus-1-dev, [18:03] libsoup-3.0-tests, libtemplates-parser16-dev, libxmlada-dom12-dev, libxmlada-input12-dev, libxmlada-sax12-dev, libxmlada-schema12-dev, libxmlada-unicode12-dev, ubuntu-desktop, ubuntu-desktop-minimal, ubuntu-desktop-raspi, ubuntu-gnome-desktop, vanilla-gnome-desktop [18:03] mwhudson: so run that to ground? [18:04] because it's a blocker even if we add armhf to broken arches [18:04] vorlon: ok [18:04] Oof, the raspi world broke. === NotEickmeyer is now known as Eickmeyer [18:05] and it looks like mutter regressed in migratability [18:07] actually it progressed and now other things are broken? [18:07] because this is an ABI change, yeah [18:08] doko: ^ I'm not sure why you uploaded mutter, nothing on my side said it was a blocker for getting glib2.0 unstuck. I'm going to block it in britney now so we can calculate an upgrade without it === gusnan_ is now known as gusnan [18:12] vorlon: libcanberra t64 [18:12] hmm [18:12] doko: ok [18:13] vorlon: oh the ada nonsense is because britney is trying to migrate gnat [18:13] doko: I'm not sure if we should trigger rebuilds now of the mutter revdeps or if we should remove their armhf binaries from release pocket; do you have time to analyze this in the next couple of hours? If not I'll look at it at some point today [18:13] the ada armhf binaries are removed, those that don't build [18:13] mwhudson: shall I block that too? [18:13] I'll go paste the corrected queue count with duplicates and obsoletes removed and triggers merged: [18:13] amd64 input: 7722 output: 7474 [18:13] i386 input: 797 output: 268 [18:13] ppc64el input: 448 output: 365 [18:13] s390x input: 430 output: 359 [18:13] arm64 input: 1046 output: 873 [18:14] armhf input: 0 output: 0 [18:14] vorlon: i think so, i don't think gnat is entangled in t64 stuff [18:14] mwhudson: done [18:15] I've asked for i386 to run a merge again so the queue is not at almost 3 times the real size, which should help, even if the current britney run still queues again [18:15] but this way i386 should be done soon [18:16] more space for amd64 [18:16] right caffeine [18:16] mwhudson: I don't think gnat blocking is needed. and it's entangled via plplot [18:17] doko: libadasockets12-dev depends on gnat (<< 13), gnat (>= 12), [18:17] doko: it's not entangled [18:17] on !armhf [18:17] which is the priority [18:17] we can rip out arbitrary quantities of non-ubuntu-core armhf packages from the release pocket to unblock the transition [18:18] mwhudson: libadasockets12-dev is gone [18:19] vorlon: I'm not sure what you're asking about mutter. why extra test triggers? [18:22] doko: trigger rebuilds, I said nothing about test triggers [18:22] doko: mutter can now migrate, but it breaks its revdeps on !armhf because of abi change [18:22] have you already scheduled builds for these? [18:22] no, can do [18:22] doko: but the question is *should we* right now or should we just remove armhf binaries [18:23] because we can block mutter and remove armhf binaries, or we can unblock mutter but then its revdeps have to all be rebuilt on !armhf [18:23] so there's a decision to make [18:23] I'll do the uploads first [18:24] can we just calculate the minimal set we need for core and rip out *everything* else [18:24] juliank: that risks having to rebootstrap more stuff so no [18:25] and if you mean "demote everything to proposed and let britney sort it out" also no ;) [18:26] telling britney to let armhf break in the release pocket really is the shortest path when we get to this point [18:27] ack [18:28] I can't wait until we see the real test queue throughput with next britney run [18:28] coq-iris uploaded, looks like it was the last straggler dependency of libcoq [18:28] (transitive dep) [18:28] no more interference with requeuing will be nice :) [18:29] ummm more rust uninstallables than there were before, including librust-libslirp-dev, which juliank will recognize [18:29] so what did I miss with my previous removals [18:29] hmm [18:29] rust-zbus-1 != rust-zbus, maybe that's it [18:30]