[00:00] $ ls -ld ubuntu/dists/hirsute/main/binary-amd64/Packages.gz [00:00] -rw-rw-r-- 1 cdimage cdimage 1799322 Feb 15 02:44 ubuntu/dists/hirsute/main/binary-amd64/Packages.gz [00:00] I think this may be a stuck anonftpsync lock [00:00] $ [00:00] * vorlon nods [00:00] There's an oldish cron.daily-live process running [00:01] oh well wxl didn't poke me [00:01] *shoves wxl out the window* [00:01] Probably just needs killing and then let everything settle? [00:01] cjwatson: the oldest I saw was from Feb 28 which doesn't account for things being stuck since the 15th? [00:02] IDK, maybe there were image build issues then or something [00:02] Or multiple overlapping stuck builds [00:02] Probably less interesting to track down the precise sequence of events than to just mash it into working again [00:02] There should be indications of held locks in CD build logs if people really want to go looking [00:03] it's an irritating thing that i would hope would have been caught somehow in the iso build process, but tbh i've never once seen something similar so i don't know if we need to sweat it too much [00:03] cjwatson: what sort of strings should i be looking for? "lock" something or other? [00:03] IIRC it's a hard-to-resolve requirements clash between (a) that and (b) supporting parallel image builds during release preparation [00:04] "unable to start rsync; lock file exists" IIRC [00:04] or maybe "Parallel build; waiting" [00:04] I think the latter actually [00:05] I'm running a manual sync now [00:05] seems to be doing its thing [00:05] I suspect the easiest fix would be some kind of timeout so that processes can't sit around for days like this [00:05] cjwatson: you mean like a watchdog that has an overarching timer on it, and when it sits for eternity past $LIMIT it hardkills and errors? [00:05] Parallel build; waiting for Lubuntu mirror to sync [00:05] overarching watchdog daemon* [00:05] *goes to fetch more caffeine* [00:06] that's right at the head of https://people.canonical.com/~ubuntu-archive/cd-build-logs/lubuntu/hirsute/daily-live-20210301.1.log [00:06] though nowadays nie mój cyrk, nie moje malpy [00:06] cjwatson: i couldn't agree more [00:06] (s/malpy/małpy/, bah) [00:06] eh that parallel build thing seems to be in every log so meh [00:07] It will be in a fair few of them but shouldn't be in all [00:07] Though I suppose it may depend on crontab timing [00:07] well ok, i did it rather quickly [00:07] well our crontab is always the same so XD [00:08] The general idea is that the first in a set of parallel builds syncs the local archive mirror, and all the others wait for it to sync and use the same version [00:08] Since having one of them sync the mirror while some other build is running could be bad news [00:08] here's one that doesn't have it https://people.canonical.com/~ubuntu-archive/cd-build-logs/lubuntu/hirsute/daily-live-20210104.1.log [00:08] took me a long while to find one that didn't [00:09] And that one was probably started by hand off the usual cron schedule [00:09] Given the .1 [00:09] yep but not all .1's are like that [00:09] including last night's [00:09] Sure, no particular reason they should be [00:09] cjwatson: my Polish isn't strong but lol at that being understandable [00:10] You might have to correlate builds of multiple projects with reference to the cdimage source to properly track it down :-/ [00:12] so am i going to need to manually rebuild or what's the deal? [00:13] Or wait for the next automatic build. But I'd recommend waiting for vorlon to say that his manual mirror sync has finished before starting a rebuild [00:13] This problem has existed since 2005 in one form or another. It's only actually noticeable when a build gets stuck though [00:15] teward: watchdog> well I mean if you want to overengineer it sure, but I was thinking of timeout(1) :-) [00:15] :P [00:15] that makes sense [00:16] May need a bit of care to ensure that locks get properly cleared etc. [00:16] (Though I think cdimage deals with that since generally it's possible to kill stuck builds to clear locks and this would be no different) [00:17] cjwatson: well, that sounds like then you need a script to handle it, something simple that makes sure that, whether killed by timeout, etc. cleans up the scripts, unless cdimage does that anyways. The core part would be setting the hard timeout limit (i.e. timeout -k 6h 12h ...) [00:18] (where 6h after the initial SIGTERM is issued a SIGKILL comes back if it hasn't died) [00:18] or w/e times are preferable [00:18] (just thinking outloud) [00:18] but makes sense [00:18] I think cdimage does that anyway so no need to make it complicated [00:18] *writes notes about 'timeout' command so he can decomplicate his scripts* [00:19] It might be better to do it in Python though, so it can go in the build_image_set function and automatically work for everything [00:20] That just amounts to something involving https://docs.python.org/3/library/signal.html#signal.alarm though [00:30] vorlon, thanks for the info about testbed. Regarding puma, I filed bugs upstream and I am helping one of the maintainers with debugging. For now I am preparing an upload to skip the problematic tests and allow its migration. [01:04] teward, wxl[m]: ftp mirror is updated as of Mar 2 00:01 UTC [01:06] awesome [01:06] so do we need to kick off a manual rebuild or wait for an autobuild? [01:06] for the Lubuntu ISOs [01:23] -queuebot:#ubuntu-release- New binary: rustc [amd64] (hirsute-proposed/universe) [1.49.0+dfsg1+llvm-0ubuntu1] (i386-whitelist) [01:26] Next auto is tomorrow so probably best today [04:01] -queuebot:#ubuntu-release- New: accepted rustc [amd64] (hirsute-proposed) [1.49.0+dfsg1+llvm-0ubuntu1] [04:01] -queuebot:#ubuntu-release- New: accepted rustc [armhf] (hirsute-proposed) [1.49.0+dfsg1+llvm-0ubuntu1] [04:01] -queuebot:#ubuntu-release- New: accepted rustc [ppc64el] (hirsute-proposed) [1.49.0+dfsg1+llvm-0ubuntu1] [04:01] -queuebot:#ubuntu-release- New: accepted rustc [s390x] (hirsute-proposed) [1.49.0+dfsg1+llvm-0ubuntu1] [04:01] -queuebot:#ubuntu-release- New: accepted rustc [arm64] (hirsute-proposed) [1.49.0+dfsg1+llvm-0ubuntu1] [04:01] -queuebot:#ubuntu-release- New: accepted rustc [riscv64] (hirsute-proposed) [1.49.0+dfsg1+llvm-0ubuntu1] [04:01] -queuebot:#ubuntu-release- New: accepted rustc [i386] (hirsute-proposed) [1.49.0+dfsg1+llvm-0ubuntu1] [06:49] RAOF: hey, I uploaded new directx-headers to ubuntu anyway, as a new ftpmaster review could take time.. if you have a chance of having a look, that would be great :) [08:57] hello Release Team, since we know libreoffice test failure is not related to poppler and its already being worked on... [08:57] what about hinting poppler to go in? [09:16] LocutusOfBorg: feels bad, then we get failing libreoffice tests and have no idea if there is a further regression or not in future [09:24] Laney, the tests are fixed in the vcs and there is an upload planned for 7.1.1 this week, it's just going to delay a few days if we wait for that one [09:25] it's a packaging error which leads to have java things not installed, doesn't impact the desktop [09:26] seb128: next question, why do we test these things if the argument when they fail is that the failures don't matter? [09:30] or maybe said in a different way, if these missing java parts aren't relevant, could they not be shipped? [09:34] macsyfinder tests fail with autopkgtest [11:52:36]: ERROR: erroneous package: rules extract failed with exit code 1 [09:35] I think that's because it's Architecture: all in release pocket, but amd64 only in proposed [09:36] So they work on amd64, but can't find the source package on other archs [09:36] There's only one reverse-recommends, from med-bio, so I guess we can just drop macsyfinder from the release pocket? [09:37] That will then make the new macsyfinder run its tests correctly [09:37] that's weird [09:37] we can hint the /unknowns though [09:37] Laney: I know, but it's not unprecedented [09:38] which part? [09:39] Laney: I think I read a bug in Debian that had the same cause and effect [09:40] Laney: I think the best approach is to force-skiptest the macsyfinder/2.0~rc1-3 upload, then that migrates, and autopkgtest for other packages should work again [09:40] juliank: sure, that works, can you file a mp? [09:40] That one passed on amd64, the failures are on dropped architectures [09:40] on a call atm [09:42] or just give me a commit, doesn't need a merge proposal [09:43] Laney: git pull git+ssh://git.launchpad.net/~juliank/britney/+git/hints-ubuntu [09:44] cannot find remote ref HEAD [09:44] Laney: git pull git+ssh://git.launchpad.net/~juliank/britney/+git/hints-ubuntu devel [09:44] extra devel [09:44] :D [09:45] :> [09:45] merci [09:45] done [09:46] Hmm do I still need to force-badtest the other unknown/$port ones? I just realized britney will still expect those to succeed for other uploads [09:50] Laney: I pushed force-badtest for all port arches to the same branch now [09:50] Laney, juliank: macsyfinder is only built for amd64 (arch dependent). why is britney even trying to run them? [09:50] doko: It's Architecture: all in release pocket [09:51] doko: So both britney and autopkgtest get confused [09:51] doko: britney tries to run the new one on architectures it does not exist for; and autopkgtest fails to find the old source package [09:52] Laney, it's worth asking how useful those tests are, I just see it as a tradeoff to accept a test regression for a few days to unblock a transition if we are confident that's not going to have a visible impact for users. At the same time I don't feel strongly enough to argue if people disagree, I was just providing some context I know about on the issue (the fact that it's pending upload and not impacting desktop users) === didrocks999 is now known as didrocks [10:14] -queuebot:#ubuntu-release- Unapproved: accepted gnutls28 [source] (groovy-proposed) [3.6.15-4ubuntu2.1] [10:16] it should be requesting the set in the source archive (proposed), there must just be a bug there [10:18] -queuebot:#ubuntu-release- Unapproved: accepted gnutls28 [source] (focal-proposed) [3.6.13-2ubuntu1.4] [10:58] Laney, can we please hint golang-github-coreos-gexpect? regressed in release? [10:58] https://autopkgtest.ubuntu.com/packages/golang-github-coreos-gexpect/hirsute/amd64 [10:58] only on amd64 [10:59] passed only two times in the history [11:01] LocutusOfBorg: probably, be nice to file a bug in debian too [11:01] but what's wrong with ubuntu that it doesn't look like https://ci.debian.net/packages/g/golang-github-coreos-gexpect/unstable/amd64/ ? [11:01] can you file a merge request to the hints? [11:03] probably it regressed with default golang-any pointing to 16 [11:03] looks like some sort of timeouts [11:03] I think an hint reset might be good [11:10] https://code.launchpad.net/~costamagnagianfranco/britney/+git/hints-ubuntu/+merge/398965 [11:10] does this sound good Laney ^^ ? [11:11] this makes etcd migrate and fixes patroni autopkgtests "forever" [11:11] and with macsyfinder fixed sphinx migrates too [11:12] thanks juliank for looking at it [11:12] We still need the force-badtest merged I think [11:20] what force-badtest? [11:20] you people are confusing [11:20] I did force-test-reset [11:21] not sure what force-badtest is [11:21] my goal is to make etcd migrate and I need golang-github-coreos-gexpect stop complaining [11:22] fine, but I dunno what juliank is talking about (or why you invoked him) [11:23] ah I see for the latter, you were talking about mac thing [11:23] Laney: I pushed force-badtest/macsyfinder/!amd64 (expanded) to my branch, because I don't believe that the !amd64 tests will suddenly disappear once it's amd64 only in release [11:23] But I don't know for sure [11:23] I think they will [11:24] the required tests for a migration are essentially recomputed each time [11:24] ah [11:24] we'll see then :D [11:51] thanks lets see [11:51] I invoked him because his attempt makes sphinx migrate together with my hint [11:53] and also setuptools, python-packaging, macsyfinder, etcd, golang-github-creack-pty [12:22] -queuebot:#ubuntu-release- New binary: linux-signed-oracle-5.4 [amd64] (bionic-proposed/main) [5.4.0-1039.42~18.04.1] (no packageset) [12:23] -queuebot:#ubuntu-release- New binary: linux-signed-gkeop-5.4 [amd64] (bionic-proposed/main) [5.4.0-1011.12~18.04.2] (no packageset) [12:24] -queuebot:#ubuntu-release- New binary: linux-signed-oem-5.10 [amd64] (focal-proposed/main) [5.10.0-1016.17] (no packageset) [12:57] hello bdmurray, I just want to verify if there is any update on this Xenial unattended-upgrades SRU: [12:57] https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1915547 [12:57] Ubuntu bug 1915547 in unattended-upgrades (Ubuntu Xenial) " sru unattended-upgrades ( 1.1ubuntu1.18.04.7~16.04.6 update to 1.1ubuntu1.18.04.7~16.04.7 ) Xenial" [High,Triaged] [13:09] -queuebot:#ubuntu-release- New: accepted linux-signed-gkeop-5.4 [amd64] (bionic-proposed) [5.4.0-1011.12~18.04.2] [13:09] -queuebot:#ubuntu-release- New: accepted linux-signed-oem-5.10 [amd64] (focal-proposed) [5.10.0-1016.17] [13:09] -queuebot:#ubuntu-release- New: accepted linux-signed-oracle-5.4 [amd64] (bionic-proposed) [5.4.0-1039.42~18.04.1] [13:42] -queuebot:#ubuntu-release- New binary: linux-signed-azure [amd64] (hirsute-proposed/main) [5.8.0-1024.26+21.04.1] (core, kernel) [13:42] -queuebot:#ubuntu-release- New binary: linux-signed-kvm [amd64] (groovy-proposed/main) [5.8.0-1020.22] (core) [13:53] Should wait for tests relating to macsyfinder 2.0~rc1-3, but forced by ubuntu-release [13:53] got it Laney ^^ [13:53] so now retrying the tests will work? [13:53] -queuebot:#ubuntu-release- New: accepted linux-signed-kvm [amd64] (groovy-proposed) [5.8.0-1020.22] [14:58] xnox: debian-installer doesn't have any rdeps anymore. can we remove it, or is it still used in obscure places? [15:00] LocutusOfBorg: just wait for the next run, see if it clears off [15:04] ok [17:04] rbasak, hi! thanks for uploading the fixes for west-chamber-dkms. They are still in the unapproved queue though, should I ping someone else to approve it? (sorry, I'm still trying to figure out how the SRU process work :)) [17:08] klebers: it was blocked on https://bugs.launchpad.net/ubuntu/+source/xtables-addons/+bug/1915177/comments/9 but I see you responded to that just now [17:08] Ubuntu bug 1915177 in xtables-addons (Ubuntu Focal) "xtables-addons fails to build after ip_route_me_harder() signature change" [High,In progress] [17:08] I'll take a proper look in my SRU shift for you tomorrow if nobody gets to it today [17:09] rbasak, ok, that makes sense. thanks! [17:14] britney is soo slooow [18:05] doko: do not remove debian-installer, it is an html placeholder page only which directs users where to find the installer. [18:06] doko: thank you! === ijohnson is now known as ijohnson|lunch [18:24] -queuebot:#ubuntu-release- New binary: freedict [amd64] (hirsute-proposed/universe) [2021.01.05-2ubuntu1] (no packageset) [18:26] -queuebot:#ubuntu-release- New: accepted freedict [amd64] (hirsute-proposed) [2021.01.05-2ubuntu1] [18:27] -queuebot:#ubuntu-release- New: accepted git-filter-repo [sync] (hirsute-proposed) [2.29.0-1] [18:30] -queuebot:#ubuntu-release- New binary: git-filter-repo [amd64] (hirsute-proposed/none) [2.29.0-1] (no packageset) [18:32] -queuebot:#ubuntu-release- New: accepted git-filter-repo [amd64] (hirsute-proposed) [2.29.0-1] [19:08] -queuebot:#ubuntu-release- New sync: grub2-unsigned (xenial-proposed/primary) [2.04-1ubuntu42] [19:09] -queuebot:#ubuntu-release- New: accepted grub2-unsigned [sync] (xenial-proposed) [2.04-1ubuntu42] [20:16] vorlon: please can you ignore the autopkgtest failure for android-sdk-meta/i386? not built anymore [20:29] -queuebot:#ubuntu-release- Unapproved: linux-firmware (focal-proposed/main) [1.187.9 => 1.187.10] (core, kernel) === ijohnson|lunch is now known as ijohnson [20:37] doko: done [20:41] -queuebot:#ubuntu-release- Unapproved: linux-firmware (bionic-proposed/main) [1.173.19 => 1.173.20] (core, kernel) [21:13] KeyError: 'autopkg_fails' [21:14] that's a new error from sru report generation [21:42] -queuebot:#ubuntu-release- Packageset: Removed boost1.74 from i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Removed libpadwalker-perl from i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Removed llvm-toolchain-11 from i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Removed postgresql-13 from i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Removed python3.10 from i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Removed boost1.74 from i386-whitelist in groovy [21:43] -queuebot:#ubuntu-release- Packageset: Removed libpadwalker-perl from i386-whitelist in groovy [21:43] -queuebot:#ubuntu-release- Packageset: Removed postgresql-13 from i386-whitelist in groovy [21:43] -queuebot:#ubuntu-release- Packageset: Removed python3.10 from i386-whitelist in groovy [21:43] -queuebot:#ubuntu-release- Packageset: Removed dpatch from i386-whitelist in hirsute [21:43] -queuebot:#ubuntu-release- Packageset: Removed gnome-icon-theme from i386-whitelist in hirsute [21:43] -queuebot:#ubuntu-release- Packageset: Removed libprelude from i386-whitelist in hirsute [21:43] -queuebot:#ubuntu-release- Packageset: Removed lowdown from i386-whitelist in hirsute [21:43] -queuebot:#ubuntu-release- Packageset: Removed qtnetworkauth-everywhere-src from i386-whitelist in hirsute [21:43] -queuebot:#ubuntu-release- Packageset: Removed readline5 from i386-whitelist in hirsute [21:43] -queuebot:#ubuntu-release- Packageset: Removed xcftools from i386-whitelist in hirsute [21:43] -queuebot:#ubuntu-release- Packageset: Added egl-wayland to i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Added gcc-11 to i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Added gcc-11-cross to i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Added gcc-11-cross-ports to i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Added gcc-8 to i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Added libdeflate to i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Added libffi7 to i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Added libmd to i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Added llvm-toolchain-12 to i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Added lowdown to i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Added nvidia-graphics-drivers-460-server to i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Added openjdk-17 to i386-whitelist in focal [21:43] -queuebot:#ubuntu-release- Packageset: Added egl-wayland to i386-whitelist in groovy [21:43] -queuebot:#ubuntu-release- Packageset: Added gcc-11 to i386-whitelist in groovy [21:43] -queuebot:#ubuntu-release- Packageset: Added gcc-11-cross to i386-whitelist in groovy [21:43] -queuebot:#ubuntu-release- Packageset: Added gcc-11-cross-ports to i386-whitelist in groovy [21:43] -queuebot:#ubuntu-release- Packageset: Added gcc-8 to i386-whitelist in groovy [21:43] -queuebot:#ubuntu-release- Packageset: Added libdeflate to i386-whitelist in groovy [21:43] -queuebot:#ubuntu-release- Packageset: Added libffi7 to i386-whitelist in groovy [21:43] -queuebot:#ubuntu-release- Packageset: Added libmd to i386-whitelist in groovy [21:43] -queuebot:#ubuntu-release- Packageset: Added llvm-toolchain-12 to i386-whitelist in groovy [21:43] -queuebot:#ubuntu-release- Packageset: Added lowdown to i386-whitelist in groovy [21:43] -queuebot:#ubuntu-release- Packageset: Added nvidia-graphics-drivers-460-server to i386-whitelist in groovy [21:43] -queuebot:#ubuntu-release- Packageset: Added openjdk-17 to i386-whitelist in groovy === rharper_ is now known as rharper [22:16] there are an awful lot of packages still in hirsute that build-depend on dh-systemd which is NBS; is anyone working on this? Are these all Ubuntu-specific? [22:29] vorlon, not all of them I think [22:29] I looked at them some days ago [22:29] reverse-depends -r sid -b dh-systemd [22:29] 11 comes from debian too [22:30] but probably the debian ones are all out from testing [22:32] and some of the debian ones seem to be alternative deps [22:32] (apt-cacher-ng) [22:38] -queuebot:#ubuntu-release- Unapproved: gnutls28 (focal-proposed/main) [3.6.13-2ubuntu1.4 => 3.6.13-2ubuntu1.5] (core, i386-whitelist) [22:43] -queuebot:#ubuntu-release- Unapproved: gnutls28 (focal-proposed/main) [3.6.13-2ubuntu1.4 => 3.6.13-2ubuntu1.5] (core, i386-whitelist) [22:52] -queuebot:#ubuntu-release- Unapproved: rejected gnutls28 [source] (focal-proposed) [3.6.13-2ubuntu1.5] [22:59] juliank, xnox: ^^ fyi please make grub2 build-depend on debhelper (>= 9.20160709) | dh-systemd, so that build-deps are satisfied back as far as xenial and also we can drop dh-systemd from hirsute [23:02] rbalint, smb: hmm, does one of you want to merge kerneloops? :) [23:34] -queuebot:#ubuntu-release- Unapproved: rejected neutron [source] (groovy-proposed) [2:17.1.0-0ubuntu1] [23:42] -queuebot:#ubuntu-release- Unapproved: accepted west-chamber [source] (groovy-proposed) [20100405+svn20111107.r124-12ubuntu0.2] [23:51] -queuebot:#ubuntu-release- Unapproved: accepted gnome-shell [source] (groovy-proposed) [3.38.3-2ubuntu0.20.10.2] [23:55] -queuebot:#ubuntu-release- Unapproved: zfs-linux (groovy-proposed/main) [0.8.4-1ubuntu11.1 => 0.8.4-1ubuntu11.2] (core, kernel-dkms) [23:56] -queuebot:#ubuntu-release- Unapproved: zfs-linux (bionic-proposed/main) [0.7.5-1ubuntu16.10 => 0.7.5-1ubuntu16.11] (core, kernel-dkms) [23:56] -queuebot:#ubuntu-release- Unapproved: zfs-linux (focal-proposed/main) [0.8.3-1ubuntu12.6 => 0.8.3-1ubuntu12.7] (core, kernel-dkms) [23:57] -queuebot:#ubuntu-release- Unapproved: zfs-linux (xenial-proposed/main) [0.6.5.6-0ubuntu28 => 0.6.5.6-0ubuntu29] (kernel-dkms)