[00:02] is pyyaml built against python3-dev instead of python3-all-dev? [00:03] we might want to fix that for future proofing [00:03] (that would also let pyyaml migrate now without python3-defaults, and make things less horrible) [00:06] hmm it already build-depends on python3-all-dev and should be installable, but the -proposed version with python3.9 support hasn't migrated yet [00:27] -queuebot:#ubuntu-release- New binary: libmatio [s390x] (hirsute-proposed/universe) [1.5.19-2] (no packageset) [00:35] -queuebot:#ubuntu-release- New: accepted libmatio [amd64] (hirsute-proposed) [1.5.19-2] [00:35] -queuebot:#ubuntu-release- New: accepted libmatio [riscv64] (hirsute-proposed) [1.5.19-2] [00:35] -queuebot:#ubuntu-release- New: accepted libmatio [ppc64el] (hirsute-proposed) [1.5.19-2] [00:35] -queuebot:#ubuntu-release- New: accepted libmatio [s390x] (hirsute-proposed) [1.5.19-2] [01:22] -queuebot:#ubuntu-release- New binary: nvidia-cuda-toolkit [ppc64el] (hirsute-proposed/multiverse) [11.1.1-2] (no packageset) [01:54] -queuebot:#ubuntu-release- New binary: vtk9 [amd64] (hirsute-proposed/universe) [9.0.1+dfsg1-1] (no packageset) [02:32] -queuebot:#ubuntu-release- New binary: vtk9 [s390x] (hirsute-proposed/universe) [9.0.1+dfsg1-1] (no packageset) [02:35] ghc migrated \o/ [02:35] (that's a thousand less packages for britney to nag me about) [02:46] yay [02:48] scipy still looks pretty unhappy [02:48] maybe that's a job for monday [02:55] -queuebot:#ubuntu-release- New binary: vtk9 [ppc64el] (hirsute-proposed/universe) [9.0.1+dfsg1-1] (no packageset) [03:57] -queuebot:#ubuntu-release- New: accepted vtk9 [ppc64el] (hirsute-proposed) [9.0.1+dfsg1-1] [03:57] -queuebot:#ubuntu-release- New: accepted vtk9 [s390x] (hirsute-proposed) [9.0.1+dfsg1-1] [03:57] -queuebot:#ubuntu-release- New: accepted nvidia-cuda-toolkit [ppc64el] (hirsute-proposed) [11.1.1-2] [03:57] -queuebot:#ubuntu-release- New: accepted vtk9 [amd64] (hirsute-proposed) [9.0.1+dfsg1-1] [04:40] -queuebot:#ubuntu-release- New binary: libpgm [s390x] (hirsute-proposed/universe) [5.3.128~dfsg-2] (i386-whitelist, kubuntu) [04:40] -queuebot:#ubuntu-release- New binary: libpgm [amd64] (hirsute-proposed/universe) [5.3.128~dfsg-2] (i386-whitelist, kubuntu) [04:40] -queuebot:#ubuntu-release- New binary: libpgm [ppc64el] (hirsute-proposed/universe) [5.3.128~dfsg-2] (i386-whitelist, kubuntu) [04:40] -queuebot:#ubuntu-release- New binary: libpgm [i386] (hirsute-proposed/universe) [5.3.128~dfsg-2] (i386-whitelist, kubuntu) [04:59] -queuebot:#ubuntu-release- New binary: libpgm [riscv64] (hirsute-proposed/universe) [5.3.128~dfsg-2] (i386-whitelist, kubuntu) [05:37] -queuebot:#ubuntu-release- New: accepted libpgm [amd64] (hirsute-proposed) [5.3.128~dfsg-2] [05:37] -queuebot:#ubuntu-release- New: accepted libpgm [ppc64el] (hirsute-proposed) [5.3.128~dfsg-2] [05:38] -queuebot:#ubuntu-release- New: accepted libpgm [s390x] (hirsute-proposed) [5.3.128~dfsg-2] [05:38] -queuebot:#ubuntu-release- New: accepted libpgm [i386] (hirsute-proposed) [5.3.128~dfsg-2] [05:38] -queuebot:#ubuntu-release- New: accepted libpgm [riscv64] (hirsute-proposed) [5.3.128~dfsg-2] [06:43] Hi doko, could the recent https://launchpad.net/ubuntu/+source/python3-defaults/3.9.0-3 cause "Missing build dependencies: python3 (< 3.9)" due to indirect dependency breakage? [06:44] sounds like something that needs a rebuild against python3.9 as default [06:45] yeah, but the package that broke for me has no python* in deps - so it must be one of the base set of common build dependencies [06:45] I'm checking which one it is and if a rebuild might already be in progress [06:48] BD -> sssd-common -> python3-sss it is [06:48] python3-sss : Depends: python3 (< 3.9) but 3.9.0-3 is to be installed [06:49] plusone doing +1 today, too, for a few hours because i lost a few hours on wednesday in my shift to regular work [06:49] hi rbalint [06:50] cpaelzer, o/ [06:50] usually the one driving the transition is doing the rebuilds against the new version - I'm not interfering by triggering an sssd rebuild now as something might not yet be ready [06:54] mwhudson, scipy looks just a hint away from release pocket, that's not bad :-) [06:59] mwhudson, hm, no, it is failing dask, indeed [07:09] plusone looking at r-base [07:10] yes, but builders in a very bad shape [07:19] cpaelzer: sssd requires samba ... [07:22] life is a circle .. of bugs [07:23] sergiodj: ^^ FYI [07:23] doko: even the rebuild of sssd needs samba? [07:23] according to the transition tracker, yes [07:23] / sigh [07:32] sergiodj: tdb/s390x ftbfs would need investigation first [07:46] oSoMoN: I'm not uploading LO for python 3.9 before the datacenter move [07:51] tdb/s390x succeeded on retry === Guest16270 is now known as _hc [09:49] Laney: is britney happily running for 4h? [09:50] Laney, juliank: are the ppc64el autopkg testers alive? [09:50] a new run just started [09:50] let's look [09:50] doko: just for information, you can see the same logs as me https://people.canonical.com/~ubuntu-archive/proposed-migration/log/hirsute/2020-11-20/ [09:51] I can't look, on a call, will check later [09:51] doko: ppc64el workers are running, yes [09:51] the bos02 ones [09:52] 20 workers [09:54] that's all we have atm, normally we have 20 more workers in bos01 [09:55] Laney: ahh, ok so the 7am run timed out with urllib.error.URLError: [09:55] and the current one still running [09:56] right [09:57] and that code already has retries, so ... [09:58] maybe there was downtime ? [09:58] -queuebot:#ubuntu-release- New binary: linux-signed-azure-4.15 [amd64] (bionic-proposed/main) [4.15.0-1100.111] (no packageset) [09:58] at least buildd outage, that's why I was asking about the autopkg testers as well [10:00] different DC AFAIK [10:25] -queuebot:#ubuntu-release- New: accepted linux-signed-azure-4.15 [amd64] (bionic-proposed) [4.15.0-1100.111] [11:03] just thought [11:04] the email policy should catch that exception and just warn shouldn't it [11:04] rather than crashing the whole run, failing to be able to send an email isn't that bad [11:20] -queuebot:#ubuntu-release- New binary: php-gettext [amd64] (hirsute-proposed/none) [1.0.12-1] (no packageset) [11:32] ah since pyyaml migrated, I guess I can retry those /unknown results ? [11:32] doko: ^ right? [11:36] Laney: why was pyyaml needed? [11:36] ubuntu-minimal transitively depends on it [11:36] via netplan possibly, I didn't look [11:37] sure, I think I only gave back a handful of unknown's this morning [11:37] ok, I have some code here to add a --only-unknown to retry-autopkgtest-regressions [11:38] I'll push that [11:48] Laney: we should force-skiptest dbus-python/1.2.16-4 so it can migrate? software-properties doesn't seem to work on armhf right now, and we need the new dbus-python to unbreak aptdaemon in release pocket [11:49] For completeness: aptdaemon has python3.9 shebangs, but dbus-python in release pocket is not built for 3.9 yet, causing it to fail [11:50] aptdaemon needs to run some extra debhelper that does the shebang rewriting [11:50] so future versions get unversioned shebangs [11:51] I think [11:51] I thought dh_python3 did that [11:52] confusing [11:53] https://autopkgtest.ubuntu.com/packages/s/software-properties/hirsute/armhf [11:53] why did everyone just spam retry on that multiple times [11:54] juliank: feels like a reset-test or badtest for s-p/armhf? [11:54] Laney: maybe, but it looks like launchpad issues today and I don't want to make a general call on that [11:55] don't understand what you mean there [11:55] Laney, i triggered r-bioc-sva/ppc64el and it may have lost [11:56] Laney, you said that i should not retry in those cases, could you please take a look? [11:56] juliank: What do Launchpad issues have to do with autopkgtests? [11:56] Laney, the trigger was hello [11:57] Oh right, 403s from ppa.launchpad.net [11:57] Yeah, 403s I think and some timeouts [11:57] I haven't looked at all of them [11:57] weren't those proxy issues? [11:57] Not so much Launchpad issues per se, but there's been GS2 network maintenance this morning [11:57] Almost certainly that [11:57] I don't know, but possible [11:58] It's supposed to be over for the time being, but it's part of preparing for the Boston datacentre move [11:59] rbalint: I don't believe you did trigger that [11:59] so Laney what I meant is that I did not look at the other issues and I think it might be temporary, so i was thinking about only getting that urgent dbus-python fix so aptdaemon works again [11:59] because I think the others did not cause that crazy regressions likely :D [12:00] but yeah, force-reset-test I guess would work [12:01] rbalint: The only mention of that package name in access.log is a s390x from bing which we 403ed [12:01] thanks very much bing [12:01] juliank: reset-test feels good, I will do that [12:05] Laney, thanks, interesting [12:06] Laney, oh, it was yesterday [12:07] Laney, i triggered r-bioc-sva/s390x that went through [12:08] right, there's no 'hello' request from yesterday either [12:08] ubuntu@juju-prod-ues-proposed-migration-machine-2:~$ egrep request.*ppc64el.*r-bioc-sva.*hello /var/log/apache2/access.log.1 [12:08] ubuntu@juju-prod-ues-proposed-migration-machine-2:~$ [12:09] but your other three(!) requests are all in the log [12:13] Laney, interesting, thanks [12:25] rbalint: btw, you should know about retry-autopkgtest-regressions --no-proposed I guess, instead of using 'hello [12:25] ' [12:28] ok retried all the unknowns [12:33] Laney, thanks, but hello works for packages in proposed [12:39] right [12:40] are you planning to automate this or do it regularly? [12:44] Laney, no, i just use it to mass trigger a range of failures, usually in combination with --log-regex or with | grep package=... [12:45] right, I'm worried you are implementing baseline retesting from the outside [12:45] Eickmeyer[m]: https://launchpadlibrarian.net/507846803/buildlog_ubuntu-hirsute-amd64.agordejo_0.1.1-0ubuntu2_BUILDING.txt.gz [12:45] if not then ok [13:23] Laney, could you please check https://code.launchpad.net/~rbalint/britney/+git/hints-ubuntu/+merge/394180 for rbase and others? [13:23] Laney, also https://code.launchpad.net/~rbalint/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/394209 [13:55] May I bring this i386-whitelist change request to any AAs attention? https://bugs.launchpad.net/ubuntu/+source/pycryptodome/+bug/1904987 [13:55] Ubuntu bug 1904987 in pycryptodome (Ubuntu) "Please add this package to i386-whitelist" [Undecided,New] [14:26] sil2100, could you please check the merges i pinged Laney with? [14:28] plusone r-base should be ready to migrate after the hints are merged [14:46] cpaelzer, retried cyrus-imapd/amd64 because is was still sad [14:46] rbalint: looking in a moment [14:46] slyon: and here as well [14:48] sil2100: thanks o/ AFAIU a no-change rebuild is needed after the package is accepted into the whitelist. Debdiff is attached to the bug [15:14] -queuebot:#ubuntu-release- Unapproved: s390-tools (xenial-proposed/main) [1.34.0-0ubuntu8.10 => 1.34.0-0ubuntu8.11] (no packageset) [15:17] sil2100, added more hints [15:18] s390x builders are going down shortly for the DC move - I've put them all on manual [15:18] rbalint: sorry, I'm on a call [15:19] but yes, I should do that with autopkgtest workers too [15:19] rbalint: ok, looking at it now anyway [15:20] slyon: will sponsor in a moment! [15:21] ok, s390x autopkgtesters off too [15:24] rbalint: and what's up with ganeti? I don't see it having any i386 binaries? [15:30] doko: would it be possible to file bugs for what you want me to do? I'm very busy with other stuff right now, but taking care of samba & sssd are definitely at the top of my TODO list [15:31] sil2100, test is still listed as regressing for ganet/proposed [15:33] sergiodj: https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1905048 [15:33] Ubuntu bug 1905048 in samba (Ubuntu) "samba ftbfs in hirsute, needs merge" [High,Confirmed] [15:35] however, --no-proposed fails if the package being triggered has a version in -proposed [15:35] Laney, rbalint: ^^ [15:36] yeah, the doc string notes that [15:36] migration-reference/0! [15:36] +1 [15:36] until then hello is good enough [15:48] * vorlon nods === Laney changed the topic of #ubuntu-release to: Released: Groovy 20.10, Focal 20.04.1, Bionic 18.04.5 | Archive: Open but no s390x | Highlight ubuntu-archive for archive admin help | Hirsute Release Coordination | We accept payment in cash, cheque or gin | melius malum quod cognoscis [16:20] -queuebot:#ubuntu-release- Packageset: Removed confuse from i386-whitelist in hirsute [16:20] -queuebot:#ubuntu-release- Packageset: Removed ruby-setup from i386-whitelist in hirsute [16:20] -queuebot:#ubuntu-release- Packageset: Added fonts-noto to i386-whitelist in hirsute [16:20] -queuebot:#ubuntu-release- Packageset: Added fonts-noto-color-emoji to i386-whitelist in hirsute [16:20] -queuebot:#ubuntu-release- Packageset: Added pycryptodome to i386-whitelist in hirsute [16:20] -queuebot:#ubuntu-release- Packageset: Added sphinx-autoapi to i386-whitelist in hirsute [16:20] -queuebot:#ubuntu-release- Packageset: Added unidecode to i386-whitelist in hirsute [16:25] rbalint: the cyrus issue this adressed was only ppc64 [16:25] rbalint: I haven't looked at its amd64 results - might be a different issue there [16:25] but at least in the PKG that got me to cyrus only ppc64 failed, all other arches were ok [16:41] Laney, vorlon: are transition tracker updates running? [16:43] sil2100, vorlon please merge to let us retry petsc4py https://code.launchpad.net/~rbalint/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/394209 [16:45] sil2100, vorlon Laney i pressed retry, please merge before the package hits the workers [16:45] is was queued anyway for python3-defaults [16:45] I call it a week ... [16:46] doko: should be, I'll look in a minute, maybe something got stuck since there were network outages earlier [16:47] doko: thanks [16:47] rbalint: why press retry before it's merged? you'll just get it not run on a large instance. [16:47] as just happened [16:47] :( [16:51] Laney, i thought the mapping happens when it is picked from the queue [16:51] it does [16:51] i see no info on http://autopkgtest.ubuntu.com/running about queued items being mapped [16:52] to length of vm sizes [16:52] but now it's running [16:52] and the thing is not merged [16:52] Laney, could you please merge it? [16:53] in a minute [16:53] Laney, thanks [16:53] but still, don't press the button until you get the merge in future please [16:53] Laney, ok, but please merge quicker [16:53] rbalint: didn't one of your earlier changes here fail to fix a failure? iirc [16:53] * Laney shrugs [16:54] if that's right, can you revert it please? having too many of these large tgests can be painful [16:54] they take up 4x as much of our quota than regular ones [16:54] Laney, that was a different package [16:55] which? [16:56] Laney, cross-toolchain-base, checking [16:59] doko: That means there was a fundamental change in Python between when I submitted the package (over two weeks ago!) and now. I'll let uptream know. [16:59] THIS is why we need more efficiency for package reviews. [17:00] Laney, i'd like to wait for a successful run of cross-toolchain-base because it passed after ~8 hours when using the small vm and this is a very significant delay in migration [17:01] Laney, let's see how much time it needs to pass on a big vm [17:02] delaying 3 other tests for the whole length of the run should also weigh into your reasoning [17:03] but ok, if it is a significant gain then it could be justified [17:19] rbalint: right, I see a ton of failures before your changes and a ton after, including a screenful of parallel large tests 😱 so I think you just managed to fail a *little* bit faster [17:19] Laney, regarding failures my proposal is LP :#1903913 [17:20] Laney, regarding failures my proposal is LP: #1903913 [17:20] Launchpad bug 1903913 in Auto Package Testing "Don't run tests that never pass (or don't run tests with force-badtest)" [Undecided,New] https://launchpad.net/bugs/1903913 [17:20] if it's forced, why are you bothered about retrying it? [17:20] Laney, running on big vms will help with successful runs [17:21] Laney, it got forced later [17:22] rbalint: ok, since your bug there is not addressed atm I'm going to remove it from big_packages and once it is fixed and verified on arm64 e.g. using Canonistack please ping me back and we can look to re-add it then [17:23] * rbalint shrugs [17:24] if you've followed the logs and saw 'quota exceeded' as much as I have you'd also be concerned about controlling resource usage :-) [17:24] s/you've/you'd/ [17:26] Laney, then don't run the failing tests? [17:27] Laney, i do care about making the most out of the quota but plaing with the big package list does not help as did not enough in the past years either [17:27] Laney, what really helps is not running tests with no signal and pushing out excess tests to clouds [17:28] yeah thanks, one thing is a task I can do in 5 seconds, the other takes thinking about, planning, testing and doing so they're not equivalent [17:28] Laney, does not make the obsolete 5s task worth doing it [17:29] it's not obsolete, not sure why you would say that [17:29] anyway this feels like it's getting close to an argument, so I think we should both step away [17:30] Laney, ok, the 5s task has a tiny tiny impact === ijohnson is now known as ijohnson|lunch [18:14] Laney, sorry, just would like to improve the testing infra significally, let's discuss the bigger tasks at the next +1 meeting [19:42] Eickmeyer[m]: It would just show up in a test rebuild anyway, or maybe surprise you when you least expect it and you're trying to fix something else small - better to find out earlier :) [19:43] -queuebot:#ubuntu-release- Unapproved: pcl (xenial-proposed/universe) [1.7.2-14build1 => 1.7.2-14build1.1] (no packageset) [19:43] cjwatson: Agreed, but I still could've been on it sooner. Turns out upstream runs Arch which is still on Python 3.8 (ironically). [19:44] hello SRU team, could someone please approve pcl in the xenial upload queue into -proposed, it's the second part of bug 1573234 [19:44] bug 1573234 in pcl (Ubuntu Xenial) "unable to link: cannot find libvtkproj4" [Medium,Confirmed] https://launchpad.net/bugs/1573234 [19:46] doko: what is the addition of libgtop2 for in update-i386-whitelist? [20:01] mdeslaur: this is not a very reproducible test case [20:01] mdeslaur: I shouldn't have to chase down build-dependencies one-by-one to reproduce the failure [20:02] (and I'm taking a close look at this one because the idea of no-change SRU rebuilds magically fixing things makes me uneasy) [20:04] arm64/armhf/ppc64el jobs are failing with no logs [20:04] retrying doesn't help [20:09] possibly related to the datacenter move happening this weekend. Those bits aren't scheduled to be moved yet but there could be transient network misconfigurations etc [20:09] cjwatson, wgrant: ^^ can you speak to the state of ARM/ppc64el builders in LP? [20:13] mdeslaur: git clone of the referenced perception_pcl project, followed by manually futzing with build-deps, and then 'cmake pcl_conversions', gives me a makefile with an empty 'all' target. I'm not willing to advance this without a cleaner test case === ijohnson|lunch is now known as ijohnson [20:26] vorlon: Not specifically. I can see logs that suggest network issues but don't know exactly what, and I'm also EOW - best ask IS [20:28] But it's all somewhat at risk at the moment due to network changes being made to prepare for the move [20:29] right, thanks [20:39] -queuebot:#ubuntu-release- Unapproved: livecd-rootfs (focal-proposed/main) [2.664.8 => 2.664.9] (desktop-core, i386-whitelist) [20:48] vorlon: ok, thanks, I'll let the reporter know [21:00] mdeslaur, vorlon: please take another look at the test case on bug 1573234, it's been greatly simplified and hopefully makes more sense [21:00] bug 1573234 in pcl (Ubuntu Xenial) "unable to link: cannot find libvtkproj4" [Medium,Confirmed] https://launchpad.net/bugs/1573234 [21:02] vorlon, note that the no-change rebuild propogates the dependency fix to properly use the system libproj instead of the vtk-vendored version, which isn't actually used by vtk at all [21:02] vorlon, but pcl ended up with the vendored lib in its cmake config files [21:10] vorlon, so the situation in Xenial right now is that vtk is built against the system libproj, but anyone linking against it tries to link against its vendored version [21:31] -queuebot:#ubuntu-release- Unapproved: livecd-rootfs (xenial-proposed/main) [2.408.61 => 2.408.62] (desktop-core) [21:34] -queuebot:#ubuntu-release- Unapproved: livecd-rootfs (bionic-proposed/main) [2.525.47 => 2.525.48] (desktop-core) [21:57] -queuebot:#ubuntu-release- Unapproved: livecd-rootfs (xenial-proposed/main) [2.408.61 => 2.408.63] (desktop-core) [22:57] -queuebot:#ubuntu-release- Unapproved: ubuntustudio-menu (groovy-proposed/universe) [0.49 => 0.50~20.10.1] (ubuntustudio) [22:57] ^ SRU bug 1905061 [22:57] bug 1905061 in ubuntustudio-menu (Ubuntu Groovy) "[SRU] Media Playback menu is overpopulated" [High,In progress] https://launchpad.net/bugs/1905061