[00:02] <vorlon> is pyyaml built against python3-dev instead of python3-all-dev?
[00:03] <vorlon> we might want to fix that for future proofing
[00:03] <vorlon> (that would also let pyyaml migrate now without python3-defaults, and make things less horrible)
[00:06] <vorlon> hmm it already build-depends on python3-all-dev and should be installable, but the -proposed version with python3.9 support hasn't migrated yet
[00:27] -queuebot:#ubuntu-release- New binary: libmatio [s390x] (hirsute-proposed/universe) [1.5.19-2] (no packageset)
[00:35] -queuebot:#ubuntu-release- New: accepted libmatio [amd64] (hirsute-proposed) [1.5.19-2]
[00:35] -queuebot:#ubuntu-release- New: accepted libmatio [riscv64] (hirsute-proposed) [1.5.19-2]
[00:35] -queuebot:#ubuntu-release- New: accepted libmatio [ppc64el] (hirsute-proposed) [1.5.19-2]
[00:35] -queuebot:#ubuntu-release- New: accepted libmatio [s390x] (hirsute-proposed) [1.5.19-2]
[01:22] -queuebot:#ubuntu-release- New binary: nvidia-cuda-toolkit [ppc64el] (hirsute-proposed/multiverse) [11.1.1-2] (no packageset)
[01:54] -queuebot:#ubuntu-release- New binary: vtk9 [amd64] (hirsute-proposed/universe) [9.0.1+dfsg1-1] (no packageset)
[02:32] -queuebot:#ubuntu-release- New binary: vtk9 [s390x] (hirsute-proposed/universe) [9.0.1+dfsg1-1] (no packageset)
[02:35] <vorlon> ghc migrated \o/
[02:35] <vorlon> (that's a thousand less packages for britney to nag me about)
[02:46] <mwhudson> yay
[02:48] <mwhudson> scipy still looks pretty unhappy
[02:48] <mwhudson> maybe that's a job for monday
[02:55] -queuebot:#ubuntu-release- New binary: vtk9 [ppc64el] (hirsute-proposed/universe) [9.0.1+dfsg1-1] (no packageset)
[03:57] -queuebot:#ubuntu-release- New: accepted vtk9 [ppc64el] (hirsute-proposed) [9.0.1+dfsg1-1]
[03:57] -queuebot:#ubuntu-release- New: accepted vtk9 [s390x] (hirsute-proposed) [9.0.1+dfsg1-1]
[03:57] -queuebot:#ubuntu-release- New: accepted nvidia-cuda-toolkit [ppc64el] (hirsute-proposed) [11.1.1-2]
[03:57] -queuebot:#ubuntu-release- New: accepted vtk9 [amd64] (hirsute-proposed) [9.0.1+dfsg1-1]
[04:40] -queuebot:#ubuntu-release- New binary: libpgm [s390x] (hirsute-proposed/universe) [5.3.128~dfsg-2] (i386-whitelist, kubuntu)
[04:40] -queuebot:#ubuntu-release- New binary: libpgm [amd64] (hirsute-proposed/universe) [5.3.128~dfsg-2] (i386-whitelist, kubuntu)
[04:40] -queuebot:#ubuntu-release- New binary: libpgm [ppc64el] (hirsute-proposed/universe) [5.3.128~dfsg-2] (i386-whitelist, kubuntu)
[04:40] -queuebot:#ubuntu-release- New binary: libpgm [i386] (hirsute-proposed/universe) [5.3.128~dfsg-2] (i386-whitelist, kubuntu)
[04:59] -queuebot:#ubuntu-release- New binary: libpgm [riscv64] (hirsute-proposed/universe) [5.3.128~dfsg-2] (i386-whitelist, kubuntu)
[05:37] -queuebot:#ubuntu-release- New: accepted libpgm [amd64] (hirsute-proposed) [5.3.128~dfsg-2]
[05:37] -queuebot:#ubuntu-release- New: accepted libpgm [ppc64el] (hirsute-proposed) [5.3.128~dfsg-2]
[05:38] -queuebot:#ubuntu-release- New: accepted libpgm [s390x] (hirsute-proposed) [5.3.128~dfsg-2]
[05:38] -queuebot:#ubuntu-release- New: accepted libpgm [i386] (hirsute-proposed) [5.3.128~dfsg-2]
[05:38] -queuebot:#ubuntu-release- New: accepted libpgm [riscv64] (hirsute-proposed) [5.3.128~dfsg-2]
[06:43] <cpaelzer> Hi doko, could the recent https://launchpad.net/ubuntu/+source/python3-defaults/3.9.0-3 cause "Missing build dependencies: python3 (< 3.9)" due to indirect dependency breakage?
[06:44] <tumbleweed> sounds like something that needs a rebuild against python3.9 as default
[06:45] <cpaelzer> yeah, but the package that broke for me has no python* in deps - so it must be one of the base set of common build dependencies
[06:45] <cpaelzer> I'm checking which one it is and if a rebuild might already be in progress
[06:48] <cpaelzer> BD -> sssd-common -> python3-sss it is
[06:48] <cpaelzer>  python3-sss : Depends: python3 (< 3.9) but 3.9.0-3 is to be installed
[06:49] <rbalint> plusone doing +1 today, too, for a few hours because i lost a few hours on wednesday in my shift to regular work
[06:49] <cpaelzer> hi rbalint
[06:50] <rbalint> cpaelzer, o/
[06:50] <cpaelzer> usually the one driving the transition is doing the rebuilds against the new version - I'm not interfering by triggering an sssd rebuild now as something might not yet be ready
[06:54] <rbalint> mwhudson, scipy looks just a hint away from release pocket, that's not bad :-)
[06:59] <rbalint> mwhudson, hm, no, it is failing dask, indeed
[07:09] <rbalint> plusone looking at r-base
[07:10] <doko> yes, but builders in a very bad shape
[07:19] <doko> cpaelzer: sssd requires samba ...
[07:22] <cpaelzer> life is a circle .. of bugs
[07:23] <cpaelzer> sergiodj: ^^ FYI
[07:23] <cpaelzer> doko: even the rebuild of sssd needs samba?
[07:23] <doko> according to the transition tracker, yes
[07:23] <cpaelzer>  / sigh
[07:32] <doko> sergiodj: tdb/s390x ftbfs would need investigation first
[07:46] <doko> oSoMoN: I'm not uploading LO for python 3.9 before the datacenter move
[07:51] <doko> tdb/s390x succeeded on retry
[09:49] <doko> Laney: is britney happily running for 4h?
[09:50] <doko> Laney, juliank: are the ppc64el autopkg testers alive?
[09:50] <LocutusOfBorg> a new run just started
[09:50] <juliank> let's look
[09:50] <Laney> doko: just for information, you can see the same logs as me https://people.canonical.com/~ubuntu-archive/proposed-migration/log/hirsute/2020-11-20/
[09:51] <Laney> I can't look, on a call, will check later
[09:51] <juliank> doko: ppc64el workers are running, yes
[09:51] <juliank> the bos02 ones
[09:52] <juliank> 20 workers
[09:54] <juliank> that's all we have atm, normally we have 20 more workers in bos01
[09:55] <doko> Laney: ahh, ok so the 7am run timed out with urllib.error.URLError: <urlopen error _ssl.c:629: The handshake operation timed out>
[09:55] <doko> and the current one still running
[09:56] <Laney> right
[09:57] <Laney> and that code already has retries, so ...
[09:58] <Laney> maybe there was downtime ?
[09:58] -queuebot:#ubuntu-release- New binary: linux-signed-azure-4.15 [amd64] (bionic-proposed/main) [4.15.0-1100.111] (no packageset)
[09:58] <doko> at least buildd outage, that's why I was asking about the autopkg testers as well
[10:00] <Laney> different DC AFAIK
[10:25] -queuebot:#ubuntu-release- New: accepted linux-signed-azure-4.15 [amd64] (bionic-proposed) [4.15.0-1100.111]
[11:03] <Laney> just thought
[11:04] <Laney> the email policy should catch that exception and just warn shouldn't it
[11:04] <Laney> rather than crashing the whole run, failing to be able to send an email isn't that bad
[11:20] -queuebot:#ubuntu-release- New binary: php-gettext [amd64] (hirsute-proposed/none) [1.0.12-1] (no packageset)
[11:32] <Laney> ah since pyyaml migrated, I guess I can retry those /unknown results ?
[11:32] <Laney> doko: ^ right?
[11:36] <doko> Laney: why was pyyaml needed?
[11:36] <Laney> ubuntu-minimal transitively depends on it
[11:36] <Laney> via netplan possibly, I didn't look
[11:37] <doko> sure, I think I only gave back a handful of unknown's this morning
[11:37] <Laney> ok, I have some code here to add a --only-unknown to retry-autopkgtest-regressions
[11:38] <Laney> I'll push that
[11:48] <juliank> Laney: we should force-skiptest dbus-python/1.2.16-4 so it can migrate? software-properties doesn't seem to work on armhf right now, and we need the new dbus-python to unbreak aptdaemon in release pocket
[11:49] <juliank> For completeness: aptdaemon has python3.9 shebangs, but dbus-python in release pocket is not built for 3.9 yet, causing it to fail
[11:50] <juliank> aptdaemon needs to run some extra debhelper that does the shebang rewriting
[11:50] <juliank> so future versions get unversioned shebangs
[11:51] <juliank> I think
[11:51] <juliank> I thought dh_python3 did that
[11:52] <juliank> confusing
[11:53] <Laney> https://autopkgtest.ubuntu.com/packages/s/software-properties/hirsute/armhf
[11:53] <Laney> why did everyone just spam retry on that multiple times
[11:54] <Laney> juliank: feels like a reset-test or badtest for s-p/armhf?
[11:54] <juliank> Laney: maybe, but it looks like launchpad issues today and I don't want to make a general call on that
[11:55] <Laney> don't understand what you mean there
[11:55] <rbalint> Laney, i triggered r-bioc-sva/ppc64el and it may have lost
[11:56] <rbalint> Laney, you said that i should not retry in those cases, could you please take a look?
[11:56] <cjwatson> juliank: What do Launchpad issues have to do with autopkgtests?
[11:56] <rbalint> Laney, the trigger was hello
[11:57] <cjwatson> Oh right, 403s from ppa.launchpad.net
[11:57] <juliank> Yeah, 403s I think and some timeouts
[11:57] <juliank> I haven't looked at all of them
[11:57] <rbalint> weren't those proxy issues?
[11:57] <cjwatson> Not so much Launchpad issues per se, but there's been GS2 network maintenance this morning
[11:57] <cjwatson> Almost certainly that
[11:57] <juliank> I don't know, but possible
[11:58] <cjwatson> It's supposed to be over for the time being, but it's part of preparing for the Boston datacentre move
[11:59] <Laney> rbalint: I don't believe you did trigger that
[11:59] <juliank> so Laney what I meant is that I did not look at the other issues and I think it might be temporary, so i was thinking about only getting that urgent dbus-python fix so aptdaemon works again
[11:59] <juliank> because I think the others did not cause that crazy regressions likely :D
[12:00] <juliank> but yeah, force-reset-test I guess would work
[12:01] <Laney> rbalint: The only mention of that package name in access.log is a s390x from bing which we 403ed
[12:01] <Laney> thanks very much bing
[12:01] <Laney> juliank: reset-test feels good, I will do that
[12:05] <rbalint> Laney, thanks, interesting
[12:06] <rbalint> Laney, oh, it was yesterday
[12:07] <rbalint> Laney, i triggered r-bioc-sva/s390x that went through
[12:08] <Laney> right, there's no 'hello' request from yesterday either
[12:08] <Laney> ubuntu@juju-prod-ues-proposed-migration-machine-2:~$ egrep request.*ppc64el.*r-bioc-sva.*hello /var/log/apache2/access.log.1
[12:08] <Laney> ubuntu@juju-prod-ues-proposed-migration-machine-2:~$
[12:09] <Laney> but your other three(!) requests are all in the log
[12:13] <rbalint> Laney, interesting, thanks
[12:25] <Laney> rbalint: btw, you should know about retry-autopkgtest-regressions --no-proposed I guess, instead of using 'hello
[12:25] <Laney> '
[12:28] <Laney> ok retried all the unknowns
[12:33] <rbalint> Laney, thanks, but hello works for packages in proposed
[12:39] <Laney> right
[12:40] <Laney> are you planning to automate this or do it regularly?
[12:44] <rbalint> Laney, no, i just use it to mass trigger a range of failures, usually in combination with --log-regex  or with  | grep package=...
[12:45] <Laney> right, I'm worried you are implementing baseline retesting from the outside
[12:45] <doko> Eickmeyer[m]: https://launchpadlibrarian.net/507846803/buildlog_ubuntu-hirsute-amd64.agordejo_0.1.1-0ubuntu2_BUILDING.txt.gz
[12:45] <Laney> if not then ok
[13:23] <rbalint> Laney, could you please check https://code.launchpad.net/~rbalint/britney/+git/hints-ubuntu/+merge/394180 for rbase and others?
[13:23] <rbalint> Laney, also https://code.launchpad.net/~rbalint/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/394209
[13:55] <slyon> May I bring this i386-whitelist change request to any AAs attention? https://bugs.launchpad.net/ubuntu/+source/pycryptodome/+bug/1904987
[14:26] <rbalint> sil2100, could you please check the merges i pinged Laney with?
[14:28] <rbalint> plusone r-base should be ready to migrate after the hints are merged
[14:46] <rbalint> cpaelzer, retried cyrus-imapd/amd64 because is was still sad
[14:46] <sil2100> rbalint: looking in a moment
[14:46] <sil2100> slyon: and here as well
[14:48] <slyon> sil2100: thanks o/ AFAIU a no-change rebuild is needed after the package is accepted into the whitelist. Debdiff is attached to the bug
[15:14] -queuebot:#ubuntu-release- Unapproved: s390-tools (xenial-proposed/main) [1.34.0-0ubuntu8.10 => 1.34.0-0ubuntu8.11] (no packageset)
[15:17] <rbalint> sil2100, added more hints
[15:18] <cjwatson> s390x builders are going down shortly for the DC move - I've put them all on manual
[15:18] <Laney> rbalint: sorry, I'm on a call
[15:19] <Laney> but yes, I should do that with autopkgtest workers too
[15:19] <sil2100> rbalint: ok, looking at it now anyway
[15:20] <sil2100> slyon: will sponsor in a moment!
[15:21] <Laney> ok, s390x autopkgtesters off too
[15:24] <sil2100> rbalint: and what's up with ganeti? I don't see it having any i386 binaries?
[15:30] <sergiodj> doko: would it be possible to file bugs for what you want me to do?  I'm very busy with other stuff right now, but taking care of samba & sssd are definitely at the top of my TODO list
[15:31] <rbalint> sil2100, test is still listed as regressing for ganet/proposed
[15:33] <doko> sergiodj: https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1905048
[15:35] <vorlon> however, --no-proposed fails if the package being triggered has a version in -proposed
[15:35] <vorlon> Laney, rbalint: ^^
[15:36] <Laney> yeah, the doc string notes that
[15:36] <Laney> migration-reference/0!
[15:36] <rbalint> +1
[15:36] <rbalint> until then hello is good enough
[15:48]  * vorlon nods
[16:20] -queuebot:#ubuntu-release- Packageset: Removed confuse from i386-whitelist in hirsute
[16:20] -queuebot:#ubuntu-release- Packageset: Removed ruby-setup from i386-whitelist in hirsute
[16:20] -queuebot:#ubuntu-release- Packageset: Added fonts-noto to i386-whitelist in hirsute
[16:20] -queuebot:#ubuntu-release- Packageset: Added fonts-noto-color-emoji to i386-whitelist in hirsute
[16:20] -queuebot:#ubuntu-release- Packageset: Added pycryptodome to i386-whitelist in hirsute
[16:20] -queuebot:#ubuntu-release- Packageset: Added sphinx-autoapi to i386-whitelist in hirsute
[16:20] -queuebot:#ubuntu-release- Packageset: Added unidecode to i386-whitelist in hirsute
[16:25] <cpaelzer> rbalint: the cyrus issue this adressed was only ppc64
[16:25] <cpaelzer> rbalint: I haven't looked at its amd64 results - might be a different issue there
[16:25] <cpaelzer> but at least in the PKG that got me to cyrus only ppc64 failed, all other arches were ok
[16:41] <doko> Laney, vorlon: are transition tracker updates running?
[16:43] <rbalint> sil2100, vorlon please merge to let us retry petsc4py https://code.launchpad.net/~rbalint/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/394209
[16:45] <rbalint> sil2100, vorlon Laney  i pressed retry, please merge before the package hits the workers
[16:45] <rbalint> is was queued anyway for python3-defaults
[16:45] <doko> I call it a week ...
[16:46] <Laney> doko: should be, I'll look in a minute, maybe something got stuck since there were network outages earlier
[16:47] <sergiodj> doko: thanks
[16:47] <Laney> rbalint: why press retry before it's merged? you'll just get it not run on a large instance.
[16:47] <Laney> as just happened
[16:47] <Laney> :(
[16:51] <rbalint> Laney, i thought the mapping happens when it is picked from the queue
[16:51] <Laney> it does
[16:51] <rbalint> i see no info on http://autopkgtest.ubuntu.com/running about queued items being mapped
[16:52] <rbalint> to length of vm sizes
[16:52] <Laney> but now it's running
[16:52] <Laney> and the thing is not merged
[16:52] <rbalint> Laney, could you please merge it?
[16:53] <Laney> in a minute
[16:53] <rbalint> Laney, thanks
[16:53] <Laney> but still, don't press the button until you get the merge in future please
[16:53] <rbalint> Laney, ok, but please merge quicker
[16:53] <Laney> rbalint: didn't one of your earlier changes here fail to fix a failure? iirc
[16:53]  * Laney shrugs
[16:54] <Laney> if that's right, can you revert it please? having too many of these large tgests can be painful
[16:54] <Laney> they take up 4x as much of our quota than regular ones
[16:54] <rbalint> Laney, that was a different package
[16:55] <Laney> which?
[16:56] <rbalint> Laney, cross-toolchain-base, checking
[16:59] <Eickmeyer[m]> doko: That means there was a fundamental change in Python between when I submitted the package (over two weeks ago!) and now. I'll let uptream know.
[16:59] <Eickmeyer[m]> THIS is why we need more efficiency for package reviews.
[17:00] <rbalint> Laney, i'd like to wait for a successful run of cross-toolchain-base because it passed after ~8 hours when using the small vm and this is a very significant delay in migration
[17:01] <rbalint> Laney, let's see how much time it needs to pass on a big vm
[17:02] <Laney> delaying 3 other tests for the whole length of the run should also weigh into your reasoning
[17:03] <Laney> but ok, if it is a significant gain then it could be justified
[17:19] <Laney> rbalint: right, I see a ton of failures before your changes and a ton after, including a screenful of parallel large tests 😱 so I think you just managed to fail a *little* bit faster
[17:19] <rbalint> Laney, regarding failures my proposal is LP :#1903913
[17:20] <rbalint> Laney, regarding failures my proposal is LP: #1903913
[17:20] <Laney> if it's forced, why are you bothered about retrying it?
[17:20] <rbalint> Laney, running on big vms will help with successful runs
[17:21] <rbalint> Laney, it got forced later
[17:22] <Laney> rbalint: ok, since your bug there is not addressed atm I'm going to remove it from big_packages and once it is fixed and verified on arm64 e.g. using Canonistack please ping me back and we can look to re-add it then
[17:23]  * rbalint shrugs
[17:24] <Laney> if you've followed the logs and saw 'quota exceeded' as much as I have you'd also be concerned about controlling resource usage :-)
[17:24] <Laney> s/you've/you'd/
[17:26] <rbalint> Laney, then don't run the failing tests?
[17:27] <rbalint> Laney, i do care about making the most out of the quota but plaing with the big package list does not help as did not enough in the past years either
[17:27] <rbalint> Laney, what really helps is not running tests with no signal and pushing out excess tests to clouds
[17:28] <Laney> yeah thanks, one thing is a task I can do in 5 seconds, the other takes thinking about, planning, testing and doing so they're not equivalent
[17:28] <rbalint> Laney, does not make the obsolete 5s task worth doing it
[17:29] <Laney> it's not obsolete, not sure why you would say that
[17:29] <Laney> anyway this feels like it's getting close to an argument, so I think we should both step away
[17:30] <rbalint> Laney, ok, the 5s task has a tiny tiny impact
[18:14] <rbalint> Laney, sorry, just would like to improve the testing infra significally, let's discuss the bigger tasks at the next +1 meeting
[19:42] <cjwatson> Eickmeyer[m]: It would just show up in a test rebuild anyway, or maybe surprise you when you least expect it and you're trying to fix something else small - better to find out earlier :)
[19:43] -queuebot:#ubuntu-release- Unapproved: pcl (xenial-proposed/universe) [1.7.2-14build1 => 1.7.2-14build1.1] (no packageset)
[19:43] <Eickmeyer[m]> cjwatson: Agreed, but I still could've been on it sooner. Turns out upstream runs Arch which is still on Python 3.8 (ironically).
[19:44] <mdeslaur> hello SRU team, could someone please approve pcl in the xenial upload queue into -proposed, it's the second part of bug 1573234
[19:46] <vorlon> doko: what is the addition of libgtop2 for in update-i386-whitelist?
[20:01] <vorlon> mdeslaur: this is not a very reproducible test case
[20:01] <vorlon> mdeslaur: I shouldn't have to chase down build-dependencies one-by-one to reproduce the failure
[20:02] <vorlon> (and I'm taking a close look at this one because the idea of no-change SRU rebuilds magically fixing things makes me uneasy)
[20:04] <LocutusOfBorg> arm64/armhf/ppc64el jobs are failing with no logs
[20:04] <LocutusOfBorg> retrying doesn't help
[20:09] <vorlon> possibly related to the datacenter move happening this weekend.  Those bits aren't scheduled to be moved yet but there could be transient network misconfigurations etc
[20:09] <vorlon> cjwatson, wgrant: ^^ can you speak to the state of ARM/ppc64el builders in LP?
[20:13] <vorlon> mdeslaur: git clone of the referenced perception_pcl project, followed by manually futzing with build-deps, and then 'cmake pcl_conversions', gives me a makefile with an empty 'all' target.  I'm not willing to advance this without a cleaner test case
[20:26] <cjwatson> vorlon: Not specifically.  I can see logs that suggest network issues but don't know exactly what, and I'm also EOW - best ask IS
[20:28] <cjwatson> But it's all somewhat at risk at the moment due to network changes being made to prepare for the move
[20:29] <vorlon> right, thanks
[20:39] -queuebot:#ubuntu-release- Unapproved: livecd-rootfs (focal-proposed/main) [2.664.8 => 2.664.9] (desktop-core, i386-whitelist)
[20:48] <mdeslaur> vorlon: ok, thanks, I'll let the reporter know
[21:00] <kyrofa> mdeslaur, vorlon: please take another look at the test case on bug 1573234, it's been greatly simplified and hopefully makes more sense
[21:02] <kyrofa> vorlon, note that the no-change rebuild propogates the dependency fix to properly use the system libproj instead of the vtk-vendored version, which isn't actually used by vtk at all
[21:02] <kyrofa> vorlon, but pcl ended up with the vendored lib in its cmake config files
[21:10] <kyrofa> vorlon, so the situation in Xenial right now is that vtk is built against the system libproj, but anyone linking against it tries to link against its vendored version
[21:31] -queuebot:#ubuntu-release- Unapproved: livecd-rootfs (xenial-proposed/main) [2.408.61 => 2.408.62] (desktop-core)
[21:34] -queuebot:#ubuntu-release- Unapproved: livecd-rootfs (bionic-proposed/main) [2.525.47 => 2.525.48] (desktop-core)
[21:57] -queuebot:#ubuntu-release- Unapproved: livecd-rootfs (xenial-proposed/main) [2.408.61 => 2.408.63] (desktop-core)
[22:57] -queuebot:#ubuntu-release- Unapproved: ubuntustudio-menu (groovy-proposed/universe) [0.49 => 0.50~20.10.1] (ubuntustudio)
[22:57] <Eickmeyer[m]> ^ SRU bug 1905061