[00:00] well, the porter chroots I have access to are broken, and I can't reproduce the sagetex autopkgtest failures (uninstallabilities) w/ chdist [00:03] oh, I /can/ reproduce the armhf one [00:04] aha sagemath [00:19] slangasek: Speaking of that, I would probably have a fix for this telepathy-logger-qt issue if I had access to an Ubuntu-based porterbox... :P [00:20] Maybe spin up an LXD container or something? ;) [00:20] ¯\_(ツ)_/¯ [01:31] -queuebot:#ubuntu-release- New binary: libreoffice [armhf] (cosmic-proposed/main) [1:6.1.0-0ubuntu1] (kubuntu, ubuntu-desktop) [02:37] -queuebot:#ubuntu-release- New binary: libreoffice [arm64] (cosmic-proposed/main) [1:6.1.0-0ubuntu1] (kubuntu, ubuntu-desktop) [03:01] slangasek: Could you please kick the can on python-fluids/0.1.72-2? Regressed autopkgtests in release, and it'll make sphinx (plus a few friends) migrate. [03:06] Hah, so it seems like imexam/0.8.0-2's s390x autopkgtest fails with the same error as the FTBFS for the no-change rebuild... [03:06] * tsimonq2 retries against itself in release [03:09] * tsimonq2 bets we can kick the can on python-scipy as well. [03:11] Same situation with skimage/0.14.0-0ubuntu2/armhf and skimage/0.14.0-0ubuntu2/i386... autopkgtests fail on the same architectures with the same error that it's FTBFS with after a no-change rebuild. Retrying against release... [03:12] After that, python-scipy *should* migrate. [03:23] ...wat? Why does imexam/0.8.0-2/s390x pass? O_o [03:26] uh wat [03:27] * tsimonq2 is confused at the fact that both my skimage retry and my imexam retry have higher versions than their triggers... [03:28] tsimonq2: i *think* there's an endian bug in scipy but as so many things ignore the results of tests during build (angryface) it's hard to know what's going on [03:28] mwhudson: Maybe if I could get the autopkgtest infra to play nice I'd be able to confirm that. ;P [03:29] mwhudson: Oh, right, python-scipy won't be done for a while npw. [03:29] *now [03:30] 12:52:46 AM < tsimonq2> slangasek: fflas-ffpack badtest on armhf? Looks like you retried against release: http://autopkgtest.ubuntu.com/packages/f/fflas-ffpack/cosmic/armhf [03:31] Perhaps a UK/European/other side of the pond ~ubuntu-release member like apw can look ^ [03:41] tsimonq2: not sure how an Ubuntu-based ppc64el porterbox would help you, given that I already said it wasn't reproducible for me... [03:41] tsimonq2: python-fluids> if by 'kick the can' you mean 'badtest', no; that's a python3.7+scipy-related test failure, I'm retriggering the tests with --all-proposed [03:41] Oh hey. [03:42] tsimonq2: skimage> mwhudson was already looking at this yesterday and believes this is a regression in scipy. [03:43] and i've looked a bit today and am absolutely no further into understanding it btw [03:43] slangasek: ppc64el> ack [03:43] fflas, let's see [03:43] slangasek: python-fluids, skimage> ack as well. [03:43] i am currently of the opinion that scientists should not write software but i'll probably get over that [03:43] hah [03:44] yeah, hinting the ffpack [03:45] \o/ [03:45] Thanks. [03:45] mwhudson: I return to that assessment every single time I touch a Debian Med or Debian Science package, then I have to remind myself that their goals are different than mine [03:45] right [03:45] and also, that the real bug is for almost any modern scientific software to be built for architectures with a 32-bit address space [03:46] but the tests fail -> disable the tests mentality is ... surely contrary to their goals too [03:46] that's more of a maintainer thing than an upstream thing of course [03:46] slangasek: telepathy-logger-qt> If I could figure out what exactly error 25 even *is*, that'd be a good first step, heh... [03:51] ohh [03:51] Just had to look further up, jeez: [03:51] dh_acc: abi-compliance-checker -q -l libtelepathy-logger-qt-dev -v1 17.08.0-2build1 -dump debian/libtelepathy-logger-qt-dev.acc -dump-path debian/libtelepathy-logger-qt-dev/usr/lib/powerpc64le-linux-gnu/dh-acc/libtelepathy-logger-qt-dev_17.08.0-2build1.abi.tar.gz returned exit code 2 [03:52] slangasek: How far did your debugging attempts go down the acc rabbit hole? :P [03:53] tsimonq2: I ran the autopkgtest and it didn't exit non-zero. [03:53] so there was no rabbit hole [03:55] I have a feeling if I passed -debug to acc it'll point directly a problem... if I had a porterbox I could just do it via shell, but I don't have a PPA big enough to copy over qtbase (bug 1789361) so I either need to wait for my PPA expansion request to be processed or use Bileto (which it isn't meant for)... [03:55] bug 1789361 in Auto Package Testing "When queueing tests against a PPA, if a trigger isn't found it should fall back to devel-proposed" [Undecided,New] https://launchpad.net/bugs/1789361 [03:56] Oh. [03:56] So looks like the error can be easily found when retrying against itself, due to my no-change rebuild... [03:56] Mkay, so it *is* doable with a PPA then... [03:59] Hmm, this is a nice little TODO in the dh_acc code :P "# TODO if next command fails, should output debug/log info?" [04:25] tsimonq2: you can get artifacts output. e.g. https://git.launchpad.net/~kubuntu-packagers/kubuntu-packaging/+git/khtml/tree/debian/tests/acc?h=kubuntu_cosmic_archive [04:26] acheronuk: TIL, thanks, but first I want to see if this acc hack works :P [04:26] figured out why armhf autopkgtest runners seemed to be going awol; we'd had a cowboyed edit to the cronjob to restart the runner systemd units hourly (and why do I not now find the related bug report?), and the lxd worker was recently reprovisioned so lost that bit. [04:26] acheronuk: Darn, just looked and it didn't... I'll try that. [04:26] so we should be back to not having a 3h window where armhf queues are not progressing. [04:26] \o/ [04:27] acheronuk: This is actually super sweet... thanks. [04:27] hmmm that was https://bugs.launchpad.net/auto-package-testing/+bug/1735878 and it was reported closed. [04:27] Ubuntu bug 1735878 in Auto Package Testing "worker auto restarting should be handled by systemd unit, not by cronjob" [Medium,Fix released] [04:28] tsimonq2: say that if if gives useful output for this ;) [04:28] acheronuk: hehe [04:29] acheronuk: It very likely will, it seems. [04:29] ... so the pkill was removed but the crontabs were never actually redeployed [04:33] I think this actually means the lxd worker wasn't actually redeployed recently... it was redeployed in May which means this was broken for a while until it got noticed [04:49] slangasek: So I found our error... [04:49] Creating library ABI dump ... [04:49] ERROR: can't pack the ABI dump: Cannot allocate memory [04:49] O_O [04:49] See https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-cosmic-tsimonq2-upload-testing/cosmic/ppc64el/t/telepathy-logger-qt/20180829_044637_96ed1@/artifacts.tar.gz [04:53] If it's truly an OOM issue, I'm not sure what the next step is here. [04:53] ...can someone with access to the infra cowboy a resource bump or something weird like that? [04:53] ¯\_(ツ)_/¯ [05:27] tsimonq2: well, in theory that gives me a path to reproduce it, then I can look at what's taking memory and if it's fixable [05:30] slangasek: Awesome. [05:42] fwiw the ppc64el runners have 1.5GiB of total memory [05:50] hmm [05:50] acc is implemented in perl and it uses this much memory? sad [05:54] I'm willing to bet it's glibc's fault, fwiw. :P [05:54] highwatermark of memory usage for a-c-c on ppc64el is 768960k according to /usr/bin/time [05:55] tsimonq2: I've run the tests every which way; -proposed glibc is not a requirement for reproducing [05:55] huh [05:56] slangasek: But if there's 1.5 GB *total*, that means if there's any other test running it'll oom? [05:56] no [05:56] it's 1 autopkgtest per VM [05:56] ah [05:56] but other processes take up memory, so I'm not sure what the effective available memory is [06:04] tsimonq2: I just traced a running instance of the autopkgtest and there was never less than 840MB free on the node [06:05] O_o [06:05] I'm out of ideas if oom isn't it. [06:05] Maybe an acc bug if it's not that.. [06:07] so I *can* make a-c-c OOM, but not in an autopkgtest runner context [06:09] the actual OOM happens when calling 'zip', hmm [06:10] Sounds about right: https://github.com/lvc/abi-compliance-checker/blob/master/abi-compliance-checker.pl#L9366 [06:10] that's what I was going by [06:10] tar does seem more likely, which the code below has. [06:11] ah, zip is only used on windows, ok [06:11] Because that code's ran on Windows. [06:11] jinx [06:16] ah of course [06:16] clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x75b7f8544520) = -1 ENOMEM (Cannot allocate memory) [06:16] because forking tar means creating a copy of the process first and that process is oversized :P [06:16] freeing memory before invoking tar would fix it [06:17] and the failure should be generically reproducible with a cgroup (but not with ulimits, which was my first attempt) [06:20] slangasek: suddenly getting some armhf tests fail with "mkdir: cannot create directory ‘//.config’: Permission denied" [06:20] any issue you know of? [06:20] no [06:22] slangasek: ok. thanks. retried 1, and it passed. retrying another but takes a while. === ogra_ is now known as ogra [07:59] -queuebot:#ubuntu-release- Unapproved: brltty (bionic-proposed/main) [5.5-4ubuntu2 => 5.5-4ubuntu2.0.1] (core) [08:50] -queuebot:#ubuntu-release- New binary: linux-signed-oem [amd64] (bionic-proposed/main) [4.15.0-1018.21] (kernel) [08:59] -queuebot:#ubuntu-release- New binary: linux-signed [amd64] (xenial-proposed/main) [4.4.0-135.161] (core, kernel) [09:00] -queuebot:#ubuntu-release- New: accepted linux-signed-oem [amd64] (bionic-proposed) [4.15.0-1018.21] [09:01] -queuebot:#ubuntu-release- New: accepted linux-signed [amd64] (xenial-proposed) [4.4.0-135.161] [09:26] -queuebot:#ubuntu-release- Unapproved: avahi (bionic-proposed/main) [0.7-3.1ubuntu1 => 0.7-3.1ubuntu1.1] (core) === tomwardill_ is now known as tomwardill [10:38] -queuebot:#ubuntu-release- New binary: python-cffi [amd64] (cosmic-proposed/main) [1.11.5-2] (kubuntu, ubuntu-desktop, ubuntu-server) [11:56] LocutusOfBorg, it seems erlang now can be a sync, how do you feel about updating it for cosmic? Debian changes are mostly bugfix releases [12:30] rbalint, ack [12:31] can you please double check why is it a sync? [13:31] -queuebot:#ubuntu-release- Unapproved: glib2.0 (bionic-proposed/main) [2.56.1-2ubuntu1 => 2.56.2-0ubuntu0.18.04.1] (core) (sync) [14:23] -queuebot:#ubuntu-release- New binary: linux-signed-lts-xenial [amd64] (trusty-proposed/main) [4.4.0-135.161~14.04.1] (kernel) [14:42] -queuebot:#ubuntu-release- New: accepted linux-signed-lts-xenial [amd64] (trusty-proposed) [4.4.0-135.161~14.04.1] [14:52] -queuebot:#ubuntu-release- New: accepted cmake-vala [sync] (cosmic-proposed) [1-1] [14:53] -queuebot:#ubuntu-release- New: accepted ironic [amd64] (cosmic-proposed) [1:11.1.0-0ubuntu3] [14:53] -queuebot:#ubuntu-release- New: accepted libminc [arm64] (cosmic-proposed) [2.4.03-1] [14:53] -queuebot:#ubuntu-release- New: accepted libminc [i386] (cosmic-proposed) [2.4.03-1] [14:53] -queuebot:#ubuntu-release- New: accepted libminc [s390x] (cosmic-proposed) [2.4.03-1] [14:53] -queuebot:#ubuntu-release- New: accepted libreoffice [arm64] (cosmic-proposed) [1:6.1.0-0ubuntu1] [14:53] -queuebot:#ubuntu-release- New: accepted libreoffice [i386] (cosmic-proposed) [1:6.1.0-0ubuntu1] [14:53] -queuebot:#ubuntu-release- New: accepted libreoffice [s390x] (cosmic-proposed) [1:6.1.0-0ubuntu1] [14:53] -queuebot:#ubuntu-release- New: accepted murano-agent [amd64] (cosmic-proposed) [1:3.5.1-0ubuntu2] [14:53] -queuebot:#ubuntu-release- New: accepted panko [amd64] (cosmic-proposed) [5.0.0-0ubuntu2] [14:53] -queuebot:#ubuntu-release- New: accepted ironic [amd64] (cosmic-proposed) [1:11.1.0-0ubuntu2] [14:53] -queuebot:#ubuntu-release- New: accepted libminc [armhf] (cosmic-proposed) [2.4.03-1] [14:53] -queuebot:#ubuntu-release- New: accepted libreoffice [amd64] (cosmic-proposed) [1:6.1.0-0ubuntu1] [14:53] -queuebot:#ubuntu-release- New: accepted libreoffice [ppc64el] (cosmic-proposed) [1:6.1.0-0ubuntu1] [14:53] -queuebot:#ubuntu-release- New: accepted node-csv-spectrum [sync] (cosmic-proposed) [1.0.0-2] [14:53] -queuebot:#ubuntu-release- New: accepted python-cffi [amd64] (cosmic-proposed) [1.11.5-2] [14:53] -queuebot:#ubuntu-release- New: accepted libminc [amd64] (cosmic-proposed) [2.4.03-1] [14:53] -queuebot:#ubuntu-release- New: accepted libreoffice [armhf] (cosmic-proposed) [1:6.1.0-0ubuntu1] [14:53] -queuebot:#ubuntu-release- New: accepted panko [amd64] (cosmic-proposed) [5.0.0-0ubuntu3] [14:53] -queuebot:#ubuntu-release- New: accepted libminc [ppc64el] (cosmic-proposed) [2.4.03-1] [14:53] -queuebot:#ubuntu-release- New: accepted llvm-toolchain-7 [sync] (cosmic-proposed) [1:7~+rc2-1~exp1] [14:55] -queuebot:#ubuntu-release- New binary: node-csv-spectrum [amd64] (cosmic-proposed/universe) [1.0.0-2] (no packageset) [14:55] -queuebot:#ubuntu-release- New: accepted node-csv-spectrum [amd64] (cosmic-proposed) [1.0.0-2] [15:06] -queuebot:#ubuntu-release- New binary: linux-signed-gcp [amd64] (bionic-proposed/main) [4.15.0-1019.20] (no packageset) [15:06] anybody looking at the twisted migration? [15:07] I could help perhaps [15:07] current version in cosmic (17.x) fails with py3.7, the new one has that fixed [15:07] or at least that particular failure [15:08] ugh, http://autopkgtest.ubuntu.com/packages/t/twisted/cosmic/amd64 has one lonely green only, from 2017 [15:08] -queuebot:#ubuntu-release- New: accepted linux-signed-gcp [amd64] (bionic-proposed) [4.15.0-1019.20] [16:02] -queuebot:#ubuntu-release- New binary: linux-signed-azure [amd64] (bionic-proposed/main) [4.15.0-1023.24] (kernel) [16:02] -queuebot:#ubuntu-release- New binary: linux-signed-hwe [amd64] (xenial-proposed/main) [4.15.0-34.37~16.04.1] (kernel) [16:02] -queuebot:#ubuntu-release- New binary: linux-signed-hwe [ppc64el] (xenial-proposed/main) [4.15.0-34.37~16.04.1] (kernel) [16:11] -queuebot:#ubuntu-release- New: accepted linux-signed-hwe [amd64] (xenial-proposed) [4.15.0-34.37~16.04.1] [16:11] -queuebot:#ubuntu-release- New: accepted linux-signed-hwe [ppc64el] (xenial-proposed) [4.15.0-34.37~16.04.1] [16:12] -queuebot:#ubuntu-release- Unapproved: ubuntu-release-upgrader (bionic-proposed/main) [1:18.04.24 => 1:18.04.25] (core) [16:23] -queuebot:#ubuntu-release- Unapproved: accepted ubuntu-release-upgrader [source] (bionic-proposed) [1:18.04.25] [16:25] -queuebot:#ubuntu-release- Unapproved: accepted ubuntu-image [source] (bionic-proposed) [1.4+18.04ubuntu2] [16:25] xnox, bdmurray: ^--- Can we verify that (a) s390x now works and (b) snap migration on amd64 still works? If both are actually tested, I'm happy with that SRU being fasttracked post-verification. [16:27] infinity: Sure I can test the snap migration still working bit. [16:29] -queuebot:#ubuntu-release- Unapproved: accepted ubuntu-image [source] (xenial-proposed) [1.4+16.04ubuntu2] [16:52] slangasek: Did you look at all into the python-cffi/i386 thing? [16:52] slangasek: Just to make sure glibc isn't off the deepend, I did this: https://paste.ubuntu.com/p/VRXRzT9gKK/ [16:52] infinity: no; did mwhudson mutter about it? I don't recall now [16:52] slangasek: So now very curious where it's pulling NaN from. [16:58] slangasek: Oh. So it does that cos(1.23) in two dlopen tests, and it's only one (dlopen_filename) that's returning NaN, so I think what it's actually doing is failing to dlopen by path. [16:58] Which also seems weird, but slightly easier to debug. [16:59] Maybe. [16:59] infinity, do-release-upgrade xenial->bionic fails; do-release-upgrade --proposed xenial->bionic goes past that point and is downloading all the things for me. So I can reproduce original bug; and proposed stuff fixes everything. [16:59] on s390x [17:00] xnox: Shiny. Tell the bug, please. [17:00] xnox: Though maybe without the --proposed thing, don't want IBM people telling their customers that's how to upgrade. :P [17:01] hahahhahhaha [17:02] infinity, the bug is commented [17:02] Ta. [17:31] slangasek, ubuntu-cloud-keyring binary from ubuntu-keyring needs to move from universe to main; it is seeded in bionic and cosmic. cosmic is easy. [17:31] slangasek, but i don't know if there is components-missmatch report for bionic, or how to copy/publish it from bionic-release/universe to bionic-security|updates/main ? [17:33] slangasek: There is no c-m for stable series. And the copy trick needs to be done by an AA. [17:33] Err. [17:33] xnox: ^ [17:33] No can brain. [17:34] infinity, yeah, that's what a vague recalled, but this happens to me only once every 4 years, so details are fuzzy. [17:34] vaguely [17:34] anyway volleyball [17:35] xnox: I'm a bit confused. [17:36] xnox: Oh. Less confused now. [17:37] -queuebot:#ubuntu-release- New binary: llvm-toolchain-7 [ppc64el] (cosmic-proposed/universe) [1:7~+rc2-1~exp2] (no packageset) [17:41] -queuebot:#ubuntu-release- New: accepted llvm-toolchain-7 [ppc64el] (cosmic-proposed) [1:7~+rc2-1~exp2] [17:55] infinity: btw telepathy-logger-qt/ppc64el is definitely ignorable, if we get to that point. hitting the threshold where abi-compliance-checker uses > 50% of free memory in the testbed is not a regression in any of the involved packages [17:56] slangasek: Fun. [17:57] (I'm trying to work around the failure in a-c-c, but perl) [17:57] One has to write some spastically bad Perl to eat all your RAM. [17:57] https://accidentallyquadratic.tumblr.com/ ? [17:58] it's using Data::Dumper for the objects that represent the complete ABI of a very dense C++ lib [17:58] so yes, that's less than ideal [17:58] but I'm not going to refactor that code to teach it how to stream to a file [18:03] infinity: xnox and I have verified bug 1787668 [18:03] bug 1787668 in Ubuntu on IBM z Systems "ubuntu-release-upgrader crashed with KeyError: "The cache has no package named 'ubuntu-desktop'"" [High,In progress] https://launchpad.net/bugs/1787668 [18:06] -queuebot:#ubuntu-release- New binary: linux-signed-azure [amd64] (xenial-proposed/main) [4.15.0-1023.24~16.04.1] (kernel) [18:31] bdmurray: Danke. Releasing. [19:36] -queuebot:#ubuntu-release- New binary: llvm-toolchain-7 [amd64] (cosmic-proposed/universe) [1:7~+rc2-1~exp2] (no packageset) [20:40] -queuebot:#ubuntu-release- Unapproved: fwupdate (cosmic-proposed/main) [12-3ubuntu1 => 12-3ubuntu1] (core) [20:40] -queuebot:#ubuntu-release- Unapproved: fwupdate (cosmic-proposed/main) [12-3ubuntu1 => 12-3ubuntu1] (core) [20:40] -queuebot:#ubuntu-release- Unapproved: fwupdate (cosmic-proposed/main) [12-3ubuntu1 => 12-3ubuntu1] (core) [20:41] -queuebot:#ubuntu-release- Unapproved: fwupdate (cosmic-proposed/main) [12-3ubuntu1 => 12-3ubuntu1] (core) [20:42] -queuebot:#ubuntu-release- Unapproved: fpc (bionic-proposed/universe) [3.0.4+dfsg-18ubuntu1 => 3.0.4+dfsg-18ubuntu2] (no packageset) [21:08] ugh, fwupdate probably shouldn't have been uploaded at all [21:09] slangasek: infinity: do you recall what the state is for fwupdate vs. fwupd? [21:13] cyphermox: I've said that going forward we will not sign uefi binaries for the unseeded one; however I see that both are in main, I could've sworn I had already processed that demotion [21:14] I did remove fwupdate-signed [21:14] fwupdate needs fixed to not spit out uefi signing requests [21:15] infinity, slangasek: i haven't looked at cffi [21:15] -queuebot:#ubuntu-release- Unapproved: rejected fwupdate [amd64] (cosmic-proposed) [12-3ubuntu1] [21:15] -queuebot:#ubuntu-release- Unapproved: rejected fwupdate [armhf] (cosmic-proposed) [12-3ubuntu1] [21:15] -queuebot:#ubuntu-release- Unapproved: rejected fwupdate [arm64] (cosmic-proposed) [12-3ubuntu1] [21:15] cyphermox: so, I've rejected the above [21:15] -queuebot:#ubuntu-release- Unapproved: rejected fwupdate [i386] (cosmic-proposed) [12-3ubuntu1] [21:15] i can, in a couple hours, if you want [21:16] i still hate everything about scipy and am going to ignore that today so i actually achieve something [21:17] slangasek: ok [21:17] slangasek: can you block shim-signed 1.37 in proposed? LP is timing out here... [21:18] maybe I'm being extra paranoid, but I wouldn't mind just breaking my system if it turns out I made some error. [21:21] nvm, got LP to let me do what I want after all [21:21] ok [21:23] slangasek, infinity - so can one of you please promote ubuntu-cloud-kerying into bionic-updates/main somehow? [21:23] slangasek, infinity - so can one of you please promote ubuntu-cloud-keyring into bionic-updates/main somehow? [21:23] xnox: what references it in bionic? [21:24] same thing as in cosmic, supported-misc-servers [21:24] but no reports... [21:27] copying [21:28] and promoted [21:41] cyphermox: I see shim-signed having migrated to cosmic, was that what you expected? [22:01] nah [22:01] but it's fine [22:02] everything can migrate. [22:03] slangasek, linux-image-generic|lowlatency|oem do not depend on intel-microcode in bionic. still. [22:04] they appear to do so in cosmic. [22:04] slangasek, is this expected?! [22:05] xnox: which version? I see the dep from 4.15.0.33.35 linux-image-generic in bionic [22:05] reverse-depends -r bionic intel-microcode must be telling me lies then! [22:06] let me finish upgrading this instance and double check with apt [22:06] xnox: does reverse-depends know about -updates? [22:07] slangasek, yeah, it's there, thanks!