[00:03] bdmurray, re bug 1781242 -> done [00:03] bug 1781242 in Ubuntu on IBM z Systems "as segfault with invalid -march= option" [Low,In progress] https://launchpad.net/bugs/1781242 [00:36] -queuebot:#ubuntu-release- New binary: knot [s390x] (cosmic-proposed/universe) [2.7.2-2] (no packageset) [00:37] -queuebot:#ubuntu-release- New binary: knot [ppc64el] (cosmic-proposed/universe) [2.7.2-2] (no packageset) [00:42] -queuebot:#ubuntu-release- New binary: knot [amd64] (cosmic-proposed/universe) [2.7.2-2] (no packageset) [00:52] -queuebot:#ubuntu-release- New binary: knot [armhf] (cosmic-proposed/universe) [2.7.2-2] (no packageset) [00:55] -queuebot:#ubuntu-release- New binary: knot [arm64] (cosmic-proposed/universe) [2.7.2-2] (no packageset) [01:00] -queuebot:#ubuntu-release- New binary: knot [i386] (cosmic-proposed/universe) [2.7.2-2] (no packageset) [03:14] slangasek: fwiw i'm planning on doing proposed-migration things on monday so feel free to highlight me with anything you think i should look at [03:14] infinity: you too ^ [03:14] infinity: i'll look at that r-cran-rgenoud thing first probably [03:50] mwhudson: on Monday> I'm sure it'll look all different by then ;) [04:16] Maybe glibc will be migrated? *cough* ;) [05:55] -queuebot:#ubuntu-release- New: accepted knot [amd64] (cosmic-proposed) [2.7.2-2] [05:55] -queuebot:#ubuntu-release- New: accepted knot [armhf] (cosmic-proposed) [2.7.2-2] [05:55] -queuebot:#ubuntu-release- New: accepted knot [ppc64el] (cosmic-proposed) [2.7.2-2] [05:55] -queuebot:#ubuntu-release- New: accepted knot [arm64] (cosmic-proposed) [2.7.2-2] [05:55] -queuebot:#ubuntu-release- New: accepted knot [s390x] (cosmic-proposed) [2.7.2-2] [05:55] -queuebot:#ubuntu-release- New: accepted knot [i386] (cosmic-proposed) [2.7.2-2] [05:55] slangasek: kernel uninstallability in -proposed didn't sort itself then? [05:55] :/ [05:56] I don't know of any reason it hasn't [05:56] ah because linux-signed is still stuck in NEW [05:56] the efi blobs got accepted but not the packages, sorry [05:57] aha. right [05:57] fixed now; allow a bit for publishing [05:57] great. thanks [05:57] -queuebot:#ubuntu-release- New: accepted linux-signed [amd64] (cosmic-proposed) [4.18.0-7.8] [05:57] -queuebot:#ubuntu-release- New: accepted linux-signed [ppc64el] (cosmic-proposed) [4.18.0-7.8] [07:54] -queuebot:#ubuntu-release- New binary: ros-pcl-msgs [amd64] (cosmic-proposed/universe) [0.2.0-8] (no packageset) [07:55] -queuebot:#ubuntu-release- New binary: ros-dynamic-reconfigure [amd64] (cosmic-proposed/universe) [1.5.49-4] (no packageset) [07:56] -queuebot:#ubuntu-release- New binary: ros-navigation-msgs [amd64] (cosmic-proposed/universe) [1.13.0-8] (no packageset) [08:10] -queuebot:#ubuntu-release- New binary: knot-resolver [ppc64el] (cosmic-proposed/universe) [3.0.0-2] (no packageset) [08:11] -queuebot:#ubuntu-release- New binary: knot-resolver [i386] (cosmic-proposed/universe) [3.0.0-2] (no packageset) [08:11] -queuebot:#ubuntu-release- New binary: linux-signed-gcp [amd64] (xenial-proposed/main) [4.15.0-1019.20~16.04.1] (no packageset) [08:13] -queuebot:#ubuntu-release- New binary: knot-resolver [armhf] (cosmic-proposed/universe) [3.0.0-2] (no packageset) [08:14] -queuebot:#ubuntu-release- New binary: knot-resolver [amd64] (cosmic-proposed/universe) [3.0.0-2] (no packageset) [08:14] -queuebot:#ubuntu-release- New binary: knot-resolver [arm64] (cosmic-proposed/universe) [3.0.0-2] (no packageset) [08:30] -queuebot:#ubuntu-release- New: accepted linux-signed-gcp [amd64] (xenial-proposed) [4.15.0-1019.20~16.04.1] [08:52] hello, ping wrt diaspora-installer test hints [08:58] slangasek, cminpack is the last glibc blocker, and testsuite seems flaky enough on almost all architectures, with combo that is a dead upstream.... hint? [08:58] they seems to be the usual "floating point precision issues" that comes with every new glibc [09:01] we might fix the testsuite by updating the numbers, but I don't know how to double check the math behind with another tool [09:02] for sure a change from 1.8410966e-16 to 1.4686870e-16 is not that important, (did anybody really care about this precision level?) [09:02] infinity, ^^ [09:16] -queuebot:#ubuntu-release- New: accepted knot-resolver [amd64] (cosmic-proposed) [3.0.0-2] [09:16] -queuebot:#ubuntu-release- New: accepted knot-resolver [armhf] (cosmic-proposed) [3.0.0-2] [09:16] -queuebot:#ubuntu-release- New: accepted knot-resolver [ppc64el] (cosmic-proposed) [3.0.0-2] [09:16] -queuebot:#ubuntu-release- New: accepted ros-navigation-msgs [amd64] (cosmic-proposed) [1.13.0-8] [09:16] -queuebot:#ubuntu-release- New: accepted knot-resolver [arm64] (cosmic-proposed) [3.0.0-2] [09:16] -queuebot:#ubuntu-release- New: accepted ros-dynamic-reconfigure [amd64] (cosmic-proposed) [1.5.49-4] [09:16] -queuebot:#ubuntu-release- New: accepted knot-resolver [i386] (cosmic-proposed) [3.0.0-2] [09:16] -queuebot:#ubuntu-release- New: accepted ros-pcl-msgs [amd64] (cosmic-proposed) [0.2.0-8] [12:19] infinity, here you have your disapora-installer testsuite regression with new glibc http://autopkgtest.ubuntu.com/packages/diaspora-installer [12:36] LocutusOfBorg, well, clearly the bug is upstream.... and sigar bit needs upgrading. [12:37] no? [12:37] hmmm but there is no new sigar.... [12:47] xnox, exactly [12:47] we can't easily patch something that downloads broken stuff from the internet I would say [12:47] I can open a sigar bug, but that is all [12:48] rubygems doesn't have like, inline patch functionality or something? =) to like patch things on the fly. [12:48] or like fail, patch things inline, continue build =) [12:48] who speaks ruby? :) I don't! [12:59] Would/could someone get the systemd/i386 failure on http://people.canonical.com/~ubuntu-archive/proposed-migration/cosmic/update_excuses.html#iproute2 overridden/ignored? From the various logs it fails for different reasons than iproute2 changes [13:01] -queuebot:#ubuntu-release- Unapproved: accepted debian-installer [source] (trusty-proposed) [20101020ubuntu318.44] [13:04] mfo, ^ d-i will start building in trusty-proposed [13:04] slashd, cool, thanks for the ping! [13:27] mfo, there will be some delay for 'd-i', it failed to build due to following bug : https://bugs.launchpad.net/ubuntu/+source/shim/+bug/1708245 [13:27] Ubuntu bug 1708245 in shim (Ubuntu Artful) "shim can't enable validation and enroll keys in one sitting" [Undecided,Fix committed] [13:27] amd64 only failed ^ [13:29] mfo, once that bug fix, you can request the rebuild of 'd-i' for amd64 [13:31] slashd, ack. interesting, there's no trusty track for the shim/signed bug. i'll take a look. [13:50] remind me if we're under FeatureFreeze, should I upload *and* file the FFe bug? [13:50] or should I file the bug and wait for the approval>? [13:51] (this is for NGINX) [14:07] -queuebot:#ubuntu-release- New binary: neutron-taas [amd64] (cosmic-proposed/universe) [4.0.0~a1~git2018083141.84846d5-0ubuntu1] (no packageset) [15:27] xpdf fixed! poppler is now ok [16:28] slangasek, infinity - any difference in e-16 is irrelevant, and is in fact encoding specific glibc version / cpu values. Hence it never passed on other arches. [16:28] imho cminpack should be overriden, cause the value changes are insignificant. or e.g. do you want to update assertions for all of our achres to have it always green? [16:29] it would need to be refreshed with next major compiler/glibc/-lm changes [16:34] doko, https://bitbucket.org/cffi/cffi/issues/378/float-test-failures-with-gcc8-on-ppc64el should this be escalated to gcc ppc64el maintainers? [16:44] xnox: can you offer a citation for this claim re: cminpack? because previously an amd64 it passed with glibc 2.21-2.27, not a single autopkgtest failure back to xenial [16:45] slangasek, correct. but the test suite difference are beyond the architecture precision capabilities... [16:45] slangasek, i can study that futher. [16:46] xnox: I'm fp-dumb, but if you can provide a reference for this "beyond the architecture precision capabilities" I will override [16:46] (we should file a bug against the package in Debian at the same time) [17:11] LocutusOfBorg: and in what sense is cminpack the last blocker? there are 5 autopkgtest regressions reported on update_excuses [17:16] xnox: retried all the failed systemd-triggered tests so far; nothing particularly interesting there, a couple flaky and some needing --all-proposed or similar [17:16] (glibc entanglement, basically) [17:49] slangasek, indeed, I missed them because too huge queue, perl and glibc are tricky to look at [17:49] btw some stuff still needs rebuilds, e.g. dante and something else [17:52] -queuebot:#ubuntu-release- Unapproved: gnome-flashback (bionic-proposed/universe) [3.28.0-1ubuntu1 => 3.28.0-1ubuntu1.1] (edubuntu) [17:59] LocutusOfBorg: bro; apitrace; unscd; dante. Done. [18:02] slangasek, infinity - i hate that all udebs get >= latest glibc dep by default instead of proper >= acient-glibc-any-ubuntu-provides based on shlibdeps [18:03] and hence getting stuck behind glibc, needlessly. [18:03] xnox: I have a solution for that, it begins with s and ends with biquity [18:07] slangasek: https://code.launchpad.net/~brian-murray/ubuntu-archive-tools/ubuntu-security-bugs/+merge/354138 [18:30] -queuebot:#ubuntu-release- Unapproved: accepted nodejs [source] (bionic-proposed) [8.10.0~dfsg-2ubuntu0.3] [18:32] bdmurray: landed, thanks [19:04] dante was already good but meh, thanks! [19:10] LocutusOfBorg: heh, ok [19:12] xnox: wow, very interesting failure in the open-iscsi autopkgtest w/ --all-proposed, it appears to implicate finalrd + liblz4? [19:14] xnox: or like, systemd's own lib deps fail to be copied to the finalrd for /lib/systemd/systemd-shutdown ? [19:23] * amd64: ayatana-indicator-printers, bluez-cups, boomaga, cloudprint, cloudprint-service, cups, cups-core-drivers, cups-filters, cups-filters-core-drivers, cups-tea4cups, cups-x2go, hplip, hplip-gui, indicator-printers, lsb, lsb-printing, lubuntu-desktop, minc-tools, pdf2djvu, printer-driver-all-enforce, printer-driver-cups-pdf, printer-driver-gutenprint, printer-driver-hpcups, printer-driver-postscript-hp, [19:23] printer-driver-splix, ubuntu-mate-core, ubuntu-mate-desktop [19:23] what the hell is going on with cups-filters? [19:23] why is update_output_notest even worse now [19:31] because it's picking local minima that are suboptimal [19:31] it'll clean up with a 'hint glibc' when the time comes, but I don't want to add that early since it'll just increase the time to generate _notest.tx [19:31] t [19:42] jamespage, coreycb: I'm retrying the neutron/s390x autopkgtest failure against the new python-oslo.reports, but it might be good if neutron's autopkgtests didn't spit out generic messages on failure telling users to look at logs they'll never be able to get to [19:43] (https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-cosmic/cosmic/s390x/n/neutron/20180831_100538_5c078@/log.gz) [19:44] slangasek: i agree that would be useful [20:03] ack thanks [20:04] slangasek, wrt diaspora installer is it ok to hint? [20:07] LocutusOfBorg: I don't know; I hadn't seen any information to suggest it was ok, it looked to my like it was regressing with packages in -proposed. What's your reason for thinking it would be ok? [20:12] Hrm, python-cffi/i386 is annoying. Reducing the testcase to something more easily iterable fails to reproduce the bug. [20:13] slangasek, it is indeed a bug [20:13] but the test downloads ruby gems fom the internet [20:13] so we can't do anything wrt this [20:14] I think I'll open a bug here https://github.com/hyperic/sigar/issues?q=is%3Aissue+is%3Aclosed [20:14] close, but no sigar [20:14] LocutusOfBorg: If it's straight up broken, it should be demoted, not hinted. [20:14] (or temporarily removed) [20:15] "the test downloads ruby gems from the Internet" is not a good justification for removing functioning software from the release [20:15] s/test/package/ [20:16] It's an *installer* package. [20:16] If it's broken upstream, we no can fix. [20:16] Unless it's literally just the test part that is broken. [20:16] ok [20:16] https://github.com/hyperic/sigar/issues/122 [20:16] hyperic issue 122 in sigar "Build failure with glibc 2.28" [Open] [20:17] I don't honestly know if the package installed works or not TBH [20:17] it is a distributed social network site, so it might be needing some gems for who knows which corner case [20:18] but as infinity said, downloading stuff from the internet is difficult to fi [20:18] *fix [20:18] there is a possible entry point with cflags [20:18] su diaspora -s /bin/sh -c "bundle config --local build.sigar '--with-cppflags=\"-fgnu89-inline\"'" [20:18] but I don't think we can "add a flag to fix a build failure" in this case [20:19] LocutusOfBorg: You could add flags to ignore all the warnings, but those warning are legit bugs. [20:19] So... [20:19] ah, so the openssl-entangled failures are actually glibc 2.28-caused failures, because new openssl requires new glibc and sigar ftbfs with new glibc [20:19] so that is a rationale for skiptest'ing openssl [20:19] but openssl won't migrate until glibc is migratable [20:19] infinity, linux_sigar.c:1178:22: error: called object ‘minor’ is not a function or [20:19] function pointer [20:19] this is not a warning... [20:19] and when glibc migrates it will break diaspora-installer [20:19] so [20:20] LocutusOfBorg: Ahh, I missed that one. ;) [20:20] :) [20:20] LocutusOfBorg: Which is direct fallout from the warnings above. [20:20] * LocutusOfBorg goes to sleep, cheers! [20:20] yep sure [20:20] yes, the "test" failure is that the package fails to install [20:20] no, we should not badtest that [20:21] slangasek: FWIW, I stared at cminpack some more (and looked at upstream commit history), and I agree with xnox on his assessment. [20:22] infinity: ok cool. you'll file a bug on the Debian package entitled "don't do stupid test"? [20:22] nice :) [20:22] slangasek: It looks like upstream has adjusted those ref files in the past for glibc precision changing. We only have ref files for one arch (which just happen to also work on ARMv7 because reasons), and the current results appear to show *improved* precision. So, it's failing due to a progression. [20:22] slangasek: Yeah, I will do something along those lines. [20:22] WHY DO YOU HATE ME, PYTHON-CFFI. [20:23] WHY. [20:24] yes, why would a library whose raison d'être is to propagate segfaults from C code into an interpreted language be anything but sunshine and light [20:24] From what I can tell, the library actually works right. [20:25] In that, if I tear out the offending test and run it in isolation, it works. [20:25] So, the test harness is doing something Very Wrong. But only on i386. And wat? [20:26] I think I'm in favor of demoting diaspora-installer, adding a temporary blocks, skiptesting openssl, retesting diaspora-installer to get it to a state that p-m will agree it should be blocked based on test state, remove explicit blocks, let things migrate [20:26] then when upstream gets itself together, a re-test should let it migrate on its own [20:26] infinity: not enough -ffloat-store? [20:27] Needs more -funrolling of loops. [20:35] LP: #1790215 [20:35] Launchpad bug 1790215 in diaspora-installer (Ubuntu) "upstream gems broken with glibc 2.28, fails to install" [Undecided,New] https://launchpad.net/bugs/1790215 [20:35] infinity: was a serious suggestion, if that wasn't obvious. If not using -ffloat-store, one fp-related test could taint another's state [20:37] slangasek: The failure looks more like a complete failure to dlopen(libm.so.6) [20:37] The why is harder to figure out. [20:37] ok [20:40] diaspora-installer demoted etc etc up to "retesting diaspora-installer" [20:41] infinity: do you want another set of eyeballs on it? how are you currently trying to iterate? [20:43] slangasek: Well, I started with isolating the test (which then made it pass, *sigh*), then rant the testsuite with LD_DEBUG=all, which might not be helpful. Tracing back through the twisty maze of abstraction to reconstruct the exact test environment, but for only the failing test, might be helpful. [20:43] Debugging through a python interpreter is not my forte. [20:44] how did you try isolating the test? [20:44] slangasek: Literal copy and paste. [20:44] slangasek, ouch [20:45] slangasek: http://paste.ubuntu.com/p/jXBN4fKNRZ/ <-- Minus the print statements, that *is* the test. [20:45] ok [20:46] xnox: btw, https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-cosmic/cosmic/i386/b/bluez-qt/20180831_184226_c6600@/log.gz - there must be some magic here wrt pinning of base packages, but I'm afraid I don't know where it is happening [20:47] (the failure due to libc6 dep, fine; but libdbus is obviously from dbus source) [20:49] * infinity does that again, but logs the result so he can actually search through it... [20:56] xnox: addendum: only i386 shows that problem with libdbus pinning [20:58] slangasek, lz4 thing looks a bit crazy [20:59] slangasek, but i would blame incomplete injection of systemd from this environment, into the child VM as prepared by the autopkgtest [21:00] as in like a bug in ./debian/tests/patch-image or some such [21:07] cminpack hinted and Debian bug filed. [21:10] slangasek: Okay, LD_DEBUG stomped on my theory. [21:10] slangasek: https://paste.ubuntu.com/p/9n5nGh4KYG/ <-- Looks like it's dlopen()ing and binding the symbol just fine. [21:11] infinity: ok. and I've spent the last half hour intermittently wondering why I couldn't reproduce the failure before noticing that I was testing on amd64 [21:11] \o/ [21:13] slangasek: Aaaaand, you're right. [21:14] slangasek: Adding another float test right before breaks it. [21:14] fun [21:15] slangasek: Specifically, a test that does x = m.sin(1.23) [21:15] Which is, indeed, nan. [21:16] slangasek: Reproducer: http://paste.ubuntu.com/p/3kH6Wf3kY5/ [21:16] is this a problem then of libm itself not being built with -ffloat-store? [21:17] or is that unrelated [21:23] slangasek: I feel like the entire libm testsuite would fail on i386 if that was the issue. :P [21:23] fair [21:24] Also, gcc docs seem to imply that ffloat-store is a really bad idea except in rare cases where you want it to fix precision. [21:24] And this isn't a precision issue. [21:31] slangasek, it looks to me like our cloud has changed. And VMs used to come up with a graphical display such that gdm3 could start on them. But not anymore. [21:31] slangasek, what's the right way to enumerate and assert that "there are no displays here"? [21:31] or maybe systemd's test to install gdm3 and expect it to be started is not a good one? [21:32] xnox: I'm not aware of any recent changes on the cloud [21:32] Command '['ps', 'u', '-C', 'gdm-x-session']' returned non-zero exit status 1. [21:32] or gdm3 not liking anymore what's there.... locally above does work in a VM, if i add a virtio graphics/display. [21:33] hmmmm [21:33] unless i broke something [21:33] hunting this things is not fun [21:34] slangasek, how can i get credentials to launch VMs into the autopkgtest user and ssh into them? [21:34] slangasek, specifically with autopkgtest neutron network-id and machine size? [21:35] xnox: non-trivially. But if you want me to remote hands a VM I will [21:35] slangasek, ok. let me prepare two requests then. [21:35] they will come in an email.... [21:43] * infinity decides his brain won't work without food, and goes to remedy that. [21:43] slangasek: I [21:43] Erm. [21:43] slangasek: I'm getting dangerously close to thinking it might be an okay idea to bisect where glibc broke this, cause I'm not doing well with attempts to reason it out. [21:44] I guess bisection will also start out with me seeing if vanilla upstream has the same bug. [21:44] ok [21:44] Since I'm sure as heck not bisecting with Debian patches on top of every iteration. [21:45] slangasek: I mean, I'm not convinced it's a glibc bug, but if it's a cffi bug, I don't think I'll understand why or how until I see the glibc commit that broke it. :P [21:47] slangasek: So, sadly, I think that might be an "over-the-weekend" project, cause even if I throw the builds at one of the kernel team's stupid huge machines, it'll take a while to drill down. [21:48] Plus, I need to write a quick recipe to build upstream in a way that will kinda work on Debian/Ubuntu without all of userspace blowing up. [21:48] (someone remind me to upstream multiarch six years ago, thanks) [21:51] infinity, i use rpath / ld preload just that one library for the binary under test; when rebuilding systemd things without patches for same reason. [21:51] such that _everything_ else still uses sensible things. [21:52] and yeah build, copy into a vm or whatnot, run hackish shell script test... continue regression. [22:05] infinity: no concerns, obviously it holds up migration for a bit but things are looking good to me overall and it takes as long as it takes. You know anything on timing of rebuild tests? [22:05] slangasek: "rebuild tests" as in bisection passes, or as in doko's rebuild tests of the world? [22:06] infinity: the latter [22:06] Not sure. [22:06] I haven't heard him complain yet that he's waiting on glibc :) [22:06] Heh. [22:06] He will soon. [22:06] But he could also copy glibc from proposed for that, so that's not blocking him. [22:06] also he uploaded binutils so he must not be in a hurry [22:16] slangasek, re: starts with s and ends with biquity -> no arch support this cycle either! [22:18] -queuebot:#ubuntu-release- Unapproved: update-manager (bionic-proposed/main) [1:18.04.11.4 => 1:18.10.4] (core) [22:38] -queuebot:#ubuntu-release- Unapproved: rejected update-manager [source] (bionic-proposed) [1:18.10.4] === Guest70189 is now known as RAOF