[00:02] <slangasek> new review> hmmm I guess we're very far outside the boundaries of the kernel SRU process if that didn't get done automatically
[00:03] <slangasek> mwhudson: NEW done
[00:03] -queuebot:#ubuntu-release- New: accepted linux-signed-azure [amd64] (bionic-proposed) [4.15.0-1015.15]
[00:03] -queuebot:#ubuntu-release- New: accepted linux-signed [ppc64el] (bionic-proposed) [4.15.0-26.28]
[00:03] -queuebot:#ubuntu-release- New: accepted linux-signed [amd64] (bionic-proposed) [4.15.0-26.28]
[00:17] <mwhudson> slangasek: hurrah
[05:27] <doko> apw: linux-gcp has old binaries in proposed
[06:06] <doko> LocutusOfBorg: haskell-store ftbfs. old qgis binaries removed
[06:10] <ginggs> doko: [20:14:38] <LocutusOfBorg> can anybody please kick haskell-store out from release and see the magic happen again?
[06:10] <doko> ahh, ok
[06:24] <doko> slangasek: ^^^ I think this needs a manual hint again, once the haskell-store haskell-stack reomvals are published
[07:11] <LocutusOfBorg> <3
[07:14] <LocutusOfBorg> while waiting for mariadb autopkgtests being fixed, I'll sync qgis
[07:56] <LocutusOfBorg> and haskell  is gone! thanks doko!
[09:17] <juliank> Laney: So, what happened to mariadb-10.1 arm64? Does this still need investigating? I don't see any timing out issues in the page at http://autopkgtest.ubuntu.com/packages/m/mariadb-10.1/cosmic/arm64 - it just fails to find packages there
[09:17] <juliank> well, maybe it did not go in there
[09:19] <juliank> Forgot about it yesterday
[10:01] <LocutusOfBorg> Laney, sorry for bothering, can you please have a look or give me hint about pinetry testsuite failing?
[10:01] <LocutusOfBorg> autopkgtest for pinentry/1.1.0-1build2: amd64: Pass, arm64: Regression ♻ , armhf: Pass, i386: Regression ♻ , ppc64el: Always failed, s390x: Regression ♻
[10:01] <LocutusOfBorg> it fails on arm64 i386 s390x, and I blame some autopkgtestsuite /dev wrong binding or permission
[10:02] <LocutusOfBorg> reasons for this are: 1) I can't reproduce on pbuilder i386 environment, 2) they seems to have been started failing without code changes
[10:02] <LocutusOfBorg> so, something in the infrastructure might have changed in the meanwhile...
[10:03] <LocutusOfBorg> I blame /proc or tty issues :)
[10:06] <apw> doko, ta
[10:43] <LocutusOfBorg> can anybody please tell me why gphoto2 is sad for britney?
[11:02] <LocutusOfBorg> indeed, somebody accepted libcdk5 when it was not built everywhere, so gphoto2 has  been built against the old version on some slow architectures
[11:02] <LocutusOfBorg> rebuild issues
[11:02] <LocutusOfBorg> *ed
[11:02] <doko> no
[11:03] <LocutusOfBorg> Get:61 http://ftpmaster.internal/ubuntu cosmic/universe armhf libcdk5-dev armhf 5.0.20161210-5build1 [250 kB]
[11:03] <doko> libcdk5 depends on the perl version in proposed
[11:03] <LocutusOfBorg> doko, look at arm64 log
[11:03] <LocutusOfBorg> trust me, I setup an arm64 chroot just to look :)
[11:04] <LocutusOfBorg> in fact, britney in the big hint, shows gphoto2 only on arm* and somewhere else, not on amd64 i386 and elsewhere
[11:04] <doko> maybe that too, but it won't migrate
[11:04] <LocutusOfBorg> it will
[11:04] <LocutusOfBorg> perl is candidate
[11:04] <LocutusOfBorg> in the big hint, only mariadb is blocking now
[11:04] <LocutusOfBorg>     * amd64: mariadb-client, mariadb-client-10.1, mariadb-plugin-connect, mariadb-plugin-cracklib-password-check, mariadb-plugin-gssapi-client, mariadb-plugin-gssapi-server, mariadb-plugin-mroonga, mariadb-plugin-oqgraph, mariadb-plugin-spider, mariadb-plugin-tokudb, mariadb-server, mariadb-server-10.1, mariadb-server-core-10.1
[11:04] <LocutusOfBorg> this is what is currently blocking
[11:05] <LocutusOfBorg> and arm* has     * arm64: gphoto2, mariadb-client-10.1, mariadb-plugin-connect, mariadb-plugin-cracklib-password-check, mariadb-plugin-gssapi-client, mariadb-plugin-gssapi-server, mariadb-plugin-mroonga, mariadb-plugin-oqgraph, mariadb-plugin-spider, mariadb-server-10.1, mariadb-server-core-10.1
[11:06] <Laney> juliank: Unless there are results on excuses, yes, it needs investigating.
[13:15] <Laney> LocutusOfBorg: This reproduces using an autopkgtest-buildvm-ubuntu-cloud build i386 image
[13:30] <LocutusOfBorg> Laney, do you have a command to reproduce? or is somebody working on that?
[13:30] <LocutusOfBorg> this is really the last stopper
[13:36] -queuebot:#ubuntu-release- Unapproved: lxd (xenial-backports/main) [3.0.1-0ubuntu1~16.04.2 => 3.0.1-0ubuntu1~16.04.3] (edubuntu, ubuntu-server)
[13:46] -queuebot:#ubuntu-release- Unapproved: accepted lxd [source] (xenial-backports) [3.0.1-0ubuntu1~16.04.3]
[14:29] -queuebot:#ubuntu-release- New source: boost1.67 (cosmic-proposed/primary) [1.67.0-0ubuntu1]
[14:35] <juliank> I just launched a mariadb autopkgtest and got it hanging
[14:36] <juliank> it starts a mysqld and then just hangs in there
[14:38] <juliank> It's waiting inside a mutex inside libjemalloc2
[14:39] <juliank> says Backtrace stopped: previous frame inner to this frame (corrupt stack?)
[14:39] <juliank> though
[14:42] <LocutusOfBorg> :/
[14:44] <juliank> There's only one thread, how can that pthread_mutex lockup????
[14:44] <LocutusOfBorg> juliank, https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=871446 ?
[14:44] <LocutusOfBorg> I'm trying to see if upstream has fixes https://github.com/jemalloc/jemalloc/compare/5.1.0...dev
[14:45] <juliank> LocutusOfBorg: I think we can just add arm64 to the blacklist in debian/rules and try again
[14:46] -queuebot:#ubuntu-release- New binary: linux-signed-azure [amd64] (bionic-proposed/main) [4.15.0-1016.16] (kernel)
[14:54] <juliank> Laney: Any idea how to get a PPA in there?
[14:54] <juliank> In the commandline launching autopkgtest
[14:54] <juliank> I could just upload to proposed, but if it does not work, we just spent a lot of cycles retesting all mariadb rdeps
[14:58] <LocutusOfBorg> juergh, they are empty, proposed proposed please, migration! :)
[14:59] <LocutusOfBorg> juliank, ^^ autopkgtest queue is empty
[14:59] <LocutusOfBorg> build queue daemons too...
[14:59] <LocutusOfBorg> and lots is waiting for this
[15:00] <juliank> LocutusOfBorg: OK
[15:00] <juliank> I have to leave anyway
[15:01] <juliank> So I uploaded it
[15:02] <juliank> https://launchpad.net/ubuntu/+source/mariadb-10.1/1:10.1.29-6ubuntu1
[15:02] <juliank> Start in 55 minutes
[15:02] <juliank> Build score:3260 (What's this?)
[15:02] <juliank> is what LP says
[15:03] <juliank> now it's 28 mins
[15:04] <juliank> so it should start soon
[15:04] <juliank> :D
[15:08] <LocutusOfBorg> yep! thanks for doing it!
[15:08] <LocutusOfBorg> I hope we can end this stuff today
[15:08] <LocutusOfBorg> bad juliank is bad :)
[15:08] <LocutusOfBorg> let me reupload
[15:09] <LocutusOfBorg> you didn't remove libjemalloc-dev from b-d on arm64 :o
[15:09] <juliank> LocutusOfBorg: hmm?
[15:09] <juliank> Shouldn't matter
[15:09] <LocutusOfBorg> yep, true
[15:09] <LocutusOfBorg> I hope :)
[15:09] <LocutusOfBorg> if the switch is good
[15:09]  * LocutusOfBorg goes AFK, and waits for the magic to happen again hopefully
[15:10] <juliank> Seems like servers are busy
[15:10] <juliank> I mean, one is building, not sure what the others are doing
[15:10] <LocutusOfBorg> https://launchpad.net/builders
[15:10] <LocutusOfBorg> cleaning, probably means some "hey lets do something on infra, like changing bits"
[15:11] <juliank> I was looking for that page!
[15:11] <juliank> :D
[15:11] <LocutusOfBorg> but they have empty queues, so it should start soon, they don't disable all the infra usually
[15:11] <juliank> we should just cancel everything elswe
[15:11] <juliank> :D
[15:12] <LocutusOfBorg> loooooool
[15:12] <LocutusOfBorg> cancel everything, let mariadb build, and then retry
[15:12] <LocutusOfBorg> (btw mariadb is now building everywhere)
[15:12]  * LocutusOfBorg leaves for train
[15:13] <juliank> OK
[15:37] <cjwatson> LocutusOfBorg: no, "cleaning" is usually that a VM reset crashed at some point
[15:37] <cjwatson> (if it's for more than a few minutes)
[15:48] -queuebot:#ubuntu-release- New: accepted linux-signed-azure [amd64] (bionic-proposed) [4.15.0-1016.16]
[15:56] -queuebot:#ubuntu-release- New binary: linux-signed-oem [amd64] (bionic-proposed/main) [4.15.0-1010.13] (kernel)
[16:31] <bdmurray> I forget is there some tool to rerun a bunch of autopkgtests?
[16:32] <acheronuk> bdmurray: retry-autopkgtest-regressions in ubuntu-archive-tools bzr repo?
[16:34] <bdmurray> This is for an SRU, so not cosmic.
[16:37] -queuebot:#ubuntu-release- Unapproved: installation-guide (bionic-proposed/main) [20160121ubuntu4.1 => 20160121ubuntu4.2] (core)
[16:39] <acheronuk> bdmurray: optional arguments: -s SERIES, --series SERIES: Ubuntu series (default: cosmic)
[16:40] <acheronuk> that implies can be used for other than the dev series, though I admit I have not tried
[16:41] -queuebot:#ubuntu-release- Unapproved: accepted evolution-data-server [source] (bionic-proposed) [3.28.3-0ubuntu0.18.04.1]
[16:43] <acheronuk> seems to give me a retry list that corresponds to regressions on http://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html
[16:44] -queuebot:#ubuntu-release- Unapproved: accepted gnome-control-center [source] (bionic-proposed) [1:3.28.2-0ubuntu0.18.04.1]
[16:44] <bdmurray> acheronuk: Cool, I try it out
[16:47] -queuebot:#ubuntu-release- Unapproved: accepted gnome-disk-utility [source] (bionic-proposed) [3.28.3-0ubuntu1~18.04.1]
[16:50] <bdmurray> coreycb: Is your nova sru for bionic meant to supersede the existing one?
[16:55] <coreycb> bdmurray: that should probably wait until nova clears from proposed
[16:55] -queuebot:#ubuntu-release- Unapproved: partman-auto (xenial-proposed/main) [134ubuntu1.2 => 134ubuntu1.3] (core)
[17:04] <bdmurray> coreycb: okay
[17:15] -queuebot:#ubuntu-release- Unapproved: grub-installer (bionic-proposed/main) [1.128ubuntu8.18.04.1 => 1.128ubuntu8.18.04.2] (core)
[17:42] -queuebot:#ubuntu-release- Unapproved: accepted squashfs-tools [source] (bionic-proposed) [1:4.3-6ubuntu0.18.04.1]
[17:43] <wxl> is there a reason we don't have a mini.iso for 18.10?
[17:43] -queuebot:#ubuntu-release- Unapproved: rejected python2.7 [source] (bionic-proposed) [2.7.15-0ubuntu1]
[17:44] -queuebot:#ubuntu-release- Unapproved: accepted squashfs-tools [source] (xenial-proposed) [1:4.3-3ubuntu2.16.04.2]
[17:45] <wxl> oh nevermind i found it
[17:45] <ginggs> the iso, or the reason?
[18:44] -queuebot:#ubuntu-release- Unapproved: accepted netplan.io [source] (bionic-proposed) [0.36.3]
[19:16] <jdstrand> bdmurray: thanks for processing the squashfs-tools srus! I'm not sure how the queue is managed, but if you could also look at the trusty one, that would be great (curiously, that is actually the most important one atm since this is for the snap store and its still on trusty)
[19:16] <jdstrand> it's*
[19:35] -queuebot:#ubuntu-release- Unapproved: accepted partman-auto [source] (xenial-proposed) [134ubuntu1.3]
[19:38] <bdmurray> jdstrand: For some reason I forget about that release. ;-)
[19:38] <jdstrand> bdmurray: I had a suspicion it might not be looked at as often :)
[19:38] <bdmurray> jdstrand: Well, I should have noticed the bug task.
[19:39] -queuebot:#ubuntu-release- Unapproved: accepted squashfs-tools [source] (trusty-proposed) [1:4.2+20130409-2ubuntu0.14.04.3]
[19:50] <jdstrand> bdmurray: thanks!
[19:53] -queuebot:#ubuntu-release- New binary: libseccomp [s390x] (cosmic-proposed/main) [2.3.3-3ubuntu1] (core)
[19:55] -queuebot:#ubuntu-release- New binary: libseccomp [ppc64el] (cosmic-proposed/main) [2.3.3-3ubuntu1] (core)
[20:00] -queuebot:#ubuntu-release- New binary: libseccomp [i386] (cosmic-proposed/main) [2.3.3-3ubuntu1] (core)
[20:01] -queuebot:#ubuntu-release- New binary: libseccomp [amd64] (cosmic-proposed/main) [2.3.3-3ubuntu1] (core)
[20:01] -queuebot:#ubuntu-release- New binary: libseccomp [armhf] (cosmic-proposed/main) [2.3.3-3ubuntu1] (core)
[20:04] -queuebot:#ubuntu-release- New binary: libseccomp [arm64] (cosmic-proposed/main) [2.3.3-3ubuntu1] (core)
[20:21] <juliank> LocutusOfBorg: so, it turns out the build of mariadb failed on arm64 with test suite issu
[20:21] <juliank> mariabackup: malloc.c:2401: sysmalloc: Assertion `(old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)' failed.
[20:21] <juliank> nice
[20:22] -queuebot:#ubuntu-release- New: accepted linux-signed-oem [amd64] (bionic-proposed) [4.15.0-1010.13]
[20:37] <juliank> so, I'm not sure what's happening there
[21:00] <juliank> I'm off now, so if somebody wants to play more with mariadb-10.1, please do so
[21:08] -queuebot:#ubuntu-release- Unapproved: rejected grub-installer [source] (bionic-proposed) [1.128ubuntu8.18.04.2]
[21:08] -queuebot:#ubuntu-release- Unapproved: grub-installer (bionic-proposed/main) [1.128ubuntu8.18.04.1 => 1.128ubuntu8.18.04.2] (core)
[21:11] <slangasek> well for my part, I was just considering removing it to let the transition through
[21:12] <slangasek> and also grumbling at the packages that depend on mariadb* instead of default-mysql*
[21:12] <slangasek> but the mariadb -> hoel -> glewlwyd -> * dependency chain is annoying
[21:16] <slangasek> juliank: what hanging tests were you trying to fix on arm64 with this upload?  because 1:10.1.29-6build2 built successfully on arm64, and mariadb-10.1's autopkgtests are force-badtest
[21:19] <juliank> slangasek: the mariadb-10.1 ones, they are always in progress
[21:19] <juliank> They don't get to the state where the force badtest takes effect
[21:20] <slangasek> juliank: ah.  well, that's a bug in the infrastructure then, which I'm happy to route around with a different hint
[21:20] <slangasek> I'm rolling back to -6build2, and will see about hinting this through
[21:22] <juliank> slangasek: ok.
[22:06] -queuebot:#ubuntu-release- New binary: aodh [amd64] (cosmic-proposed/main) [6.0.1-0ubuntu3] (openstack, ubuntu-server)
[22:08] <LocutusOfBorg> slangasek, <3
[22:09] <LocutusOfBorg> I don't see the whole migrating, lets wait some more time
[22:09] <LocutusOfBorg> in the meanwhile I'm trying to understand if that test is good to fail
[22:10] <LocutusOfBorg> juliank, my wild guess is:
[22:10] <LocutusOfBorg> # Ignore test suite exit code on unstable platforms
[22:10] <LocutusOfBorg> ifeq ($(DEB_HOST_ARCH),$(filter $(DEB_HOST_ARCH), mips mipsel mips64el alpha powerpc sh4 hurd-i386 sparc64 kfreebsd-i386 kfreebsd-amd64))
[22:10] <LocutusOfBorg> it seems to contain all the architectures where jmalloc is disabled (and some more). I don't think this is a "coincidence" :)
[22:12] <LocutusOfBorg> slangasek, did you already place the hint?
[22:13] <slangasek> LocutusOfBorg: no, I haven't identified anything yet that needs hinting.  Currently I see that mariadb-10.1 arm64 is 'Test in progress (always failed)', which should definitely not block; and catci and dbconfig-common have previous test runs that failed, which I'm now retrying.
[22:13] <LocutusOfBorg> slangasek, retrying makes the test hang
[22:13] <slangasek> (failed due to uninstallable test deps, not due to anything with test hangs)
[22:14] <LocutusOfBorg> but it will hang, like it did the last few days
[22:14] <slangasek> what will hang?
[22:14] <LocutusOfBorg> and it doesn't hang on my qemu
[22:14] <slangasek> I'm not looking at hung tests here
[22:14] <LocutusOfBorg> postinst of mariadb
[22:14] <LocutusOfBorg> seems that mysql has issues with a racy mutex in jmalloc
[22:14] <LocutusOfBorg> this is why juliank disabled jmalloc on arm64
[22:15] <LocutusOfBorg> my proposal is to 1) ignore tests there and force mariadb-10.1 and then 2) reupload mariadb-10.1 with jmalloc disabled on arm64 and tests ignored because of what I wrote above
[22:15] <LocutusOfBorg> so we can re-enable autosync, and do octave transition during the weekend
[22:17] <LocutusOfBorg> I uploaded a "fixed" mariadb in my ppa some seconds ago, this one should make tests pass
[22:18] <LocutusOfBorg> if juliank analized correctly the situation :)
[22:18]  * LocutusOfBorg goes to sleep
[22:18] -queuebot:#ubuntu-release- Unapproved: ndiswrapper (xenial-proposed/universe) [1.60-3~ubuntu16.04.2 => 1.60-3~ubuntu16.04.3] (no packageset)
[22:22] <slangasek> LocutusOfBorg: so, what I currently see in mariadb-10.1 autopkgtest results does not show any hung tests blocking the package.  Only some straight up failures of reverse-dependencies' tests, and a lost test result for slurm-llnl.
[22:25] -queuebot:#ubuntu-release- Unapproved: xtables-addons (xenial-proposed/universe) [2.12-0.1~16.04.2 => 2.12-0.1~16.04.3] (no packageset)
[22:28] <LocutusOfBorg> slangasek, hanging test won't appear on the page :/
[22:30] <LocutusOfBorg> slangasek, Setting up mariadb-server-10.1 (1:10.1.29-6build2) ...
[22:30] <LocutusOfBorg> your retries seems to be all stuck there
[22:30] <LocutusOfBorg> after the timeout, they will be killend and no log shown on result page
[22:31] <LocutusOfBorg> if you want to see tests not hanging, trying this mariadb might be the solution https://launchpad.net/~costamagnagianfranco/+archive/ubuntu/locutusofborg-ppa/+delete-packages
[22:32] <LocutusOfBorg> but I prefer it being hinted and fixed later
[22:34] <slangasek> LocutusOfBorg: hanging tests absolutely show on update_excuses as 'Test in progress'
[22:35] <slangasek> and the only one in that state is slurm-llnl, which if that's the hung test in question, no one has said
[22:39] <slangasek> tests killed during dep setup ought to still be reported, and it's a bug if not.  But ok, now I understand that what we're talking about here is mariadb-10.1 causing hangs in autopkgtests of other packages
[22:47] <LocutusOfBorg> [00:34:20] <slangasek> LocutusOfBorg: hanging tests absolutely show on update_excuses as 'Test in progress' <-- true in general, in this case they were reported as "bad" because dependencies weren't installable due to the old common ubuntu1 mariadb package still being around :)
[22:47] <LocutusOfBorg> btw slangasek the bug is really in jmalloc code probably, or maybe a bad mutex in mariadb that shows up only on arm64
[22:48] <LocutusOfBorg> dbconfig-common will fail exactly with the same hang
[22:48] <LocutusOfBorg> and also cacti
[22:49] <LocutusOfBorg> not sure what happens there, and I can't reproduce locally
[22:49] <slangasek> ok
[22:49] <LocutusOfBorg> we might really blame infra :p (easy answer, but consider it a joke)
[22:50] <LocutusOfBorg> disabling jmalloc was a good try, and I'm pretty sure the package in my ppa won't hang
[22:50] <LocutusOfBorg> I can try a test tomorrow morning, if the package publishes tonight
[22:50] <slangasek> so, I've added the hints now; would of course prefer mariadb-10.1 fixed, so I'm looking forward to the results of your ppa test
[22:51] <LocutusOfBorg> I don't like disabling testsuite on arm64, but testing reverse deps seems to be a superior test
[22:51] <LocutusOfBorg> (and probably the test failure is related to some test that requires jmalloc being enabled))
[22:52] -queuebot:#ubuntu-release- Unapproved: v4l2loopback (xenial-proposed/universe) [0.9.1-4 => 0.9.1-4ubuntu0.1] (no packageset)
[22:53] <LocutusOfBorg> https://autopkgtest.ubuntu.com/request.cgi?release=cosmic&arch=arm64&package=cacti&ppa=costamagnagianfranco/locutusofborg-ppa&trigger=mariadb-10.1/1:10.1.29-6ubuntu2
[22:53] <LocutusOfBorg> this might be the query to run :)
[22:58] -queuebot:#ubuntu-release- Unapproved: dahdi-linux (xenial-proposed/universe) [1:2.10.2~dfsg-1ubuntu2 => 1:2.10.2~dfsg-1ubuntu3] (no packageset)
[23:34] -queuebot:#ubuntu-release- Unapproved: oss4 (xenial-proposed/universe) [4.2-build2010-5ubuntu1~14.04.1 => 4.2-build2010-5ubuntu1~16.04.2] (no packageset)
[23:44] <slangasek> chrisccoulson: seems the libgit2 you synced from experimental is not compatible with current libgit2-glib.  did you get emails telling you that your sync was stuck in -proposed?
[23:46] <sarnold> something he said elsewhere.. "hmmm, https://launchpad.net/ubuntu/+source/libgit2-glib needs fixing" "it actually needs porting to the latest libgit2, which hasn't happened upstream"