[05:51] <Unit193> While I might dispute the validity of the NMU, someone that is gtk2-rm crazy could sync gparted from experimental.
[08:34] <Gargoyle> doko: There's a bug in MySQL 8 which seems to have been the same or re-surfaced as I have found an exact same bug un launchpad for 5.5 from 3 years ago. I've had a poke round launchpad but cannot find where to file an updated report or submit a patch. I would like to do it if someone can point me in the right direction.
[08:35] <Gargoyle> Bug is actually with the logrotate script, so probably just a packaging issue and not an issue with MySQL 8.0 itself. (https://bugs.launchpad.net/ubuntu/+source/mysql-5.5/+bug/1558471)
[09:41] <Gargoyle> I've greated a fresh patch from the code here:- https://code.launchpad.net/~usd-import-team/ubuntu/+source/mysql-8.0/+git/mysql-8.0 and I've tried to look upstream, but debian is even more confusing as they have dropped mysql from stable. :/ Now I'm lost!
[09:41] <Gargoyle> s/greated/created
[10:25] <rbasak> Gargoyle: o/
[10:25] <rbasak> Gargoyle: you can send a merge proposal against https://salsa.debian.org/mariadb-team/mysql/tree/mysql-8.0/ubuntu/develif you like
[10:25] <rbasak> Or file a bug against https://bugs.launchpad.net/ubuntu/+source/mysql-8.0/+filebug
[10:49] <jbicha> Unit193: oh I didn't realize that was an NMU. Have you tried the new gparted yet?
[10:52] <Gargoyle> Thanks rbasak. Done (https://salsa.debian.org/mariadb-team/mysql/merge_requests/28) :-)
[11:08] <rbasak> Gargoyle: thanks! Just looking at this now.
[11:09] <rbasak> Gargoyle: I can reproduce on 19.10 by running "logrotate -f /etc/logrotate.d/mysql-server" but not "logrotate -f /etc/logrotate.conf" which I find puzzling.
[11:10] <rbasak> It makes sense to take your merge as it's obviously correct, but in terms of importance for backporting to 19.10, how does it affect users in practice?
[11:12] <Skuggen> Maybe the base config already does something like this?
[11:13] <rbasak> What I'm wondering is that since in the default configuration /var/log/mysql.log isn't created (I think?), then in the default configuration logrotate would never warn like this?
[11:13] <rbasak> If so, then we should take the fix for future releases but perhaps there's no need to backport the fix?
[11:14] <Skuggen> «su root syslog» is in /etc/logrotate.conf
[11:19] <rbasak> Aha
[11:19] <rbasak> So this isn't a bug after all?
[11:19] <Skuggen> And yeah, the package default is /var/log/mysql/error.log
[11:21] <Skuggen> I don't really know much about how logrotate works :)
[11:21] <Skuggen> Hm, wait, I was looking on a 18.04 box
[11:21] <Skuggen> In 19.10 it seems to be «su root adm»
[12:26] <RikMills> cyphermox: can unity-autopilot be got rid of? it depends src:autopilot, which depends on parts of src:python-qt4 which are dropped in -proposed package
[12:33] <Gargoyle> rbasak: It stops the logrotate service from starting.
[12:33] <Gargoyle> https://paste.ubuntu.com/p/SCf58hcgYM/
[12:34] <Gargoyle> ^ Relevant lines from "journalctl -b 0"
[12:34] <rbasak> Gargoyle: could you give me full steps to reproduce please, on 19.10 presumably?
[12:35] <Gargoyle> Yup. 1 sec
[12:51] <Gargoyle> rbasak: Hmm. Nope. Apparently I cannot! With my changes removed I can still run "sudo /usr/sbin/logrotate -d /etc/logrotate.conf" without errors.
[12:52] <Gargoyle> rbasak: This would make sense because "syslog" is actually a member of "adm". :-/
[13:04] <rbasak> Gargoyle: OK, so no bug?
[13:05] <rbasak> "U2F support in OpenSSH HEAD" https://marc.info/?l=openssh-unix-dev&m=157259802529972&w=2
[13:06] <rbasak> This would be nice for Focal, but I suspect there won't be an upstream release in time. I assume cherry-picking is going to be frowned upon?
[13:07] <Gargoyle> There's a bug somewhere, rbasak. It seems like it's intermittent though:- https://paste.ubuntu.com/p/PX5Zpy4nCp/
[13:07] <Gargoyle> I'm going to do a bit more poking. Looks like it might be an issue with the post rotate and "sudo systemctl status logrotate.service" was showing me a red herring.
[13:10] <rbasak> OK
[13:10] <cjwatson> rbasak: I wouldn't be comfortable with that because (a) it's a large and complicated set of commits to CP, (b) it'd be easy to make a security-critical mistake, and (c) it wouldn't be unheard of for upstream to catch security-critical errors of their own in this sort of thing before release
[13:11] <cjwatson> I agree there's no particular assurance that there'll be an upstream release in time; the release schedule isn't regular, but somewhere around every 4-6 months is typical (though some have been a lot quicker).
[13:12] <rbasak> cjwatson: I thought as much, thanks
[13:13] <cjwatson> Interesting that they went for a new key type, which means all servers have to update too.  I guess that's another thing to add to my Twisted patch queue
[13:13] <cjwatson> (i.e. it'll be a while before we can support it in LP)
[13:15] <cjwatson> I've had basically the full patch set necessary to support Ed25519 in LP ready for a while, but the Twisted patch review cycle grinds pretty slowly :(
[13:18] <Gargoyle> rbasak: It's a race condition.
[13:25] <Gargoyle> rbasak: Not very easy to visualise, but here's a full boot log:- https://termbin.com/zn3c
[13:28] <Gargoyle> rbasak: If you search for "Starting Rotate log files", then then scroll down, "Starting MySQL Community Server" although it's still in the same second is a page and a half later. My guess would be that when mysql's logs do actually need rotating, the post rotate script tries to run "sudo /usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf flush-logs" but mysql is not yet up and running.
[13:29] <rbasak> Ah, I see
[13:29] <rbasak> So we need to add a check before the mysqladmin call to no-op if mysqld isn't running perhaps?
[13:29] <rbasak> Skuggen: ^?
[13:30] <rbasak> Seems like something that flush-logs should have an option for :)
[13:30] <rafaeldtinoco> tjaalton: ping. would u mind approving sg3-utils SRU for disco ? it has already being checked/verified/tested in bug 1833618.
[13:31] <Gargoyle> I would still like to try and create reliable steps to reproduce, but logrotate somehow knows my log files don't need rotating - even if I decompress the last rotated one
[13:36] <Skuggen> rbasak: Hm, the script runs a mysqladmin ping command just before to verify the server is up
[13:36] <Skuggen> https://salsa.debian.org/mariadb-team/mysql/blob/mysql-8.0/ubuntu/devel/debian/mysql-server-8.0.mysql-server.logrotate#L16
[13:37] <Skuggen> Actually, what exactly does that if check? That the mysqladmin command doesn't output anything to stderr?
[13:40] <Gargoyle> Skuggen: It outputs "mysql is alive". If mysql is alive - and nothing on error.
[13:42] <Skuggen> Ah, of course, that if is for the failure case. Then the flush-logs command should only be run if the server is up and ready
[13:42] <Gargoyle> So if mysql is not alive, we enter that block and test the outcome of "killall -q -s0 -umysql mysqld;"
[13:43] <Skuggen> Yeah
[13:43] <Gargoyle> Which has an exit status of 1 when mysql is not running.
[13:44] <Gargoyle> What is signal 0 ?
[13:44] <tjaalton> rafaeldtinoco: no release on a friday, sorry
[13:44] <rafaeldtinoco> ah ok. was just following what is written in the sru wiki
[13:45] <rafaeldtinoco> maybe we should update it the
[13:45] <Skuggen> So if mysql isn't running it should return success
[13:45] <Skuggen> Signal 0 is sort of a dry-run thing, if I understand it correctly
[13:45] <cyphermox> RikMills: I suppose unity-autopilot could be dropped (at a glance, it looks like only unity wants it); but this is really a question for the desktop team
[13:45] <Skuggen> i.e. it won't actually kill mysql, but will check if it's there
[13:46] <cyphermox> RikMills: as long as the tests are still runnable somehow as they were, it would probably be fine to stop shipping them as a package
[13:46] <Skuggen> So if mysqld is in the startup phase, process running but not ready yet, it makes sense that the logrotate script will fail, I think
[13:46] <Gargoyle> Skuggen: Thats what I was just typing! :D
[13:47] <Skuggen> So some test to detect that state is what we need :)
[13:48] <Skuggen> Which is basically what rbasak said :D
[13:58] <Gargoyle> Turned that postrotate script into a stand alone bash script and ran it with a while true and started mysql. Conformed that it hits the "exit 1" during mysql startup.
[14:06] <Gargoyle> Crude but effective! XD (https://paste.ubuntu.com/p/dCj5ZsvxXH/)
[14:07] <Gargoyle> I ran "service mysql start" in another term when that was running ^^
[14:10] <TomyWork> hi. are you aware of a problem connecting to DFS shares on bionic that was introduced between 2:4.7.6+dfsg~ubuntu-0ubuntu2.11 and 2:4.7.6+dfsg~ubuntu-0ubuntu2.13? That version has been out for at least a week now (I installed it on oct 24) and I can't see anything on the bug tracker about it. It worked for me on oct 22 and I noticed it only today (hadn't tried mounting that share between oct 22 and today) and going directly to the (only) share
[14:10] <TomyWork> behind it works. The person responsible for all the file servers says nothing has changed server-side since oct 22, so I'm quite confident that this was introduced with the update.
[14:10] <TomyWork> So, if any samba maintainers here are unaware of the issue, I'll write up a bug report about it.
[14:11] <rbasak> TomyWork: that's not related tohttps://lists.ubuntu.com/archives/ubuntu-server/2019-October/007967.html is it?
[14:12] <TomyWork> temporally correlated for sure :)
[14:12] <TomyWork> i don't recognize any error messages
[14:13] <rbasak> Please do file a bug, and if you're reporting a regression in stable release, tag it regression-update
[14:13] <TomyWork> btw is there any way I can downgrade to .11 so i can confirm this issue?
[14:13] <TomyWork> ...as a regression?
[14:18] <rbasak> Yes
[14:19] <rbasak> Launchpad will have the old binaries
[14:19] <rbasak> Via https://launchpad.net/ubuntu/+source/samba/+publishinghistory
[14:19] <rbasak> https://launchpad.net/ubuntu/+source/samba/2:4.7.6+dfsg~ubuntu-0ubuntu2.11 I think?
[14:19] <TomyWork> found it, thanks
[14:19] <rbasak> Then click on amd64 (I assume)
[14:20] <rbasak> You will need to work out which packages from that source you have installed (and thus need to downgrade)
[14:20] <TomyWork> yeah i'll just go through the list on packages.ubuntu.com for the source package
[14:22] <TomyWork> dpkg -l ctdb libnss-winbind libpam-winbind libparse-pidl-perl libsmbclient libsmbclient-dev libwbclient-dev libwbclient0 python-samba registry-tools samba samba-common samba-common-bin samba-dbg samba-dev samba-dsdb-modules samba-libs samba-testsuite samba-vfs-modules smbclient winbind 2>/dev/null | grep ^ii
[14:22] <TomyWork> that works :)
[14:28] <TomyWork> ok, rebooting to be sure it's live
[14:34] <Gargoyle> Skuggen: rbasak: I think I can re-write that postrotate script to check for all 3 possible outcomes (mysql not running, bad debian-sys-maint user, running and connecting ok) with a single check and eliminate the need for the second "if killall.... etc" check which is causing the race condition.
[14:36] <rbasak> Gargoyle: that sounds good! I'm working on something else right now so haven't confirmed any of the detail, but I really do appreicate any fixes you want to give us :)
[14:36] <Gargoyle> Should I open a new bug?
[14:37] <rbasak> Good question
[14:37] <rbasak> I don't mind if you do
[14:37] <rbasak> If you just want to send a Salsa merge request that's fine too, though if discussion gets complicated then that would probably best be done in a bug
[14:38] <rbasak> And then there's the question of whether the fix needs to be backported to stable Ubuntu releases
[14:38] <Gargoyle> OK. I'll get the patch/MR ready and see how well I can explain it! :-)
[14:38] <rbasak> If the conclusion is that it does, then we'll need a bug to exist to track the status of that
[14:38] <rbasak> Thanks!
[14:40] <Skuggen> Gargoyle: If you'd make a pr for that it would be great!
[14:42] <tjaalton> rafaeldtinoco: oh, you mean approve to -proposed and not -updates? yeah, that can be done
[14:42] <rafaeldtinoco> tjaalton: #) yep. thx !
[14:42] <mdeslaur> Skuggen: so it turns out my s390x problem is when mysql is built with openssl after all...I properly enabled openssl in 5.7.27 and I'm hitting the same issue
[14:43] <mdeslaur> Skuggen: I'm kind of stuck now
[14:43] <xnox> mdeslaur:  is it because openssl tries to use CPU hw acceleration and it's not available on your LPAR / VM?
[14:44] <xnox> mdeslaur:  cause that's "new" in 1.1.1 partially
[14:44] <mdeslaur> xnox: how would I find out?
[14:45] <xnox> invoke jfh
[14:46] <xnox> there are command line things to check if all the right CPU functionality is enabled and available, but i don't remember them.
[14:46] <mdeslaur> jfh?
[14:48] <TomyWork> rbasak, regression confirmed. Mounting works with .11
[14:49] <TomyWork> actually let me upgrade again first, to see whether it was just a reboot that was needed :D
[14:49] <TomyWork> (unlikely since everything else worked)
[14:52] <TomyWork> rbasak, ok forget about the whole story, I'm back on .13 and the issue isn't there anymore. I dont know what it was.
[14:52] <TomyWork> it might be an issue, but it's definitely not a regression
[14:56] <rbasak> TomyWork: OK. Thank you for investigating anyway - negative results are also good results, and now you know what to do if you do ever spot a regression :)
[15:02] <TomyWork> yeah, and how to downgrade if one is already reported but not fixed yet :)
[15:02] <Skuggen> mdeslaur: It's during the build, right? I could try to diagnose it in a ppa
[15:03] <mdeslaur> Skuggen: take the current bionic package, switch to -DWITH_SSL=system, add libssl-dev to build-depends, and it will hang during build as soon as the server is started for the tests
[15:04] <mdeslaur> Skuggen: last thing in the log is "Installing system database..."
[15:18] <Gargoyle> In the current script, it has: "test -x /usr/bin/mysqladmin || exit 0" followed by a comment "# If this fails, check debian.conf" what does that mean? It it trying to say the installation is broken because there is a rote script for a package that is not installed?
[15:19] <rbasak> I'm not sure about the comment
[15:19] <rbasak> If you're asking about the -x test, that's pretty common in Debian
[15:19] <rbasak> It's because a package removal doesn't remove its configuration files
[15:20] <rbasak> So that later reinstallation will maintain the previous configuration
[15:20] <Gargoyle> Ahh. ok.
[15:20] <rbasak> So configuration that is actually code must check for that condition and handle it gracefully
[18:42] <popey> vorlon: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1789118
[18:42] <popey> This bug appears fixed in distro, but (as per comment #20) installer images are broken, so unable to install Ubuntu in xen.
[18:47] <vorlon> popey: http://gb.archive.ubuntu.com/ubuntu/dists/bionic/main/installer-i386/current/images/ doesn't have the fix because it's the release pocket.  They want http://gb.archive.ubuntu.com/ubuntu/dists/bionic-updates/main/installer-i386/current/images/
[18:47] <vorlon> well, I mean, they shouldn't want anything i386
[18:47] <vorlon> but generally
[18:48] <gpiccoli> Hi rbasak, I'd like to discuss LP #1847924 when you have some minutes
[18:59] <popey> Ah. Thank you
[20:14] <Gargoyle> Skuggen: rbasak: This is what I have been able to do with the best of my knowledge as it stands at the moment. Bug = https://bugs.launchpad.net/ubuntu/+source/mysql-8.0/+bug/1850980, MR = https://salsa.debian.org/mariadb-team/mysql/merge_requests/29
[20:15] <Gargoyle> I decided to create the bug anyway because there's still an un-addressed edge case.