[01:01] <Fudge> hey guys, trying to spin up an image for Vinux, Linux for vision impaired with no success. keeps on failing at bootstrap stage checking debian security for disco which is not right, this is my auto/config file. https://paste.ubuntu.com/p/VJWv643hYz/
[04:24] <wxl> mdeslaur: finally submitted that merge request to make usb-creator-kde work. if i could get a quick approval on this, i can move forward with the SRU. https://code.launchpad.net/~wxl/usb-creator/usb-creator/+merge/370094
[04:32] <valorie> ooo, cool wxl
[04:32] <valorie> ask in the #kub chan for testers
[04:33] <valorie> too
[04:33] <wxl> ah yeah thanks i forgot about that
[04:33] <valorie> it nearly always worked for me
[04:33] <valorie> not WELL, but it worked and made working live sessions and well-installed ISOs
[04:34] <wxl> well it hasn't worked since xenial. i can tell you that
[04:41] <valorie> I used it at LFNW
[04:41] <valorie> and at SeaGL
=== Wryhder is now known as Lucas_Gray
[04:59] <wxl> not sure which version valorie but i know it doesn't work in lubuntu back to xenial and i can also confirm at least in bionic it doesn't work in kubuntu https://askubuntu.com/questions/1158249/puzzling-message-from-startup-disk-creator/1158252#1158252
[04:59] <valorie> OK
[04:59] <valorie> I have been upgrading for quite a few cycles
[05:00] <valorie> so I might not have the latest of that
[05:00] <wxl> it's remotely possible kubuntu's got some magic library or something in it but i doubt it given the fix
[05:01] <wxl> bug report confirms it in kubuntu 17.04, 16.04.3, 18.04
[05:11] <valorie> right, it's been inconsistent
[05:13] <cpaelzer> I'm wondering about a component mismatch that IMHO should have happen but wasn't triggered in Eoan, I summarized it in the related MIR at https://bugs.launchpad.net/ubuntu/+source/edk2/+bug/1570617/comments/28
[05:13] <ubottu> Launchpad bug 1570617 in edk2 (Ubuntu) "[MIR] edk2" [Undecided,In progress]
[05:13] <cpaelzer> apw: doko: you are the AAs that should be awake the earliest, if you (or anyone else) can help me out understanding what I'm missing in this case that would be great
[05:13] <wxl> valorie: you should check to see if it's still working and if it is, drop me a line with the version you're on
[05:14] <valorie> Sysinfo for 'valorie-Oryx-Pro': Running inside KDE Plasma 5.16.3 on Ubuntu 19.04 (Disco Dingo) powered by Linux 5.0.0-20-generic, CPU: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz at 3399-3400/3800 MHz, RAM: 11302/32115 MB, Storage: 311/1144 GB, 237 procs, 6.67h up
[05:14] <valorie> let me find a thumb key
[05:58] <tobikoch> Laney: kenvandine: vorlon: I added proper error checking, where the base of a snap is determined. Together with mvo's sanity check on seed.yaml, this should prevent inconsistently seeded images like the one in bug #1828500.
[05:58] <ubottu> bug 1828500 in livecd-rootfs (Ubuntu) "snapd fails always in Optimised Ubuntu Desktop images available in Microsoft Hyper-V gallery" [Undecided,Confirmed] https://launchpad.net/bugs/1828500
[07:56] <cpaelzer> (sorry) repeating my question from earlier this morning hoping more peopl are around now
[07:56] <cpaelzer> I'm wondering about a component mismatch that IMHO should have happen but wasn't triggered in Eoan, I summarized it in the related MIR at https://bugs.launchpad.net/ubuntu/+source/edk2/+bug/1570617/comments/28
[07:56] <ubottu> Launchpad bug 1570617 in edk2 (Ubuntu) "[MIR] edk2" [Undecided,In progress]
[07:56] <cpaelzer> apw: doko: you are the AAs that should be awake the earliest, if you (or anyone else) can help me out understanding what I'm missing in this case that would be great
[07:58] <Unit193> At 4am on a Monday?  noooo.
[08:04] <sil2100> cpaelzer: hey! So one reason why it might not be showing up is because the component-mismatches page is outdated
[08:05] <sil2100> cascardo: outdated and actually looking just weird - the timestamp is from 2019-07-12 20:30
[08:05] <sil2100> cpaelzer: ^
[08:05] <sil2100> cascardo: (ignore my earlier message, highlighted wrong person)
[08:06] <cpaelzer> sil2100: yeah you are right, but my main issue is that it actually migrated to -release
[08:06] <cpaelzer> with the known mismatch in place
[08:06] <cpaelzer> thanks for the hint on the page being outdated thou!
[08:08] <apw> cpaelzer: is  it a mismatch as recommends?
[08:08] <cpaelzer> yes
[08:08] <cpaelzer> was it changed that recommends are no more mismatches?
[08:09] <apw> or is suggests the missmatch
[08:09] <cpaelzer> they are recommends for sure, I double checked the build log and posted that in the bug that I linked
[08:10] <cpaelzer> I also checked in a eoan container and apt-cache show confirms what I expect
[08:38] <cpaelzer> apw: did you find why the qemu<->edk2 component mismatch was not triggered?
[08:39] <apw> cpaelzer, seemts the reporting is broken; being fixed right now
[08:40] <cpaelzer> In my case I waited for the mismatch and hence reported it since it migrated without
[08:41] <apw> cpaelzer, in your case you broke it :)
[08:41] <cpaelzer> do we need to be afraid that other mismatches might have slipped in the last few days?
[08:41] <cpaelzer> wut
[08:41] <cpaelzer> tell me what I did wrong so I can avoid it apw
[08:41] <apw> cpaelzer, the report was bust by your name
[08:41] <cpaelzer> lol
[08:42] <apw> cpaelzer, but also i don't think we gate on it
[08:42] <apw> we fix things in it
[08:42] <cpaelzer> no gating and fixing the report which was disliking my special char
[08:42] <cpaelzer> ok that would match what I ahve seen then
[08:42] <cpaelzer> thanks for taking care of it apw
[08:42] <apw> oh i did nothing; i am mearly a messenger
[08:46] <apw> /
[08:46] <cpaelzer> so the solution for now is to drop my lovely ubuntu logo to get things working again?
[08:46] <apw> cpaelzer, na, the report is being repaire
[08:46] <cpaelzer> or are you already fixing the script
[08:46] <cpaelzer> thanks
=== pieq_ is now known as pieq
[09:13] <LocutusOfBorg> doko good morning, syncpackage -f python3-stdlib-extensions?
[09:53] <rbasak> cpaelzer, apw: while you're on the topic, kronosnet shows in https://people.canonical.com/~ubuntu-archive/component-mismatches-proposed.svg as not approved, but the bug https://bugs.launchpad.net/ubuntu/+source/kronosnet/+bug/1811139 is "In Progress" which the documentation defines as approved. I couldn't explain to rafaeldtinoco why the report disagrees.
[09:53] <ubottu> Launchpad bug 1811139 in kronosnet (Ubuntu) "[MIR] kronosnet" [High,In progress]
[09:58] <cpaelzer> rbasak: its status is approved
[09:58] <cpaelzer> it waits for an AA to resolve it by promoting
[09:58] <cpaelzer> but it doesn't show up in https://people.canonical.com/~ubuntu-archive/component-mismatches either (maybe due to the same issue)
[09:58] <cpaelzer> apw: said he fixed the script
[09:59] <cpaelzer> so it might show up when this is regrenerated
[10:36] <Laney> it is now
[10:41] <Laney> As for rbasak's observation, if that's the correct interpretation of MIR process then it'd be fixed in the script at around https://bazaar.launchpad.net/~ubuntu-archive/ubuntu-archive-tools/trunk/view/head:/component-mismatches#L386
[10:43] <cpaelzer> arr you are right, since the change that triggers the mismatch is in the archive it needs to move to Fix committed
[10:52] <cpaelzer> fixed the status on the bug
[10:52] <rbasak> cpaelzer: I got that from your diagram (can't find the link right now)
[10:54] <cpaelzer> rbasak: there https://wiki.ubuntu.com/MIRTeam#Process_states
[10:55] <cpaelzer> we missed step 4->5
[10:55] <cpaelzer> since in the case of kronosnet the change triggering that has already happened
[10:56] <rbasak> Ah. I see.
[10:56] <rbasak> The report should be able to accomodate that though, right?
[10:56] <rbasak> If the MIR bug is In Progress but the component mismatch is present, then treat it as if it's Fix Committed.
[10:57] <cpaelzer> the line that Laney referred showed me that without the status change (which I now have done) it will not be added to "approved_mirs"
[10:57] <cpaelzer> not sure where in the report that makes it end up differently without reading the rest
[10:58] <mdeslaur> wxl: sure! I'll look at it today
[10:58] <cpaelzer> rbasak: I tihnk on the next run you should now get "yellow" instead of "darkkhaki"
[11:03] <LocutusOfBorg> ahasenack, I uploaded squid with a workaround for LP: #1835831
[11:03] <ubottu> Launchpad bug 1835831 in squid (Ubuntu) "FTBFS: gcc9 stringop-truncation and others" [High,New] https://launchpad.net/bugs/1835831
[11:03] <LocutusOfBorg> so we unblock a bunch of other stuff
[11:03] <LocutusOfBorg> since the issues are there since the begin...
[11:15] <rbasak> cpaelzer: thanks!
[11:33] <cpaelzer> rbasak: the updated report is online - your color changed, but kronosnet is still not listed in the text output
[11:33] <cpaelzer> rbasak: you might want to run the tool from ubuntu-archive-tools yourself and check why?
=== ricab is now known as ricab|lunch
[13:08] <kenvandine> tobikoch: thanks
[13:09] <LocutusOfBorg> doko, I'm moving ncurses transition to old, I don't think it is useful anymore, right?
[13:13] <LocutusOfBorg> xnox, I'm removing also some ssl1.0 transition tracker...
=== ricab|lunch is now known as ricab
[14:05] <xnox> sure
[14:39] <LocutusOfBorg> thanks
[14:40] <LocutusOfBorg> xnox, if you have time, can you please tell me why dh-cargo needs that .buildinfo hack?
[14:41] <LocutusOfBorg> "Touch .cargo_vcs_info.json to update timestamp"
[14:41] <LocutusOfBorg> I kept the delta, but I'm happy to drop it
[15:53] <xnox> LocutusOfBorg:  because launchpad rejects builds with bogus zero unix timestamp.
[15:53] <xnox> LocutusOfBorg:  whilst dak was patched to whitelist dh-cargo .cargO_vcs_info.json file
[16:07] <LocutusOfBorg> xnox, interesting, how can I check if in the meanwhile LP has bee fixed?
[16:07] <LocutusOfBorg> because I tried to upload a dh-cargo in sync in my ppa and the deb was published
[16:11] <cjwatson> We don't consider this a bug in LP
[16:12] <cjwatson> The check is still present - if you didn't run into it then your test was bad
[16:43] <LocutusOfBorg> cjwatson, it seems obvious that my test is not good, because I don't know "which" deb is getting rejected... I guess not dh-cargo itself
[16:43] <LocutusOfBorg> https://launchpad.net/~costamagnagianfranco/+archive/ubuntu/locutusofborg-ppa/+build/17272858
[16:43] <cjwatson> I dunno, sorry
[16:43] <cjwatson> I would assume it's things that are built using dh-cargo
[16:43] <LocutusOfBorg> yes, probably, but apt-file shows really few files
[16:44] <LocutusOfBorg> anyway, I hope xnox will answer me, so I can update the changelog with some more verbose information, or open a bug explaining what is going on and how to reproduce
[16:44] <LocutusOfBorg> the fix might be good for debian too, even if they patched dak
[16:44] <cjwatson> Also pretty sure that xnox is wrong that dak was patched to whitelist those files, since there's no mention of "cargo" in dak
[16:45] <cjwatson> It's possible that Dinstall::PastCutoffYear is configured to something that isn't what's in the git tree
[16:45] <cjwatson> Or that it's accidentally broken for some other reason, or I dunno, something else
[16:46] <LocutusOfBorg> I think we should ideally have one Ubuntu bug for each package that has delta... so we can track if we can discard it or not
[16:46] <LocutusOfBorg> or at least verbose changelogs, or debian bugs...
[16:50] <LocutusOfBorg> probably new rust packages are the culprit, my bionic apt-file didn't show that many as the eoan version
[16:51] <ginggs> I've needed to touch files to fix timestamps rejected by LP in another package, dak was happy with them.
[16:52] <ahasenack> xnox: hi, in case you are not aware: https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1836329
[16:52] <ubottu> Launchpad bug 1836329 in apache2 (Ubuntu) "Regression running ssllabs.com/ssltest causes 2 apache process to eat up 100% cpu, easy DoS" [Critical,In progress]
[16:53] <LocutusOfBorg> cjwatson, why can't the infra accept them to avoid that kind of troubles?
[16:55] <LocutusOfBorg> ginggs, you don't remember which one, right?
[16:56] <ginggs> LocutusOfBorg: no, possibly some node thing
[16:56] <cjwatson> LocutusOfBorg: It's very clearly wrong and is often a mistake
[16:56] <cjwatson> It would be good to figure out why dak is accepting them, because according to a plain reading of its code AFAICS it should be rejecting them
[16:56] <cjwatson> It may be a dak bug
[16:57] <cjwatson> grep for PastCutoffYear in dak and you'll find a check that is very similar to the one in Launchpad (which is in lp.archiveuploader.nascentuploadfile:BaseBinaryUploadFile.verifyDebTimestamp)
[16:57] <cjwatson> In fact we got the check from dak in the first place - it wasn't a Launchpad invention
[16:59] <LocutusOfBorg> ok, the reasoning behind what I say is that we should understand if there is a bug and fix it, and I'm willing to forward it in Debian if needed of course
[17:00] <LocutusOfBorg> in the meanwhile I could reproduce it
[17:00] <LocutusOfBorg> https://launchpadlibrarian.net/433190036/upload_17272877_log.txt
[17:01] <cjwatson> IMO the correct fix is on the dak side to make it reject such binaries, since from its code it clearly intends to do so
[17:01] <cjwatson> Then it will be obvious that packages need to be fixed in Debian in the small number of cases they haven't been already
[17:02] <LocutusOfBorg> sure, I fully agree
[17:02] <LocutusOfBorg> this is why I raised the question about this delta... thanks for helping me
[17:02] <cjwatson> Ah, so there does exist a Lintian exception for cargo_vcs_info.json
[17:03] <cjwatson> What's not clear is why dak's BinaryTimestampCheck apparently isn't being used
[17:04] <cjwatson> It seems bogus to me to make exceptions, because the original justification for the extreme-time-travel check was that it could cause errors on extraction (perhaps with non-dpkg tools, I'm not sure)
[17:09] <xnox> cjwatson:  dak autorejects got moved to lintian autorejects, which whitelists cargo.
[17:10] <cjwatson> xnox: I don't see the relevant check being disabled in dak though
[17:10] <xnox> was my understanding of how it now works.
[17:10] <cjwatson> Can you demonstrate it to me from the dak code?
[17:10] <xnox> cjwatson:  ok. I will trace all the pieces then.
[17:11] <cjwatson> As I say, BinaryTimestampCheck still seems to exist and to be used
[17:19] <rafaeldtinoco> ahasenack: for the samba gvfs regressions, I can't see anything apart from a flaky test for async/scheduled calls (when doing a Gio.File.new_for_path()->trash() call)
[17:19] <ahasenack> rafaeldtinoco: ok, let me retry them
[17:19] <rafaeldtinoco> ahasenack: if they still fail in that point I can suggest a sleep time in between the thrash call and the assert
[17:19] <ahasenack> rafaeldtinoco: mark the timestamps of the current red tests (it's part of the log filename)
[17:20] <rafaeldtinoco> ok
[17:20] <ahasenack> rafaeldtinoco: so you know if the log you are seeing is the one from now, or from the retry
[17:31] <xnox> cjwatson:  https://paste.ubuntu.com/p/XzCGS66ycH/ that check only checks timestamps on the control tar members; not on the data tar members.
[17:31] <xnox> (i adjusted the callback locally, and ran it against debian's deb with a similar file)
[17:32] <xnox> if it was interated on the data, it would have raised a reject.
[17:32] <xnox> *itterated
[17:32] <xnox> https://github.com/Debian/dak/blob/master/daklib/checks.py#L530 -> iterates control tarball only
[17:49] <ahasenack> xnox: vorlon: hi, I need some guidance wrt https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1836329
[17:49] <ubottu> Launchpad bug 1836329 in apache2 (Ubuntu) "Regression running ssllabs.com/ssltest causes 2 apache process to eat up 100% cpu, easy DoS" [Critical,In progress]
[17:49] <ahasenack> I reproduced the bug
[17:49] <ahasenack> I tried one patch which might have had something to do with it, https://github.com/apache/httpd/commit/524608b65ec410e797a7283e6e142f8e5a74be26
[17:49] <ahasenack> but that didn't fix it
[17:49] <ahasenack> now I can keep digging, or revert, and thus reopen https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1833039
[17:49] <ubottu> Launchpad bug 1833039 in apache2 (Ubuntu Cosmic) "18.04/Apache2: rejecting client initiated renegotiation due to openssl 1.1.1" [High,Fix released]
[17:50] <ahasenack> the subject of that latter bug is misleading, it basically affects clients using ssl certificate authentication. Each https connection gets delayed 15s or so
[17:50] <xnox> ahasenack:  .... use nginx? =)
[17:51] <xnox> ahasenack:  first day back from holiday =)))))) sorry for the party mood still =)
[17:51] <vorlon> when I learned that apache2 doesn't have support for being an SNI reverse-proxy, I became an nginx convert :P
[17:51] <vorlon> ahasenack: I think the correct answer here is that we need to keep digging
[17:51] <ahasenack> xnox: same for me
[17:51] <ahasenack> (day back after pto)
[17:52] <ahasenack> vorlon: question is basically if the 100% cpu usage for a process after the scan is worse than the 15s delay for people using client cert auth
[17:52] <vorlon> ahasenack: both regressions are unacceptable to leave in -updates; if you want me with my SRU hat to make a call on which regression is worse and if it warrants a revert, I'll take a minute to think about it
[17:52] <ahasenack> (for the uploading a revert point)
[17:53] <ahasenack> vorlon: ok. I have a ppa setup to fix the first case, which affected people have tried and vouched that it fixed their ssl client cert auth case
[17:53] <vorlon> ahasenack: if you can keep working on finding a fix in the meantime, I'll chew on the question of reverting - thanks for escalating
[17:53] <ahasenack> hm, except I deleted it
[17:53] <ahasenack> vorlon: ok
[18:08] <xnox> vorlon:  ahasenack: well, one way to revert to tls1.2 only is to rebuild apache against openssl1.0
[18:09] <ahasenack> xnox: tls 1.3 is not available in that apache version
[18:09] <ahasenack> even the first bug, with client cert auth, was on tls 1.2
[18:10] <xnox> ahasenack:  indeed, that's why revert from openssl 1.1.1 to 1.0.2
[18:10] <ahasenack> why not 1.1.0?
[18:10] <ahasenack> or whatever was there before
[18:10] <ahasenack> same soname?
[18:10] <xnox> we don't have 1.1.0 available
[18:10] <xnox> we only have 1.0.2 and 1.1.1 available now.
[18:11] <xnox> (and lots of things depending on libssl1.1 > 1.1.1 now, hence can't downgrade libssl1.1 to 1.1.0 easily anymore)
[18:11] <xnox> however
[18:11] <xnox> i don't know if apache modules link against libssl directly or indirectly too, and if they all have to have the same libssl (i.e. mod_python mod_perl etc..)
[18:12] <xnox> ahasenack:  also does that ssl test cpu usage reproducible on eoan too?
[18:12] <xnox> apache master?
[18:12] <ahasenack> xnox: something for me to test, I was also wondering
[18:13] <ahasenack> but eoan doesn't have latest upstream yet
[18:13] <ahasenack> 2.4.38, upstream is at 2.4.39
[18:13] <ahasenack> we are with debian still on this one
[18:14] <ahasenack> xnox: good point on the indirect deps, that could be a mess if mod_ssl only is using an older ssl, and some other module is linked with 1.1.1
[18:14] <TJ-> ahasenack: was trying to debug it on a server of mine (18.04) that has 2.4.29-1ubuntu4.7 but there are no dbgsym packages for that version, which has stumped me. I was going to attach gdb to the spinning processes
[18:14] <ahasenack> TJ-: did you add the dbg repo?
[18:23] <TJ-> ahasenack: oh indeed, the lists are in /var/lib/apt/lists too but there's no "apache2-bin-dbgsym"
[18:25] <TJ-> ahasenack: looking at the raw ddebs.ubuntu.com the versions are 2.4.38-2ubuntu2 2.4.18-2ubuntu3.10 2.4.7-1ubuntu4.22
[18:25] <ahasenack> TJ-: I don't control any of that, I just upload the source. I'd expect this to be just taken care of automatically
[18:28] <TJ-> ahasenack: hmmm! the build log shows it built apache2-dbg (so no -dbgsym)
[18:29] <TJ-> ahasenack: so its apache2-dbg/bionic-updates,bionic-proposed 2.4.29-1ubuntu4.7
[18:35] <TJ-> ahasenack: link to the gdb "bt full" https://iam.tj/projects/ubuntu/apache2-libssl1.1-spinning.txt
[18:38] <ahasenack> during a renegotiation test
[18:38] <ahasenack> I was really hoping this would fix it: https://github.com/apache/httpd/commit/524608b65ec410e797a7283e6e142f8e5a74be26
[18:38] <ahasenack> at least it sounded related
[18:39] <TJ-> ahasenack: and here's the other thread - this looks more like it - stuck in _int_malloc() https://iam.tj/projects/ubuntu/apache2-libssl1.1-spinning-malloc.txt
[18:54] <TJ-> ahasenack: in apache's ssl_engine_io.c there's an interesting comment: "We rely on SSL_get_error() after the read, which requires an empty error queue before the read in order to work properly." ... so if rc = SSL_read(inctx->filter_ctx->pssl, buf + bytes, wanted - bytes); returns < 0 the code hits the else {...} block ... I suspect then its following if (ssl_err == SSL_ERROR_SYSCALL) { ... and then
[18:54] <TJ-> hitting the if (APR_STATUS_IS_EAGAIN(inctx->rc) {...} and then finally "continue;  /* Blocking and nothing yet?  Try again. */" -- which means it'll spin inside the while(1) {} endlessly
[18:59] <TJ-> ahasenack: OK, it's not the SSL_ERROR_SYSCALL its the SSL_ERROR_WANT_READ clause. I added a breakpoint in there are line 715, gdb reports "ssl_err = 2" which is SSL_ERROR_WANT_READ
[19:00] <ahasenack> I see some mentions of clearing ssl errors in the commit log
[19:00] <ahasenack> https://svn.apache.org/repos/asf/httpd/httpd/trunk@1845768
[19:04] <ahasenack> let me try 2.4.39 in the meantime
[19:04] <ahasenack> latest upstream
[19:20] <ahasenack> TJ-: xnox: vorlon: apache 2.4.38 from eoan works fine
[19:20] <ahasenack> I'll keep trying some interesting commits
[19:21] <ahasenack>     * modules/ssl/ssl_engine_io.c (bio_filter_out_write,
[19:21] <ahasenack>       bio_filter_in_read): Clear retry flags before aborting
[19:21] <ahasenack>       on client-initiated reneg.
[19:21] <ahasenack> like that
[19:40] <TJ-> ahasenack: This one looks likely: https://github.com/apache/httpd/commit/524608b65ec410e797a7283e6e142f8e5a74be26#diff-f9e8db5f4b7fae788784f623962bf96a
[19:44] <ahasenack> TJ-: I tried it :(
[19:44] <ahasenack> TJ-: it's in https://launchpad.net/~ahasenack/+archive/ubuntu/apache-regression-test-2/
[19:44] <ahasenack> unless I made a silly mistake applying the patch
[19:45] <ahasenack> dpkg-source: info: applying ssl-read-rc-value-openssl-1.1.1.patch
[19:45] <ahasenack> maybe multiple will be needed
[19:47] <TJ-> ahasenack:  looking more closely, that patch doesn't touch the if (ssl_err == SSL_ERROR_WANT_READ) { path
[19:47] <TJ-> ahasenack: maybe the same approach is needed for SSL_ERROR_WANT_READ
[19:48] <TJ-> whoever thought "while(1)" was a good idea needs shooting!
[19:57] <TJ-> ahasenack: right, a BIG clue. ssl_io_input_read() is being called with len == 0 !
[19:57] <TJ-> (gdb) p len
[19:57] <TJ-> $2 = (apr_size_t *) 0xffe62aa8
[19:57] <TJ-> (gdb) p *len
[19:57] <TJ-> $3 = 0
[19:57] <TJ-> ahasenack: in which case there's never going to be any more data to read
[19:58] <ahasenack> what's the defined behavior in that case for that ..._read() call?
[20:08] <TJ-> I'm not sure...but *len is copied into bytes and wanted and those are used to call SSL_read() and as I currently understand it, the code "SSL_read(inctx->filter_ctx->pssl, buf + bytes, wanted - bytes" is effectively "SSL_read(inctx->filter_ctx->pssl, buf + 0, 0 - 0"
[20:25] <TJ-> ahasenack: digging deeper, SSL_read() is returning -1, which originates from ssl_read_internal() and one place where that obviously can be caused is if (s->handshake_func == NULL) {...} but also it could be one of the two 'return' statements, such as "return s->method->ssl_read(s, buf, num, readbytes);"
[20:25] <ahasenack> ssl ain't easy
[20:26] <TJ-> note though the value of 'num' here is 8192: Breakpoint 4, SSL_read (s=0x57164400, buf=0xf5691040, num=8192) at ../ssl/ssl_lib.c:1771
[20:27] <TJ-> which implies wanted == 8192 and bytes == 0 or similar
[20:28] <TJ-> ahasenack: hypothesis:  if the cause is  if (s->handshake_func == NULL) {...} might that imply a protocol mismatch of some form?
[20:28] <ahasenack> or an aborted connection, the log is full of those
[20:30] <TJ-> I think we can rule this one out - I dropped a breakpoint inside that block and it doesn't trigger
[20:32] <ahasenack> I'm now testing https://github.com/apache/httpd/commit/7fa21ea6602b30cc43d4f485777545dd73bb25a6 and the one about the SSL_read() rc
[20:33] <TJ-> it isn't s->mode == SSL_MODE_ASYNC clause either. s->mode == 16 which is I think # define SSL_MODE_RELEASE_BUFFERS 0x00000010U
[20:35] <TJ-> which makes it the: " return s->method->ssl_read(s, buf, num, readbytes); "
[20:35] <TJ-> which *looks like* TLSv1 ?
[20:35] <TJ-> (gdb) p s->method
[20:35] <TJ-> $11 = (const SSL_METHOD *) 0xf704dba0 <tlsv1_server_method_data>
[20:35] <TJ-> (gdb) p s->method->ssl_read
[20:35] <TJ-> $12 = (int (*)(SSL *, void *, size_t, size_t *)) 0xf6fdda20 <ssl3_read>
[20:43] <ahasenack> TJ-: hey, that last one might have fixed it
[20:43] <ahasenack> TJ-: wanna give it a try to confirm? https://launchpad.net/~ahasenack/+archive/ubuntu/apache-regression-test-2/+packages
[20:49] <TJ-> ahasenack: for some reason add-apt-repository is hanging on me! hope this isn't related to the bug!
[20:49] <ahasenack> TJ-: haha
[20:49] <ahasenack> lp could be slow
[20:49] <ahasenack> or fetching the gpg key for the ppa
[20:49] <ahasenack> from the key server
[20:50] <ahasenack> xnox: vorlon: I applied https://github.com/apache/httpd/commit/524608b65ec410e797a7283e6e142f8e5a74be26#diff-f9e8db5f4b7fae788784f623962bf96a and https://github.com/apache/httpd/commit/7fa21ea6602b30cc43d4f485777545dd73bb25a6 and that seems to work
[20:50] <ahasenack> maybe just one is needed, remains to be checked, but looks like progress
[20:54] <vorlon> \o/
[21:00] <ahasenack> TJ-: I'm eod, but will peek in here later tonight
[21:22] <TJ-> ahasenack: sorry, turns out my host has lost IPv6 connectivity; I'll test it once I've got that fixed, and I'll comment on bug  #1836329
[21:22] <ubottu> bug 1836329 in apache2 (Ubuntu Bionic) "Regression running ssllabs.com/ssltest causes 2 apache process to eat up 100% cpu, easy DoS" [Critical,In progress] https://launchpad.net/bugs/1836329