[10:02] <cjwatson> wgrant: Ugh, doing a proper whole-request timeout with requests manages to be even more of a nightmare than it is with urllib2.
[10:08] <wgrant> cjwatson: ... srsly?
[10:08] <cjwatson> The sock.shutdown dance is still needed, AFAICS, and it's incredibly hard to pick through the layers to get at the actual socket.
[10:09] <wgrant> How is this not something that everyone wants :(
[10:09] <cjwatson> The response object isn't initialised until after the first recv, so a naïve attempt hangs in lp.services.tests.test_timeout.TestTimeout.test_urlfetch_timeout_after_listen.
[10:09] <cjwatson> See https://github.com/kennethreitz/requests/issues/239#issuecomment-22233420 and cry
[10:10] <cjwatson> I'm seriously tempted to switch it over to using a subprocess rather than a thread so that we can just abruptly kill the thing.
[10:10] <wgrant> Heh :/
[10:11] <wgrant> requests is meant to be the sane thing to use :(
[10:11] <wgrant> I began to doubt that while trying to work out how to, you know, close its connections.
[10:11] <wgrant> And failing.
[10:12] <wgrant> Oh
[10:12] <wgrant> That very issue, in fact :/
[10:12] <cjwatson> The only other decent idea I have so far is to pick through the urllib3 connection pool and forcibly close everything in it.
[10:13] <wgrant> I think I tried that but gave up.
[10:13] <cjwatson> But I think we have to fix this before using GitHostingClient much more ...
[10:13] <wgrant> That may have just been because I was immensely frustrated with the ongoing production incident, rather than because it was fundamentally too difficult.
[10:16] <cjwatson> Maybe if I subclass stuff all the way down.
[10:18] <cjwatson> Subclass HTTPConnectionPool etc. so that I can customise its actual pool and add code to clean it up properly; subclass PoolManager to use my modified HTTPConnectionPool; subclass requests.adapters.HTTPAdapter to use my modified PoolManager
[10:18] <cjwatson> It might actually be possible with only the advertised API.
[10:19] <cjwatson> But seriously.
[10:28] <wgrant> Yup.
[10:28] <wgrant> At least we don't have an ongoing production outage because of it this time!
[10:31] <cjwatson> Was that with swift?
[10:31] <cjwatson> Oh god I'm going to have to subclass Queue.LifoQueue somehow.
[10:33] <wgrant> Right, the librarian EMFILE incident with Swift connections hanging around.
[11:31] <cjwatson> OMG I think this foul contraption might work.
[11:32] <wgrant> I don't believe you.
[12:18] <cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/timeout-with-requests/+merge/252570 - I think it's still a net improvement to be able to use other parts of requests, but ugh
[12:21] <cjwatson> Reassuring that the tests caught my early insufficient attempts, at least.