| cjwatson | wgrant: Ugh, doing a proper whole-request timeout with requests manages to be even more of a nightmare than it is with urllib2. | 10:02 |
|---|---|---|
| wgrant | cjwatson: ... srsly? | 10:08 |
| cjwatson | The sock.shutdown dance is still needed, AFAICS, and it's incredibly hard to pick through the layers to get at the actual socket. | 10:08 |
| wgrant | How is this not something that everyone wants :( | 10:09 |
| cjwatson | The response object isn't initialised until after the first recv, so a naïve attempt hangs in lp.services.tests.test_timeout.TestTimeout.test_urlfetch_timeout_after_listen. | 10:09 |
| cjwatson | See https://github.com/kennethreitz/requests/issues/239#issuecomment-22233420 and cry | 10:09 |
| cjwatson | I'm seriously tempted to switch it over to using a subprocess rather than a thread so that we can just abruptly kill the thing. | 10:10 |
| wgrant | Heh :/ | 10:10 |
| wgrant | requests is meant to be the sane thing to use :( | 10:11 |
| wgrant | I began to doubt that while trying to work out how to, you know, close its connections. | 10:11 |
| wgrant | And failing. | 10:11 |
| wgrant | Oh | 10:12 |
| wgrant | That very issue, in fact :/ | 10:12 |
| cjwatson | The only other decent idea I have so far is to pick through the urllib3 connection pool and forcibly close everything in it. | 10:12 |
| wgrant | I think I tried that but gave up. | 10:13 |
| cjwatson | But I think we have to fix this before using GitHostingClient much more ... | 10:13 |
| wgrant | That may have just been because I was immensely frustrated with the ongoing production incident, rather than because it was fundamentally too difficult. | 10:13 |
| cjwatson | Maybe if I subclass stuff all the way down. | 10:16 |
| cjwatson | Subclass HTTPConnectionPool etc. so that I can customise its actual pool and add code to clean it up properly; subclass PoolManager to use my modified HTTPConnectionPool; subclass requests.adapters.HTTPAdapter to use my modified PoolManager | 10:18 |
| cjwatson | It might actually be possible with only the advertised API. | 10:18 |
| cjwatson | But seriously. | 10:19 |
| wgrant | Yup. | 10:28 |
| wgrant | At least we don't have an ongoing production outage because of it this time! | 10:28 |
| cjwatson | Was that with swift? | 10:31 |
| cjwatson | Oh god I'm going to have to subclass Queue.LifoQueue somehow. | 10:31 |
| wgrant | Right, the librarian EMFILE incident with Swift connections hanging around. | 10:33 |
| cjwatson | OMG I think this foul contraption might work. | 11:31 |
| wgrant | I don't believe you. | 11:32 |
| cjwatson | wgrant: https://code.launchpad.net/~cjwatson/launchpad/timeout-with-requests/+merge/252570 - I think it's still a net improvement to be able to use other parts of requests, but ugh | 12:18 |
| cjwatson | Reassuring that the tests caught my early insufficient attempts, at least. | 12:21 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!