=== ojwb is now known as Guest55901 === Guest55901 is now known as ojwb === ojwb is now known as Guest74151 === Guest74151 is now known as ojwb [19:50] mōrena [20:01] morning [20:02] morning [20:58] morning [21:09] morning [21:10] got bitten badly by google last week. turns out that their RSS web crawler ignores robots.txt files. asking for an RSS feed of 5000 epubs every 3 minutes without caching makes my server sad when someone leaves their google homepage open for a couple of days [21:10] eep [21:11] * ojwb can see an argument for ignoring robots.txt there, but hitting every 3 minutes isn't sane [21:17] the answer, naturally, is caching [21:17] :) [21:39] morning [22:45] It's interesting to find weaknesses due to third parties activities. I discovered that I wasn't caching nicegear's rather heavy 404 page and just about brought the server down when some PCI-DSS company ran a stupid vulnerability scan. [22:45] And, morning. [22:46] snail: well, maybe make epub cheaper to deliver ? [22:58] lifeless: the issue was that the RSS feed was based on a solr search. the ePubs are already stored as flat files on the disk [23:05] ah [23:13] morning [23:22] Morning.