Okay, so I'm using over 300 threads that u开发者_开发百科se buffer readers to get information from over 300 sites at a reasonable speed. So basically, it spams this exception a whole bunch of times each second. Meanwhile, it's only ending up getting less than 50% of the information from the sites. So, I was wondering what would be a good sleep time to wait until starting a new thread.
...only experience will tell I guess. This depends a lot on your infrastructure, the quality of you connection, the performance of the OS to manage the sockets...
Just try some small sleeps or less simultaneous connections and tune them to satisfy your needs.
You could also try some simple "sleep and retry" policy, where successive errors would increase the sleep time each time before the next request.
I think your problem is not sleep time, but the number of simultanous operations doing reads. You can use as much threads as you want, but allow only N of them to do I/O operations. You can consider using Semaphore class for entering I/O sections or commons-pool from apache or anything similar.
![Interactive visualization of a graph in python [closed]](https://www.devze.com/res/2023/04-10/09/92d32fe8c0d22fb96bd6f6e8b7d1f457.gif)
加载中,请稍侯......
精彩评论