开发者

Multithreaded Downloading Through Proxies In Python

开发者 https://www.devze.com 2022-12-08 20:13 出处:网络
What would be the best library for multithreaded h开发者_开发百科arvesting/downloading with multiple proxy support? I\'ve looked at Tkinter, it looks good but there are so many, does anyone have a spe

What would be the best library for multithreaded h开发者_开发百科arvesting/downloading with multiple proxy support? I've looked at Tkinter, it looks good but there are so many, does anyone have a specific recommendation? Many thanks!


Twisted


Is this something you can't just do by passing a URL to newly spawned threads and calling urllib2.urlopen in each one, or is there a more specific requirement?


Also take a look at http://scrapy.org/, which is a scraping framework built on top of twisted.

0

精彩评论

暂无评论...
验证码 换一张
取 消