I'm a graduate student whose research is complex network. I am working on a project that involves analyzing connections between users (follwing and follwe开发者_Go百科rs). Is it possible to write a crawler for twitter based on friendship information?
I looked around but couldn't find any things useful so far. thanks Leila
I don't know whether something "out of the box" exists in this case.
But, some frameworks exist in which it is quite easy to write crawlers/spiders.
For instance:
In Python: http://scrapy.org/
In Java: http://java.sun.com/developer/technicalArticles/ThirdParty/WebCrawler/
I appreciate Scrapy for its simplicity in this case. But you might want to look for a library suited for the languages you are familiar with.
精彩评论