can i use a distributed DB for my WebCrawler,what al开发者_开发知识库l are the available Open Source Distributed Databases?? can i use MySQL as my distributed DB? how amount of data can be stored?
MongoDb might be right for you. It allows sharding and replication.
As said earlier in a comment: use Sphinx (for instance).
精彩评论