开发者

Scraping data straight to Cassandra database

开发者 https://www.devze.com 2023-03-15 12:35 出处:网络
Can anyone help me with Cassandra databa开发者_如何学运维se? Actually, I want to through the scraped data directly toCassandra. Currently, I am using scrapy for parsing data from different websites.

Can anyone help me with Cassandra databa开发者_如何学运维se? Actually, I want to through the scraped data directly to Cassandra. Currently, I am using scrapy for parsing data from different websites.

Thanks-in advance.

ssich


In Scrapy, once you have scraped the data you can run it through an item pipeline to store it.

The documentation is easy to understand: http://doc.scrapy.org/en/latest/topics/item-pipeline.html

Just use the python driver for Cassandra within you custom item pipeline to store the results.


Edit: Updated the dead link with working one.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号