Insert performance with Cassandra

sorry for my English in advance.

I am a beginner with Cassandr开发者_开发技巧a and his data model. I am trying to insert one million rows in a cassandra database in local on one node. Each row has 10 columns and I insert those only in one column family.

With one thread, that operation took around 3 min. But I would like do the same operation with 2 millions rows, and keeping a good time. Then I tried with 2 threads to insert 2 millions rows, expecting a similar result around 3-4min. bUT i gor a result like 7min...twice the first result. As I check on differents forums, multithreading is recommended to improve performance. That is why I am asking that question : is it useful to use multithreading to insert data in local node (client and server are in the same computer), in only one column family?

Some informations : - I use pycassa - I have separated commitlog repertory and data repertory on differents disks - I use batch insert for each thread - Consistency Level : ONE - Replicator factor : 1

---

It's possible you're hitting the python GIL but more likely you're doing something wrong.

For instance, putting 2M rows in a single batch would be Doing It Wrong.

---

Try running multiple clients in multiple processes, NOT threads.

Then experiment with different insert sizes.

1M inserts in 3 mins is about 5500 inserts/sec, which is pretty good for a single local client. On a multi-core machine you should be able to get several times this amount provided that you use multiple clients, probably inserting small batches of rows, or individual rows.

---

You might consider Redis. Its single-node throughput is supposed to be faster. It's different from Cassandra though, so whether or not it's an appropriate option would depend on your use case.

---

The time taken doubled because you inserted twice as much data. Is it possible that you are I/O bound?