.net: Create/store a big amount data paralleling to SQL server

I'm going to create/save loads of records (30,000) to database tables in each batch. If I use paralleling programm for reducing the process time, how can I manage the fol开发者_JS百科lowing issues?

• the db tables will be blocked that the performance will not actually be as good as expected
• Is there any pattern I can take a reference?

Thanks in advance.

---

Are you sure you need parallel processing? 30000 may not be that big. I suggest you look at the SqlBulkCopy class first. Here is a link that explains how to use it: Using SqlBulkCopy To Perform Efficient Bulk SQL Operations

---

Firstly, most database professionals wouldn't consider 30K records a large data set - are you sure you need to worry about performance?

Depending on a whole bunch of factors, database servers actually deal really well with multiple processes inserting/updating data in the same table - it's kind of their primary purpose in life. I'm assuming you do mean a DB server product like MS SQL Server, Oracle or MySQL.

Simon Mourier's suggesting of BulkCopy is probably the fastest way of banging a large data set into a table.

If you do need to run parallel processing, the key is transaction management - if each thread starts a transaction and doesn't commit it for a long time, you may run into locking issues. Read up on your database's implementation, and try to have short-running transactions.