开发者

Slow Import of Large MySQL Dump

开发者 https://www.devze.com 2023-03-23 20:02 出处:网络
I\'m trying to im开发者_Go百科port a very large MySQL dump file into a new MySQL server.However, after a certain point, it seems to bog down.I\'m about 250M rows in, and it\'s taking over 10-12 second

I'm trying to im开发者_Go百科port a very large MySQL dump file into a new MySQL server. However, after a certain point, it seems to bog down. I'm about 250M rows in, and it's taking over 10-12 seconds to run even a single line of the dump. It seems to hang on "update".

Am I missing something I should do to make this import go faster? It's a pretty beefy server so I don't think it's the I/O..


If you're sure that your data is correct, try disabling foreign checks so the process can be speed up:

SET @@FOREIGN_KEY_CHECKS = 0;


What seemed to work for me with a 'beefy' server, was splitting the file into several smaller and importing them all in parallel. Altough that may break your row ordering. (and results may vary depending on storage engine)

IIRC there was some kind of script to do that, but it was unmaintained. So when I was messing with that, I used just head/tail and pipes.

If you are inserting into multiple tables and not using foreign keys, i could probably provide you my simple perl script that I use to split one big dump into per table dumps, which can in turn be imported in parallel.

Also this may help. Either done manually (insert disable at the beginning and enable at the end of your dump file) or when dumping the db. This should speed up your inserts, but it will take quite some time to build indexes.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号