开发者

ibatis: improve insert performance

开发者 https://www.devze.com 2023-01-20 23:47 出处:网络
I am using ibatis for my sql insert stmt. In my code I am parsing files line by line from a folder. Each line that matches criteria, need to be inserted into database.

I am using ibatis for my sql insert stmt. In my code I am parsing files line by line from a folder. Each line that matches criteria, need to be inserted into database. Total number of insert in a single run of program can be any where along 200k.

    SqlSession sess = null;    
    this.sess = sf.openSession(ExecutorType.BATCH, false);
    for (each file) {
     for( each line matching criteria ){
         this.sess.insert("com.logs.util.insertFileInfo", fileData);
         insertcount++;
         if(insert cou开发者_如何学运维nt == 10)
              this.sess.commit();
         }    
      }
      if(insert count > 0){
           this.sess.commit();
      }   
    }

This style slowly takes up lot of memory and after some times throws OutOfMemory exception. How can I improve performance here?


Is it your intention to commit after every 10 inserts? It looks like you only do so after the first 10 inserts. I guess that you would need something like

if ((insertCount % 10) == 0) {
   this.sess.commit();
}

Those uncommitted changes have to be stored somewhere. I'm not familiar with Ibatis but if the uncommitted changes are being stored in a buffer allocated by Ibatis then you will eventually run out of memory if you don't commit the changes.

0

精彩评论

暂无评论...
验证码 换一张
取 消