开发者

Why is ostringstream faster than ofstream

开发者 https://www.devze.com 2023-02-14 23:20 出处:网络
To write many piece of data to file, I have 2 approaches: Write to ofstream one by one directly ofstream file(\"c:\\\\test.txt\");

To write many piece of data to file, I have 2 approaches:

  1. Write to ofstream one by one directly

    ofstream file("c:\\test.txt");
    for (int i = 0; i < 10000; ++i)
    {
        file << data[i];
    }
    
  2. Write to istringstream first, and then write to ofstream at once

    ostringstream strstream;
    for (int i = 0; i < 10000; ++i)
    {
        strstream 开发者_StackOverflow<< data[i];
    }
    ofstream file("c:\\test.txt");
    file << strstream.str();
    

Not surprisingly, the second approach is faster, in fact, it is 4 times faster than the first approach on my HP7800 machine.

But why? I know ofstream is using filebuf inside, and ostringstream is using stringbuf - as a buffer they should all reside in memory thus should have no difference.

What is the difference under the hood?


Are you using std::endl a lot instead of '\n'? std::endl does two things: it inserts a '\n' into the stream and then flushes the buffer to disk. I've seen code talking a severe performance hit by doing so. (The code ran 5-10 times faster after that was fixed.)
Flushing to a string buffer will be much faster than flushing to the disk, so that would explain your findings.

If that's not the case you might consider is increasing the buffer size:

const std::size_t buf_size = 32768;
char my_buffer[buf_size];
ofstream file("c:\\test.txt");
file.rdbuf()->pubsetbuf(my_buffer, buf_size);

for (int i = 0; i < 10000; ++i)
{
    file << data[i];
}


Disk is slow. Many small writes are more expensive than one large.


It can be implementation issue with specific OS. Also I guess the ofstream buffer(buflen) is smaller than 10000, a typical value of which is 4095. So try running with i<4096 and the response time should be quite same!

The reason why it's faster in the second case:

In the first case when the buffer is full ( buflen=4095bytes) it is written to disk. So for i<10000 it'd have caused it to be flushed 3 times.

While in the second case, all data is first prepared in the RAM and in one go flushed to the harddisk. So two flushes have been saved!

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号