Speed of writing to a text file or via key/value database

I tested writing to a text file simply via shell command or php. Using a while loop for 1 million cycles, appending cycle number into the file (as $i\n). I thought this must be fastest way to store data on the disk. Where in shell or php, it took about 40s for 1 million records.

Alternatively, I used fast key/value databases such as QDBM, GDBM, etc. The overall 开发者_StackOverflow中文版time was even lesser (15-30s). What is the theoretical explanation for faster storage of data via simple database systems? I thought the rate-limiting should be hardware (e.g the speed of hard disk or CPU).

Another Question: When increasing the cycle from 100k to 1M, the overall time was increased 10 times; but when increasing from 1M to 10M, it took 100 times longer. What threshold makes the system slow this way?

---

The bottleneck for writing data to disk is usually the disk I/O. The fastest continuous output can be achieved when writing contiguous blocks to the disk(s).

I assume that the file was opened and closed 1 million times in your shell script. Each close issues a synchronization of the file to disk, which probably causes a wait (depending on file system and storage solution).