what could be reason for virtual bytes to grow 2x private bytes?

An application's virtual bytes gr开发者_如何学运维ow 2-times the private bytes.

does this indicate memory leak? bad application design?

OS is 32Bit

any thoughts are welcome. application is stream database.

---

Fragmentation.

If you allocate the following chunks of memory:

• 16KB
• 8KB
• 16KB

and you then free the chunk of 8KB, your application will have 32 KB of private bytes, but 40 KB bytes of virtual memory, which is actually the highest virtual memory address that has ever been in use by your process (ignoring the other memory parts for sake of simplicity).

Consider (if possible) using another memory manager. Some alternatives are:

• The Windows Low-fragementation heap (see http://msdn.microsoft.com/en-us/library/aa366750%28VS.85%29.aspx for more info)
• The Doug-Lea open source memory manager
• Commercial alternatives like Hoard

A fourth alternative is to write your own memory manager. It's not that easy, but if done right, it can have quite some benefits. Especially for certain niche or special applications, writing your own memory manager can be useful.

---

An application's virtual bytes grow 2-times the private bytes.

If application allocates only heap, then to me it would be the sign that application allocates lots of memory but never actually touches it. For example:

void *p = malloc( 16u<<20 );

would eat up 16MB of virtual memory. But as long as application doesn't perform any actions with the memory block, OS wouldn't even attempt to map the virtual memory to the RAM. Simplest way to force the actual allocation of private memory is to memset() it:

void *p = malloc( 16u<<20 );
memset( p, 0, 16u<<20 );

does this indicate memory leak? bad application design?

Or both. Or neither.

The longer variant of the response: unknown, depends on what memory application allocates, what other resources application uses, OS, h/w platform, etc.

If unsure, use a memory leak analysis tools to investigate, e.g. valgrind. Read up SO for more information on memory leak analysis in C++.

---

Memory allocation has overhead to store management information about what was allocated. If you're allocating very small buffers the extra information can be a significant percentage of the total. That might be what you're seeing.

---

One possibility is if you set a large stack reserve size for your threads with linker option /STACK:reserve_bytes and then you start a lot of threads.

For example, if you have an ATL service, it automatically starts 4*numberOfCores apartment message dispatching threads by default. Compile and link such a service with /STACK:12000000 (12 megabytes), then run it on a 16-core server and it will start 64 threads, each with a 12MB stack, immediately consuming 768MB of virtual address space, although the actual committed memory may be much lower.