Data structure Memory Mapped or DB? (millions of items)

I have a data structure which is essentially a lookup for some calculations that take a really long time (100ms) to calculate and need to be used over and over. I have roughly 6,000,000 of these calculations and want to load them into memory when my application starts (I will precalculate them all).

The question is can I store this as a memory mapped file (dictionary of something) or should I store it in a db and then load it into ram on program start up? Ho开发者_如何学JAVAw fast would binary serialization be?

What are my options?

---

Binary serialization is fast, specially if you only have to load it once. Load speed from a database really depends on how the data is structured. The advantage to using a database is easy of management. If you want to easily manage, change, track changes, or use these values with multiple clients, then a DB would be the way to go. If they are never going to change, a file would suffice.

---

You have to try different approaches and measure for yourself. There is no other road to solve performance problems. Note that you need to have some concreate goal in mind (like 1 second for load/1ms for lookup).

Options:

• calculate all values on startup and sotre in some lookup (preallocated array/dictionary)
• calculate on demand and sotre in some lookup (preallocated array/dictionary)
• calculate in advance and load uncompressed (note that amount of data you have is large and will take noticable 1-3 seconds time to load)
• calculate in advance and load on demand
• calculate in advance and load compressed data

I'd recommned trying to compute all values on load and see if it works fast enough - most likely easiest way to go.