Keep serialized and compressed Objects in-memory

I'm currently working on a Part of an Application where "a lot" of data must be selected for further work and I have the impression that the I/O is limiting and not the following work. My idea is now to have all these objects in memory but serialized an compressed. The question is, if accessing the objects like this would be faster than direct Database access and if it is a good idea or not. 开发者_开发知识库(and if it is feasble in terms of memory consumption = serialized form uses less memory than normal object)

EDIT February 2011:

The creation of the objects is the slow part and not the database access itself. Having all in memory is not possible and using ehcache option to "overflow to disk" is actually slower than just getting the data from the database. Standard java serialization is also unusable. it is also a lot slower. So basically nothing I can do about it...

---

You're basically looking for an in-memory cache or an in-memory datagrid. There are plenty of APIs/products for this sort of thing. ehcache/hibernate chace/gridgain etc etc

---

The compressed serialized form will use less memory, if it is a large object. However for smaller objects e.g. which use primtives. The original object will be much smaller.

I would first check whether you really need to do this. e.g. Can you just consume more memory? or restructure your objects so they use less memory.

---

"I have the impression that the I/O is limiting and not the following work. " -> I would be very sure of this before starting implementing such a thing.

The simpler approach I can suggest you is to use ehcache with the option to store on disk when the size of the cache get too big.

Another completely different approach could be using some doc based nosql db like couchdb to store objects selected "for further work"