Java solutions to record / replay executed code in a JVM

We have a bug we're trying to find that happens non-deterministically (well, it's deterministic, but we just don't know what's actually causing it) and it only happens once every couple hours.

We read a lot of network data, and we have many threads, so there's a likelihood it's from an input, race condition, or combination of both, but for the most part, unreproducible.

I'm wondering if there are any JVM recorders / replayers out there that can store everything that happened in a JVM so we can then go through it step by step and recreate the steps to the exception.

I've found one. (I'm not going to post it so people don't think I'm trying to advertise a product.), but I'm wondering if there are others, and more importantly, others that people have used, and can report that work well.

Edit to add:

I've found Replay Solutions. I've never used it, and I've never heard of it, so I don't know how good it actually is.

We use JProfiler, but I 开发者_开发技巧don't think it actually has support to record/replay everything.

---

I would suggest using Chronon it worked well for me, but I haven't used it extensively yet.

---

It is not possible (except maybe for micro-benchmarks) to record everything, especially because of the thread interleavings: the performance degradation/slowdown would be too severe. If the problem you are trying to reproduce is a concurrency bug you may want to try systems that perform partial logging and complete the interleaving using SMT solving:

http://www.gsd.inesc-id.pt/~nmachado/software/Symbiosis_Tutorial.html

To the extent of my knowledge, this is the most recent system, and is available open-source (in java, c and c++).