I have written a Java program that builds up a huge arraylist (with more than 7 million elements). Then iterating on the elements of that arraylist, I am building up a second arraylist whose size is also like to be in mill开发者_如何学运维ions. At some point during the iteration, I get outof memory exception with GC overhead limit exceeded.
I have thought of a solution like whenever the size of the 2nd arraylist hits 10 thousand, I will save its elements to a file or database table and invoke removeall() on that arraylist. But will this invocation force garbage collection of the elements ? If not then my problem remains unsolved.
Thanks in advance for any useful suggestion.
Calling removeAll
will not trigger garbage collection, and manually triggering garbage collection won't help either. Let garbage collection happen as required. (You can be sure that it will run before your application gets an OOME. The OOME is only thrown if there is not enough space left after running a full GC.)
What you need to do is reduce your the number and size of reachable objects. That will mean that when the garbage collection does run, it will reclaim more garbage and your application will be able to continue.
It sounds like your strategy of limiting the size of the 2nd array could help by reducing the working set of reachable objects. Another possibility is to use ArrayList.set(pos, null)
to null the entries in the first list that are no longer needed. Unfortunately, there aren't enough details in your question to be more helpful.
Alternatively, just increase your JVM's maximum heap size.
only if all the references to the objects in the list are removed.
The garbage collector will automatically run as needed. You can manually call it via System.gc()
, but that won't help in your case.
There's a post that explains what the error means and specifically how to disable it:
The parallel / concurrent collector will throw an OutOfMemoryError if too much time is being spent in garbage collection: if more than 98% of the total time is spent in garbage collection and less than 2% of the heap is recovered, an OutOfMemoryError will be thrown. This feature is designed to prevent applications from running for an extended period of time while making little or no progress because the heap is too small. If necessary, this feature can be disabled by adding the option -XX:-UseGCOverheadLimit to the command line.
精彩评论