I have a 25GB Clover file that I'm trying to read using the CloverDataReader and keep getting hit with Java heap space issues. It appears the component is putting all of the records in memory as it reads??? I tried using a filter right after the CloverDataReader in hopes or reducing the JVM foot print but that doesn't seem to work either. I've increased the JVM size as much as I can (~7GB) but still no dice. Also tried change the edge types and some defaultProperties. Any other ideas or advise are certainly welcomed.
ERROR [WatchDog] - Node CLOVER_READER0 finished with status: ERROR caused by: Java heap space
ERROR [WatchDog] - Node CLOVER_READER0 error details:
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
Thanks,
Chris
ERROR [WatchDog] - Node CLOVER_READER0 finished with status: ERROR caused by: Java heap space
ERROR [WatchDog] - Node CLOVER_READER0 error details:
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
Thanks,
Chris
-
Hi ccasano,
CloverDataReader is reading file sequentially, only requested records at time. But we are aware of bug https://bug.javlin.eu/browse/CL-2252 causing similar issues as you have. It will be fixed in 3.3M3 released within month.
Please sign in to leave a comment.
Comments 1