Graph executed many times leaks memory

esegal

• June 22, 2014 00:00

• Answered

Follow

Another thing please - my application run Clover Graph multiple times - mainly for aggregation purpose.
As I see by JConsole the application consume more and more memory until I get:
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:633)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
at org.jetel.graph.DirectEdge.init(DirectEdge.java:107)
at org.jetel.graph.Edge.init(Edge.java:265)
at org.jetel.graph.Phase.init(Phase.java:147)
at org.jetel.graph.TransformationGraph.init(TransformationGraph.java:463)
at org.jetel.graph.runtime.EngineInitializer.initGraph(EngineInitializer.java:176)

When I set the JVM for enlarge the Direct buffer memory I get the same result - but after more time.
I took dump file of the application after running a while and I see 41,040 instances of org.jetel.component.DataReader (!!!)
- Well something is malfunction here - I think I use Clover as needed - mybe I'm wrong ?
my code call EngineInitializer.initEngine() only once, and after each graph running call also:
Map<String,Node> nodes= graph.getNodes();
for (Node node : nodes.values())
node.free();
graph.free();

Is any one know about memory-leak in CloverETL?? or am I doing something wrong ??

Thanks in advance !
Segal

• 

  jurban

  • June 26, 2014 09:48

  0

  Comment actions Permalink

  Hi esegal,
  
  can you please post the graph you're using? Then we'll analyse it.
  
  Regards,
  Jaro
• 

  esegal

  • June 26, 2014 14:10

  0

  Comment actions Permalink

  Hi,
  Please find attached the Graph I use.
  Thanks !
  Segal

  • AGGREGATE_NRTRDE.grf
• 

  jurban

  • June 26, 2014 15:07

  0

  Comment actions Permalink

  I didn't find anything suspicious in the graph. However, you're running the graph multiple times - how are you doing that? Are you running it from your own Java code? You might be not calling proper cleanup after the graph execution (e.g. you should call graph.free()).
• 

  esegal

  • June 30, 2014 09:53

  0

  Comment actions Permalink

  The graph is running multiple times by Java code.
  I mentioned before that I call graph.free(), and even free() for each node in graph...
  I'm trying to compile Clover code in order to try to debug what's happening .
  I'll update what's going on.
  Thanks !
• 

  mahendran

  • December 29, 2015 10:53

  0

  Comment actions Permalink

  Hi esegal,
  
  Can you update us on what issue you found and how you fixed it.
• 

  stella365

  • November 25, 2018 20:28

  0

  Comment actions Permalink

  How did you fix the issue, esegal?

Please sign in to leave a comment.