Clover graphs are stuck for a long time

chaitanya_r

• February 18, 2014 00:00

• Answered

Follow

Hi


I am facing an issue where my graph execution clover are stuck for a long time and I don't see any logs which gives details on why the graphs are stuck for a long time.

Now if I wait for some more time then I am getting an error message saying:

error executing graph
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:691)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1371)
	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:132)
	at com.cloveretl.server.c.b.b.executeWatchDog(Unknown Source)
	at com.cloveretl.server.c.h.f(Unknown Source)
	at com.cloveretl.server.c.h.a(Unknown Source)
	at com.cloveretl.server.c.h.executeGraph(Unknown Source)
	at com.cloveretl.server.c.j.executeGraph(Unknown Source)
	at com.cloveretl.server.c.b.c.a(Unknown Source)
	at com.cloveretl.server.c.b.c.executeGraph(Unknown Source)
	at com.cloveretl.server.c.b.c.executeGraphSync(Unknown Source)
	at org.jetel.component.RunGraph.runGraphThisInstance(RunGraph.java:521)
	at org.jetel.component.RunGraph.runSingleGraph(RunGraph.java:418)
	at org.jetel.component.RunGraph.execute(RunGraph.java:263)
	at org.jetel.graph.Node.run(Node.java:453)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:722)

Please help me how to debug this issue? Is there any logs that I need to check other than server logs to see why the graph is stuck? I am using Weblogic server.

• 

  novakj

  • February 19, 2014 15:17

  0

  Comment actions Permalink

  Hi chaitanya_r,
  
  Please verify if you are not hitting OS related limit "max user processes" - on Linux counts also threads. You can see the limit on Linux by command:

  ulimit -u

  
  
  Default limits are usually low and will prevent you to create let's say more than 1024 threads per Java process.
  
  Another possible reason of this exception and your slow graph execution could be too much concurrently running graphs.
  
  For more detailed answer, could you please specify some details regarding your environment? For example, operating system, CloverETL Designer/Server version, JVM memory settings etc.
• 

  chaitanya_r

  • February 28, 2014 11:39

  0

  Comment actions Permalink

  Thanks a lot, increasing the max user processes on my Linux box has fixed my issue.

Please sign in to leave a comment.