Limiting jobflow concurrent executions

azbweidman

• October 24, 2019 00:00

• Answered

Follow

We have setup a Event Listener to watch for arrival of files and to start a jobflow when they arrive.
We need to have a jobflow only run one at a time, but multiple files can land in short order.


We created the file listener:
Type of Check: File Added
Filename match type: Regex filename match
Filename Pattern:
Check every seconds: 300
Trigger task when file has not changed for X Check: Unchecked
Ignore Empty: Checked
Sandbox:
JobFlow: jobflow/XXX.jbf


We configured the jobflow:
max_running_concurrently 1
enqueue_executions true

but they run concurrently.
(The jobflow calls other jobflows, and a graph.) The toplevel jobflow the Event Listener is configured to start is what we change to configuration on to say only run 1, but queue up others if multiple files are present.

• 

  darvehng

  • November 04, 2019 19:22

  0

  Comment actions Permalink

  Hi azbweidman,
  
  Thank you for your inquiry. Please see below to the list of questions that I would appreciate you responses to.
  
  List of questions:
  __________________
  1. Which version of CloverDX Server are you running? In addition, are you running it as a stand-alone server or a cluster?
  2. Can you attach the run-log of the two concurrently running jobs? I'd like to inspect the difference in the execution time of the two jobs to ensure that the "enqueue_executions" property is working the way it's suppose.
  3. Are the files that are being checked local or remote?
  
  Best,

Please sign in to leave a comment.