Continue JobFlow If A Single Node Fails

hesske

• October 20, 2023 18:54

Follow

I have a jobflow that has a DeleteFiles component in the first phase. Sometimes, there is no file for this component to delete which will cause the component and the jobflow to fail. Is there a way to have the jobflow continue if this 1 node fails?

 

• 

  Tomas Pavlas

  • October 23, 2023 13:06

  1

  Comment actions Permalink

  Hi,

  If you want the jobflow to continue even if the DeleteFiles component does not have anything to delete, you should do two things:

  1. Set the Stop processing on fail property to false. So even if the file doesn’t exist, the component will continue to process the next input.
     
     
     
  2. If you don’t want to have error messages or job failures, you can prevent that by simply connecting another component to Output port 1 of the DeleteFiles.
     The DeleteFiles will send the error message on the edge to the other component. You can use the FlatFilerWriter so those error messages will be stored in a File, or if you use something like a Success component, they can be reviewed in Job Inspector, but will not be stored.
     

  I hope this will help.

  Best regards, Tom.

• 

  hesske

  • October 23, 2023 15:15

  0

  Comment actions Permalink

  Thank you so much Tomas! This solution worked just as desired. I chose to add a FlatFileWriter to output 1 on the DeleteFiles component so I could capture the error messages in case there was something else that happened down the road. 

  Thanks again!

Please sign in to leave a comment.