Jobflow failures after upgrading to 4.0.1.20 from 3.4.2.11

rama

• February 04, 2015 00:00

• Answered

Follow

Subsequent to our upgrade from 3.4.2.11 to 4.0.1.20 version of the CloverETL server, an execute graph component on error is causing the entire job flow to fail. It used to only report a failure at the graph level but not cause the entire job flow to fail. I couldn't find any settings that would help restore the previous behavior. Let me know if you need any additional details.

Thanks,
- Rama

• 

  imriskal

  • February 05, 2015 11:51

  0

  Comment actions Permalink

  Hello Rama,
  
  ExecuteGraph has a property called Stop processing on fail which can ignore errors in the graph when set to False.
  
  Hope this helps.
• 

  rama

  • February 05, 2015 21:36

  0

  Comment actions Permalink

  Hi,
  We tested the "Stop processing on fail" flag even prior to starting this post. Unfortunately it doesn't seem to impact the behavior, the entire job flow fails on the graph that is used in the execute graph component.
  
  Thanks,
  - Rama
• 

  imriskal

  • February 06, 2015 09:28

  0

  Comment actions Permalink

  This seems odd, it should not behave this way. Could you please send me the jobflow, the graph and the error log? I will look into them and hopefully find the issue.
  
  Thanks.
• 

  rama

  • February 06, 2015 21:20

  0

  Comment actions Permalink

  Hi Lubos,
  Here are the files that you requested the jobflow, graph and the corresponding error logs for them. Couldn't add one of the log files because of the 3 file max limitation. I will add that in the next response.
  
  Thanks,
  - Rama

  • ProcessReplicationScope.jbf
  • CompareTableScheme.grf
  • run_log_389066.log
• 

  rama

  • February 06, 2015 21:33

  0

  Comment actions Permalink

  I could not upload the error log for the jobflow because of the size limitation. Let me know if you need the tail of that file with the failure details.
  
  Thanks,
  - Rama
• 

  admin

  • February 09, 2015 12:45

  0

  Comment actions Permalink

  Hi Rama,
  
  It seems that your graph fails during initialization phase - so it is not runnable. That is not covered by "Stop processing on fail".
  
  The problem is visible in log:
  
  

  
  java.sql.SQLException: Invalid column name 'visible_ind'.
  

  
  
  There are dynamic metadata DynamicMetadata2 which does sql query "select ${FIELD_LIST} from ${TARGET_PROCESSED_TABLE_WITH_SCHEMA}" during initialization phase. Parameters for it are passed from JobFlow. So please inspect what parameters values are passed. Obviously it is "visible_ind" which is not available in DB.
  
  I hope that helps.
• 

  rama

  • February 09, 2015 18:18

  0

  Comment actions Permalink

  Hi Lubos,
  Thanks for your assistance thus far with this issue.
  
  This is the same code that was used in 3.4.2.11 and even though the graph would fail in the initialization phase the job flow wasn't impacted. We haven't changed anything in this code since the upgrade roughly two weeks ago. We compare the source and target tables scoped for replication during this phase. This ETL process was developed by Clover's consulting team and we haven't modified anything in this graph ever since.
  
  Thanks,
  - Rama
• 

  imriskal

  • February 18, 2015 07:42

  0

  Comment actions Permalink

  Hi,
  
  The change does not have to be in the graph, it can be in the DB as well. Did the database table change lately?
  
  We did multiple tests here and the result was always as expected - if the graph fails during the initialization, the jobflow ends with an error no matter the version of CloverETL. Therefore, I would like to ask you to send tail of the jobflow run log.
  
  Thanks.

Please sign in to leave a comment.