Hi,
Looking for best-practice approach to help with the following scenario:
A source file supplied must have each record pass all validation checks, or the entire data set is to be rejected.
Upon rejection, the users are to be emailed (got it) and the file moved to a 'rejected' folder (can't figure out how; SystemExecute always runs even with phase tweaks).
Also - if the graph fails due to any system issue (database, OS, filesystem, network, server hiccup, other etc.) then the IT team is to be notified, via email, and the source file rejected.
Presently my graph detects data issues and forces a raiseError ("Data invalid error") if one (or more) bad record is detected .
An 'Error-handler' graph could launch on GRAPH_ERROR, but how would it determine the cause of the error, and respond appropriately?
Looking for best-practice approach to help with the following scenario:
A source file supplied must have each record pass all validation checks, or the entire data set is to be rejected.
Upon rejection, the users are to be emailed (got it) and the file moved to a 'rejected' folder (can't figure out how; SystemExecute always runs even with phase tweaks).
Also - if the graph fails due to any system issue (database, OS, filesystem, network, server hiccup, other etc.) then the IT team is to be notified, via email, and the source file rejected.
Presently my graph detects data issues and forces a raiseError ("Data invalid error") if one (or more) bad record is detected .
An 'Error-handler' graph could launch on GRAPH_ERROR, but how would it determine the cause of the error, and respond appropriately?
-
Hi twaldock,
I would create 2 graphs:
1st takes file path to be processed and execute second graph (RunGraph component). Depending on result of run it will act (so move file, send mail, ...). RunGraph have 2 outputs - success/fail - this will give you 2 execution branches you need. Description of problem can be obtained from log file (optional parameter of RunGraph : see http://www.cloveretl.com/documentation/ ... graph.html). If you will put some extra signature into you log lines, it can be quite easily parsed/extracted.
2nd will take file, try to validate it and throw error in case of problems. Before throwing error, it will log issue(s) by function "printLog" (see here http://www.cloveretl.com/documentation/ ... -ctl2.html )
Please sign in to leave a comment.
Comments 2