I know there is a way to skip the graph configuration, using skip_check_config.
But is there a way to have a graph skip the initialization, and just start running?
Basically, I have a child graph with dynamic metadata using a sql query - SELECT * FROM SOURCE_TBL WHERE 1=0;
Before running the graph, one of the first things clover does is initialize it, by checking the Metadata.
To do this, it needs to check and connect to the database connection.
If the database is down, the metadata fails, causing a "graph initialization" error.
Because it does this initialization check, before passing any parameters, or much of anything else, the log is very sparse if the metadata fails. It also makes it difficult to add any retry logic.
I was just curious if there is a setting available, or some other work-around, that allows you to skip this. I want the graph to actually start running before receiving the database error.
But is there a way to have a graph skip the initialization, and just start running?
Basically, I have a child graph with dynamic metadata using a sql query - SELECT * FROM SOURCE_TBL WHERE 1=0;
Before running the graph, one of the first things clover does is initialize it, by checking the Metadata.
To do this, it needs to check and connect to the database connection.
If the database is down, the metadata fails, causing a "graph initialization" error.
Because it does this initialization check, before passing any parameters, or much of anything else, the log is very sparse if the metadata fails. It also makes it difficult to add any retry logic.
I was just curious if there is a setting available, or some other work-around, that allows you to skip this. I want the graph to actually start running before receiving the database error.
-
Hello Hewills,
unfortunately, it is not possible to force CloverETL Designer to skip the initialization phase which is by design. In other words, it is not possible to make the graph start running if the connection to a database cannot be established. Although there is no workaround within the CloverETL Designer there is a way to achieve this within the CloverETL Server.
Let’s assume that there are parts of the graph which you want to run even though a connection to the database is not established when trying to pull the dynamic metadata. You could divide the graph into several smaller graphs in order to separate out those parts that should be independent of the successful connection and those that need the successful connection to run. In the CloverETL Server, you would use the Execute Graph component to orchestrate running of the consecutive graph parts. Then you would want to modify the attribute of the Execute Graph components called “Stop processing on fail” and set it up to “false”. This will ensure that those graph parts that are independent of the successful connection will still run in spite of the connection failure.
Best regards,
Please sign in to leave a comment.
Comments 1