I'm running multiple graphs in seperate JVM's via the RUN_GRAPH component from a "master" graph. Each of the graphs have links to external properties and metadata files. These links get defined as relative paths from the project directory, and work as expected when run individually. When I run through the "master" they fail to find the linked files. If I hard code the absolute path in the individual graphs they work from the "master", but automatically get reset to the relative path when they are run seperately (which in turn breaks any subsequent run from the "master").
I've noticed this in the log file: "PROJECT_DIR property: null"
I'm thinking that if this were defined I could use it in conjunction with the relative path info to resolve the links correctly for either scenario. I've tried a few different ways to pass this to the JVM in the RUN_GRAPH components, but to no avail. Is this the correct approach, and if so could you point me to a working example of how to correctly pass the PROJECT_DIR property?
I've noticed this in the log file: "PROJECT_DIR property: null"
I'm thinking that if this were defined I could use it in conjunction with the relative path info to resolve the links correctly for either scenario. I've tried a few different ways to pass this to the JVM in the RUN_GRAPH components, but to no avail. Is this the correct approach, and if so could you point me to a working example of how to correctly pass the PROJECT_DIR property?
-
Till CloverETL 2.9, PROJECT_DIR was internal clover parameter, that defined root directory for all resources. So you can left relative paths in the graph, and set the PROJECT_DIR parameter when running graph from RunGraph component (-P:PROJECT_DIR=absolute_path_to_project in the 2nd input field or in cloverCmdLineArgs attribute). Since CloverETL 3.0 the internal parameter was replaced with -contexturl switch. -
I've tried both suggestions without success.
In the example graph, the first run sets the path in the data generator, in the second it is set as an argument.
What am I doing wrong?
<?xml version="1.0" encoding="UTF-8"?>
<Graph author="287207-admin" created="Wed Oct 27 12:23:37 CDT 2010" guiVersion="2.9.1" id="1288366414517" licenseCode="CLGXDINITI279458BY" licenseType="initiate" modified="Fri Oct 29 13:18:19 CDT 2010" modifiedBy="287207-admin" name="testRun" revision="1.12">
<Global>
<Metadata id="Metadata67" previewAttachment="${INPUT_DIR}/runGraph.txt" previewAttachmentCharset="ISO-8859-1">
<Record fieldDelimiter="|" name="runGraphIn" previewAttachment="${INPUT_DIR}/runGraph.txt" previewAttachmentCharset="ISO-8859-1" recordDelimiter="\r\n" type="delimited">
<Field name="graph" type="string"/>
<Field name="parameters" type="string"/>
</Record>
</Metadata>
<Metadata id="Metadata66" previewAttachmentCharset="ISO-8859-1">
<Record name="runGraphOut" previewAttachmentCharset="ISO-8859-1" recordSize="-1" type="delimited">
<Field delimiter=";" name="graph" nullable="true" type="string"/>
<Field delimiter=";" name="result" nullable="true" type="string"/>
<Field delimiter=";" name="description" nullable="true" type="string"/>
<Field delimiter=";" name="message" nullable="true" type="string"/>
<Field delimiter="\n" name="duration" nullable="true" type="decimal"/>
</Record>
</Metadata>
<Property id="GraphParameter0" name="INPUT_DIR" value="C:/Documents and Settings/Administrator/production_graphs/RPM/CustomerFiles/Contact"/>
</Global>
<Phase number="2">
<Node enabled="enabled" guiHeight="0" guiName="DataGenerator" guiWidth="0" guiX="36" guiY="24" id="DATA_GENERATOR2" recordsNumber="1" type="DATA_GENERATOR">
<attr name="generate"><![CDATA[//#TL
// Generates output record.
function generate() {
$0.graph := "${INPUT_DIR}" + "/RPM_Company.grf";
$0.parameters := "-plugins \"C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/plugins\";\"C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.initiatesystems.etl.gui_9.2.0/lib/plugins\" -P:PROJECT_DIR=\"C:/Documents and Settings/Administrator/production_graphs/RPM/\"";
}
// Called to return a user-defined error message when an error occurs.
// function getMessage() {}
// Called during component initialization.
// function init() {}
// Called after the component finishes.
// function finished() {}
]]></attr>
</Node>
<Node enabled="enabled" guiHeight="0" guiName="RunCompanyGraph" guiWidth="0" guiX="192" guiY="24" id="RUN_GRAPH1" logAppend="false" logFile="runGraph.log" sameInstance="false" type="RUN_GRAPH"/>
<Node enabled="enabled" guiHeight="0" guiName="Trash" guiWidth="0" guiX="360" guiY="24" id="TRASH2" type="TRASH"/>
<Node enabled="enabled" guiHeight="0" guiName="Trash" guiWidth="0" guiX="372" guiY="60" id="TRASH4" type="TRASH"/>
<Edge debugMode="false" fromNode="DATA_GENERATOR2:0" guiBendpoints="" id="Edge6" inPort="Port 0 (in regular mode: names of the graphs to be executed, otherwise nothing)" metadata="Metadata67" outPort="Port 0 (out)" toNode="RUN_GRAPH1:0"/>
<Edge fromNode="RUN_GRAPH1:0" guiBendpoints="" id="Edge13" inPort="Port 0 (in)" metadata="Metadata66" outPort="Port 0 (regular mode: status messages, pipeline mode: an information record in case of successful termination, otherwise nothing)" toNode="TRASH2:0"/>
<Edge fromNode="RUN_GRAPH1:1" guiBendpoints="" id="Edge14" inPort="Port 0 (in)" metadata="Metadata66" outPort="Port 1 (in pipeline mode an information record in case of an error, otherwise nothing)" toNode="TRASH4:0"/>
</Phase>
<Phase number="4">
<Node enabled="enabled" guiHeight="0" guiName="DataGenerator" guiWidth="0" guiX="36" guiY="168" id="DATA_GENERATOR1" recordsNumber="1" type="DATA_GENERATOR">
<attr name="generate"><![CDATA[//#TL
// Generates output record.
function generate() {
$0.graph := "${INPUT_DIR}" + "/RPM_Hierarchy.grf";
$0.parameters := "-plugins \"C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/plugins\";\"C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.initiatesystems.etl.gui_9.2.0/lib/plugins\"";
}
// Called to return a user-defined error message when an error occurs.
// function getMessage() {}
// Called during component initialization.
// function init() {}
// Called after the component finishes.
// function finished() {}
]]></attr>
</Node>
<Node cloverCmdLineArgs="-P:PROJECT_DIR=\"C:/Documents and Settings/Administrator/production_graphs/RPM/\"" enabled="enabled" guiHeight="0" guiName="RunHierarchyGraph" guiWidth="0" guiX="192" guiY="168" id="RUN_GRAPH2" logAppend="true" logFile="runGraph.log" sameInstance="false" type="RUN_GRAPH"/>
<Node enabled="enabled" guiHeight="0" guiName="Trash" guiWidth="0" guiX="360" guiY="168" id="TRASH1" type="TRASH"/>
<Node enabled="enabled" guiHeight="0" guiName="Trash" guiWidth="0" guiX="372" guiY="204" id="TRASH5" type="TRASH"/>
<Edge debugMode="false" fromNode="DATA_GENERATOR1:0" guiBendpoints="" id="Edge15" inPort="Port 0 (in regular mode: names of the graphs to be executed, otherwise nothing)" metadata="Metadata67" outPort="Port 0 (out)" toNode="RUN_GRAPH2:0"/>
<Edge fromNode="RUN_GRAPH2:0" guiBendpoints="" id="Edge16" inPort="Port 0 (in)" metadata="Metadata66" outPort="Port 0 (regular mode: status messages, pipeline mode: an information record in case of successful termination, otherwise nothing)" toNode="TRASH1:0"/>
<Edge fromNode="RUN_GRAPH2:1" guiBendpoints="" id="Edge17" inPort="Port 0 (in)" metadata="Metadata66" outPort="Port 1 (in pipeline mode an information record in case of an error, otherwise nothing)" toNode="TRASH5:0"/>
</Phase>
</Graph> -
It seems to be problem with RunGraph component and quoted parameters. I've created an issue in our bug tracking system (http://bug.cloveretl.org/view.php?id=5346) for it.
The only workaround I can see is to place all needed resources in the folders with paths without spaces and don't quote the parameters. -
It must be something else, since I'm using quoted parameters for the -plugins and these are located a loaded without a hitch.
I've also tried relocating the project to C:/workspace/RPM, and using this non-space-location doesn't work either.
Here is some output from the master graph log:INFO [RUN_GRAPH0] - Running graph C:/workspace/RPM/CustomerFiles/NewTest/RPM_Initialize.grf in separate instance of clover.
INFO [RUN_GRAPH0] - Executing command: "[java, -cp, /C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/cloveretl.engine.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/annotations.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/cloveretl.engine.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/commons-cli-1.1.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/commons-codec-1.4.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/commons-logging.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/commons-net-1.4.1.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/dom4j-1.6.1.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/icu4j-normalizer.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/jakarta-oro-2.0.8.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/janino.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/javolution.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/jms.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/joda-time-1.6.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/jsch-0.1.39.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/jsr173_1.0_api.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/jxl.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/log4j-1.2.12.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/ooxml-schemas-1.0.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/openxml4j-1.0-beta.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/poi-3.5.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/poi-ooxml-3.5.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/tar.jar;/C:/Program Files (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/lib/xmlbeans-2.3.0.jar;C:\workspace\RPM\bin;C:\Program Files (x86)\InitiateSystems\Workbench9.2.0\plugins\com.initiatesystems.workbench_9.2.0\lib\madapi.jar;C:\Program Files (x86)\InitiateSystems\Workbench9.2.0\plugins\com.initiatesystems.workbench_9.2.0\lib\madhandlers.jar, org.jetel.main.runGraph, -plugins, "C:/Program, Files, (x86)/InitiateSystems/Workbench9.2.0/plugins/com.cloveretl.gui_2.9.1/lib/plugins";"C:/Program, Files, (x86)/InitiateSystems/Workbench9.2.0/plugins/com.initiatesystems.etl.gui_9.2.0/lib/plugins", -P:PROJECT_DIR=C:/workspace/RPM, -P:PROJECT_DIR=null, -nodebug, C:/workspace/RPM/CustomerFiles/NewTest/RPM_Initialize.grf]"
INFO [RUN_GRAPH0] - C:/workspace/RPM/CustomerFiles/NewTest/RPM_Initialize.grf: Processing with an ERROR:
: Process exit value = -1
Notice that PROJECT_DIR is defined (along with -plugins) and then immediately set to null!
and this from the graph that was being run:INFO [main] - Graph definition file: C:/workspace/RPM/CustomerFiles/NewTest/RPM_Initialize.grf
INFO [main] - PROJECT_DIR property: null
ERROR [main] - Error in reading graph from XML !
org.jetel.exception.XMLConfigurationException: Can't load property definition from ${PROJECT_DIR}/z/zcustomer.prm
at org.jetel.graph.TransformationGraphXMLReaderWriter.instantiateProperties(TransformationGraphXMLReaderWriter.java:757)
at org.jetel.graph.TransformationGraphXMLReaderWriter.read(TransformationGraphXMLReaderWriter.java:328)
at org.jetel.graph.TransformationGraphXMLReaderWriter.read(TransformationGraphXMLReaderWriter.java:284)
at org.jetel.graph.TransformationGraphXMLReaderWriter.loadGraph(TransformationGraphXMLReaderWriter.java:213)
at org.jetel.main.runGraph.main(runGraph.java:334)
Caused by: java.io.FileNotFoundException: null\${PROJECT_DIR}\z\zcustomer.prm (The system cannot find the path specified)
I've tried everything I can think of to pass in the PROJECT_DIR, and always get "PROJECT_DIR property: null". Without this functionality it makes it extremely problematic to use external links for parameters, metadata, etc.
Anything else I should try? -
I see. Unfortunately you have CloverETL version with bug. You need newer CloverETLEngine (see download site).
Please sign in to leave a comment.
Comments 5