I had been using a plugin version of CloverETL withing Eclipse. However, I need the ability to run a graph from the command line (so I can schedule it via Windows Task Manager). So, today, I downloaded the standalone version:
Version: 4.0.0.026M1
I searched for a clover.bat file which is apparently needed to run from a command line and no such file was found. Did I get a corrupted download or has something major changed in this version (I'm assuming a major changes since you jumped to v4.0.0...).
Here's is the forum topic I was following:
http://forum.cloveretl.com/viewtopic.php?f=4&t=4986#p7850
Version: 4.0.0.026M1
I searched for a clover.bat file which is apparently needed to run from a command line and no such file was found. Did I get a corrupted download or has something major changed in this version (I'm assuming a major changes since you jumped to v4.0.0...).
Here's is the forum topic I was following:
http://forum.cloveretl.com/viewtopic.php?f=4&t=4986#p7850
-
Hi skaszyk,
The scripts have been indeed removed in the release 4.0.0-M1. They were by-product of our build process and not needed by the Designer application. Additionally we are in the process of redesigning our embedded runtime in Designer, so the scripts stop working with the next release 4.0.0-M2.
If you came across the scripts and used them, please also be aware they do not initialize plugin system correctly and graphs containing components that are not part of open source Engine do not run. In case you want to keep using CloverETL from command-line, consider switching to the open source Engine instead, where clover.bat originates from. The open source Engine however does not contain all components and does not support Jobflow or Subgraphs; those are only supported by our commercial products. -
I downloaded the open source Engine as you suggested but I just can't seem to get the command correct. No matter how I format the command, it doesn't get parsed correctly and I either get the usage printout or I get an error that it can't find the .grf (because the path isn't parsed correctly).
Can you provide some guidance for a full command that works on Windows?
Here's the usage result with my command in bold:
C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\graph\LPL File Loaders>clover.bat "-P:PROJECT_PATH=C:\cloverETL\AdvisorDataLoads\" "graph\LPLFileLoaders\LoadDailyLPLPositionFile.grf"
CLOVER_HOME=C:\cloverETL\bin\.\..
CLOVER_CMD_LINE_ARGS="-P:PROJECT_PATH=C:\cloverETL\AdvisorDataLoads\" "graph\LPLFileLoaders\LoadDailyLPLPositionFile.grf"
JAVA_CMD_LINE_ARGS=
USER_CLASSPATH=
"C:\Program Files\Java\jdk1.7.0_45\bin\java.exe" -classpath ";;C:\cloverETL\bin\;C:\Program Files\Java\jdk1.7.0_45\lib\tools.jar;;C:\cloverETL\bin\.\../lib/annotations.jar;C:\cloverETL\bin\.\../lib/cloveretl.engine.jar;C:\cloverETL\bin\.\../lib/commons-cli-1.2.jar;C:\cloverETL\bin\.\../lib/commons-codec-1.6.jar;C:\cloverETL\bin\.\../lib/commons-collections-3.2.1.jar;C:\cloverETL\bin\.\../lib/commons-compress-1.7.jar;C:\cloverETL\bin\.\../lib/commons-io-2.2.jar;C:\cloverETL\bin\.\../lib/commons
-io-LICENSE.txt;C:\cloverETL\bin\.\../lib/commons-logging-1.1.3.jar;C:\cloverETL\bin\.\../lib/commons-net-3.3.jar;C:\cloverETL\bin\.\../lib/commons-pool-1.6.jar;C:\cloverETL\bin\.\../lib/dom4j-1.6.1.jar;C:\cloverETL\bin\.\../lib/ftp4j-1.7.2.jar;C:\cloverETL\bin\.\../lib/ftp4j-LICENSE.txt;C:\cloverETL\bin\.\../lib/httpclient-4.3.2.jar;C:\cloverETL\bin\.\../lib/httpclient-LICENSE.txt;C:\cloverETL\bin\.\../lib/httpcore-4.3.1.jar;C:\cloverETL\bin\.\../lib/httpcore-LICENSE.txt;C:\cloverETL\bin\.\../l
ib/httpmime-4.3.2.jar;C:\cloverETL\bin\.\../lib/httpmime-LICENSE.txt;C:\cloverETL\bin\.\../lib/icu4j-52_1.jar;C:\cloverETL\bin\.\../lib/icu4j-LICENSE.html;C:\cloverETL\bin\.\../lib/icu4j-unicode-LICENSE.txt;C:\cloverETL\bin\.\../lib/jackson-core-asl-1.9.3.jar;C:\cloverETL\bin\.\../lib/javolution.jar;C:\cloverETL\bin\.\../lib/jaxen-1.1.6.jar;C:\cloverETL\bin\.\../lib/jcifs-1.3.17.jar;C:\cloverETL\bin\.\../lib/jets3t-0.9.0.jar;C:\cloverETL\bin\.\../lib/jms.jar;C:\cloverETL\bin\.\../lib/joda-time-2
.1.jar;C:\cloverETL\bin\.\../lib/jsch-0.1.50.jar;C:\cloverETL\bin\.\../lib/jxl.jar;C:\cloverETL\bin\.\../lib/LICENSE.txt;C:\cloverETL\bin\.\../lib/log4j-1.2.17.jar;C:\cloverETL\bin\.\../lib/poi-3.8-20120326.jar;C:\cloverETL\bin\.\../lib/poi-ooxml-3.8-20120326.jar;C:\cloverETL\bin\.\../lib/poi-ooxml-schemas-3.8-20120326.jar;C:\cloverETL\bin\.\../lib/sardine-5.0.2.jar;C:\cloverETL\bin\.\../lib/signpost-commonshttp4-1.2.1.2.jar;C:\cloverETL\bin\.\../lib/signpost-core-1.2.1.2.jar;C:\cloverETL\bin\.\
../lib/slf4j-api-1.7.2.jar;C:\cloverETL\bin\.\../lib/slf4j-log4j12-1.7.2.jar;C:\cloverETL\bin\.\../lib/truezip-driver-file-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-driver-zip-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-file-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-kernel-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-path-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-swing-7.7.5.jar;C:\cloverETL\bin\.\../lib/xmlbeans-2.5.0-modified-for-WSC.jar;C:\cloverETL\bin\.\../lib/XmlSchema-1.4.7-modified-
for-WSC.jar;C:\cloverETL\bin\.\../lib/XmlSchema-LICENSE.txt" "-Dclover.home=C:\cloverETL\bin\.\.." org.jetel.main.runGraph -plugins "C:\cloverETL\bin\.\..\plugins" "-P:PROJECT_PATH=C:\cloverETL\AdvisorDataLoads\" "graph\LPLFileLoaders\LoadDailyLPLPositionFile.grf"
Usage: runGraph [-(v|cfg|logcfg|loglevel|P:|tracking|info|plugins|pass)] <graph definition file>
Options:
-v be verbose - print even graph layout
-P:<key>=<value> add definition of property to global graph's property list
-cfg <filename> load definitions of properties from specified file
-logcfg <filename> load log4j configuration from specified file;
if not specified, "log4j.properties" should be in classpath
-loglevel <level> overrides log4j configuration and sets specified logging level for rootLogger;
possible levels are: ALL | TRACE | DEBUG | INFO | WARN | ERROR | FATAL | OFF
-tracking <seconds> how frequently output the graph processing status
-info print info about Clover library version
-plugins directory where to look for plugins/components
-pass password for decrypting of hidden connections passwords
-stdin load graph definition from STDIN
-loghost define host and port number for socket appender of log4j (log4j library is required); i.e. localhost:4445
-checkconfig only check graph configuration
-skipcheckconfig skip checking of graph configuration
-noJMX turns off sending graph tracking information
-config <filename> load default engine properties from specified file
-nodebug turns off all runtime debugging e.g edge debugging
-debugdirectory <foldername> directory dedicated to store temporary debug data; default is java's temporary folder
Note: <graph definition file> can be either local filename or URL of local/remote file
Here's where the graph can't be found when I try to use full paths in double quotes:
C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\graph\LPL File Loaders>clover.bat "-P:PROJECT_PATH=C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\" "C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\graph\LPLFileLoaders\graph\LPLFileLoaders\LoadDailyLPLPositionFile.grf"
CLOVER_HOME=C:\cloverETL\bin\.\..
CLOVER_CMD_LINE_ARGS="-P:PROJECT_PATH=C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\" "C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\graph\LPLFileLoaders\graph\LPLFileLoaders\LoadDailyLPLPositionFile.grf"
JAVA_CMD_LINE_ARGS=
USER_CLASSPATH=
"C:\Program Files\Java\jdk1.7.0_45\bin\java.exe" -classpath ";;C:\cloverETL\bin\;C:\Program Files\Java\jdk1.7.0_45\lib\tools.jar;;C:\cloverETL\bin\.\../lib/annotations.jar;C:\cloverETL\bin\.\../lib/cloveretl.engine.jar;C:\cloverETL\bin\.\../lib/commons-cli-1.2.jar;C:\cloverETL\bin\.\../lib/commons-codec-1.6.jar;C:\cloverETL\bin\.\../lib/commons-collections-3.2.1.jar;C:\cloverETL\bin\.\../lib/commons-compress-1.7.jar;C:\cloverETL\bin\.\../lib/commons-io-2.2.jar;C:\cloverETL\bin\.\../lib/commons
-io-LICENSE.txt;C:\cloverETL\bin\.\../lib/commons-logging-1.1.3.jar;C:\cloverETL\bin\.\../lib/commons-net-3.3.jar;C:\cloverETL\bin\.\../lib/commons-pool-1.6.jar;C:\cloverETL\bin\.\../lib/dom4j-1.6.1.jar;C:\cloverETL\bin\.\../lib/ftp4j-1.7.2.jar;C:\cloverETL\bin\.\../lib/ftp4j-LICENSE.txt;C:\cloverETL\bin\.\../lib/httpclient-4.3.2.jar;C:\cloverETL\bin\.\../lib/httpclient-LICENSE.txt;C:\cloverETL\bin\.\../lib/httpcore-4.3.1.jar;C:\cloverETL\bin\.\../lib/httpcore-LICENSE.txt;C:\cloverETL\bin\.\../l
ib/httpmime-4.3.2.jar;C:\cloverETL\bin\.\../lib/httpmime-LICENSE.txt;C:\cloverETL\bin\.\../lib/icu4j-52_1.jar;C:\cloverETL\bin\.\../lib/icu4j-LICENSE.html;C:\cloverETL\bin\.\../lib/icu4j-unicode-LICENSE.txt;C:\cloverETL\bin\.\../lib/jackson-core-asl-1.9.3.jar;C:\cloverETL\bin\.\../lib/javolution.jar;C:\cloverETL\bin\.\../lib/jaxen-1.1.6.jar;C:\cloverETL\bin\.\../lib/jcifs-1.3.17.jar;C:\cloverETL\bin\.\../lib/jets3t-0.9.0.jar;C:\cloverETL\bin\.\../lib/jms.jar;C:\cloverETL\bin\.\../lib/joda-time-2
.1.jar;C:\cloverETL\bin\.\../lib/jsch-0.1.50.jar;C:\cloverETL\bin\.\../lib/jxl.jar;C:\cloverETL\bin\.\../lib/LICENSE.txt;C:\cloverETL\bin\.\../lib/log4j-1.2.17.jar;C:\cloverETL\bin\.\../lib/poi-3.8-20120326.jar;C:\cloverETL\bin\.\../lib/poi-ooxml-3.8-20120326.jar;C:\cloverETL\bin\.\../lib/poi-ooxml-schemas-3.8-20120326.jar;C:\cloverETL\bin\.\../lib/sardine-5.0.2.jar;C:\cloverETL\bin\.\../lib/signpost-commonshttp4-1.2.1.2.jar;C:\cloverETL\bin\.\../lib/signpost-core-1.2.1.2.jar;C:\cloverETL\bin\.\
../lib/slf4j-api-1.7.2.jar;C:\cloverETL\bin\.\../lib/slf4j-log4j12-1.7.2.jar;C:\cloverETL\bin\.\../lib/truezip-driver-file-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-driver-zip-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-file-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-kernel-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-path-7.7.5.jar;C:\cloverETL\bin\.\../lib/truezip-swing-7.7.5.jar;C:\cloverETL\bin\.\../lib/xmlbeans-2.5.0-modified-for-WSC.jar;C:\cloverETL\bin\.\../lib/XmlSchema-1.4.7-modified-
for-WSC.jar;C:\cloverETL\bin\.\../lib/XmlSchema-LICENSE.txt" "-Dclover.home=C:\cloverETL\bin\.\.." org.jetel.main.runGraph -plugins "C:\cloverETL\bin\.\..\plugins" "-P:PROJECT_PATH=C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\" "C:\Users\skaszyk.IFPARTNERS\workspace\Advisor Data Loads\graph\LPLFileLoaders\graph\LPLFileLoaders\LoadDailyLPLPositionFile.grf"
INFO [main] - *** CloverETL framework/transformation graph, (c) 2002-2014 Javlin a.s. ***
INFO [main] - Running with CloverETL library version 4.0.0.M1 build#026 compiled 10/04/2014 02:48:33
INFO [main] - Running on 8 CPU(s), OS Windows 7, architecture amd64, Java version 1.7.0_45, max available memory for JVM 3715584 KB
INFO [main] - Loading default properties from: defaultProperties
DEBUG [main] - Plugin org.jetel.bulkloader loaded.
id - org.jetel.bulkloader
version - 4.0.0
.... (more logging)
ERROR [main] -
----------------------------------------------------- Error details -------------------------------------------------------
Error - graph definition file can't be read
Loads\graph\LPLFileLoaders\graph\LPLFileLoaders\LoadDailyLPLPositionFile.grf (The system cannot find the path specified)
---------------------------------------------------------------------------------------------------------------------------
(notice it's not parsing the path correctly) -
Hi skaszyk,
for project stored in "c:\test\sample project" (space in path is fine) having graph on subpath "graph\WebServicesHTTP.grf" you may execute engine by this command:
c:\test\cloverETL\bin>clover.bat -contexturl "c:\test\sample project" graph\WebServicesHTTP.grf
I hope this helps. -
If you came across the scripts and used them, please also be aware they do not initialize plugin system correctly and graphs containing components that are not part of open source Engine do not run...
Hi Jaroslav,
So if using some of the additional transformation components present in the full Designer we must use the Server product to schedule the graphs, you cannot call the Designer from the command line / a batch file and use all transformations ?.
Thanks. -
Hi Fatboy40,
Yes, it is forbidden by EULA to execute graphs on command line using Engine of commercial CloverETL Designer. You can either prepare graphs using OpenSource components and execute on OpenSource Engine or purchase CloverETL Server.
Please let me know if you have further questions.
Please sign in to leave a comment.
Comments 6