Hi Everyone,
Currently in my project we are using CloverETL 3.2 version and we are not using any graphical user interface for development.
As per requirement I have to read the following xml file and store it into csv file. But the problem is I am unable to do it as I am not aware of full function of CloverETL. I have gone through XMLExtract class but I don't understand how to create Graph and run the XMLExtract.
The ones who developed the code in CloverETL are left the company and don't any one who can help me to understand the workflow of the CloverETL.
I tried in online but don't find the any source which does not using CloverETL GUI for developing.
This is my sample XML file,
<?xml version="1.0" encoding="ISO-8859-1"?>
<FIXML s="2012-04-23" v="FIX.5.0SP2">
<Batch ID="RPTTA111PUBLI20170509">
<MktDef MktID="XEUR" MktSegID="14" EfctvBizDt="2017-05-11" NxtEfctvBizDt="2017-05-15" MktSeg="CONF" MarketSegmentDesc="FUT 8-13 Y. SWISS GOV.BONDS 6%" Sym="CH0002741988" ParentMktSegmID="FBND" Ccy="CHF" MktSegStat="1" USFirmFlag="Y" PartID="2">
<MtchRules MtchRuleProdCmplx="5" MtchAlgo="PT"/>
<MtchRules MtchRuleProdCmplx="1" MtchAlgo="PT"/>
<FlexProdEligs FlexProdEligCmplx="5" FlexProdElig="Y"/>
<BaseTrdgRules QtSideInd="1" FastMktPctg="0">
<TickRules TickRuleProdCmplx="1" StartTickPxRng="0" EndTickPxRng="99999.9999" TickIncr="0.01"/>
<TickRules TickRuleProdCmplx="5" StartTickPxRng="0" EndTickPxRng="99999.9999" TickIncr="0.01"/>
<QuotSizeRules MinBidSz="1" MinOfrSz="1" FastMktInd="0"/>
<QuotSizeRules MinBidSz="1" MinOfrSz="1" FastMktInd="1"/>
<PxRngRules PxRngRuleID="75" PxRngProdCmplx="1" StartPxRng="0" EndPxRng="99999.9999" PxRngValu="0.15"/>
<PxRngRules PxRngRuleID="347" PxRngProdCmplx="5" StartPxRng="0" EndPxRng="99999.9999" PxRngValu="0.12"/>
</BaseTrdgRules>
<MDFeedTyps MDFeedTyp="HS" MDBkTyp="2" MktDepth="10" MDRcvryTmIntvl="120000" SvcLctnID1="224.0.50.102" SvcLctnSubID1="59500" SvcLctnID2="224.0.50.230" SvcLctnSubID2="59500"/>
<MDFeedTyps MDFeedTyp="HI" MDBkTyp="2" MktDepth="10" MktDepthTmIntvl="0" SvcLctnID1="224.0.50.103" SvcLctnSubID1="59501" SvcLctnID2="224.0.50.231" SvcLctnSubID2="59501"/>
<MDFeedTyps MDFeedTyp="HI" MDBkTyp="3" MktDepthTmIntvl="0" SvcLctnID1="224.0.114.97" SvcLctnSubID1="59501" SvcLctnID2="224.0.114.113" SvcLctnSubID2="59501"/>
<MDFeedTyps MDFeedTyp="HS" MDBkTyp="3" SvcLctnID1="224.0.114.96" SvcLctnSubID1="59500" SvcLctnID2="224.0.114.112" SvcLctnSubID2="59500"/>
<MDFeedTyps MDFeedTyp="L" MDBkTyp="2" MktDepth="5" MktDepthTmIntvl="3500" MDRcvryTmIntvl="30000" SvcLctnID1="224.0.50.89" SvcLctnSubID1="59500" SvcLctnID2="224.0.50.217" SvcLctnSubID2="59500"/>
</MktDef>
<SecDef PriSetPx="158.39">
<Instrmt ID="408805" Src="M" SecTyp="FUT" Status="1" ProdCmplx="1" CFI="FFMPSX" MatDt="2017-06-08" MMY="201706" Mult="1" ValMeth="FUT" SettlMeth="P" SettlSubMeth="4" PxPrcsn="2" MinPxIncr="0.01" MinPxIncrAmt="10">
<AID AltID="1048612" AltIDSrc="M"/>
<AID AltID="XF000001RQD8" AltIDSrc="4"/>
<Evnt EventTyp="7" Dt="2017-06-08"/>
</Instrmt>
<MktSegGrp MktSegID="14">
<SecTrdgRules>
<BaseTrdgRules>
<PxRngRules PxRngRuleID="75"/>
</BaseTrdgRules>
</SecTrdgRules>
</MktSegGrp>
</SecDef>
</Batch>
</FIXML>
Can any one help me on understanding the workflow and help me to create graph and run the XMLExtract class.
Or at least any working examples for custom developed sample project to understand the workflow.
Please do needful as soon as possible.
Thank you in Advance!
Sai Chaitanya
Currently in my project we are using CloverETL 3.2 version and we are not using any graphical user interface for development.
As per requirement I have to read the following xml file and store it into csv file. But the problem is I am unable to do it as I am not aware of full function of CloverETL. I have gone through XMLExtract class but I don't understand how to create Graph and run the XMLExtract.
The ones who developed the code in CloverETL are left the company and don't any one who can help me to understand the workflow of the CloverETL.
I tried in online but don't find the any source which does not using CloverETL GUI for developing.
This is my sample XML file,
<?xml version="1.0" encoding="ISO-8859-1"?>
<FIXML s="2012-04-23" v="FIX.5.0SP2">
<Batch ID="RPTTA111PUBLI20170509">
<MktDef MktID="XEUR" MktSegID="14" EfctvBizDt="2017-05-11" NxtEfctvBizDt="2017-05-15" MktSeg="CONF" MarketSegmentDesc="FUT 8-13 Y. SWISS GOV.BONDS 6%" Sym="CH0002741988" ParentMktSegmID="FBND" Ccy="CHF" MktSegStat="1" USFirmFlag="Y" PartID="2">
<MtchRules MtchRuleProdCmplx="5" MtchAlgo="PT"/>
<MtchRules MtchRuleProdCmplx="1" MtchAlgo="PT"/>
<FlexProdEligs FlexProdEligCmplx="5" FlexProdElig="Y"/>
<BaseTrdgRules QtSideInd="1" FastMktPctg="0">
<TickRules TickRuleProdCmplx="1" StartTickPxRng="0" EndTickPxRng="99999.9999" TickIncr="0.01"/>
<TickRules TickRuleProdCmplx="5" StartTickPxRng="0" EndTickPxRng="99999.9999" TickIncr="0.01"/>
<QuotSizeRules MinBidSz="1" MinOfrSz="1" FastMktInd="0"/>
<QuotSizeRules MinBidSz="1" MinOfrSz="1" FastMktInd="1"/>
<PxRngRules PxRngRuleID="75" PxRngProdCmplx="1" StartPxRng="0" EndPxRng="99999.9999" PxRngValu="0.15"/>
<PxRngRules PxRngRuleID="347" PxRngProdCmplx="5" StartPxRng="0" EndPxRng="99999.9999" PxRngValu="0.12"/>
</BaseTrdgRules>
<MDFeedTyps MDFeedTyp="HS" MDBkTyp="2" MktDepth="10" MDRcvryTmIntvl="120000" SvcLctnID1="224.0.50.102" SvcLctnSubID1="59500" SvcLctnID2="224.0.50.230" SvcLctnSubID2="59500"/>
<MDFeedTyps MDFeedTyp="HI" MDBkTyp="2" MktDepth="10" MktDepthTmIntvl="0" SvcLctnID1="224.0.50.103" SvcLctnSubID1="59501" SvcLctnID2="224.0.50.231" SvcLctnSubID2="59501"/>
<MDFeedTyps MDFeedTyp="HI" MDBkTyp="3" MktDepthTmIntvl="0" SvcLctnID1="224.0.114.97" SvcLctnSubID1="59501" SvcLctnID2="224.0.114.113" SvcLctnSubID2="59501"/>
<MDFeedTyps MDFeedTyp="HS" MDBkTyp="3" SvcLctnID1="224.0.114.96" SvcLctnSubID1="59500" SvcLctnID2="224.0.114.112" SvcLctnSubID2="59500"/>
<MDFeedTyps MDFeedTyp="L" MDBkTyp="2" MktDepth="5" MktDepthTmIntvl="3500" MDRcvryTmIntvl="30000" SvcLctnID1="224.0.50.89" SvcLctnSubID1="59500" SvcLctnID2="224.0.50.217" SvcLctnSubID2="59500"/>
</MktDef>
<SecDef PriSetPx="158.39">
<Instrmt ID="408805" Src="M" SecTyp="FUT" Status="1" ProdCmplx="1" CFI="FFMPSX" MatDt="2017-06-08" MMY="201706" Mult="1" ValMeth="FUT" SettlMeth="P" SettlSubMeth="4" PxPrcsn="2" MinPxIncr="0.01" MinPxIncrAmt="10">
<AID AltID="1048612" AltIDSrc="M"/>
<AID AltID="XF000001RQD8" AltIDSrc="4"/>
<Evnt EventTyp="7" Dt="2017-06-08"/>
</Instrmt>
<MktSegGrp MktSegID="14">
<SecTrdgRules>
<BaseTrdgRules>
<PxRngRules PxRngRuleID="75"/>
</BaseTrdgRules>
</SecTrdgRules>
</MktSegGrp>
</SecDef>
</Batch>
</FIXML>
Can any one help me on understanding the workflow and help me to create graph and run the XMLExtract class.
Or at least any working examples for custom developed sample project to understand the workflow.
Please do needful as soon as possible.
Thank you in Advance!
Sai Chaitanya
-
Hello Sai,
there are many sources of information on how to create graphs that would read and parse the content of a XML file such as the one you reported. To name a few:
• The Help documents describing the basics of creating and running graphs in CloverETL Designer are available here and here.
• The Help document describing the basic functionality of the XMLExtract component (which is the most commonly used XML file reader component in CloverETL Designer thanks to its GUI and lower memory consumption) can be found here.
• Within the very same article you might want to review the section called ‘XMLExtract Mapping Editor and XSD Schema’ that would provide you with a better insight into what the XSD schema is and how you can take advantage of it in the Designer.
• If you are interested specifically in XML reading components without the GUI, you might want to review the following Help documents for the XMLReader component and the XMLXPathReader component.
• There are also a handful of articles that concern reading and parsing of XML files on our CloverETL Forum. Scouring them can help you resolve a variety of XML specific issues that other CloverETL users encountered in the past and that have already been answered by our CloverCare Support.
From what you mentioned, however, you are currently using the 3.2 version of CloverETL which is fairly obsolete and our support of this version has already been discontinued. Upgrading to a more recent production version of CloverETL such as 4.5 or 4.6 is strongly recommended not only because of the overall improved functionality but also because it contains a series of handy tutorials that can help you comprehend the Designer functionality basics.
As for your current situation, may I clarify a couple of details?
1. May I know the reason why you develop graphs without the GUI?
2. Do you run the graphs that you developed within the GUI or not?
3. Do you run them using the open source engine?
4. Do you run them on the server or locally?
Best, -
Hi Barton,
Thanks for your reply.
Please find the details for what you need below,
As for your current situation, may I clarify a couple of details?
1. May I know the reason why you develop graphs without the GUI?
Reason: Because our project is very large and is having the combination of different tools and technologies.
When the project goes to production, they won't be doing anything in GUI, directly they will run the Jars. So, for that purpose
we included the CloverETL jars instead of GUI in my project.
And one more reason we are using CloverETL jars instead of GUI is because we are developing custom classes using the Jars.
2. Do you run the graphs that you developed within the GUI or not?
Yes, we run the graphs without using the GUI and running on server.
3. Do you run them using the open source engine?
We are running them using shell and python scripting.
4. Do you run them on the server or locally?
We run them in server.
I hope you get some clarity on what I am doing without CloverETL GUI.
And, as you mentioned you guys are not providing support to the version 3.2. If needed I can upgraded to latest version of CloverETL jars.
But please help me how can I write code to use XMLExtract or XMLReader or XMLXpathReader components in my project.
I have opened the source code of those and unable to understand what parameters to pass to the methods inside those components.
And I am unable to understand how to write the graph for those components.
Because I didn't find anything in internet to get sample project to run on server using those components.
Forgive me if I did any mistake in the way to asking the questions.
Please help me on this Barton.
Thank you very much!
Best Regards,
Sai Chaitanya -
Hello Sai,
thank you for the detailed explanation. I understand why you don’t want to use Clover GUI for your projects. However, running the jars outside of the GUI by copying them and running them elsewhere is not what they have been designed for. In fact, such manipulation is in contradiction with EULA. Typically, CloverETL users would use the CloverETL Designer (the GUI) to design the graphs by placing various components in the GUI and by setting up various properties in order to design a data transformation. They would also create jobflows in the GUI that would help them design and manage a graph orchestration. Ultimately, they would utilize the CloverETL Server to have the jobflows or graphs run automatically at predefined times or circumstances on a server. For those purposes, you can take advantage of the simple HTTP API or the launch services.
Nevertheless, I can imagine it would be cumbersome to comprehend how the software works only from the code and API perspective. Therefore, I would certainly suggest installing the latest 4.6 version of the CloverETL Designer and CloverETL Server and taking advantage of the tutorials to see how the software is designed and how you can take the real advantage of it in your projects. We will be more than glad to even provide you with an expert on demand who could help you design your projects on site.
Regards, -
Excellent answer by bartonv and great topic regarding Java. Thank you.
Please sign in to leave a comment.
Comments 4