Customer Portal

Problem in implementing CompanyTransactionsTutorial Example

Comments 11

  • Avatar
    admin
    0
    Comment actions Permalink
    Hi megha,

    you probably can't open XML because is too big for Eclipse editor (even files in order of MB can be problematic - there is huge overhead due to DOM+syntax highlighting). There is message "caused by: java.lang.OutOfMemoryError: Java heap space". You can try to adjust -Xmx parameter (http://docs.oracle.com/javase/1.4.2/doc ... /java.html) of your Designer (pass -Xmx{mem in MB}m to command line when starting).

    There is just one file, because you removed "Records per file" and "Max number of records" parameters from XmlWriter. Take a look on default graph for reference.
  • Avatar
    jurban
    0
    Comment actions Permalink
    Here's a hint for passing the Xmx argument to Designer - open the CloverETLDesigner.ini file in your Designer installation and you'll find the Xmx argument there. If you installed Designer from an update site into an existing Eclipse, then the file will be called eclipse.ini.

    Regards,
    Jaro
  • Avatar
    megha
    0
    Comment actions Permalink
    Thanks jurban and kubosj for your reply. It really helped me a lot.
    Well i have gone through the refernce graphs and got that i have to set Records per file and Max number of records field.
    In the reference graph Records per file is set to 1 and Max number of records is set to {MAPPINGS}.
    Well, when i am setting the Max number of records to {Mappings} and clicking OK button then the value gets disappeared due to which m getting very large number of XML files approximately 100.000 files. :(
    Can anyone help me out and guide me how do i set Max number of files field to {MAPPINGS} and restrict the number of output XML files to 14.


    Thanks
  • Avatar
    admin
    0
    Comment actions Permalink
    Hi megha,

    the value of "Records per file" should be "${MAPPINGS}" not "{MAPPINGS}". Please take also a look to its value - in sample it is 15.

    For reference I attached unaltered graph. Are you sure you didn't do any changes in it?
  • Avatar
    megha
    0
    Comment actions Permalink
    Hey Kubosj,
    Thanx for your response and guidance. Oops sry but by mistake I wrote the value of Max number of records as {MAPPINGS}.
    Actually I created a new parameter known as MAPPINGS and put the value as 15. And then i substituted the value of Max number of records as ${MAPPINGS}. Well now I am getting 15 XML files as o/p but the content in those XML files are not the same as the one in CompanyTransactionsTutorial graph. And also I am getting Varying number of XML files same o/p ie. the first 3 XML o/p files content is same and then the next few XML o/p files are same.
    I have done the mapping the same way as it is done in CompanyTransactionsTutorial graph.
    But still I am not getting the o/p as per expected.
    Whereas in the CompanyTransactionsTutorial graph they are getting 15 XML o/p files distinct and correct.
    I am not able to identify where I am going wrong. :(
    Can you please help me out in this.
  • Avatar
    admin
    0
    Comment actions Permalink
    Hi megha,

    can you send me samples of "wrong" files? I also do not understand abbreviation "o/p" :)
  • Avatar
    megha
    0
    Comment actions Permalink
    Hey Kubosj,

    I am greatful to you for your continuous response to my post.
    With O/p I mean to say output. :)

    And in output I am getting duplicate XML files means with same content. I am sending you some of my output files below:

    CustEmplFamAm000.xml:

    <?xml version="1.0" encoding="UTF-8" ?>
    - <Customers>
    - <customers CustomerID="1" CustomerState="Virginia" CustomerSurname="Driggers" CustomerFirstname="Tauni">
    - <employee EmployeeID="12" EmployeeSurname="Schmidt" EmployeeFirstname="Daniel">
    <spouse SpouseName="Rosemary" />
    <Children />
    <transactions AmountID="98920" Amount="39.14" CustomerID="1" EmployeeID="12" />
    </employee>
    </customers>
    </Customers>


    CustEmplFamAm001.xml:

    <?xml version="1.0" encoding="UTF-8" ?>
    - <Customers>
    - <customers CustomerID="1" CustomerState="Virginia" CustomerSurname="Driggers" CustomerFirstname="Tauni">
    - <employee EmployeeID="12" EmployeeSurname="Schmidt" EmployeeFirstname="Daniel">
    <spouse SpouseName="Rosemary" />
    <Children />
    <transactions AmountID="98920" Amount="39.14" CustomerID="1" EmployeeID="12" />
    </employee>
    </customers>
    </Customers>


    CustEmplFamAm002.xml:

    <?xml version="1.0" encoding="UTF-8" ?>
    - <Customers>
    - <customers CustomerID="1" CustomerState="Virginia" CustomerSurname="Driggers" CustomerFirstname="Tauni">
    - <employee EmployeeID="12" EmployeeSurname="Schmidt" EmployeeFirstname="Daniel">
    <spouse SpouseName="Rosemary" />
    <Children />
    <transactions AmountID="98920" Amount="39.14" CustomerID="1" EmployeeID="12" />
    </employee>
    </customers>
    </Customers>


    CustEmplFamAm003.xml:

    <?xml version="1.0" encoding="UTF-8" ?>
    - <Customers>
    - <customers CustomerID="2" CustomerState="Montana" CustomerSurname="Keane" CustomerFirstname="Wathalee">
    - <employee EmployeeID="2" EmployeeSurname="Galvin" EmployeeFirstname="William">
    <spouse SpouseName="Valerie" />
    - <Children>
    <child ChildName="Amanda" />
    <child ChildName="Zoe" />
    </Children>
    <transactions AmountID="2" Amount="768.5" CustomerID="2" EmployeeID="2" />
    </employee>
    - <employee EmployeeID="7" EmployeeSurname="Salazar" EmployeeFirstname="Jacqueline">
    <spouse SpouseName="Rowan" />
    - <Children>
    <child ChildName="Daniel" />
    <child ChildName="Stanley" />
    <child ChildName="Zoe" />
    </Children>
    <transactions AmountID="96624" Amount="376.2" CustomerID="2" EmployeeID="7" />
    </employee>
    - <employee EmployeeID="17" EmployeeSurname="Jans" EmployeeFirstname="Amy">
    <spouse SpouseName="James" />
    - <Children>
    <child ChildName="Agatha" />
    <child ChildName="Frank" />
    </Children>
    <transactions AmountID="88057" Amount="836.88" CustomerID="2" EmployeeID="17" />
    </employee>
    - <employee EmployeeID="37" EmployeeSurname="Zwier" EmployeeFirstname="James">
    <spouse SpouseName="Francene" />
    - <Children>
    <child ChildName="Amy" />
    <child ChildName="Gilbert" />
    <child ChildName="Peter" />
    </Children>
    <transactions AmountID="72212" Amount="364.13" CustomerID="2" EmployeeID="37" />
    </employee>
    - <employee EmployeeID="75" EmployeeSurname="Tusting" EmployeeFirstname="Leonardo">
    <spouse SpouseName="Francene" />
    - <Children>
    <child ChildName="Robert" />
    <child ChildName="Rosemary" />
    </Children>
    <transactions AmountID="36880" Amount="239.55" CustomerID="2" EmployeeID="75" />
    </employee>
    </customers>
    </Customers>


    CustEmplFamAm004.xml:

    <?xml version="1.0" encoding="UTF-8" ?>
    - <Customers>
    - <customers CustomerID="2" CustomerState="Montana" CustomerSurname="Keane" CustomerFirstname="Wathalee">
    - <employee EmployeeID="2" EmployeeSurname="Galvin" EmployeeFirstname="William">
    <spouse SpouseName="Valerie" />
    - <Children>
    <child ChildName="Amanda" />
    <child ChildName="Zoe" />
    </Children>
    <transactions AmountID="2" Amount="768.5" CustomerID="2" EmployeeID="2" />
    </employee>
    - <employee EmployeeID="7" EmployeeSurname="Salazar" EmployeeFirstname="Jacqueline">
    <spouse SpouseName="Rowan" />
    - <Children>
    <child ChildName="Daniel" />
    <child ChildName="Stanley" />
    <child ChildName="Zoe" />
    </Children>
    <transactions AmountID="96624" Amount="376.2" CustomerID="2" EmployeeID="7" />
    </employee>
    - <employee EmployeeID="17" EmployeeSurname="Jans" EmployeeFirstname="Amy">
    <spouse SpouseName="James" />
    - <Children>
    <child ChildName="Agatha" />
    <child ChildName="Frank" />
    </Children>
    <transactions AmountID="88057" Amount="836.88" CustomerID="2" EmployeeID="17" />
    </employee>
    - <employee EmployeeID="37" EmployeeSurname="Zwier" EmployeeFirstname="James">
    <spouse SpouseName="Francene" />
    - <Children>
    <child ChildName="Amy" />
    <child ChildName="Gilbert" />
    <child ChildName="Peter" />
    </Children>
    <transactions AmountID="72212" Amount="364.13" CustomerID="2" EmployeeID="37" />
    </employee>
    - <employee EmployeeID="75" EmployeeSurname="Tusting" EmployeeFirstname="Leonardo">
    <spouse SpouseName="Francene" />
    - <Children>
    <child ChildName="Robert" />
    <child ChildName="Rosemary" />
    </Children>
    <transactions AmountID="36880" Amount="239.55" CustomerID="2" EmployeeID="75" />
    </employee>
    </customers>
    </Customers>


    The same output continues till CustEmplFamAm011.xml.
    Then the next output file is below:

    CustEmplFamAm012.xml:

    <?xml version="1.0" encoding="UTF-8" ?>
    - <Customers>
    - <customers CustomerID="3" CustomerState="South Carolina" CustomerSurname="Barile" CustomerFirstname="Marlowe">
    - <employee EmployeeID="3" EmployeeSurname="Barrier" EmployeeFirstname="Agatha">
    <spouse SpouseName="John" />
    <Children />
    <transactions AmountID="3" Amount="297.59" CustomerID="3" EmployeeID="3" />
    </employee>
    - <employee EmployeeID="94" EmployeeSurname="Ashbrook" EmployeeFirstname="William">
    <spouse SpouseName="Valerie" />
    - <Children>
    <child ChildName="Flora" />
    <child ChildName="Janessa" />
    </Children>
    <transactions AmountID="72293" Amount="359.69" CustomerID="3" EmployeeID="94" />
    </employee>
    - <employee EmployeeID="122" EmployeeSurname="Cammack" EmployeeFirstname="Raquel">
    <spouse SpouseName="Daniel" />
    <Children />
    <transactions AmountID="51760" Amount="578.4" CustomerID="3" EmployeeID="122" />
    </employee>
    - <employee EmployeeID="124" EmployeeSurname="Trouse" EmployeeFirstname="Arnold">
    <spouse SpouseName="Vanessa" />
    <Children />
    <transactions AmountID="91667" Amount="447.11" CustomerID="3" EmployeeID="124" />
    </employee>
    </customers>
    </Customers>


    And now the above output continues till CustEmplFamAm014.xml... Means CustEmplFamAm012.xml, CustEmplFamAm013.xml, CustEmplFamAm014.xml contains same output.


    Now I am not able to understand why I am getting such outputs?? :(

    The correct output files should have distinct output and you might also be knowing what exactly I should get in output XML files.


    I would be great if u can help me out with this issue.

    Thanks,
    Megha
  • Avatar
    admin
    0
    Comment actions Permalink
    Hi Megha,

    I am starting to be confused by amount of inputs. So lets clarify:

    1] does original&unaltered "A05_CreatingXMLEmplFamCustAm" work as expected with your installation?
    2] you derived your graph from "A05_CreatingXMLEmplFamCustAm" - can't you compare it against original and find difference? (this is hard for me because I am missing e.g. external metadata you created)
    3] maybe you can derive your graph again and check outputs after each small change
    4] you can use http://www.cloveretl.com/documentation/ ... edges.html for debugging
    5] you should take a care of each property of each component - even small change may have major impact (type of join, ...)
  • Avatar
    megha
    0
    Comment actions Permalink
    Hi Kubosj,
    Thanks for ur reply.

    Well i have checked the original & unaltered graph from CompanyTransactionsTutorial. It is giving the expected output.

    I have designed the same graph with same configuration and same metadata fields and checked all the parameters for each component and have kept all the parameters same as given in the Original Graph. But still I am not getting the exact output which i should get.

    In the original unaltered graph, 15 XML files are generated and none of the XML file contains the same output. All of the 15 XML files have distinct output which means no duplicacy of output exists.

    When I am running my graph, I am also getting 15 XML files but more than 1 Xml files consists of same output. I am not getting all the 15 XML files distinct.

    Can you please figure out in what field I may be going wrong?? :(


    Thanks,
    Megha
  • Avatar
    admin
    0
    Comment actions Permalink
    Hi megha,

    did you tried to debug your graph? You should see number of records send through each edge, as well as values. Then you should follow your data from input components to output components. Check records count and records data after each component. Compare between original and your graph. When you find discrepancy, then try to find out what is wrong with problematic component - what is difference between original and your graph. It may be one small settings.

    Also take a look on link from my last post.
  • Avatar
    megha
    0
    Comment actions Permalink
    Hey kubosj,

    Thankyou so much for ur suggestion. Finally, I have identified my mistake and now I am getting correct output. :)

Please sign in to leave a comment.