Partition component Question

oldforum

• July 16, 2007 00:00

• Answered

Follow

Hello!

I am trying to use the PARTITION component (using "graphPartition.grf" as example) to load a database table but it is not working. The strange part is that I get no errors: no record are loaded into my database table. This is what my graph looks like

<?xml version="1.0" encoding="UTF-8"?>
...
<Phase number="0">
<Node id="INPUT" type="DB_INPUT_TABLE" dbConnection="OracleSRC" url="/sql/products.sql"/>
<Node id="BROADCAST" type="SIMPLE_COPY"/>
<Node id="PARTITION1" type="org.jetel.component.Partition" partitionKey="TYPE" ranges="2;3"/>
<Node id="OUTPUT1" type="DB_OUTPUT_TABLE" dbConnection="OracleTGT" dbTable="Sales" />
<Node id="OUTPUT2" type="DB_OUTPUT_TABLE" dbConnection="OracleTGT" dbTable="Sales" />
<Edge id="INEDGE1" fromNode="INPUT:0" toNode="BROADCAST:0" metadata=.../>
<Edge id="INEDGE2" fromNode="BROADCAST:0" toNode="PARTITION1:0" metadata=.../>
<Edge id="INNEREDGE1" fromNode="PARTITION1:0" toNode="OUTPUT1:0" metadata=.../>
<Edge id="INNEREDGE2" fromNode="PARTITION1:1" toNode="OUTPUT2:0" metadata=.../>
</Phase>
</Graph>

Has anyone been able to use this component with success against a database table? Please note that my example above works successfully if I output to a set of files (again...using "graphPartition.grf" as example) instead of a database table.

Is using this component against a database table not supported? I welcome any help and/or suggestion. Thank you.

Alan.

• 

  oldforum

  • July 16, 2007 08:23

  0

  Comment actions Permalink

  Hello Alan !
  
  Your set up of Partition component suggests three intervals - ie. three output edges should be connected:
  
  *first interval ...3>
  *second interval (3...5>
  *third interval (5...
  
  see following example:
  ******
  <Phase number="0">
  <Node id="INPUT1" type="DELIMITED_DATA_READER_NIO" DataPolicy="Strict" >
  <attr name="fileURL">data/employees.dat</attr>
  </Node>
  <Node id="PARTITION" type="org.jetel.component.Partition">
  <attr name="ranges">3;5</attr>
  <attr name="partitionKey">EmployeeID</attr>
  </Node>
  <Node id="TRASH1" type="TRASH" />
  <Node id="TRASH2" type="TRASH"/>
  <Node id="TRASH3" type="TRASH"/>
  <Edge id="INEDGE1" fromNode="INPUT1:0" toNode="PARTITION:0" metadata="InMetadata"/>
  <Edge id="INNEREDGE1" fromNode="PARTITION:0" toNode="TRASH1:0" metadata="InMetadata"/>
  <Edge id="INNEREDGE2" fromNode="PARTITION:1" toNode="TRASH2:0" metadata="InMetadata"/>
  <Edge id="INNEREDGE3" fromNode="PARTITION:2" toNode="TRASH3:0" metadata="InMetadata"/>
  </Phase>
  </Graph>
  ******
  
  David.
• 

  oldforum

  • July 16, 2007 08:23

  0

  Comment actions Permalink

  Hello Alan !
  
  This is very strange in deed ! If you see data coming out of trash, it means that PARTITION component sends them out.
  
  One thing I noticed - you are sharing the same DB connection among all DB_OUTPUT_TABLE. Try to use "threadSafeConnectio" option or simply create three DBConnections and allow each DB_OUTPUT_TABLE to use its own - simply copy&paste the one you have now and specify different name/id for each.
  
  Also send me the original graph tracking output (with logging configured to output even DEBUG ingo) to david.pavlis<at>centrum.cz
  
  David.
• 

  oldforum

  • July 16, 2007 08:23

  0

  Comment actions Permalink

  David,
  
  I understand what you are saying. I changed my graph as per your email:
  
  <?xml version="1.0" encoding="UTF-8"?>
  ...
  <Phase number="0">
  <Node id="INPUT1" type="DB_INPUT_TABLE" dbConnection="OracleSRC" url="/sql/products.sql"/>
  <Node id="PARTITION" type="org.jetel.component.Partition" partitionKey="TYPE" ranges="2;3"/>
  <Node id="OUTPUT1" type="DB_OUTPUT_TABLE" dbConnection="OracleTGT" dbTable="Sales" />
  <Node id="OUTPUT2" type="DB_OUTPUT_TABLE" dbConnection="OracleTGT" dbTable="Sales" />
  <Node id="OUTPUT3" type="DB_OUTPUT_TABLE" dbConnection="OracleTGT" dbTable="Sales" />
  <Edge id="INEDGE1" fromNode="INPUT1:0" toNode="PARTITION:0" metadata="..."/>
  <Edge id="INNEREDGE1" fromNode="PARTITION:0" toNode="OUTPUT1:0" metadata="..."/>
  <Edge id="INNEREDGE2" fromNode="PARTITION:1" toNode="OUTPUT2:0" metadata="..."/>
  <Edge id="INNEREDGE3" fromNode="PARTITION:2" toNode="OUTPUT3:0" metadata="..."/>
  </Phase>
  </Graph>
  
  ZERO ROWS are loaded to my target table and I got NO ERRORS...BUT...Using TRASH output as option (instead of database table) works very well.
  
  Does this mean that the PARTITION component is not able to work when connected to a DB_OUTPUT component?
  
  Thanks in advance for your advice.
  
  Alan.

Please sign in to leave a comment.