EXT_MERGE_JOIN cannot perform Full Outer Join?

achan

• March 24, 2008 00:00

• Answered

Follow

graph file looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<Graph name="Test" description="POC" revision="1.0">
<Global>
  <Metadata fileURL="output.fmt" id="OutMetadata"/>
  <Metadata fileURL="data1.fmt" id="InMetadata1"/>
  <Metadata fileURL="data2.fmt" id="InMetadata2"/>
</Global>
<Phase number="0">
  <Node id="INPUT1" type="DATA_READER" fileURL="data1" quoteStrings="true"/>
  <Node id="INPUT2" type="DATA_READER" fileURL="data2" quoteStrings="true"/>
  <Node id="SORT_INPUT1" type="EXT_SORT" sortKey="REFERENCE"/>
  <Node id="SORT_INPUT2" type="EXT_SORT" sortKey="REFERENCE"/>
  <Node id="JOIN" type="EXT_MERGE_JOIN" joinKey="REFERENCE" joinType="fullOuter" transformClass="myPackage.ReformatJoinTest"/>
  <Node id="OUTPUT" type="DELIMITED_DATA_WRITER" append="false" fileURL="output.txt"/>

  <Edge id="INEDGE1" fromNode="INPUT1:0" toNode="SORT_INPUT1:0" metadata="InMetadata1"/>
  <Edge id="INEDGE2" fromNode="INPUT2:0" toNode="SORT_INPUT2:0" metadata="InMetadata2"/>
  <Edge fromNode="SORT_INPUT1:0" id="INNEREDGE1" metadata="InMetadata1" toNode="JOIN:0"/>
  <Edge fromNode="SORT_INPUT2:0" id="INNEREDGE2" metadata="InMetadata2" toNode="JOIN:1"/>
  <Edge fromNode="JOIN:0" id="OUTEDGE" metadata="OutMetadata" toNode="OUTPUT:0"/>
</Phase>
</Graph>

data1 file:
"000000010272 ","WASILLA","AK",99654
"000000010273 ","WASILLA","AK",99654

data2 file:
"000000010274 ","address1"
"000000010273 ","address2"

output.txt result:
"000000010272 ","WASILLA","AK",99654,
"000000010273 ","WASILLA","AK",99654,"address2"


looks like it's doing a left-outer join instead of performing a full-outer join??

any help is greatly appreciated :-)

Thanks!

al

• 

  avackova

  • March 25, 2008 13:21

  0

  Comment actions Permalink

  Hello,
  it's really bug. I've found where is the problem and I'm fixing it.
• 

  achan

  • March 27, 2008 15:52

  0

  Comment actions Permalink

  Hi Agata,
  
  Do you have an estimated date of when this bug will be fixed? Please let me know what I need to do after you fix it...
  
  Thanks,
  al
• 

  avackova

  • March 28, 2008 08:22

  0

  Comment actions Permalink

  Hi,
  fix is available in SVN branch 2.4 (svn://svn.berlios.de/svnroot/repos/clov ... .component) and will be in nearest patch release.
• 

  achan

  • March 28, 2008 16:13

  0

  Comment actions Permalink

  Hi Agata,
  
  I could not built (using ant) from the SVN you gave me... Could you send me the JAR file instead?
  
  Do you have a date on the next patch release? if it's in the near future, I can wait for that too...
  
  Thanks so much for your help :-)
  
  al
• 

  avackova

  • March 31, 2008 08:08

  0

  Comment actions Permalink

  Hi,
  patch will be released today or tomorrow :-) , so I'll not send you the jar file.
• 

  achan

  • April 11, 2008 21:10

  0

  Comment actions Permalink

  Hi Agata,
  
  After downloading CloverETL rel 2.4.3, I am still not getting the correct result for full-outer join...
  
  Using the same example,
  
  data1 file:
  "000000010272 ","WASILLA","AK",99654
  "000000010273 ","SACRAMENTO","CA",95825
  
  data2 file:
  "000000010274 ","address1"
  "000000010273 ","address2"
  
  the output.txt is:
  "000000010273 ","SACRAMENTO","CA",95825,"address2"
  "000000010274 ","SACRAMENTO","CA",95825,"address1"
  
  
  However, for full-outer join, the correct result should be:
  
  "000000010272 ","WASILLA","AK",99654,,
  "000000010273 ","SACRAMENTO","CA",95825,"address2"
  "000000010274 ",,,,"address1"
  
  right?? Please advice...
  
  Thanks,
  albert

Please sign in to leave a comment.