Writing a file of Fixed Length Records

wmdbowler

• May 14, 2008 00:00

• Answered

Follow

I am using CloverETL 2.4.3 and I have tried using the Fixed Data Length Writer and Universal Writer and neither are working as expected.

Fixed Length Data Writer is placing 'null' value (^0 in unix) and when I transfer that to windows, Textpad shows that as a big black pipe in each throughout the data.

Universal Data Writer doesn't write out anything ugly like that null value in the Fixed, but it doesn't stop at the end of the record at 223 bytes. I have a record delimiter set up on the metadata but it continues to write all the data out on one line. Why is that?

Here is a simple test graph I have been trying to configure to get this to work before putting into my final graph.
<Global>
<Metadata id="Metadata0">
<Record name="GLOBAL_EXTRACT" recordDelimiter="\r\n" recordSize="225" type="fixed">
<Field locale="en" name="EMPI_ID" nullable="true" shift="0" size="12" type="string"/>
<Field name="MEDIPAC_CPI" nullable="true" size="9" type="string"/>
<Field name="SPACE" nullable="true" size="1" type="string"/>
<Field name="MEDIPAC_MRN" nullable="true" size="9" type="string"/>
<Field name="SPACE1" nullable="true" size="1" type="string"/>
<Field name="EPIC_MRN" nullable="true" size="9" type="string"/>
<Field name="SPACE2" nullable="true" size="1" type="string"/>
<Field name="WPIC_CPI" nullable="true" size="9" type="string"/>
<Field name="SPACE3" nullable="true" size="1" type="string"/>
<Field name="WPIC_MRN" nullable="true" size="9" type="string"/>
<Field name="SPACE4" nullable="true" size="1" type="string"/>
<Field name="TDS_MRN" nullable="true" size="9" type="string"/>
<Field name="HP_ID" nullable="true" size="11" type="string"/>
<Field name="SPACE5" nullable="true" size="1" type="string"/>
<Field name="MAGEE_MRN" nullable="true" size="9" type="string"/>
<Field name="SPACE6" nullable="true" size="1" type="string"/>
<Field name="LASTNAME" nullable="true" size="25" type="string"/>
<Field name="FIRSTNAME" nullable="true" size="25" type="string"/>
<Field name="DOB" nullable="true" size="10" type="string"/>
<Field name="SEX" nullable="true" size="1" type="string"/>
<Field name="SSN" nullable="true" size="11" type="string"/>
<Field name="STATE" nullable="true" size="2" type="string"/>
<Field name="SPACE7" nullable="true" size="10" type="string"/>
<Field name="ZIPCODE" nullable="true" size="10" type="string"/>
<Field name="EXPIRED" nullable="true" size="1" type="string"/>
<Field name="SPACE8" nullable="true" size="1" type="string"/>
<Field name="CHILDERNS_MRN" nullable="true" size="9" type="string"/>
<Field name="SPACE9" nullable="true" size="3" type="string"/>
<Field name="PASSAVANT_MRN" nullable="true" size="9" type="string"/>
<Field name="SPACE10" nullable="true" size="3" type="string"/>
<Field delimiter="\r\n" name="BEST_ID" nullable="true" shift="0" size="10" type="string"/>
</Record>
</Metadata>
<Metadata id="memhead">
<Record name="memhead" recordSize="-1" type="delimited">
<Field delimiter="|" name="curentrecno" nullable="true" shift="0" type="long"/>
<Field delimiter="|" name="caudrecno" nullable="true" shift="0" type="long"/>
<Field delimiter="|" name="maudrecno" nullable="true" shift="0" type="long"/>
<Field delimiter="\n" name="memidnum" type="string"/>
</Record>
</Metadata>
</Global>
<Phase number="0">
<Node enabled="enabled" fileURL="ext_enthead_bedford.unl" guiHeight="0" guiName="Delimited Data Reader" guiWidth="0" guiX="43" guiY="40" id="DELIMITED_BEDFORD_READER" type="DELIMITED_DATA_READER"/>
<Node enabled="enabled" fileURL="ext_enthead_best.unl" guiHeight="0" guiName="Delimited Data Reader" guiWidth="0" guiX="46" guiY="114" id="DELIMITED_BEST_READER" type="DELIMITED_DATA_READER"/>
<Edge debugMode="false" fromNode="DELIMITED_BEDFORD_READER:0" guiBendpoints="" id="Edge0" inPort="Port 0 (driver)" metadata="memhead" outPort="Port 0 (out)" toNode="EXT_MERGE_JOIN0:0"/>
<Edge debugMode="false" fromNode="DELIMITED_BEST_READER:0" guiBendpoints="" id="Edge1" inPort="Port 1 (slave)" metadata="memhead" outPort="Port 0 (out)" toNode="EXT_MERGE_JOIN0:1"/>
</Phase>
<Phase number="1">
<Node enabled="enabled" guiHeight="0" guiName="Ext Merge Join" guiWidth="0" guiX="194" guiY="42" id="EXT_MERGE_JOIN0" joinKey="curentrecno;#curentrecno;#" type="EXT_MERGE_JOIN">
<attr name="transform"><![CDATA[${out.0.EMPI_ID} = ${in.0.memidnum};
${out.0.BEST_ID} = ${in.1.memidnum};
]]></attr>
</Node>
<Edge fromNode="EXT_MERGE_JOIN0:0" guiBendpoints="" id="Edge2" inPort="Port 0 (in)" metadata="Metadata0" outPort="Port 0 (out)" toNode="DATA_WRITER0:0"/>
</Phase>
<Phase number="2">
<Node append="false" enabled="enabled" fileURL="Local_test.txt" guiHeight="0" guiName="Universal Data Writer" guiWidth="0" guiX="302" guiY="278" id="DATA_WRITER0" type="DATA_WRITER"/>
</Phase>
</Graph>

• 

  wmdbowler

  • May 14, 2008 19:06

  0

  Comment actions Permalink

  I've also tried this changing the record delimiter to \r\n and it had no affect. The output of the universal data writer continues to place all data onto one line.
  </Metadata>
  <Metadata id="memhead">
  <Record name="memhead" recordSize="-1" type="delimited">
  <Field delimiter="|" name="curentrecno" nullable="true" shift="0" type="long"/>
  <Field delimiter="|" name="caudrecno" nullable="true" shift="0" type="long"/>
  <Field delimiter="|" name="maudrecno" nullable="true" shift="0" type="long"/>
  <Field delimiter="\r\n" name="memidnum" type="string"/>
  </Record>
  </Metadata>
• 

  avackova

  • May 15, 2008 08:26

  0

  Comment actions Permalink

  It seems there are bugs in the writers. I'm afraid you can't do it with current Clover version.
• 

  avackova

  • May 15, 2008 09:38

  0

  Comment actions Permalink

  Both bugs were fixed. The fix is available in SVN branch 2.4 and will be in nearest release.
• 

  wmdbowler

  • May 15, 2008 16:04

  0

  Comment actions Permalink

  Thank you for the quick response. When is the next 2.4 release? I tested the 2.4.4 version and I experienced the same behavior.
• 

  avackova

  • May 16, 2008 07:22

  0

  Comment actions Permalink

  Version 2.4.5 was released yesterday.

Please sign in to leave a comment.