Hello. I'm new to clover ETL and I have the following problem, basically what I want to do is to lookup for a register in a db, compare it to another one in a txt file, and if they arent equals, filter it and sent it to a trash. But I have a fatal error in the writer and I dont know why, besides, there is not much clover info on net from what I can see.
ERROR [WatchDog] - Phase initialization failed with reason: WRITER ...FATAL ERROR !
WRITER ...FATAL ERROR !
at org.jetel.graph.Phase.init(Phase.java:163)
at org.jetel.graph.runtime.WatchDog.executePhase(WatchDog.java:565)
at org.jetel.graph.runtime.WatchDog.call(WatchDog.java:155)
at org.jetel.graph.runtime.WatchDog.call(WatchDog.java:70)
at java.util.concurrent.FutureTask$Sync.innerRun(libgcj.so.81)
at java.util.concurrent.FutureTask.run(libgcj.so.81)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(libgcj.so.81)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(libgcj.so.81)
at java.lang.Thread.run(libgcj.so.81)
Caused by: java.lang.RuntimeException: Failed initialization of filler buffers :java.lang.IllegalStateException
at org.jetel.data.formatter.FixLenDataFormatter.initRecordFiller(FixLenDataFormatter.java:134)
at org.jetel.data.formatter.FixLenDataFormatter.init(FixLenDataFormatter.java:171)
at org.jetel.util.MultiFileWriter.prepareTargets(MultiFileWriter.java:165)
at org.jetel.util.MultiFileWriter.init(MultiFileWriter.java:126)
at org.jetel.component.FixLenDataWriter.init(FixLenDataWriter.java:218)
at org.jetel.graph.Phase.init(Phase.java:158)
The grf is:
[code]<?xml version="1.0" encoding="UTF-8"?>
<Graph created="Fri Jan 12 08:41:50 CET 2007" description="This graph illustrates how to join data using reformat component." guiVersion="1.9" id="1204558846009" licenseType="Evaluation license." modified="Tue Mar 11 13:03:10 CET 2008" name="Testing Hash Joiner" revision="1.8">
<Global>
<Metadata id="Metadata1">
<Record fieldDelimiter=";" name="Cliente" recordDelimiter="\n" recordSize="-1" type="delimited">
<Field name="Codigo_abonado" nullable="true" shift="0" type="string"/>
<Field name="Numero_abonado_A" nullable="false" shift="0" type="string"/>
</Record>
</Metadata>
<Metadata fileURL="${META_DIR}/fixlen/clientes.fmt" id="Metadata0"/>
<Metadata fileURL="${META_DIR}/fixlen/clientes.fmt" id="Metadata2"/>
<Connection dbConfig="${CONN_DIR}/mysql.cfg" id="Connection0" type="JDBC"/>
<Property fileURL="/home/erik/Escritorio/CloverU/cloverETL/workspace.prm" id="GraphParameter0"/>
<LookupTable dbConnection="Connection0" id="LookupTable0" metadata="Metadata1" name="name" type="dbLookup">
<attr name="sqlQuery"><![CDATA[select * from clientes where Numero_abonado_A=?]]></attr>
</LookupTable>
</Global>
<Phase number="0">
<Node dataPolicy="Strict" enabled="enabled" fileURL="${DATAIN_DIR}/fixlen/formato1.csv" guiHeight="25" guiName="INPUT1" guiWidth="50" guiX="20" guiY="20" id="INPUT1" type="FIXLEN_DATA_READER"/>
<Node enabled="enabled" guiHeight="25" guiName="JOIN" guiWidth="50" guiX="228" guiY="20" id="JOIN" type="REFORMAT">
<attr name="transform"><![CDATA[
import org.jetel.component.DataRecordTransform;
import org.jetel.data.DataRecord;
import org.jetel.data.RecordKey;
import org.jetel.data.lookup.LookupTable;
import org.jetel.exception.JetelException;
import org.jetel.graph.TransformationGraph;
public class reformatTest extends DataRecordTransform{
LookupTable mylookup;
public boolean init(){
try {
mylookup = graph.getLookupTable("LookupTable0");
mylookup.init();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// create key
String[] lookupKeyStr={"Numero_abonado_A"};
RecordKey key=new RecordKey(lookupKeyStr,sourceMetadata[0]);
key.init();
mylookup.setLookupKey(key);
return true;
}
public boolean transform(DataRecord[] source, DataRecord[] target)
{
DataRecord employee;
employee=mylookup.get(source[0]);
if (employee==null) {// skip this one
errorMessage = "Order: " + source[0].getField(0).getValue() + " - no corresponding record in lookup table";
System.out.println("Dentro del source: "+source[0]);
return false;
}
//target[0].getField(0).setValue(source[0].getField(0).getValue());
//target[0].getField(1).setValue(source[0].getField(1).getValue());
//target[0].getField(2).setValue(source[0].getField(2).getValue().toString());
//target[0].getField(3).setValue(employee.getField(0).getValue().toString());
//target[0].getField(4).setValue(employee.getField(1).getValue());
return true;
}
}
]]></attr>
</Node>
<Node append="false" fileURL="${DATAOUT_DIR}/joined_data_hash.out" id="WRITER" type="FIXLEN_DATA_WRITER"/>
<Edge debugMode="false" fromNode="INPUT1:0" guiBendpoints="" id="INEDGE1" inPort="Port 0 (in)" metadata="Metadata2" outPort="Port 0 (out)" toNode="JOIN:0"/>
<Edge fromNode="JOIN:0" guiBendpoints="" id="OUTEDGE" inPort="Port 0 (in)" metadata="Metadata0" outPort="Port 0 (out)" toNode="WRITER:0"/>
</Phase>
</Graph>
And the metada is:
<?xml version="1.0" encoding="UTF-8"?>
<Record name="Clientes" recordSize="92" type="fixed">
<Field name="Estado_llamada" nullable="true" shift="0" size="1" type="string"/>
<Field name="Causa_de_salida" nullable="true" shift="0" size="1" type="string"/>
<Field name="Numero_abonado_A" nullable="true" shift="0" size="10" type="string"/>
<Field name="Numero_abonado_B" nullable="true" shift="0" size="18" type="string"/>
<Field name="Categoria_abonado_A" nullable="true" shift="0" size="2" type="string"/>
<Field name="Categoria_abonado_B" nullable="true" shift="0" size="2" type="string"/>
<Field name="Parte_que_paga" nullable="true" shift="0" size="1" type="string"/>
<Field name="Fecha_inicio_tarificacion" nullable="true" shift="0" size="6" type="string"/>
<Field name="Hora_inicio_tarificacion" nullable="true" shift="0" size="6" type="string"/>
<Field name="Hora_fin_tarificacion" nullable="true" shift="0" size="6" type="string"/>
<Field name="Duracion_llamada" nullable="true" shift="0" size="6" type="string"/>
<Field name="Codigo_falla" nullable="true" shift="0" size="5" type="string"/>
<Field name="Ruta_saliente" nullable="true" shift="0" size="7" type="string"/>
<Field name="Ruta_entrante" nullable="true" shift="0" size="7" type="string"/>
<Field name="Clase_tarifa" nullable="true" shift="0" size="3" type="string"/>
<Field name="Abonado_x" nullable="true" shift="0" size="10" type="string"/>
</Record>
This is based on graphDBLookup.grf of the examples.
Thanks so much. I really need this.[code][/code][/quote][code][/code]
ERROR [WatchDog] - Phase initialization failed with reason: WRITER ...FATAL ERROR !
WRITER ...FATAL ERROR !
at org.jetel.graph.Phase.init(Phase.java:163)
at org.jetel.graph.runtime.WatchDog.executePhase(WatchDog.java:565)
at org.jetel.graph.runtime.WatchDog.call(WatchDog.java:155)
at org.jetel.graph.runtime.WatchDog.call(WatchDog.java:70)
at java.util.concurrent.FutureTask$Sync.innerRun(libgcj.so.81)
at java.util.concurrent.FutureTask.run(libgcj.so.81)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(libgcj.so.81)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(libgcj.so.81)
at java.lang.Thread.run(libgcj.so.81)
Caused by: java.lang.RuntimeException: Failed initialization of filler buffers :java.lang.IllegalStateException
at org.jetel.data.formatter.FixLenDataFormatter.initRecordFiller(FixLenDataFormatter.java:134)
at org.jetel.data.formatter.FixLenDataFormatter.init(FixLenDataFormatter.java:171)
at org.jetel.util.MultiFileWriter.prepareTargets(MultiFileWriter.java:165)
at org.jetel.util.MultiFileWriter.init(MultiFileWriter.java:126)
at org.jetel.component.FixLenDataWriter.init(FixLenDataWriter.java:218)
at org.jetel.graph.Phase.init(Phase.java:158)
The grf is:
[code]<?xml version="1.0" encoding="UTF-8"?>
<Graph created="Fri Jan 12 08:41:50 CET 2007" description="This graph illustrates how to join data using reformat component." guiVersion="1.9" id="1204558846009" licenseType="Evaluation license." modified="Tue Mar 11 13:03:10 CET 2008" name="Testing Hash Joiner" revision="1.8">
<Global>
<Metadata id="Metadata1">
<Record fieldDelimiter=";" name="Cliente" recordDelimiter="\n" recordSize="-1" type="delimited">
<Field name="Codigo_abonado" nullable="true" shift="0" type="string"/>
<Field name="Numero_abonado_A" nullable="false" shift="0" type="string"/>
</Record>
</Metadata>
<Metadata fileURL="${META_DIR}/fixlen/clientes.fmt" id="Metadata0"/>
<Metadata fileURL="${META_DIR}/fixlen/clientes.fmt" id="Metadata2"/>
<Connection dbConfig="${CONN_DIR}/mysql.cfg" id="Connection0" type="JDBC"/>
<Property fileURL="/home/erik/Escritorio/CloverU/cloverETL/workspace.prm" id="GraphParameter0"/>
<LookupTable dbConnection="Connection0" id="LookupTable0" metadata="Metadata1" name="name" type="dbLookup">
<attr name="sqlQuery"><![CDATA[select * from clientes where Numero_abonado_A=?]]></attr>
</LookupTable>
</Global>
<Phase number="0">
<Node dataPolicy="Strict" enabled="enabled" fileURL="${DATAIN_DIR}/fixlen/formato1.csv" guiHeight="25" guiName="INPUT1" guiWidth="50" guiX="20" guiY="20" id="INPUT1" type="FIXLEN_DATA_READER"/>
<Node enabled="enabled" guiHeight="25" guiName="JOIN" guiWidth="50" guiX="228" guiY="20" id="JOIN" type="REFORMAT">
<attr name="transform"><![CDATA[
import org.jetel.component.DataRecordTransform;
import org.jetel.data.DataRecord;
import org.jetel.data.RecordKey;
import org.jetel.data.lookup.LookupTable;
import org.jetel.exception.JetelException;
import org.jetel.graph.TransformationGraph;
public class reformatTest extends DataRecordTransform{
LookupTable mylookup;
public boolean init(){
try {
mylookup = graph.getLookupTable("LookupTable0");
mylookup.init();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// create key
String[] lookupKeyStr={"Numero_abonado_A"};
RecordKey key=new RecordKey(lookupKeyStr,sourceMetadata[0]);
key.init();
mylookup.setLookupKey(key);
return true;
}
public boolean transform(DataRecord[] source, DataRecord[] target)
{
DataRecord employee;
employee=mylookup.get(source[0]);
if (employee==null) {// skip this one
errorMessage = "Order: " + source[0].getField(0).getValue() + " - no corresponding record in lookup table";
System.out.println("Dentro del source: "+source[0]);
return false;
}
//target[0].getField(0).setValue(source[0].getField(0).getValue());
//target[0].getField(1).setValue(source[0].getField(1).getValue());
//target[0].getField(2).setValue(source[0].getField(2).getValue().toString());
//target[0].getField(3).setValue(employee.getField(0).getValue().toString());
//target[0].getField(4).setValue(employee.getField(1).getValue());
return true;
}
}
]]></attr>
</Node>
<Node append="false" fileURL="${DATAOUT_DIR}/joined_data_hash.out" id="WRITER" type="FIXLEN_DATA_WRITER"/>
<Edge debugMode="false" fromNode="INPUT1:0" guiBendpoints="" id="INEDGE1" inPort="Port 0 (in)" metadata="Metadata2" outPort="Port 0 (out)" toNode="JOIN:0"/>
<Edge fromNode="JOIN:0" guiBendpoints="" id="OUTEDGE" inPort="Port 0 (in)" metadata="Metadata0" outPort="Port 0 (out)" toNode="WRITER:0"/>
</Phase>
</Graph>
And the metada is:
<?xml version="1.0" encoding="UTF-8"?>
<Record name="Clientes" recordSize="92" type="fixed">
<Field name="Estado_llamada" nullable="true" shift="0" size="1" type="string"/>
<Field name="Causa_de_salida" nullable="true" shift="0" size="1" type="string"/>
<Field name="Numero_abonado_A" nullable="true" shift="0" size="10" type="string"/>
<Field name="Numero_abonado_B" nullable="true" shift="0" size="18" type="string"/>
<Field name="Categoria_abonado_A" nullable="true" shift="0" size="2" type="string"/>
<Field name="Categoria_abonado_B" nullable="true" shift="0" size="2" type="string"/>
<Field name="Parte_que_paga" nullable="true" shift="0" size="1" type="string"/>
<Field name="Fecha_inicio_tarificacion" nullable="true" shift="0" size="6" type="string"/>
<Field name="Hora_inicio_tarificacion" nullable="true" shift="0" size="6" type="string"/>
<Field name="Hora_fin_tarificacion" nullable="true" shift="0" size="6" type="string"/>
<Field name="Duracion_llamada" nullable="true" shift="0" size="6" type="string"/>
<Field name="Codigo_falla" nullable="true" shift="0" size="5" type="string"/>
<Field name="Ruta_saliente" nullable="true" shift="0" size="7" type="string"/>
<Field name="Ruta_entrante" nullable="true" shift="0" size="7" type="string"/>
<Field name="Clase_tarifa" nullable="true" shift="0" size="3" type="string"/>
<Field name="Abonado_x" nullable="true" shift="0" size="10" type="string"/>
</Record>
This is based on graphDBLookup.grf of the examples.
Thanks so much. I really need this.[code][/code][/quote][code][/code]
-
Hi,
for joining data from flat file with database attends DBJoin Component.
See following examples:
DBJoin example
DB unload parametrized example
Lookup join example
For writing data to file use rather Universal Data Writer than Fixlen DW. In GUI you can use edge debugger for data preview.
Please sign in to leave a comment.
Comments 1