Customer Portal

EBCDIC file format

Comments 13

  • Avatar
    twaller
    0
    Comment actions Permalink
    Hi,
    you can try the IBM 1047 encoding. Maybe it is what you are searching for.
    Best regards
  • Avatar
    dpavlis
    0
    Comment actions Permalink
    The IBM 1047 codepage is (according to iana.org) - EBCDIC Latin 1/Open Systems
  • Avatar
    schabluk
    0
    Comment actions Permalink
    Thanks for an answer.
    I'm afraid that IBM 1047 is not supported by Clover.
    It's not on the Reader task Charset list, and gives error when trying to set it manualy.

    INFO  [main] - Graph definition file: graph/Serat.grf
    Exception in thread "main" ERROR [main] - Can't create object of type DATA_READER with reason: DATA_READER:DATA_READER0:IBM1047
    java.lang.RuntimeException: Can't create object of type DATA_READER with reason: DATA_READER:DATA_READER0:IBM1047
    at org.jetel.component.ComponentFactory.createComponent(ComponentFactory.java:132)
    at org.jetel.graph.TransformationGraphXMLReaderWriter.instantiateNodes(TransformationGraphXMLReaderWriter.java:465)
    at org.jetel.graph.TransformationGraphXMLReaderWriter.instantiatePhases(TransformationGraphXMLReaderWriter.java:428)
    at org.jetel.graph.TransformationGraphXMLReaderWriter.read(TransformationGraphXMLReaderWriter.java:340)
    at org.jetel.graph.TransformationGraphXMLReaderWriter.read(TransformationGraphXMLReaderWriter.java:270)
    at org.jetel.graph.TransformationGraphXMLReaderWriter.loadGraph(TransformationGraphXMLReaderWriter.java:203)
    at org.jetel.main.runGraph.main(runGraph.java:308)
  • Avatar
    dpavlis
    0
    Comment actions Permalink
    Well, it depends on platform, not Clover. Check your Java installation. On Mac (OSX) Clover offers more than 120 encodings, IBM1047 amongst them.

    On Windows Jre 1.5 does not have it, Jdk 1.5 does.You may try just to borrow charset.jar from Jdk 1.5
  • Avatar
    schabluk
    0
    Comment actions Permalink
    Ok. J have changed JRE from jre1.6 to jdk1.5. Now, i'm getting this error:


    ERROR [WatchDog] - Node DATA_READER0 finished with status: ERROR caused by: Parse error: The size of data buffer for data record is only 49152. Set appropriate parameter in defautProperties file. when parsing record #1 field zm19
    DEBUG [WatchDog] - Node DATA_READER0 error details:
    java.lang.RuntimeException: Parse error: The size of data buffer for data record is only 49152. Set appropriate parameter in defautProperties file. when parsing record #1 field zm19
    at org.jetel.data.parser.DataParser.parseNext(DataParser.java:454)
    at org.jetel.data.parser.DataParser.getNext(DataParser.java:147)
    at org.jetel.util.MultiFileReader.getNext(MultiFileReader.java:354)
    at org.jetel.component.DataReader.execute(DataReader.java:191)
    at org.jetel.graph.Node.run(Node.java:379)
    at java.lang.Thread.run(Thread.java:619)
    ERROR [WatchDog] - !!! Phase finished with error - stopping graph run !!!


    The metadata is fixed length, and summary it contains no more than 1274 characters per line.
  • Avatar
    dpavlis
    0
    Comment actions Permalink
    For some reason, the parser can't find record delimiter (probably). This message means that Clover tries to create a data record which is too large - usually means, that record/field boundaries are not found or properly defined.
  • Avatar
    schabluk
    0
    Comment actions Permalink
    Still nothing...

    DBA said, that this is a binary fie, generated from DB2 database, with codepage 870, and 0D0A(hex) as EOL character.

    Is there another way to read this file from Clover?
  • Avatar
    avackova
    0
    Comment actions Permalink
    Is the file stored in IXF format? If you mean this as binary DB2 file, I'm afraid, Clover is not able to parse it. On DB2 documentation I've found that "IXF (Integration Exchange Format, PC version) is a binary format that is used exclusively by DB2." (http://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic=/com.ibm.db2.luw.admin.cmd.doc/doc/r0008304.html). So I think that only way to get data from it, is to import them to DB2 database.
  • Avatar
    schabluk
    0
    Comment actions Permalink
    There is existing mechanism writen in SAS script language that reads this file to SAS table, so I belive it is readable outside DB2.
    My job is to migrate from SAS to Java...
  • Avatar
    avackova
    0
    Comment actions Permalink
    So maybe try to read the data directly from SAS by DBInputTable (http://support.sas.com/documentation/cdl/en/jdbcref/59666/HTML/default/n12a0cg9du45g2n1ib275rxpcusz.htm)
  • Avatar
    schabluk
    0
    Comment actions Permalink
    I can't. The SAS licence is expiring, that is why I need to migrate reader for this binary file.
  • Avatar
    avackova
    0
    Comment actions Permalink
    Only way is to write your own Reader. Clover parsers can be helpful then. Information about ixf format you can find in IBM pages ( http://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic=/com.ibm.db2.luw.admin.dm.doc/doc/r0004667.html)
  • Avatar
    schabluk
    0
    Comment actions Permalink
    One way to do that:


    import java.io.*;

    public class readEbcdic {

    /**
    * @param args
    */
    public static void main(String[] args) {
    // TODO Auto-generated method stub
    try {
    readFile("C:/input.txt");
    } catch (IOException e) {
    System.out.println(e.getMessage());
    }
    }

    static void readFile(String fileName) throws IOException
    {
    BufferedInputStream bis = null;

    try {
    bis = new BufferedInputStream(new FileInputStream(fileName));

    FileWriter fstream = new FileWriter("C:/output.txt");
    BufferedWriter out = new BufferedWriter(fstream);

    byte[] recText;
    ByteArrayOutputStream baos = new ByteArrayOutputStream();

    while(bis.read() > -1) {

    recText = new byte[1273]; // 1273 is a fixed line length
    bis.read(recText);

    baos.write(recText);
    out.write(baos.toString("Cp037") + "\n");
    baos.reset();
    }
    out.close();
    }
    finally {
    try {
    bis.close();
    }
    catch (Exception e) {}
    }
    }
    }

Please sign in to leave a comment.