Customer Portal

loader reading from memory??

Comments 13

  • Avatar
    avackova
    0
    Comment actions Permalink
    You can use Simple lookup table for storing your data and then use LookupTableWriterWriter component.
  • Avatar
    priyadarshini
    0
    Comment actions Permalink
    I got LookupTableReaderWriter but not LookupTableWriterWriter. Can u give more details.
  • Avatar
    avackova
    0
    Comment actions Permalink
    I'm sorry - I've thought LookupTableReaderWriter: If you are able to store the data in Map<HashKey, DataRecord>, then you can read them by LookupTableReaderWriter. If not you have to save data in file and then use OracleDataWriter component.
  • Avatar
    priyadarshini
    0
    Comment actions Permalink
    I am storing data in Map in Memory only, now giving name of LookupTable in property of LookupTableReaderWriter should not be sufficient ??
  • Avatar
    avackova
    0
    Comment actions Permalink
    LookuTableReaderWriter can read data only from lookup table that implements org.jetel.data.lookup.LookupTable interface.
  • Avatar
    priyadarshini
    0
    Comment actions Permalink
    Also, I m finding that DataRecord object (that is value for MAP implementing LookupTable interface) should also belong to DataRecord class of clover. So, I need to extend my DataRecord class to same of clover. Again, I need to handle metadata part of it. It seems going in a very complex direction....is there any simple way.
  • Avatar
    avackova
    0
    Comment actions Permalink
    The simplest way is to save data in flat file and them use OracleDataWriter. Or you can write a Reader, that will send data to OracleDataWriter.
  • Avatar
    priyadarshini
    0
    Comment actions Permalink
    Thats the problem...I do not want to use flat file as its extra overhead...

    Again, If I use LookupReaderWriter, then the OracleDataWriter need to added to LookupReaderWriter...to finally load the thing to DB.

    Is it a way, OracleDataWriter can directly read from memory and load to DB(using sqlldr)....pipe can help or not ?
  • Avatar
    avackova
    0
    Comment actions Permalink
    All clover writers can read data from input port (edge). If the data are saved on disk or not depends on edge type and amount of data. See useFileForExchange attribute of OracleDataWriter component
  • Avatar
    priyadarshini
    0
    Comment actions Permalink
    On Unix, UseFileToExchange is set to false by default. OracleDataWriter reads from edge ...write it to pipe and then load to sqlldr.

    I was thinking if there is any way, I can write to pipe myself taking memory data...that will be consumed by sqlldr ....
  • Avatar
    priyadarshini
    0
    Comment actions Permalink
    what I understood, loader can read from named pipe or can read from stdin as well. still exploring, how exactly can achieve it.
  • Avatar
    avackova
    0
    Comment actions Permalink
    You can easily write your Reader, which will send data to Oracle Data Writer. Something like:
    import java.util.Iterator;
    import java.util.Map;
    import java.util.Map.Entry;

    import org.jetel.data.DataRecord;
    import org.jetel.graph.Node;
    import org.jetel.graph.Result;
    import org.jetel.util.SynchronizeUtils;


    /**
    * @author avackova
    *
    */
    public class MapReader extends Node {


    public final static String COMPONENT_TYPE = "MAP_READER";

    private Map map;

    public MapReader(String id, Map data) {
    super(id);
    this.map = data;
    }

    /* (non-Javadoc)
    * @see org.jetel.graph.Node#execute()
    */
    @Override
    public Result execute() throws Exception {
    DataRecord record = new DataRecord(getOutputPort(OUTPUT_PORT).getMetadata());
    record.init();
    Iterator<Entry> data = map.entrySet().iterator();
    while (runIt && data.hasNext()) {
    getNextRecord(record, data.next());
    writeRecord(OUTPUT_PORT, record);
    SynchronizeUtils.cloverYield();
    }
    broadcastEOF();
    return runIt ? Result.FINISHED_OK : Result.ABORTED;
    }

    private void getNextRecord(DataRecord record, Entry data) {
    record.getField(0).setValue(data.getKey());
    record.getField(1).setValue(data.getValue());
    }

    /* (non-Javadoc)
    * @see org.jetel.graph.Node#getType()
    */
    @Override
    public String getType() {
    return COMPONENT_TYPE;
    }

    }

    For full instruction how to create the component see: Step by Step Component building
  • Avatar
    priyadarshini
    0
    Comment actions Permalink
    Thanks for reply...let me try this.

Please sign in to leave a comment.