Customer Portal

Clover incremental data load

Comments 3

  • Avatar
    novakj
    0
    Comment actions Permalink
    Hi,

    I prepared for you simple example of incremental reading. In this example there is used incremental reading from input file (based on date field). When you run the graph for the first time, all records will be stored into output file called incremental.dat. The second run of the graph will read only new lines from input file (please add some new lines into the input file for test).
  • Avatar
    snehangka
    0
    Comment actions Permalink
    Thanks Novakj for your prompt response.

    However this works only if a new record is added to the input file. What if the attribute values of the existing records gets modified? In this case using incremental file/incremental key is not identifying the modified records. Could you please provide any example for the same?


    Thanks,
    Snehangka
  • Avatar
    novakj
    0
    Comment actions Permalink
    Unfortunately, it is not possible to use incremental reading from file when existing records were modified. The incremental reading mechanism does not recognize these changes.

    You have to read a whole file and search for changed records. You can, for example, compare new content of your input file with a previous copy of this file (you can use, for example, DataIntersection component).

Please sign in to leave a comment.