What is and how to use incremental reading?
---------------------------------------------------------
If you have a file (url) that you want to process but at once you suppose the file is not complete (additional data will be appended at the end of file) then it is suitable to use the incremental reading.
Most of readers have incrementalFile and incrementalKey attribute.
The incrementalFile defines a file where the clover engine writes a file position.
The incrementalKey is arbitrary name you can define. That key holds the value of the file position.
The clover writes the position into incrementalFile when the phase of graph happens successfully.
---------------------------------------------------------
If you have a file (url) that you want to process but at once you suppose the file is not complete (additional data will be appended at the end of file) then it is suitable to use the incremental reading.
Most of readers have incrementalFile and incrementalKey attribute.
The incrementalFile defines a file where the clover engine writes a file position.
The incrementalKey is arbitrary name you can define. That key holds the value of the file position.
The clover writes the position into incrementalFile when the phase of graph happens successfully.
Please sign in to leave a comment.
Comments 0