How Clover support Incremental Load(CDC, SCD, ...)?

suntoe

• September 17, 2009 00:00

• Answered

Follow

Does Clover support this crucial feature? And How?

• 

  ciscochi

  • September 17, 2009 17:11

  0

  Comment actions Permalink

  Hi,
  You can see the incremental support in the DBInputTable and the UniversalDataReader components. The database support is a bit primitive in that it just lets you set a max value and include that max value your query. You may be thinking of deltas directly from a database log files. The file incremental is a bit interesting in that it allows you to read files that are still being written to. I haven't tried out this feature yet, as it isn't something I would do.
  
  I also didn't see any SCD transformers out of the box, but the features are there for you to create your own.
  
  Hope this helps.
• 

  suntoe

  • September 20, 2009 16:47

  0

  Comment actions Permalink

  Yes, I'm thinking of "deltas directly from a database log files", but it's too expensive for us to read log. As I know, informatica supports CDC by analyse database transaction log, they could support oracle, mssql, etc.
  I took a look at clover etl but didn't find that. If none, is clover going to support the transaction-log-based incremental load? I guess it should be in the roadmap
• 

  ciscochi

  • September 21, 2009 21:28

  0

  Comment actions Permalink

  Can someone from Clover comment on whether there is an out-of-the-box SCD support?
• 

  pnajvar

  • September 22, 2009 12:07

  0

  Comment actions Permalink

  Clover has templates for SCD (both type I & II). There is no single component as it can be handled by connecting some of the existing - like Data Intersection.

Please sign in to leave a comment.