Hello, i'm new in cloverETL, i would like to ask if is't possible to check if the record is already exist in the database? thnx...
Hi, here is the scenario. I have a database(db1) which will bhe migrated to a New Database(db2).. these databases have different structures... For now, i can migrate data from db1 to db2 but some of the tables in db2 contains records already. when i try to migrate data from the table which has records, an error occurs(Unique constraint). thats why i want to check first if the record that will be migrated to db2 is already exist... is it possible??? thnx...
Hi, here is the scenario. I have a database(db1) which will bhe migrated to a New Database(db2).. these databases have different structures... For now, i can migrate data from db1 to db2 but some of the tables in db2 contains records already. when i try to migrate data from the table which has records, an error occurs(Unique constraint). thats why i want to check first if the record that will be migrated to db2 is already exist... is it possible??? thnx...
-
Hi !
There are three common options how to approach it:
a) use DB lookup table and check whether the record is in DB - try to look for the key. However this may be slow as with every record you do DB query.
Faster option can be unloading from the target table just keys and use normal lookup table.
b) unload keys from the target table and use either JOIN component or Intersection component to find out what records from your source are already in target - just try to join your new data and data already in db (these unloaded)
c) use the capability of DBOutputTable which can be set to tolerate errors - records which can't be inserted (because of the PK constraint) get rejected by the DB and the component sends them out through port number 1 (second) - if it is connected.
d) .... there are probably some other scenarios, but the first three are most common. Which one to use depends on how much data you need to process and some other aspects.
David. -
Hi !
DB lookup is not a component - it is simply lookup object... You may find it under Lookups in "Outline" view in CloverGUI.
CloverGUI 1.6 has a DBJoin component, which is practically DB lookup simplified.
You wrote that you use Eclipse 3.0. CloverGUI 1.6 (and 1.5 too) need minimum Eclipse 3.1 and also Java (JVM) 1.5 is a minimum.
Please sign in to leave a comment.
Comments 3