DBoutput node speed

ntmoyer

• August 13, 2009 00:00

• Answered

Follow

i have a problem with the speed of the DBoutput node when using it with update and delete statements and would like to know if i can do anything to make it go faster.

here is the basic structure of the graph
i have a table in a source system that has a few hundred thousand rows in it that could have rows inserted, updated, or deleted on a daily basis. so i pull that table in to the graph and compare it to a master table every night and then i have 3 DBoutput nodes to make the needed changes.
the insert statement usually runs at 7000 records/second and works. the problem is the DBoutput nodes with the update and delete statements. with the batch mode on and default values it sends 1800 rows through right away and then slows down to several rows per minute. when i increase the batch size it will send the new batch size number of rows and then slow down again.

if it helps i am using ms sql server 2000 and the jdbc drivers from microsoft and i can post the graph code if i need to.

• 

  avackova

  • August 17, 2009 11:55

  0

  Comment actions Permalink

  Hello,
  it seems to be database problem. I've found on the Java Database Connectivity forum (http://forums.sun.com/thread.jspa?forum ... dID=569293), that indexing your database could help.
• 

  ntmoyer

  • August 19, 2009 13:40

  0

  Comment actions Permalink

  having the table indexed did not help any. but i dont think that is the same problem since the first set of records goes at a different speed. if i set the batch size to a value less then 10000 then i get that many rows updated right away and after that it is very slow

Please sign in to leave a comment.