Customer Portal

Too many records inserted in Mysql database

Comments 2

  • Avatar
    admin
    0
    Comment actions Permalink
    Hi Gary,

    here goes some hints:

    * One problem may be that you have defined some trigger/stored procedure which is activated when bulk loader stores record and causes other record(s) creation.
    * Isn't defined some replication with other databases?
    * What version of Clover do you use? What version of MySql?
    * You can try to replace MysqlDataWriter by DbOutputTable. Does it help? (DbOutputTable uses JDBC connection, MysqlDataWriter stores records into file and pass it to db via bulk loader)
    * You can try to replace "Path to mysql utility" of MysqlDataWriter by some interactive exe file (e.g. calc.exe for windows). When executed, Clover prepares data file and executes your exe. Of course calculator will not save anything into db, but while running, data file will be available for inspection. When closed, Clover erase it. You can find this data file in root of your project. It is dynamically named like "mysqlExchange1490219469849266860.tmp" You can inspect it to see what is going to be stored.
  • Avatar
    gaffonso
    0
    Comment actions Permalink
    Thanks for the reply.

    There are no triggers in the database. There is no replication for this db. Clover version 3.3.0.M1, Mysql 5.5.

    Thanks for the hint about replacing mysql utility, that's a good troubleshooting technique that I didn't know about.

    I did switch over to DBOutput and that gave me more descriptive errors. The problem is with two blob fields (technically mysql TEXT fields). I'm not entirely sure what's going on but I think it's a couple of things:

    1) The contents of the fields contain \r and ; characters which are the default field-delimiters used by CloverETL. Changing the CloverETL delimiters to something unique (like "CLOVERETL_COL_DELIMITER") seems to help.

    2) I think I've got some field-type problems. DBOutput is having trouble inserting into those fields. Still working on that problem.

    ---

    Thank you for your quick reply!

    - Gary

Please sign in to leave a comment.