BufferOverflowException

av686165

• May 06, 2020 00:00

• Answered

Follow

Hi ,

I am getting below error while running my graph,
Component [UniversalDataReader:UNIVERSAL_DATA_READER1] finished with status ERROR. (Out0: 0 recs)
Error when parsing record #1 field deal_foreign_ccy
Realocation of CloverBuffer failed. Requested capacity is 33554434 and maximum capacity is 33554432.
java.nio.BufferOverflowException


Kindly help.

• 

  Vladimir Barton

  • May 07, 2020 06:58

  0

  Comment actions Permalink

  Hello,
  this error message is quite common and is typically caused by one of the following two conditions:
  
  1. An error in the input file
  For instance, a record delimiter or field delimiter is missing in the input file or there is one more delimiter than it should be. This can corrupt the parsing of the next fields and records and can result in loading a huge chunk of data into a single field/record, thus causing the buffer overflow error.
  2. A valid record or field larger than 32 MB
  In this case, it is the actual size of the data being loaded either into a single field or into a single record that exceeds one of the two limits: Record.RECORD_LIMIT_SIZE and Record.FIELD_LIMIT_SIZE.​ Worth noting is that this problem does not have anything to do with the size of the input file of the UniversalDataReader. In other words, it is not the number of records (rows, if you will) that makes the difference here so breaking the files up into smaller ones would not be a remedy to the root cause.
  
  Overall, increasing the limits is, in fact, an adequate solution. Since the limit has been overflown only by a relatively low amount, it should be sufficient to double both limits. In other words, add the following lines into your engine properties and restart CloverDX. More information can be found in our documentation.
  
  

  Record.RECORD_LIMIT_SIZE = ‭67108864‬ 
  Record.FIELD_LIMIT_SIZE = 67108864‬

  
  
  Best regards,

Please sign in to leave a comment.