Identifying error in input file

saravanan77

• September 01, 2007 00:00

• Answered

Follow

Hi,

How to identify the errors in input file ? For example, i have to read a fixed length format file. Consider there are 3 fields, A,B & C and each of length 1. If the content of the file is,

123
12
1

I want to process only 123 and for other 2 rows i have to do dump to database. How can i able to do this ?

Regards,
Saravanan A

• 

  avackova

  • September 06, 2007 12:54

  0

  Comment actions Permalink

  Hello,
  default dataPolicy parameter on each reader is set to "strict" value. It means that after first BadDataFormatException graph execution is aborted. You can set foregoing parameter to "controlled" (parseable records are sent to output port, unparseable records are logged) or "lenient" (attempts to set incorrect data to default values while processing continues).
• 

  acq

  • January 08, 2008 11:47

  0

  Comment actions Permalink

  Hello,
  
  is it possible, by means of actual clover standard components, to send to a log file any record unparseable by delimited datareader?
  Actually, setting a controlled data policy would print errors encountered, not unparseable records, is it right?
  
  Thanks for any suggestion.
• 

  crfpkr

  • January 08, 2008 12:34

  0

  Comment actions Permalink

  I did this by reading the input in all string format and then passing it to Reformat component where using java class, i parse the input and attach a flag to each record as true or false. Then i pass on this to a partition component where i seperate the record in to valid or invalid based on the flag i added in reformat. Works great.,,, like this you can parse and validate your data as many ways as you wish cuz you do it in java.. hope this helps
• 

  acq

  • January 08, 2008 13:38

  0

  Comment actions Permalink

  Thank you for your answer. I did exactly the same, but though it does work, the resulting code is not as scalable as metadata approach for a huge number of different validation patterns.
  Regards,
• 

  avackova

  • January 09, 2008 08:26

  0

  Comment actions Permalink

  If you use Universal Data Reader you can save to file more info about unparseable records.

Please sign in to leave a comment.