Data Reader with Comma at EOL

gtateco

• July 01, 2011 00:00

• Answered

Follow

I'm trying to read a .csv with the Universal Data Reader that has a comma a /r and /n for each end of line(EOL). I'm not able to get the reader to go past the first record, it always fails on the last field. Here is the error message.

Parse error: The size of data buffer for data record is only 12288. Set appropriate parameter in defautProperties file. when parsing record #1 field blank

I've tried setting the last field to ,\r\n for the delimiter and tried creating a bogus blank field with just \r\n. Both return the same error which seems kind of strange to me.

Here is a sample of the data.

0,13.65,13.65,"","","","","","Nordrhein-Westfalen",

Any help would be great, thank you!

Glenn

• 

  teebag

  • July 05, 2011 07:47

  0

  Comment actions Permalink

  You need to add a 10 field to your metadata, type integer, delimiter \r\n
  
  Name Type Delimiter
  Field1 Integer ,
  ...
  Field8 String ,
  Field9 String ,
  Field10 integer \r\n
• 

  gtateco

  • July 05, 2011 16:56

  0

  Comment actions Permalink

  Thanks for the idea, that worked great. Of course I had two issues which was compounding the issue.
  
  The other issue is I'm having trouble reading the file directly from the customer. It is coming from a Linux box, probably output from some sort of scripting language. The format is UTF-8 and although it reads fine in the preview the Java engine has issues with it, ex. you can't see the field preview when editing the metadata. If I open on my windows computer in VIM and copy and paste it to a new file it works great. That workaround is OK for now, but this is a daily process that needs to be automated.
  
  Once again, any ideas appreciated!
  
  Glenn
• 

  jurban

  • July 07, 2011 08:27

  0

  Comment actions Permalink

  Hi Glenn,
  
  could you please describe your problem in more details please? Do I understand it correctly that on the same computer the preview works OK (when creating metadata) but it doesn't work when editing metadata? Do you see some error message?
  
  On first thought I would expect some problems with line delimiters - can you try just \n or just \n?
  
  Best regards,
  Jaro
• 

  avackova

  • July 12, 2011 10:42

  0

  Comment actions Permalink

  Hello Glenn,
  the problem is probably with line breaks. To make your graph working with both, linux and windows ends of line, you can use alternative delimiter \n\\|\r\n (for full information see Changing and Defining Delimiters)

Please sign in to leave a comment.