CSV file with non-standard delimiters and quote characters

andy

• March 12, 2010 00:00

• Answered

Follow

What's the best way to process a CSV file with non-standard delimiters and quote characters?

In my case the delimiter is \u0014 and the quote character is \u00fe . I'm in the metadata editor now in the designer, and I'm trying to specify \u0014 as the delimiter, and it's not splitting properly. There also doesn't seem to be a way to specify a quote character.

Thanks!

• 

  avackova

  • March 16, 2010 09:04

  0

  Comment actions Permalink

  Hello Andy,
  you can set such delimiter with ctl: set field delimiter as `'\u0014'` - all, that is between `` characters is interpreted as ctl code.
  Unfortunately only single and double quote characters are recognized as quotation by Clover.
  But you can use following workaround to parse the data:
  

  • read full record as one string field
  
  
  • translate \u00fe to " in Reformat:

    function transform() {
    	$0.field1 := translate($0.field1,'\u00fe','"');
    }

  
  
  • send translated field to Data Reader - use port:$0.field1:discrete in fileURL attribute. Remember to set eofAsDelimiter="true" on output metadata

Please sign in to leave a comment.