Delimited Reader's encoding issue

mishra.hanu@yahoo.co.in

• October 16, 2008 00:00

• Answered

Follow

Hi,
I ahve one file, encoding of that file is UTF8.
and it has data like this := 2910,"Université du Québec à Chicoutimi"

when i load this data in DB it is giving exception:-

java.lang.RuntimeException: java.io.IOException:MALFORMED[1] when converting from UTF-8 when parsing record #1 field institution_id
at org.jetel.data.parser.DelimitedDataParser.parseNext(DelimitedDataParser.java:449)
at org.jetel.data.parser.DelimitedDataParser.getNext0(DelimitedDataParser.java:173)
at org.jetel.data.parser.DelimitedDataParser.skip(DelimitedDataParser.java:595)
at org.jetel.util.MultiFileReader.nextSource(MultiFileReader.java:185)
at org.jetel.util.MultiFileReader.init(MultiFileReader.java:120)
at org.jetel.component.DelimitedDataReader.init(DelimitedDataReader.java:180)
at org.jetel.component.DelimitedDataReader.checkConfig(DelimitedDataReader.java:304)
at org.jetel.graph.Phase.checkConfig(Phase.java:204)
at org.jetel.graph.TransformationGraph.checkConfig(TransformationGraph.java:874)
at org.jetel.graph.runtime.GraphExecutor.initGraph(GraphExecutor.java:122)
at com.mps.pubstats.tool.FeedLoadGraph.runGraph(FeedLoadGraph.java:269)
at com.mps.pubstats.tool.FeedController.LoadAccountFeed(FeedController.java:514)

How to handle this problem....
and kindly tell me that how can i cahnge encding of the DelimitedReader dynamically.....


I used this code...
EncodingController encoderController=new EncodingController();
DelimitedDataReader deLimitedReader= new DelimitedDataReader("DataParser",
feedController.inputFileName,encoderController.getEncoding(feedController.inputFileName));
deLimitedReader.setSkipFirstLine(true);
deLimitedReader.setPolicyType("Controlled");
encoderController.closeFileObject();

//////////////
public String getEncoding(String fileName){

String encodeName="";
try
{
obj = new InputStreamReader(new FileInputStream(new File(fileName)));
encodeName = obj.getEncoding();
System.out.println("File Name---->"+fileName);
System.out.println("Encoding---->"+encodeName);
}
catch(Exception ex)
{
ex.printStackTrace();
}
return encodeName;
}

////

but it doesn't work..

Thanks,
Hanuman

• 

  mishra.hanu@yahoo.co.in

  • October 16, 2008 09:50

  0

  Comment actions Permalink

  Even if i set UTF-8 then it is giving same error..now got that getEncoding is not currect..it is giving the system encoding...
  
  But UTF-8 is the super in all the encding (charset) so i want to use to read and write but still it is giving the same error..
  
  java.lang.RuntimeException: java.io.IOException:MALFORMED[1] when converting from UTF-8 when parsing record #1 field institution_id
  at org.jetel.data.parser.DelimitedDataParser.parseNext(DelimitedDataParser.java:449)
  at org.jetel.data.parser.DelimitedDataParser.getNext0(DelimitedDataParser.java:173)
  
  
  Please help me to short out thsi problem
• 

  mishra.hanu@yahoo.co.in

  • October 17, 2008 09:45

  0

  Comment actions Permalink

  Hi ,
  
  please help me out how to resolve this problemm....
• 

  avackova

  • January 12, 2009 14:32

  0

  Comment actions Permalink

  Hello,
  which CloverETL version do you use? With 2.6 ver. DataReader as well as DelimitedDataReder seem to work properly.

Please sign in to leave a comment.