Duplicate records

jcuendet

• March 09, 2011 00:00

• Answered

Follow

Hi,

Currently I'd like to duplicate records without using copy.
In my example, I have 13 records that I'd like to multiply by 10'000. In order to get 130'000 records.
I cannot do it in the input file as it's an Excel file and doesn't support more than 65K lines.

What do you suggest?
Thanks

• 

  avackova

  • March 10, 2011 07:42

  0

  Comment actions Permalink

  Hello,
  I see two simple solutions:
  

  
  
  • copy these records many times to the flat file:
    [list:1pwkoc8m][*:1pwkoc8m]XLSReader --> DataWriter with append="true"
  
  
  • DataGenerator --> RunGraph

  [/*:m:1pwkoc8m]
  

  Use normalizer, that just duplicates the input records:
  

  //#CTL2
  function integer count() {
  	return ${rec_no};
  }
  
  function integer transform(integer idx) {
  	$0.* = $0.*;
      return OK;
  }
  

  [/list:o:1pwkoc8m]
• 

  jcuendet

  • March 10, 2011 11:00

  0

  Comment actions Permalink

  Thanks I used the second solution and it works fine.
  
  FYI I created around 1'000'000 records from 13. with the following functions:
  

  function int count() {
  	return 75000;
  }

  
  and

  function int transform(int idx) {
  	$0.* = $0.*;
  	if(idx != 0){
  		$0.positionType = $0.positionType + '-gen-' + num2str(idx);
  	}
      return 0;
  }

  
  My process is generating CSV files and run in 150 seconds on a powerful machine.

Please sign in to leave a comment.