SystemExecute on consecutive records

p_swiss

• July 01, 2013 00:00

• Answered

Follow

Hello,

I'm new to CloverETL, struggling to do the following: I have an input stream of multiple records, and I need to run a SystemExecute FOR EACH RECORD and generate an output based on that, where the System Command is a function of a string field of the corresponding input record. I tried to do this by connecting the reader to the input port of SystemExecute; however, this does not read consecutive input records... Any suggestions how I can do it?

Example of commands I want to run:

python myprogram.py arg1
python myprogram.py arg2
python myprogram.py arg3
...

where arg1 arg2 arg3... are the values of a field of my input stream.

Thanks a lot!

• 

  imriskal

  • July 02, 2013 11:38

  0

  Comment actions Permalink

  Hi, p_swiss,
  
  I think very similar problem is already solved here: viewtopic.php?f=4&t=5241
  
  Please, take a look on it and let me know with any questions.
  
  Best regards,
• 

  p_swiss

  • July 02, 2013 14:46

  0

  Comment actions Permalink

  Hi Lobus,
  
  Thanks a lot for your reply. I had actually seen this post. In the solution I see it's mentioned all input records are serialized and sent to SystemExecute as STDIN. But what I want is a different System Command to be executed for each record (the script depending on the field). Kind of a loop over the records... If I'm wrong, could you please explain in a bit more details how I can treat my example (arg1 arg2 arg3 ... each for one different record). Sorry if I'm being a bit dummy here :)
  
  Thanks!
• 

  admin

  • July 03, 2013 11:15

  0

  Comment actions Permalink

  Hi p_swiss,
  
  You can use approach described in mentioned topic:
  
  1. in loop parse lines out of whole stdin (line will be equal to one serialized record)
  2. if line/records contains more fields, parse them out (using fields delimiter used in metadata)
  
  Inside of the loop use if-statements (using parsed fields in conditions) selecting proper commands to execute. Syntax of parsing&loop&if highly depends on your OS and used shell script interpreter.
  
  Sample of stdin:
  

  
  script1.py|42
  script2.py|66
  

  
  
  Parsed line:
  

  
  script1.py|42
  

  
  
  Parsed fields:
  

  
  "script1.py" and "42"
  

  
  
  I hope this helps.
• 

  p_swiss

  • July 03, 2013 13:50

  0

  Comment actions Permalink

  Thank you so much for your reply. I finally got how to do it with a loop in a shell script, works like charm :)
  
  Thanks a lot!

Please sign in to leave a comment.