How to perform parallel operations into a Graph

fabrizio.orlando

• August 27, 2012 00:00

• Answered

Follow

I have an input file with customers' feedbacks related to some products that that has 2 fields: PROD_ID and PROD_COMMENT. I need to evaluate the sentiment of the feedback for each row of the file and store the results in a table for further analysis on the customer base.
I have a Web Service that accepts the text I should evaluate in input and gives a score in output, but it does not accept other parameters.
Could you please suggest me how to build the Graph in order to reconcile the Web Service output about the PROD_COMMENT with the related PROD_ID to store it on the target table?
Thanks in advance for your help,
Fabrizio

• 

  imriskal

  • August 29, 2012 08:15

  0

  Comment actions Permalink

  Dear fabrizio.orlando,
  
  try to build your graph e.g. this way:
  
  1. Use a Reformat component where you add one new column named e.g. join_id. Generate numbers from 1 to "n" (where "n" is a sum of your input records) in its transformation and store them in join_id column. This will assign a joining key to each record and it will help you later with joining two outputs together.
  2. Use a SimpleCopy component to split the output from the Reformat component.
  3. Send one stream of the output to a WebServiceClient. Important is that this component preserves the order of records. But be careful, you have to make sure that every input record will have its output. Every input record has to have its score, even the records containg errors which may generate another type of webservice response. Handle the errors as you want (e.g. assign them a negative score) but the sum of the input records has to equal to the sum of the output records.
  4. The output from the WebServiceClient component will be reformated again with another Reformat component. Add a new column join_id again, generate numbers from 1 to "n" and store them in join_id again.
  5. Use the ExtMergeJoin component to join the output from the second Reformat component with the second stream of the SimpleCopy output together with join_id as join key.
  
  Please, do not hesitate to reply if anything was unclear.
  
  Best regards,
• 

  fabrizio.orlando

  • August 30, 2012 07:05

  0

  Comment actions Permalink

  Dear imriskal,
  
  It works!
  
  Thanks,
  Fabrizio

Please sign in to leave a comment.