is there a way to separate data in to different outputs?
i want something that works similar to a router that sorts data based on a row value in a column. I see that there's a parallelPartition component, in the expression, i defined 3 different outputs 0, 1, 2. but i'm not sure how to set the edge to different writers.
i want something that works similar to a router that sorts data based on a row value in a column. I see that there's a parallelPartition component, in the expression, i defined 3 different outputs 0, 1, 2. but i'm not sure how to set the edge to different writers.
-
Hi Mohamad,
yes, there are a couple of different options on how to separate data into different outputs. Choosing the right one depends on some of the specific use case details, namely, whether there is a known number of the possible column values and then, also the number itself. In other words, if there is a known number of values, you can often use a single component. For example, if there are 2 possible values (true/false), you can use a Filter component with simple filter expression ($in.0.field1 == true). If there are 3 or more, you can use Partition or Reformat, for instance. However, if the number of values (based on which you decide the number of output ports) is not known before the graph runs, it would require a more complex approach possibly using more components. Could you provide sample input and demonstrate the desired result?
Note: ParallelPartition is not a good candidate in this case as it is designed to distribute incoming data records in the CloverDX Cluster among different cluster nodes.
Kind regards,
Please sign in to leave a comment.
Comments 1