Hi,
can someone help me ?
I'm working on Clover and the issue is that :
I have two input metadatas in the port 0 and 1.
for every lines in the port 0, I want to compare the values with all the lines in the second file
in programing language, it could be something like that :
for (I=0,i<10,i++){
for (j=0,j<10,j++){
}
}
for doing that, it's okay for the first metadata, for the second one, the issue is that :
First Loop
while ((inRecordFab2500 = readRecordFromPort(0)) != null) {
DataRecord outRecord=mappingWithJoy2000Table(outRecord, inRecordFab2500);
while ((inputDatasJoy2000 = readRecordFromPort(1)) != null) {
}
}
The problem is, after the first time, the value of readRecordFromPort(1)) remain to null and I don't know how to loop in the second while for every lines of the first while... as outputs, I have the values for only the first lines of the first while.
Can someone help me ?? thank you.
can someone help me ?
I'm working on Clover and the issue is that :
I have two input metadatas in the port 0 and 1.
for every lines in the port 0, I want to compare the values with all the lines in the second file
in programing language, it could be something like that :
for (I=0,i<10,i++){
for (j=0,j<10,j++){
}
}
for doing that, it's okay for the first metadata, for the second one, the issue is that :
First Loop
while ((inRecordFab2500 = readRecordFromPort(0)) != null) {
DataRecord outRecord=mappingWithJoy2000Table(outRecord, inRecordFab2500);
while ((inputDatasJoy2000 = readRecordFromPort(1)) != null) {
}
}
The problem is, after the first time, the value of readRecordFromPort(1)) remain to null and I don't know how to loop in the second while for every lines of the first while... as outputs, I have the values for only the first lines of the first while.
Can someone help me ?? thank you.
-
Hi,
From your message I was able to deduce that you would like to compare two records to determine if they are new or have been updated. If this is the case I would recommend that you use the following approach:- Have both records sorted via a unique key, then pass them into the input ports of the DataIntersection component. In the component you will have to specify the key that you will do the join on, the result will produce 3 outputs (e.g. 0 new records, 1 data that will be use for the updated records, 2 old records)
- Using the data in output port 1 sort it via all fields in the record using ExtSort or FastSort depending on the amount of data you have, and pass it into the first input port of a new DataIntersection. In the second input port you will pass a copy of the new data you are trying to compare with by using a SimpleCopy (make sure to sort via all fields as well). This will result in validating which records have been updated from the prior run
I have taken the liberty of adding a sample graph with this process. By using this method, you can wrap it into a subgraph, thus having a standard way of comparing records.
If for any reason this was not what you are trying to accomplish, please provide me with more information. -
Hello,
So i have the exact same issue as above, however my end goal is different.
In the 'CustomJavaTrasformer' i need to be able to integrate through all the records of the second input, for each of the records from the first.
Is there a way to reset the 'readRecordFromPort' function so that it can start at the beginning of the list of records?
Look forward to your reply!
Cheers
Marc -
Hi Garcy,
I have attached a example graph/subgraph compares two files and joins them based on category ID. The CustomJoiner.sgrf demonstrates how to read all records from one input port, store them in a temporary file and then iterate thought them repeatedly. For this example CustomJavaTransformer_iterationes.grf shows how the joiner is sued to replace product category ID with category name.
CustomJavaTransformer.zip -
Hi Garcy,
I have attached a example graph/subgraph compares two files and joins them based on category ID. The CustomJoiner.sgrf demonstrates how to read all records from one input port, store them in a temporary file and then iterate thought them repeatedly. For this example CustomJavaTransformer_iterationes.grf shows how the joiner is sued to replace product category ID with category name.
CustomJavaTransformer.zip
Hi,
That is perfect! Exactly what i was after and a very detailed example with multiple options, thank you!
I do have another (hopefully quick) query.. logging from a sub-graph, is this possible? I cannot seem to get anything logged from the sub-graph, however when the custom transformer is place onto the main graph, all logging occurs.
Is this simply not a feature yet, or is there something i need to do?
Cheers
Marc -
Hi Garcy,
I have attached and updated version of the project where we have created a method called logParent(). It will allow you to temporarily switch the logging context to the parent graph and back again.
CustomJavaTransformer_logging.zip
Please sign in to leave a comment.
Comments 6