Hi, so I am trying out CloverETL and I am having an error occur on one of my list files nodes. Which is then giving me an error in a move node. Strangely enough the actual routine is working.
So I am trying out a basic run that picks up a zip file from an ftp. Unzips the file and then moves this to a working server. I am testing with 2 files. The image shows that coming out of the second ListFiles a count of 4 when there should only be 2. Then the Error in the MoveFiles, yet it has moved the two files. I am presuming because ListFiles has doubled up on the result, the move has completed the move on the first 2 files but cannot do the next process as the file has already been removed. Why would the ListFiles double up on the results? How could I fix this?
Many Thanks
So I am trying out a basic run that picks up a zip file from an ftp. Unzips the file and then moves this to a working server. I am testing with 2 files. The image shows that coming out of the second ListFiles a count of 4 when there should only be 2. Then the Error in the MoveFiles, yet it has moved the two files. I am presuming because ListFiles has doubled up on the result, the move has completed the move on the first 2 files but cannot do the next process as the file has already been removed. Why would the ListFiles double up on the results? How could I fix this?
Many Thanks
-
Hi GavC,
a simple change that would resolve the issue that you encountered is to remove the edge between the ExecuteScript component (called "Unzip") and the second ListFiles component and push the second ListFiles component to a later phase (for example, phase #1). The ListFiles component does not need any input token in order to list the files and send the records forward to the MoveFiles component. What seems to be happening in your graph is that 2 input tokens make the ListFile component perform the file listing twice, thus doubling up the results.
I will be glad to provide you a deeper insight into the optimization of your jobflow and show you various other approaches that can be chosen but at this point, I believe it would be most beneficial if you provided me with the jobflow (the .jbf file) itself so that I can inspect the mapping in your components. Feel free to disguise or delete any sensitive information in regards to the SFTP server, such as URLs, usernames, passwords etc.
Kind regards,
Please sign in to leave a comment.
Comments 1