Hi everyone. I stuck with a simple ETL graph. I have a directory with a bunch of zip files. I need to organize such process:
1. extract all files from current archive into d:\temp directory
2. delete archive
3. run another graph to get data from extracted files
4. clean up d:\temp
5. goto step 1
I use ListFiles component to get list of zip files, then in reformat I combine list of cmd commands for 7z.exe. But I can't find a way, how to get a loop, which allow me process every row in metadata step by step. Thanks.
1. extract all files from current archive into d:\temp directory
2. delete archive
3. run another graph to get data from extracted files
4. clean up d:\temp
5. goto step 1
I use ListFiles component to get list of zip files, then in reformat I combine list of cmd commands for 7z.exe. But I can't find a way, how to get a loop, which allow me process every row in metadata step by step. Thanks.
-
Hi,
I suggest to use either "File Event Listener" or scheduler on the CloverETL Server. For instance, you can create a graph in which cloverETL will decompress files in one phase and perform the rest in the second. Make a "File Event Listener" to check the required directory for files and execute the graph when new files appear.
I hope this will help. -
Hi
I would use Jobflow http://doc.cloveretl.com/documentation/ ... bflow.html if you have it available.
The parent jobflow would use ListFiles to list all zip files, then you would use ExecuteGraph to process specific zip file (that file url you will pass through Input mapping in ExecuteGraph component, you need to create parameter in child graph.)
The child graph would process one zip file at the time, so it will unzip it with zip command, do transformation and let parent graph know it finishes successfully. Parent jobflow then could delete zip file as processed.
I would use File Event Listener to wait for new zip file to trigger this process all over again.
(Or I might misunderstood your question completely ;))
PK
Please sign in to leave a comment.
Comments 2