Hi,
Please help me with the URL for the partitioned sandbox.
Flow :
Input DB table --> Cluster Partition --> Universal Data Writer
Here my o/p file will refer to 2-node partitioned sandbox. What will be the url for the data file so that it will write partition data into two seperate directories.
Please help me with the URL for the partitioned sandbox.
Flow :
Input DB table --> Cluster Partition --> Universal Data Writer
Here my o/p file will refer to 2-node partitioned sandbox. What will be the url for the data file so that it will write partition data into two seperate directories.
-
Hi sameerj,
URLs are given by your settings in partitioned sandbox, see my screenshot:
partitioned.png
I also recommend you to read this article in our documentation: http://doc.cloveretl.com/documentation/ ... ssing.html The whole issue is described quite nicely there.
You can use also sandboxes.home.partitioned property if you want to unify them, see: http://doc.cloveretl.com/documentation/ ... rties.html
Regards, -
Let me rephrase my question again.
I have a requirement to unload the data from the DB table and create a multifile. If I use DB -- ClusterPartition --- UniversalDataWriter (2 nodes allocation), then what would be the URL for the output file.
I have created partitioned SB and have defined it in the properties but my requirement is to write two seperate files for both the nodes in seperate directory under partitioned sb.
Let's say there are 1000 records then 500 records should be written to node1 directory and 500 records to node2 directory.
I have created shared file system so that both the cluster nodes can access it and within that shared file location, I have created two seperate directories for node 1 and node2.
Thank You -
Oh, I think I understand now. You can use this syntax:
sandbox://abcd/my_partitioned_file.csv
Using the example screenshot above, a half of the records would be saved in X:/sandboxes/shared/01/abcd/my_partitioned_file.csv on node01 and the rest would be in /opt/CloverETL/sandboxes/shared/02/abcd/my_partitioned_file.csv on node02.
Is this the result you expect?
By the way, I think the shared file system is not necessary in this situation. Both nodes can work on their own piece of data, they do not have to share the data. If you want to use a shared workspace, you can use shared sandbox instead of partitioned one.
Please sign in to leave a comment.
Comments 4