I am new to CloverETL. I would like to create 1 large json file from 2 sub json files (part-a.json, part-b.json).
Sample part-a.json file is,
{
"PERSON_ENTPRS_ID" : 1001,
"PopArray" : [ {
"GNDR_CD" : "M",
"MBR_AGE_NO" : 10,
"PERSON_ENTPRS_ID" :1001,
"Year" : "2010"
}, {
"GNDR_CD" : "M",
"MBR_AGE_NO" : 10,
"PERSON_ENTPRS_ID" :1001,
"Year" : "2010"
} ],
.....
}
Sample Part-b.json
{
"DxArray" : [ {
"HLH_EVNT_SK" : "19540000",
"DX_CD" : "380.10",
"DX_TP_CD" : "P",
"PERSON_ENTPRS_ID" : 1001
}, {
"HLH_EVNT_SK" : "19540000",
"DX_CD" : "380.10",
"DX_TP_CD" : "P",
"PERSON_ENTPRS_ID" : 1001
}]
...
}
Steps I followed are as follows:
1. Read the part-a.json file using universal reader, and disable the quoting to read it as string.
2. Read the part-b.json file using universal reader, and disable the quoting to read it as string.
I am not using the JSON READER, because my large file has keys with '_'. I dont want to go through 1000 keys to change back to '-' or escape it :(
Now what I want to do is combine the outputs of two readers and write to one file. So the output file will contain json arrays from both files. Also, I just want to combine these files, no transformation are manipulation is required.
please help?
Thanks so much.
Prashant
Sample part-a.json file is,
{
"PERSON_ENTPRS_ID" : 1001,
"PopArray" : [ {
"GNDR_CD" : "M",
"MBR_AGE_NO" : 10,
"PERSON_ENTPRS_ID" :1001,
"Year" : "2010"
}, {
"GNDR_CD" : "M",
"MBR_AGE_NO" : 10,
"PERSON_ENTPRS_ID" :1001,
"Year" : "2010"
} ],
.....
}
Sample Part-b.json
{
"DxArray" : [ {
"HLH_EVNT_SK" : "19540000",
"DX_CD" : "380.10",
"DX_TP_CD" : "P",
"PERSON_ENTPRS_ID" : 1001
}, {
"HLH_EVNT_SK" : "19540000",
"DX_CD" : "380.10",
"DX_TP_CD" : "P",
"PERSON_ENTPRS_ID" : 1001
}]
...
}
Steps I followed are as follows:
1. Read the part-a.json file using universal reader, and disable the quoting to read it as string.
2. Read the part-b.json file using universal reader, and disable the quoting to read it as string.
I am not using the JSON READER, because my large file has keys with '_'. I dont want to go through 1000 keys to change back to '-' or escape it :(
Now what I want to do is combine the outputs of two readers and write to one file. So the output file will contain json arrays from both files. Also, I just want to combine these files, no transformation are manipulation is required.
please help?
Thanks so much.
Prashant
-
Dear Prashant,
To combine outputs of the readers you can use the Combine component. The component Combine takes single records from each input port and combines them to single output record according to specified transformation.
For more details about the Combine component, please see the documentation:
http://doc.cloveretl.com/documentation/ ... mbine.html
In the transformation function at the Combine component you can set desired join of the two files (add some brackets or comma to ensure the result is valid JSON file).
I prepared for you sample CloverETL project based on your input files, you can take a look into the project and you can inspire from it.
Please sign in to leave a comment.
Comments 1