Hi All,
I am receiving 25 records from a reader component and then after doing transformation(using reformat) sending the records to unviersalwriter as csv.
I need to know
1) how to count how many records came from reader component in reformat component.
2) how to add a a counter on out records (like "/opt/record"+i+".html";)
in current for loop I am getting only last value which is 24.
I want to replace 24 with total number of records received from previous node.
Thanks
Sandeep
I am receiving 25 records from a reader component and then after doing transformation(using reformat) sending the records to unviersalwriter as csv.
I need to know
1) how to count how many records came from reader component in reformat component.
2) how to add a a counter on out records (like "/opt/record"+i+".html";)
function integer transform() {
integer i;
$out.0.field = $in.0.field;
$out.0.field1 = $in.0.field1;
string strBody = "";
strBody = removeNonAscii(removeNonPrintable($in.0.field2));
$out.0.field2=strBody;
//$out.0.field3 = $in.0.field3;
for (i=0;i<=24;i++)
{
[b]$out.0.field3= "/DATA2/raw_html_rss/news"+i+".html";[/b]
}
return ALL;
}in current for loop I am getting only last value which is 24.
I want to replace 24 with total number of records received from previous node.
Thanks
Sandeep
-
Hi Sandeep,
I’m not sure if you want to get a serial number of a row or total count of all records that went through an edge. Therefore I have prepared for you an example that answers both your questions.
Basically to get a serial number of some record, you might use a simple CTL transformation within a Reformat component, where you will use a variable that you increase by one with every single incoming record.
However, if you want get total number of records, you might take into consideration using Aggregate component. See attached graph for more details.
Hope this helps. -
Thanks So much...
Now I am clear on this,
My doubt was regarding why the variable wasn't incrementing , so now I understood that the variable should also go in some outport to get incremented...
Is there any way to increment the variable, without sending variable to out port in reformat component.
Thanks So Much again
Regards,
Sandeep Thakur -
Hi Sandeep,
If I understand your question, you want to work with total record count value (integer) just like with any other variable in CTL. However, since Reformat processes records one by one, it starts processing data even before receiving all records. This approach is faster as it does not require waiting for all records to be processed by preceding component. Therefore when Reformat starts to process records it doesn't know yet how many records it will receive.
Therefore you should start thinking about another approach. Maybe if you told us what exactly you have on input of the Reformat and what is supposed to be returned on output, we might help you find a different way. -
I really appreciate the way you cleared my queries.
I am bringing in RSS feeds from a web site using xml extractor and then using Universal reader (source) to fetch the source of RSS component.Problem is that I want to extract the title, description and Url of the RSS feed using xml extractor and in parallel want to fetch the source of RSS feed URL
(Which is HTML source text ) and attaching them as one record finally. (Which I was not able to achieve).
So What I thought is, I firstly fetched source of RSS URL's and named it as rss$.html in phase 0. And in phase 1, I am fetching link{},Title{}, description{}
and then I am trying to add an file system url , which will point to each relative RSS news source in file system, so i thought of putting an variable RSS+i+.html. So automatically I will get an RSS feed pointer.
Quite complex thinking..:) I know there might be other ways, but I was not able to figure out, so I invested some time in thinking my own resolution..
(may be funny)
Please suggest some good way to achieve above..
Regards
Sandeep Thakur -
Hi Sandeep,
Please correct me if I do not understand you question. You want to fill the following metadata:Link ; Title ; Description ; Content
Where the first three fields are retrieved directly from RSS feed XML tags, and the fourth is the content of the file that is available under the link from the first field. If that is true, you do not need to do any CTL transformation. The only thing you need to do is to split the XMLExtract output stream into two records. For this purpuse you may use SimpleCopy component. On the second branch you will put UniverasalDataReader, which will return the whole content of the file (retrieved from the link tag) into one field. Since the number and order of the records on both branches are identical, we can use Combine component to merge these data together.
For more information, please see attached graph.
Hope this helps.
Please sign in to leave a comment.
Comments 6