Retrieve information vom Graph Tracking (CloverETL-Server)

svogel

• March 17, 2014 00:00

• Answered

Follow

Hi community,

I'd like to make a report about the number of records read from the database, the number of records filtered in a specific filter-component.
Now I see, that this information is already there: on the cloveretl-server there is (Execution > History, select a Job and "Tracking"-Tab).

Can I access the tracking-information somehow (especially the Records for Input/Output of specific components)?
Where is this data stored?

Thanks a lot for any hint.

• 

  admin

  • March 17, 2014 13:37

  0

  Comment actions Permalink

  Hi Stefan,
  
  You have several way how to extract this information:
  
  * tracking information of executed graph is available in output mapping of ExecuteGraph (http://doc.cloveretl.com/documentation/ ... graph.html) available in JobFlows of CloverETL Server Corporate+ from version 3.3
  * use SimpleHTTP API (http://doc.cloveretl.com/documentation/ ... p-api.html) or SOAP WS API (http://doc.cloveretl.com/documentation/ ... ap-ws.html) to download tracking info from Server
  * calculate tracking info within graph using Aggregate (http://doc.cloveretl.com/documentation/ ... egate.html) and store it somewhere (XML, DB, ...)
  
  I hope this helps.
• 

  svogel

  • March 17, 2014 15:00

  0

  Comment actions Permalink

  Hi Jaroslav,
  
  thanks for the quick response. That sounds really great. I especially like getting the data via the HTTP Connector as |-separated values.
  But now I got another issue that got me stuck.
  
  I get the complete HTML-body as "content" from the HTTP Connector, I'd like to have the lines as separate records.
  How I can I split the "one" line HTML-body (at character "\n") to obtain several rows of data (which I can split with character | to obtain the separate fields)
  
  Thanks a lot in advance and kind regards
• 

  svogel

  • March 18, 2014 07:01

  0

  Comment actions Permalink

  Hi Jaroslav,
  
  thanks for the quick reply.
  That sounds awesome. I really like the possibility to query this information via http
  
  But now I got stock with the next thing.
  
  I'm reading the |-separated report from http://.../clover/request_processor/exe ... ESCRIPTION ...
  I get the complete HTML-body in "content", now I want to work through the lines of this report (filtering some information).
  
  But how can I split the content by "\n"? I searched through the components but found no adequate.
  
  And furthermore, if I got the complete body/content splitted in lines, how can I work with this format (as it has a different number of columns, based on the "keyword" in the first row). Is there something easy to use or do I have to program a few lines of code in CTL?
• 

  imriskal

  • March 18, 2014 15:09

  0

  Comment actions Permalink

  Hello Stefan,
  
  I recommend you using parameter returnType=DESCRIPTION_XML. You do not have to split anything whis way, everything is served in a form of XML to you. And if you want, you can extract just the data you are interested in using XMLExtract component after the HTTPConnector.
  
  Regards,
• 

  admin

  • March 18, 2014 15:17

  0

  Comment actions Permalink

  Hi Stefan,
  
  I would suggest Lubos's way, but to make answer complete: For parsing non-heterogenous data you can use http://doc.cloveretl.com/documentation/ ... eader.html In this case you will get each of separate record types on separate output, each having own metadata. Records may be linked through id. Configuration of ComplextDataReader is not really straight-forward so this way may be hard.
  
  I hope this helps.
• 

  svogel

  • March 18, 2014 22:03

  0

  Comment actions Permalink

  Hi guys,
  
  thanks a lot. You are really helpful.
  I didn't know of the ComplexDataReader ... good to know.
  And well ok, I will try to use the XML, but to be honest, I always try to avoid this Xslt-thing:
  I'll have a look.
  
  Thanks for your help.

Please sign in to leave a comment.