configurable filter node

mhgrove

• May 01, 2009 00:00

• Answered

Follow

Hi. I want to filter records using a method which is not translateable to the filter expression syntax used by the default Filter node. I ended up getting around this by extending the Filter node and setting a custom RecordFilter. This works fine but I had a few points of feedback for the next version of clover.

* Can RecordFilter be an interface, or at least an abstract class that doesnt depend on a filter expression? Then current implementation become the default instantiation of that interface/abstract class? Right now, I have to pass a valid, but ultimately ignored filter expression to the constructor of my custom RecordFilter instance (which just overrides accepts) and that's kind of crufty. Plus I don't need all the filter expression logic floating around beneath the surface.
* Filter should provide an accessor method for its record filter instance. You can set it, but you can't get it.
* can XML_FILTEREXPRESSION_ATTRIBUTE be protected rather than private so a class extending Filter can use it?

Thanks.

• 

  avackova

  • May 04, 2009 07:52

  0

  Comment actions Permalink

  Hello,
  you can do it with Reformat component instead of Filter or ExtFilter. You can pass to Reformat transformation in CTL or java, so anything, that can be written in java can be passed as your filter expression. All you need is to return proper port number. See following code as a simple example:
  

  function accept(){
  	if (char_at($field1, length($field1) -1) == '1') return true;
  	return false;
  }
  
  function mappingTo0(){
  	$0.* := $0.*;
  }
  
  function mappingTo1(){
  	$1.* := $0.*;
  }
  
  
  // Transforms input record into output record.
  function transform() {
  	if (accept()) {
  		mappingTo0();
  		return 0;
  	}
  	mappingTo1();
  	return 1;
  }
  

• 

  achan

  • May 21, 2009 16:55

  0

  Comment actions Permalink

  Hi Agata,
  
  If I am using a Java class as my transformation, how can I return the proper port number since the Transform method can only return a boolean, right? So I am thinking of using a Partition node instead of a Reformat node since I can specify the port number in getOutputPort method, right?
  
  Thanks,
  al
• 

  avackova

  • May 22, 2009 07:03

  0

  Comment actions Permalink

  Hello Al,
  RecordTransform interface was changed in 2.6 version, so since this version function transform returns int instead of boolean. The meanings of return value are:
  

  • RecordTransform.ALL -- send the data record(s) to all the output ports
  
  
  • RecordTransform.SKIP -- skip the data record(s)
  
  
  • >= 0 -- send the data record(s) to a specified output port
  
  
  • < -1 -- fatal error / user defined
• 

  achan

  • May 22, 2009 16:23

  0

  Comment actions Permalink

  Hi Agata,
  
  Thanks :-)
  
  By the way, do you have plans to update the Javadoc? looks like the Javadoc for RecordTransform (http://cloveretl.berlios.de/docs/JavaDo ... sform.html) is for the previous version... or you actually have an updated doc somewhere else?
  
  al
• 

  avackova

  • May 25, 2009 07:18

  0

  Comment actions Permalink

  Hi Al,
  we are working on it :wink: (http://bug.cloveretl.org/view.php?id=2087). At time being up to date javadocs are distributed with Cloveretl engine sources.

Please sign in to leave a comment.