Fast Sort BufferOverflowException

CloverDX Customer Portal
Forum
Fast Sort BufferOverflowException

pro7

November 02, 2011 00:00

Answered

Hi,
We have a graph that has been running along just fine. Today it started throwing BufferOverflowException. We bumped both
Data.DATA_RECORDS_BUFFER_SIZE and Data.MAX_BUFFERS_ALLOCATED but still getting the errors. The number of records has not changed significantly from when the graph ran fine. What are other possibilities? Could it be one bad data record that is causing this problem? Thanks

11-02 13:20:06,485[ WatchDog] WatchDog ERROR 124553 Node SORT_GOREMAL finished with status: ERROR caused by: null
11-02 13:20:06,485[ WatchDog] WatchDog ERROR 124553 Node SORT_GOREMAL error details:
java.nio.BufferOverflowException
at java.nio.Buffer.nextPutIndex(Buffer.java:495)
at java.nio.DirectByteBuffer.put(DirectByteBuffer.java:250)
at org.jetel.util.bytes.ByteBufferUtils.encodeLength(ByteBufferUtils.java:146)
at org.jetel.graph.DirectEdge.eof(DirectEdge.java:343)
at org.jetel.graph.Edge.eof(Edge.java:439)
at org.jetel.graph.Node.closeAllOutputPorts(Node.java:854)
at org.jetel.graph.Node.broadcastEOF(Node.java:865)
at com.opensys.cloveretl.component.FastSort.b(Unknown Source)
at com.opensys.cloveretl.component.FastSort.execute(Unknown Source)
at org.jetel.graph.Node.run(Node.java:425)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

Comments 12

pro7

November 03, 2011 14:56
0

Comment actions Permalink
I changed to an ExtSort but I still get the error.
pro7

November 03, 2011 17:32
0

Comment actions Permalink
Update.... It turns out that the changes we made to defaultProperties were not picked up by the clover engine. I believe the file being used by clover engine is in a different path. I'll try to edit there.
julrych

November 03, 2011 19:25
0

Comment actions Permalink
CloverETL uses default engine properties file unless specified otherwise. In order to do so, in CloverETL designer go to menu Run | Run Configurations and on Arguments tab specify Program argument
```
-config engine_configuration_file
```
The engine_configuration_file is a full path or path relative to project.
pro7

November 04, 2011 14:16
0

Comment actions Permalink
Jan,
Thanks. We would really like to make a change to the defaultProperties file used on the server but since that is proving difficult I will ask about using the –config switch on the Arguments tab of Run Configurations. I have done this successfully in Clover ETL Designer and tested on a graph I’m using to recreate our problem. However, the graph we are having problems with is running on the server. It is scheduled to run on a regular basis on the server. How can we set the –config switch such that it will persist on the server and be used for all scheduled runs of our graph??

Thanks
mzatopek

November 07, 2011 16:02
0

Comment actions Permalink
The clover engine configuration file can be changed even on clover server, see 'engine.config.file' property at http://server-demo-ec2.cloveretl.com/clover/docs/list-of-properties.html.
pro7

November 07, 2011 16:27
0

Comment actions Permalink
When I logon to the server there is a “Monitoring” link. Clicking on this link brings me to a page with a number of tabs. The “Clover Engine properties” tab shows all the values that are being used by the engine. The “Clover Server properties” tab has no data associated with engine.config.file. See attached. Also, for the –config switch question, is it only possible to use from within designer?

On the server where we are running clover I see three copies of defaultProperties in the following paths:
E:\Archives\CloverETL\cloverETL.src.rel-3-0-0\cloverETL\cloveretl.engine\src\org\jetel\data
E:\Archives\CloverETL\cloverETL.src.rel-2-9-4\cloverETL\cloveretl.engine\src\org\jetel\data
E:\Apps

We are running 3.0.0 so I doubt the 2.9.4 file is used at all. I can change these files but any change is not being picked up by the server....I have tried.
Thanks
- clover_server_prop.JPG
pro7

November 14, 2011 16:48
0

Comment actions Permalink
Hello,
Any thoughts on the above post? Where might we find the config file being used by our server?? Also, regarding the second part of the question. Is it possible to use the -config switch for a graph that is running scheduled on the server?

Thanks
tkramolis

November 15, 2011 16:22
0

Comment actions Permalink
Hi,

You need to set path to your custom engine configuration file into the engine.config.file server property. How to do that? I suggest 2 solutions. First is simpler, second is cleaner, I guess.
- Alter the engine.config.file property of the default config.properties server configuration file that comes with the Clover Server clover.war distribution. It's located inside WEB-INF directory inside your clover.war file (which is actually ordinary zip file). Then redeploy the modified clover.war file on your application server. Check out the “Clover Server properties” tab then if the engine.config.file property has desired value.
- In case you don't want to change the clover.war file, you need to somehow specify custom Server configuration file instead of the default config.properties from your clover.war Server distribution, see http://server-demo-ec2.cloveretl.com/cl ... urces.html on possible ways how this is done.
Best regards,
Tom

Javlin a.s.
tkramolis

November 15, 2011 16:31
0

Comment actions Permalink
Regarding the -config switch, it is not possible to use it for graphs running on the Server.

--Tom
pro7

December 19, 2011 16:54
0

Comment actions Permalink
Tom,
Thanks. I have tried a number of things without much success. My apologies but this is going to be a long post.
The link you provided mentions :

[tomcat_home]/conf/Catalina/localhost/clover.xml
We are running on tomcat but I did not find the above file. I did find a file named ROOT.xml in path
E:\Apps\apache-tomcat-6.0.29\conf\Catalina\localhost\
I added :
<Parameter name="Record.MAX_RECORD_SIZE" value="24576" override="false" />
I restarted tomcat server and looked at the Clover Engine Properties tab on the server. It was still showing the old vale for Record.MAX_RECORD_SIZE (12288).
Next I searched for a clover.xml file and found one in path :
E:\Apps\apache-tomcat-6.0.29\webapps\ROOT\WEB-INF\classes\com\cloveretl\server\nodemanager\
So I tried making changes to this file and restarted server but again the Clover Engine Properties tab is showing the original value.
Next I created a custom config file with one line:
Record.MAX_RECORD_SIZE=24576
And in E:\Apps\apache-tomcat-6.0.29\webapps\ROOT\WEB-INF\config.properties
I changed
engine.config.file=
to
engine.config.file=E:\Apps\apache-tomcat-6.0.29\webapps\ROOT\WEB-INF\custconfig.properties
This change was reflected on the Clover Engine Properties tab but it was now the only property displayed on that tab. I suspect that all the default engine properties were still intact but I could not be sure. Maybe you could confirm?? Is it possible to have default, plus custom properties displayed on the tab?
Next I tried adding all the properties that had previously been displayed on the Clover Engine Properties tab to my custom config file. Like this:

ASSIGN_SIGN= :=
CLOVER_FIELD_INDICATOR= $
CLOVER_FIELD_REGEX= \$[\w]+
Component.KEY_FIELDS_DELIMITER= ;
Component.KEY_FIELDS_DELIMITER_REGEX= \s*([|;]|:(?!=))\s*
DEFAULT_BINARY_PATH= ./
DEFAULT_DATETIME_FORMAT= yyyy-MM-dd HH:mm:ss
DEFAULT_DATE_FORMAT= yyyy-MM-dd
DEFAULT_INTERNAL_IO_BUFFER_SIZE= 49152
DEFAULT_IOSTREAM_CHANNEL_BUFFER_SIZE= 2048
DEFAULT_JAVA_COMPILER= internal
DEFAULT_PATH_SEPARATOR_REGEX= \s*;\s*
DEFAULT_PLUGINS_DIRECTORY= ./plugins
DEFAULT_REGEXP_FALSE_STRING= false|F|FALSE|NO|N|f|0|no|n
DEFAULT_REGEXP_TRUE_STRING= true|T|TRUE|YES|Y|t|1|yes|y
DEFAULT_TIME_FORMAT= HH:mm:ss
Data.DATA_RECORDS_BUFFER_SIZE= 10485760
Data.MAX_BUFFERS_ALLOCATED= 99
DataFieldMetadata.DECIMAL_LENGTH= 8
DataFieldMetadata.DECIMAL_SCALE= 2
DataFormatter.DEFAULT_CHARSET_ENCODER= ISO-8859-1
DataFormatter.DELIMITER_DELIMITERS_REGEX= \\\|
DataFormatter.FIELD_BUFFER_LENGTH= 4096
DataParser.DEFAULT_CHARSET_DECODER= ISO-8859-1
DataParser.FIELD_BUFFER_LENGTH= 4096
Graph.DIRECT_EDGE_FAST_PROPAGATE_NUM_INTERNAL_BUFFERS= 4
GraphProperties.EXPRESSION_EVALUATION_ENABLED= true
GraphProperties.EXPRESSION_PLACEHOLDER_REGEX= (?<!\\)`(.*?)(?<!\\)`
GraphProperties.PROPERTY_PLACEHOLDER_REGEX= \$\{(\w+)\}
INCREMENTAL_STORE_KEY= incremental_store
InternalSortDataRecord.DEFAULT_INTERNAL_SORT_BUFFER_CAPACITY= 8000
Lookup.LOOKUP_INITIAL_CAPACITY= 512
OracleConnection.ROW_PREFETCH= 100
PACKAGES_EXCLUDED_FROM_GREEDY_CLASS_LOADING= java.;javax.;sun.misc.
PortReadingWriting.DATA_LENGTH= 2048
Record.DEFAULT_COMPRESSION_LEVEL= 5
Record.MAX_RECORD_SIZE= 12288
WatchDog.DEFAULT_WATCHDOG_TRACKING_INTERVAL= 5000
WatchDog.NUMBER_OF_TICKS_BETWEEN_STATUS_CHECKS= 1
WatchDog.WATCHDOG_SLEEP_INTERVAL= 1000

My graph started to get errors like :
Unclosed group near index 19 (?<!\)`(.*?)(?<!\)` ^
I suspect that some of the properties would need to be in double quotes? But I was not sure so I restored config.properties to look like : engine.config.file=
I also tried changing web.xml as mentioned in the linked page you provided but the Clover Engine properties tab again showed the original value.
pro7

December 27, 2011 01:30
0

Comment actions Permalink
Here is my question. Can I trust that my custom engine.config.file will only affect the properties defined in the custom file? All others will be unchanged defaults, correct? If I create a custom file is there some way to see all the settings since they are not being displayed on the Clover Engine Properties tab? thanks
jurban

January 06, 2012 15:46
0

Comment actions Permalink
Hi,

yes, we can confirm that the custom engine.config.file will only affect the properties defined in itself, and the other properties will keep their defaults. The fact that you're seeing only your custom properties is a bug, I've reported it in our bugtracker (see https://bug.javlin.eu/browse/CLS-607).

Regards,
Jaro