The memory assigned to the application server JVM is being managed on the application server level. It is strongly recommended to adjust the default limits for memory allocation by adjusting the "Xms" and "Xmx" JVM parameters. Generally speaking, the recommended total heap memory value equals half of the RAM memory of the machine, where the Server resides. If the CloverDX Server is the only application running on the physical machine, even more than half of RAM can be assigned to it. Yet, it should never be set close to the same amount as the total max. amount of the system memory.
In Java 7 and earlier, there is memory space for loading classes (so-called "PermGen space") which is separated from the heap memory, and can be set by the JVM parameter "-XX:MaxPermSize". By default, it is just 64 MB which is not enough for enterprise applications. Again, suitable memory limit depends on various criteria, but 512 MB should be enough in most cases. If the PermGen space maximum is too low, OutOfMemoryError: PermGen space may occur.
In Java 8, memory space for loading classes (so-called "Metaspace") is separated from the heap and can be set by the JVM parameter -XX:MaxMetaspaceSize. The default maximum Metaspace size is unlimited.
Please see the specific container section for details on memory settings.
The clover engine also uses direct memory for data records manipulation. For example, an underlying memory of CloverBuffer (container for serialized data records) is allocated outside of the Java heap space in direct memory. Before CloverETL 4.9.0-M2, this attribute was set to true by default to improve performance. However, direct memory is out of control of a Java virtual machine, so try to turn off usage of direct memory in case an OutOfMemory exception occurs. This is done via engine properties. In Tomcat, you basically need to create a new properties file (e.g. engine.properties) and to put "USE_DIRECT_MEMORY=false" into it. Then pass the full path to the file, to the CloverDX Server by adding the "-Dclover.engine.config.file=/full/path/to/file.properties" to the CATALINA_OPTS in the setenv file. You can verify that the settings were applied by navigating into the CloverDX web console -> Configuration -> CloverDX Info -> Engine Properties. You should see the USE_DIRECT_MEMORY parameter in the list there. This will prevent the CloverDX Server from using memory allocated out of the heap space.