Logging to external syslog server

morris

• September 21, 2020 00:00

• Answered

Follow

Hi all,

During our local process in CloverDX, we are trying to log custom created messages about the status of the process to an external syslog server. I am trying to use the CustomJavaComponent in combination with the log4j2 java library to accomplish this.

As I am quite new to Java, I have some trouble with how to set this up.

The syslog server is set up by our operations department and works well from other applications.

Thus far, I tried to test my setup with the following code within the execute function of the CustomJavaComponent:

import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.LogManager;
import java.io.*;

public class SyslogLogger extends AbstractGenericTransform
{
    private static final Logger LOG = LogManager.getLogger(SyslogLogger.class);

    public static void execute()
    {
        LOG.info("testing INFO level");
    }
}

I have placed the configuration file (log4j2.xml) in the ${PROJECT}/trans folder.
My xml file looks as follows:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO">
    <!-- Console appender -->
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %c %x - %enc{%m}{CRLF}%n"/>
        </Console>

        <Socket name="SYSLOGD" host="<myhostname>" port="514" protocol="UDP">
            <PatternLayout
                    pattern="&lt;1&gt;%d{MMM dd HH:mm:ss} ${hostName} appName: {
      &quot;host&quot;:&quot;${hostName}&quot;,
      &quot;thread&quot;:&quot;%t&quot;,
      &quot;level&quot;:&quot;%p&quot;,
      &quot;logger&quot;:&quot;%c{1}&quot;,
      &quot;line&quot;:%L,
      &quot;message&quot;:&quot;%enc{%m}&quot;,
      &quot;exception&quot;:&quot;%exception&quot;
      }%n"
            />
        </Socket>
    </Appenders>

    <Loggers>
        <!-- Root logger -->
        <Root level="INFO" additivity="false">
            <AppenderRef ref="Console"/>
            <AppenderRef ref="SYSLOGD"/>
        </Root>
    </Loggers>
</Configuration>

However, it seems like the CustomJavaComponent is not using the xml file I created as configuration, as we are not receiving the log messages on the server or in the console.

My questions are:
1) Where do I place the xml file so that it will be used as configuration for the logging? Is it possible to only log the custom messages we've created to the external syslog server, while keeping the logs CloverDX creates during a process in the Console / Cloverlogs?
2) (Or) is there an other way to log to an external syslog server more easily via CloverDX components, instead of a CustomJavaComponent with log4j?

Many thanks in advance,

Morris

• 

  admin

  • November 08, 2020 11:22

  0

  Comment actions Permalink

  Hi Morris,
  
  In fact, the CloverDX is already using the log4j2 apache logging platform. Therefore, the Designer is by default using the original log4j2.xml file stored on the workspace level. In order to use a customized version, you have to tell the system to use a different configuration file instead. It can be done by adding a system property
  -Dlog4j.configurationFile=path/to/log4j2.xml with a path to your customized version of the file. You can find more details about this topic on the Server related documentation page https://doc.cloverdx.com/latest/server/logging.html.
  
  The most important fact is that you can't actually use two different log4j2.xml files at once, but the new one completely overrides the one shipped with CloverDX. It is, therefore, best to copy out and modify the original configuration shipped with CloverDX.
  
  The documentation covers the setups in case you are working with CloverDX Server, so please let me add some details that should help you achieve the same goal in CloverDX Designer (Local Project).
  
  Location of the Log4j2 configuration file is:
  workspace_home_folder/.metadata/.plugins/com.cloveretl.gui.runtime/runtime-deploy/webapp/WEB-INF/log4j2.xml
  (in Windows, workspaces are by default in C:/Users/UserName/)
  
  System property has to be added to the CloverDXDesigner.ini file located in:
  CloverDX Designer/CloverDXDesigner.ini
  (in Windows, C:/Program Files/CloverDX Designer/)
  
  To create the customized log message as you have it in your example, I have done the following:
  
  1. I have copied the log4j2.xml file from C:\Users\Evka\workspace\workspace581\.metadata\.plugins\com.cloveretl.gui.runtime\runtime-deploy\webapp\WEB-INF\log4j2.xml and stored it in ${PROJECT}/trans folder
  
  2. I have updated the newly created file and added my own Appender into it (honestly, I have just copied one of the existing appenders in the file and just changed the name to "myAppender" updated the pattern and location of the log file)
  
  3. Then I added the new logger:
  

  <Logger name="SyslogLogger" level="info" additivity="false">
              <AppenderRef ref="myAppender" />
  </Logger>

  
  Please note that the name of the logger is the java class created in your CustomJavaComponent.
  
  4. I have then added the system property at the end of the CloverDXDesigner.ini file, the property in my example looks as follows:
  

  -Dlog4j.configurationFile=file:///c:\Users\Evka\workspace\LocalProject\trans\log4j2.xml

  
  
  5. Then I restarted the Runtime. (In the left bottom corner of the Designer there is the status of runtime - usually "Ready" - click on the arrow next to it and choose Restart Runtime. Or you can also quit the Designer and open it again.)
  Run your graph.
  
  Please give this a try and let us know if it works for you. Thanks, Eva

Please sign in to leave a comment.