Separate log files per Proxy or API in WSO2 EI

Posted by Wouter van Wijngaarden on 22 Nov, 2018

log filesIn WSO2 products you can always find all the enabled logging aggregated into the wso2carbon.log, but what if you want to split some of this logging up on the API/Proxy level? Well you’re in luck, WSO2 EI (or WSO2 ESB) supports logging per API or Proxy into their respective files and I’m here to tell you how to do that!
WSO2 products use Apache Log4j to enable logging of the components inside the product, this allows for a lot of optional customization by the user. For more info about Apache Log4j and the version WSO2 uses you can check out the apache log4j documentation.

Inside the WSO2 products the configuration file for Log4j is located in the %PRODUCT_HOME%/conf directory with the filename log4j.properties. This file contains the basic log4j configuration used by the product and from here you can modify it as you wish.

Creating a separate log file for designated proxy services

To allow the creation of an extra log file for a specific proxy you just have to add a few lines to the log4j.properties file. These lines include various parameters which I will explain below.

log4j.category.SERVICE_LOGGER.testProxy=DEBUG, YENLO_APPENDER

This line specifies which proxy the configuration is for, in this case it’s our “testProxy” on DEBUG level and it specifies which appender will be used for this proxy service. The appender is a log4j interface used to write the logging to a file, which we configure below as well.

log4j.additivity.SERVICE_LOGGER.testProxy=false

The additivity line specifies whether the logger should inherit configurations from the parent configurations also listed in the same file. We set this to “false” because otherwise we will end up with extra log files that we don’t need or want.

log4j.appender.YENLO_APPENDER=org.apache.log4j.DailyRollingFileAppender

The above designates the use of the DailyRollingFileAppender, this appender can be configured to rotate your new log files at a certain interval and rename the old file with an added date. The interval can be configured in the datePattern as a pattern. For example, for a daily log rotation the pattern should be yyyy-MM-dd and as below.
log4j.appender.YENLO_APPENDER.datePattern='.'yyyy-MM-dd 

Now that the rotation has been set up we can assign a name to our new log file. This can be done with the log4j.appender.YENLO_APPENDER.file behind which you can define the log filename. In our case: yenlo.testProxy.log.

log4j.appender.YENLO_APPENDER.File=logs/yenlo.testProxy.log

Last but not least, in our default configuration we can define a pattern layout for how the log is written inside the file. First we call the log4j.PatternLayout class after which we configure it with a specific conversion pattern.

log4j.appender.YENLO_APPENDER.layout=org.apache.log4j.PatternLayout

log4j.appender.YENLO_APPENDER.layout.ConversionPattern=%d{ISO8601} \[%X{ip}-%X{host}\] \[%t\] %5p %c{1} %m%n

The letters inside the conversion pattern string represent specific items to display in the log file.
You can find more information about these on the apache documentation under the org.apache.log4j.PatternLayout class.

The end result added to the configuration file is as follows:

log4j.category.SERVICE_LOGGER.testProxy=DEBUG, YENLO_APPENDER
log4j.additivity.SERVICE_LOGGER.testProxy=false
log4j.appender.YENLO_APPENDER=org.apache.log4j.DailyRollingFileAppender
log4j.appender.YENLO_APPENDER.datePattern='.'yyyy-MM-dd
log4j.appender.YENLO_APPENDER.File=logs/yenlo.testProxy.log
log4j.appender.YENLO_APPENDER.layout=org.apache.log4j.PatternLayout
log4j.appender.YENLO_APPENDER.layout.ConversionPattern=%d{ISO8601} \[%X{ip}-%X{host}\] \[%t\] %5p %c{1} %m%n 

These added lines will cause the WSO2 ESB/EI to create a new log file the first time the specific service is called and then keep adding to it until the DailyRollingFileAppender archives it with a date added to the file name. Afterwards the logging will appear in a brand-new file.

There are a lot more optional changes you can make which I am not going to cover here. You will be able to look at those in the apache documentation as well. Just to give an example, you can also limit the maximum amount of log files that get archived by the log rotator.

Creating a separate log file for designated API’s  

Proxy services are of course not the only thing we might want a separate log file for, luckily the exact same thing can be done for APIs.
Below you can see an example of a logger for our “yenloApi” which logs into the yenloApi.log file and creates a new log file every hour after archiving the old one with yyyy-MM-dd-HH added to the file name.

log4j.category.API_LOGGER.yenloApi=DEBUG, YENLO_API_APPENDER
log4j.additivity.API_LOGGER.yenloApi=false
log4j.appender.YENLO_API_APPENDER=org.apache.log4j.DailyRollingFileAppender
log4j.appender.YENLO_API_APPENDER.File=${carbon.home}/repository/logs/${instance.log}/TestAPI.log
log4j.appender.YENLO_API_APPENDER.datePattern='.'yyyy-MM-dd-HH
log4j.appender.YENLO_API_APPENDER.layout=org.apache.log4j.PatternLayout
log4j.appender.YENLO_API_APPENDER.layout.ConversionPattern=%d{ISO8601} \[%X{ip}-%X{host}] [%t] %5p %c{1} %m%n

I hope this information helps some people with their WSO2 EI / ESB projects, if there are any questions don’t hesitate to post them in the comments below! New Call-to-action 

Topics: WSO2 ESB, WSO2 EI, WSO2 Enterprise Service Bus, WSO2 Enterprise Integrator, Technical

Written by Wouter van Wijngaarden

Wouter van Wijngaarden
Wouter is a certified WSO2 ESB consultant with experience on a variety of projects. Using his interest in puzzles he engages in solving integration problems for our customers every day.