Troubleshooting Connection Reset Error from WSO2 Enterprise Integrator

This Configuration is applied to the WSO2 Enterprise Integrator 6.2.0 to EI 6.6.0

Blocking vs non-Blocking:

While doing HTTP calls from the Enterprise Integrator, there are 2 ways to invoke the backend services/apis. One is Blocking and the other is non-blocking. The difference is that when you invoke a backend Service in non-blocking mode, the request thread does not wait till the underlying worker thread calling the back-end returns a response whereas in the blocking mode the request thread would be blocked till the worker thread completes the back-end call and respond.

For this reason, WSO2, according to their best practices, recommends that we should always prefer to use non-blocking transport. As this is better in terms of performance and resource management.

Why is blocking neccesary sometimes?

Although WSO2 recommends using the non-blocking mode as much as possible, there are situations where we need to use the blocking mode. For example, one such scenario can be when dealing with JMS based proxy services. In a JMS based proxy if you are to make multiple calls to the backend systems, all the calls need to be in blocking mode. This is done to keep track of the JMS Transactions and so the message can be rolled back if necessary.

While making Blocking calls, WSO2 allows by default only 2 simultaneous connections per host. If the ESB needs to send more calls, each request will have to stay in line till a connection is available. This may take a lot of time and can even cause calls to fail resulting in Connection Reset errors to be thrown. The Exception you will see in the logs can be seen below:

Exeption:

java.net.SocketException: Connection reset
    at java.base/java.net.SocketInputStream.read(Unknown Source)
    at java.base/java.net.SocketInputStream.read(Unknown Source)
    at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source)
    at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source)
    at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source)
    at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source)
    at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
    at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
    at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
    at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
    at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
    at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
    at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
    at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
    at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
    at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)

What to do when an error occors?

To solve this issue, there are 2 simple things that you can do.

First in <EI_HOME>/bin directory, edit the integrator.sh or integrator.bat file. Add the following values in it.

• -Dlst_t_core=200
• -Dlst_t_max=250
• -Dsnd_t_core=200
• -Dsnd_t_max=250

This will increase the number of threads being used. If these values are not there you can just define them. These properties are related to nHTTP Protocol configurations.

Secondly, you can increase the limit of defaultMaxConnectionsPerHost parameter configuration in the <EI_HOME>/conf/axis2/axis2_blocking_client.xml file. For example, if you need to set 100 simultaneous connections per host, you can set the value of the parameter as follows:

<transportsender class="org.apache.axis2.transport.http.CommonsHTTPTransportSender" name="http">
        <parameter name="PROTOCOL">HTTP/1.1</parameter>
        <parameter name="Transfer-Encoding">chunked</parameter>
        <parameter name="cacheHttpClient">true</parameter>
        <parameter name="defaultMaxConnectionsPerHost">100</parameter>
</transportsender>

After doing both the changes, you need to restart the EI instances.

Conclusion:

By changing these settings, you will increase the number of threads and default limit of WSO2 EI connection limit per Host. Please note that there could be other reason as well for this error to be encountered. So, If the above configuration does not work for you, then there might be another underlying problem.  The most likely scenario in that case, is that there can be a difference in TLS versions between the Enterprise Integrator and the backend.

If you want to avoid the hassle of investigating these long exceptions and errors, do checkout our Connext Go Solution which provides the Integration as a Service solution. With the Connext Go, you can count on24/7 complete support of Yenlo-experts to take care of such issues at a fixed price per integration per month.

Yenlo is the leading, global, multi-technology integration specialist in the field of API-management, Integration technology and Identity Management. Known for our strong focus on best-of-breed hybrid and cloud-based iPaaS technologies.
Yenlo is the product leader and multi-award winner in WSO2, Boomi, MuleSoft and Microsoft Azure technologies and offers best-of-breed solutions from multiple leading integration vendors.