Preserve message order with multiple WSO2 JMS proxies and ActiveMQ

Problem Context

When JMS proxy services are deployed in a WSO2 EI high availability (HA) setup, all JMS proxy consumers compete to consume from the single JMS queue. This will impede preserving the message order for the target system(s). This can be problematic for certain target systems which needs to preserve message order

In above setup all JMS consumers compete for messages and the target system will receive messages out of order. This is because even though the JMS queue adheres to FIFO (first in first out), as each JMS proxy in each EI node has a different processing latency it eventually leads to out of sync message order for the target system.

Solution

TTo overcome this issue, we can use the exclusive consumer[1] feature in ActiveMQ. The exclusive consumer feature allows only one JMS consumer to be active at any given moment for a given queue. Resulting in only one consumer consuming JMS messages regardless of number of JMS consumers subscribed to the given queue. If the active consumer fails the broker will failover to next connected consumer and continue. This ensures continuous ordered JMS message delivery for the downstream systems. The exclusive consumer feature is included in ActiveMQ out of the box from ActiveMQ v4.x onwards.

To use this feature JMS session needs to be created with “consumer.exclusive=true” flag when creating the JMS consumer.

The following steps illustrate how to enable this feature within WSO2 JMS proxy consumer and verify the functionality

Implementation

Since exclusive consumer is an ActiveMQ feature, we can use any EI version as long as we use ActiveMQ version 4.x or above. Following steps explains how to test this feature in a local setup with two EI servers and ActiveMQ service.

• Start ActiveMQ server 4.x or above

• EI server setup:
  • Configure two EI servers (in ESB profile) with port offset.
  • Configure ActiveMQ transport in both EI servers [2].
  • Start EI servers.

• Configure following JMS consumer proxy in both EI servers as shown below. This proxy will create a server log when it consumes a JMS message.

 <?xml-versie = "1.0" encoding = "UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
 name="test.excl.queue"
 startOnLoad="true"
 statistieken = "uitschakelen"
 trace = "uitschakelen"
 transports="jms">
 <target>
 <inSequence>
 <log level="custom">
 <property name="JMSProxy: " value="bericht ontvangen."/>
 </log>
 </inSequence>
 </target>
 <parameter name="transport.jms.SessionAcknowledgement">CLIENT_ACKNOWLEDGE</parameter>
 <parameter name="transport.jms.DestinationType">queue</parameter>
 <parameter name="transport.jms.Destination">test1.Q?consumer.exclusive=true</parameter>
 <parameter name="java.naming.provider.url">failover:(tcp://localhost:61616)</parameter>
 <parameter name="transport.jms.SessionTransacted">true</parameter>
 <parameter name="transport.jms.CacheLevel">consumer</parameter>
 <description/>
</proxy>

Note that exclusive consumer flag is configured in JMS destination parameter as shown below.

<parameter name="transport.jms.Destination">test1.Q?consumer.exclusive=true</parameter>

Testing/Demonstration

To verify configurations are applied correctly.

• Login to ActiveMQ and navigate to Queue section. Under “test1.Q” should have two active subscriptions as below.

• Navigate to “test1.Q” JMS consumers section in ActiveMQ console. Verify it has two consumers registered with exclusive flag enabled.

• Once confirmed the configurations are applied in ActiveMQ send multiple messages via ActiveMQ console and observe console logs in both EI servers. (To send JMS messages can use ActiveMQ built-in messages sender)

Server logs confirm only one JMS consumer is actively consuming messages. Hence message order is preserved.

• To test the failover scenario from one JMS consumer to other, shut down the active JMS consumer server while messages are consuming.

Send large number (around 3000) of JMS messages to the queue via ActiveMQ console (as shown above).

While messages are consuming shutdown the EI server with active JMS consumer.

As shown per server logs ActiveMQ successfully failover to next available JMS consumer without loss of messages.

This demonstration proves ActiveMQ exclusive consumer feature can successfully be used to solve message ordering issues in high availability EI JMS proxy consumer setup.