Handling messages based on priority in WSO2 ESB / WSO2 EI

We all like to be treated with priority, right? When you are boarding a flight, visit a tourist place or queueing to enter a cinema, separate lanes for priority passengers or visitors are quite common. Visa application and approval is a business process where prioritizing applications is norm.  Vendor organizations handling visa applications prioritize based on the role of applicant (for e.g., State heads and employees in embassies get highest priority), the premium paid (if you are willing to pay premium fees, your visa is processed faster) & other special conditions. It is an important technical requirement that systems that process applications, connect through the ESB layer.

Once, such requirement came up while setting up the WSO2 Enterprise Service Bus or WSO2 Enterprise Integrator. The requirement was “apply ten times weightage for a priority message”. This simply meant for every ten priority messages only one standard message will be processed. This sound unrealistic, there may not be 10 times “priority visa applications” compared to number of “normal applications” on a workday in visa application center. Whereas it is a realistic case when messages are sent after a system maintenance week or after public holidays. There will be applications(messages) collected over a period which are sent out at once to other systems. How do we ensure the above requirement is met in such flood gate scenario within ESB?

WSO2 provides different means to realize priority processing:
1. Priority set on in-flight messages on messaging layer using JMS priority.
2. Priority processing within a mediation.

Prioritization is implemented at two levels in WSO2 ESB:
1. HTTP transport level – If users would like to use the EI as a pure router.
2. Message mediation level – If users use EI for heavy processing like XSLT and XQuery.

From the user’s perspective, key to any priority mediation is to determine the priority of an incoming message. WSO2 ESB provides priority executor to implement prioritization at message mediation level.

Priority executors

A priority has only some meaning when compared with other priorities. For example, if there are two priorities having values 10 and 1, a message with priority 10 will get 10 times more resources then messages with priority 1. These executors can be used with the Enqueue Mediator to execute sequences with a given priority. Priority executors are used in high-load scenarios when you want to execute different sequences for messages with different priorities. This approach allows you to control the resources allocated to sequences. This also prevents delaying or dropping of high-priority messages.

A simplified illustration on use of priority executors is shown below.

Source -JMS-ESB- External Web Service (Submit Visa).

• 1. The source system populates PRIORITYID value as “ABC” for messages with higher priority
• 2. The messaging layer acts as temporary storage. There are a smaller number of high priority (green color) messages compared to standard (yellow) messages.
• 3. But ESB ensures more high priority(green) messages are first processed in a high load scenario compared to number of standard messages processed.
• 4. The external web service for processing visa receive high priority applications first.

WSO2 ESB ensures high priority messages are not kept waiting in a floodgate scenario.

WSO2 ESB – Design of Process flow

1. 1. A JMS based proxy service named priority-outbound-service consume messages from JMS queue.
   1.   – Priority is set to 10 for messages having “ABC” as value of transport property  PRIORITYID
2.   – Priority is set to 1 for all other messages
3. 

• 2. Process the message through enqueue mediator visa_priority_exec
• 

The queues shown in above configurations defines separate queues for different priorities in a Thread Pool Executor.

• – core – Defines a core number of Priority Executor threads. When EI is started with the  priority executor, 2 threads will be created.
• – max – Defines the maximum (100) threads this executor will have.

When 100 threads are spawn in a high load scenario, 80 threads are allocated to messages with priority 10 compared to 20 threads allocated to messages with priority 1.

3. Enqueue mediator controls the thread to execute  sequence call-external-sequence

WSO2 ESB – Priority executor Deploy process

As every other WSO2 ESB artifacts, I expected priority executors can be packaged in a CAR file. This was not the case; the composite application project did not recognize the priority executor artifact.

Similarly, there was no folder name “priority executors” under synapse-config in eclipse project as well.  This leaves us with only option to add the executors to synapse config through the admin console.

• 1. Click on “Main” in the left menu to access the “Manage” menu.

• 2. In the “Manage” menu, click on “Priority Executors” under “Service Bus.”

• 3. In the “Priority Executors” window, click on the “Add Executor” link.
• 
  • In the next window fill in values based on the queue size, maximum number of queues and priority values (10, 1 in this sample) etc.

  Priority Executor – how it is implemented

  The priority mediation implementation is based on Queues and ThreadPoolExecutors.

  ThreadPoolExecutor accepts a BlockingQueue implementation. A custom blocking queue that can be used to order the jobs based on priority, is implemented. ThreadPoolExecutor only starts queuing when all the core threads are busy. Every message will get equal priority until all the core threads are used.

  Internally custom BlockingQueue uses multiple queues for accepting jobs with different priorities. Once jobs are put into the queue, it uses an algorithm for choosing the next job. The default algorithm chooses the jobs based on a priority-based, round-robin algorithm.

  In our use case we have two priorities, 10 and 1. This algorithm tries to fetch 10 items with priority 10 and then 1 item with the priority 1.

  Summary

  We can design and thereby control the message flow within WSO2 ESB using priority executor together with JMS priority. How priority must be handled within WSO2 ESB is now up to the use case which has to be realized.

  If you have any questions about this blogpost, contact us via the comments section of this blog. View also our other WSO2 blogs, webinars or white papers for more technical information. Or join one of our WSO2 trainings!