Imagine there are 4 different APIs exposed by WSO2 API Manager. One of them is really slow and it takes more than 60 seconds to respond. The other 3 are fast and will respond in less than 3 seconds. Now, in order to simplify very complex topic, imagine your system is capable to serve 10 requests concurrently. By that, 10 customers will be served in parallel, but the 11th and each additional request will be rejected from WSO2 API Manager as of resource limitation.
To move our story forward, consider our slow API as very popular and by that it’ll sooner or later take all 10 available threads. More likely it’ll be sooner as the response time is 60+ seconds, so our API Manager will wait and backend system(s) serving that API will work hardly. As your working power on the API Manager layer is “busy” as of waiting, nobody is available to serve clients in need for fast APIs. Imagine how good would be if you can release at least one or two from 10 available threads to serve fast APIs. If you don’t want to take care about such cases, your system will become unstable.
By that, it’s very important to keep your business and technical rules optimal. It’s not easy and there is no such solution which universally fits the best in each case. But at least here are a few steps you should do:
- You need to see if it’s anyhow possible to optimize the slow backend API. Bad programming decisions can influence a lot here so please review your algorithms, DB indexes and queries etc.
- Figure out how and where to introduce caching mechanism.
- Consider to introduce API (path) based load balancing to prevent slow requests to takeover all your resources. When you filter such traffic, you can even optimize horizontal scaling. Here you can benefit if you’re in cloud, as you can “easy” tweak power to your stack. We know in practice it doesn’t have to be so easy, but at least is doable after some effort.
- Fine-tune WSO2 API manager. I’ve already mentioned the whole set of levels where you can do enhancements. Good resource to move forward with this is this page. But please consider your own needs and use cases.
Timeout is one of the most important fine-tuning mechanisms regarding this topic. Please read very carefully all statements and limitations over there, as it’s not very intuitive to determine the priority of different parameters. Consider there are:
- Socket timeout: APIMHOMErepositoryconfpassthru-http.properties as http.socket.timeout;
- Global timeout: synapse.global_timeout_interval in APIM_HOMErepositoryconfsynapse.properties;
- API-specific timeout available in APIM_HOME/repository/deployment/server/synapse-configs/default/api per each API.
Keep in mind Socket timeout is the big boss here, so if your global timeout or API-specific timeout is longer than Socket timeout, WSO2 API Manager will follow up with Socket timeout. So, your Socket timeout should be more flexible than the rest two.
In the second part of this blog, I’ll share some benchmarks and conclusions regarding this topic. If you forget that, you are not able to connect to the H2 instance.
If you have any questions about this blogpost contact us via the comments section below. View also our WSO2 Tutorials, webinars or whitepapers for more technical information. Need support? We do deliver WSO2 Product Support, WSO2 Development Support, WSO2 Operational Support and WSO2 Training Programs.
