The WSO2 products’ installations should end with tests which proof that the system is working stable and fast. The tests can show that some settings are not correct or that installation can be tuned. It also can show that there are some problems with the backend system (like access, internal errors, etc.).
Goal
How to test a published REST APIs or web service? Usually we request the endpoints and check the results. The problem is how many time we have to request the endpoint to be sure that everything is checked? How to measure the time of responses? How many resources (people, computers) we should use for it?
In this blog will be presented how can we easy test endpoints. As an environment we will use REST API published in WSO2 API Manager 2.0 and Gatling as test stress tool.
- WSO2 API Manager is a complete solution for designing and publishing APIs, creating and managing a developer community, and for securing and routing API traffic in a scalable way.
- Gatling is a highly capable load testing tool. It’s designed for ease of use, maintainability and high performance. It comes with support of the HTTP protocol that makes it a tool for load testing any HTTP server.
Scenario
The Yenlo Car API is designed to get an information about the cars registered in Netherlands. We can get information about car for given vehicle license plate or about the cars for given mark or for given mark and type.
The Yenlo Car API we would like to be managed. It means that we want to have an API’s life cycle management, validate API payload or access control and apply security policies to it. For this purpose we will publish our API via WSO2 API Manager 2.0.0.
Requirements
- WSO2 API Manager 2.0
- Download from http://wso2.com/products/api-manager/
- Install it
$ unzip wso2am-2.0.0.zip - Run application
$ $APIM_HOME/bin/wso2server.sh
- Yenlo Cars API – can be build and run on your environment (local computer) or used by Heroku application servers.
- Local
- download project https://bitbucket.org/tszielinski/yenlo-cars
- upload schema and data to MySQL
mysql> source yenlo-cars-import/sql/schema.sql; mysql> source yenlo-cars-import/sql/cars-backup.sql; - build project (Java8 required) under yenlo-cars directory
yenlo-cars$ mvn clean install - run server
yenlo-cars$ java -jar yenlo-cars-api/target/dependency/webapp-runner.jar --path /cars --port 8080 yenlo-cars-api/target/*.war--port 8080 yenlo-cars-api/target/*.war
- Local
- WSO2 API Manager Publisher
- Login to Publisher
- Add new API
- the API’s endpoint upload using Swagger URL http://localhost:8080/cars/v2/api-docs
- the API’s endpoint upload using Swagger URL https://yenlo-cars.herokuapp.com/v2/api-docs
- the API’s endpoint upload using Swagger URL http://localhost:8080/cars/v2/api-docs
- Design, implement and configure the API
- set Endpoint Security Scheme to Secured and Endpoint Auth Type to Basic Auth with credentials user as yenlo-cars, password as yenlo-cars-demo.
- set Endpoint Security Scheme to Secured and Endpoint Auth Type to Basic Auth with credentials user as yenlo-cars, password as yenlo-cars-demo.
- Publish the API
- Login to Publisher
- WSO2 API Manager Store
- Sign-in to Store
- Subscribe to cars-api and generate the tokens.
- Sign-in to Store
Tests
The tests will check the endpoint published in WSO2 API Manager Store. The test report gives us the measure of the times between requests and responses. The same tests can be run on backend system and we can compare the results.
Test scenario
The Gatling tool gives us a possibility to create test scenario which defines the list of requests to be called and virtual users involved in it.
Let’s define our scenario. We would like to test the requests on listed URL’s:
- GET /count
- GET /count/marks
- GET /count/mark/{mark}
- GET /vrp/{vehicleRegistrationPlate}
- PUT /vrp/{vehicleRegistrationPlate}/notes
- PUT /vrp/{vehicleRegistrationPlate}/notes/clear
We want to simulate 50 users which request the URLs in nearly same time. 25 users start the test at one time and next 25 users start test during 30 seconds.
Source code
Gatling tool is written in Scala and tests have to be written in this language. The classes can be run under Scala environment or the Maven project runs in Java environment using gatling-maven-plugin.
$ mvn gatling:execute |
Test classes
The source code with the test is available on Bitbucket.
BaseSimulation.scala
package com.yenlo.cars;
import io.gatling.core.Predef._
import io.gatling.http.Predef._
abstract class BaseSimulation extends Simulation {
// car’s mark
val marks = Array("A.C.","ALFA ROMEO","ALKO","ALPINE-RENAULT","ALVIS","AMERICAN MOTORS","AUDI",
"AUDI/PORSCHE","AUSTIN","AUSTIN-HEALEY","AUTO BIANCHI","AUTO UNION","BAVARIA","BAVARIA CAMP",
"BECK","BENTLEY","BERTONE","BUERSTNER","BUICK OPEL","CHALLENGER","CHRYSLER","CLASSIC ROADSTERS",
"CMS AUTO","COXX","D.K.W.","DACIA","DAIMLER BENZ","DATSUN","DERCKS","DETHLEFFS","DIGICROSS",
"ESTEPE","EURA MOBIL","EUROPELIFT","EVOBUS","FIAT","FIAT CAM","FIAT-ABARTH","FIAT-FLEURETTE",
"FIAT-NECKAR","FIAT/CHALLENGER","FORD","FRANKIA","FRANKIA-RMB","FSM","GALLOPER","HENDRICKS",
"HONDA","HYUNDAI","IFAKRAFTFARZEUGWERK AUDI","ISUZU","IVECO","IVECO FIAT","KIA","KIA MOTORS", "KIAMASTER","LATERAL DESIGN LIMITED","LECAPITAINE","MALLAGHAN ENGINEERING LTD","MARUTI",
"MATTIG BAMBERG","MEIWA","MERCEDES","MERCEDES-AMG","MERCEDES-BENZ","MERCEDES-BENZ/SCHENK",
"MERCEDES-HYMER","MERCEDES-RAPIDO","MITSUBISHI","MITSUBISHI FUSO","MORGAN","NEBIM","NECKAR",
"NILSSON","NSU-FIAT","OPEL","OPEL MOD DRST","POESSL","POLSKI FIAT","PORSCHE","PORSCHE-DIESEL",
"QINGQUI","QUATTRO","RAMBLER RENAULT","RELIANT","RENAULT","RENAULT ALPINE","RENAULT MOD DRST",
"RENAULT MOD JPM","RENAULT RODEO","RENAULT SAMSUNG","RENAULT SPORT","RENAULT TRUCKS","ROVER",
"SANTANA","SAVIEM-RENAULT","SEAT","SIMCA-FIAT","SINGER","SKODA","SOVAB","STEYR-FIAT",
"STEYR-PUCH","STUDEBAKER","STX","SUZUKI","TEILHOL","TEMPO","TERBERG TECHNIEK","TESLA",
"TESLA MOTORS","TOMOS","TOYOTA","TOYOTA-MINICRUISER","TSCHU-TSCHU","TVR","VAN HOOL-FIAT",
"VOLKSWAGEN","VOLKSWAGEN-TABBERT","VOLKSWAGEN/DE VRIES","VOLVO","VOLVO-VERHEUL","VOLVO-ZABO",
"VOLVO/SCHENK","VW-PORSCHE","WESTFIELD","WINNEBAGO","YAMAHA","YUGO","ZIE BIJZONDERHEDEN")
// vehicle licence plate
val vrps = Array("00BBP2","00BBP9","00BBR1","ZB02VY","ZB02YX","ZB03BL","AE4781","AE4782","AE4794","1011EB","1016ZN","1017GB","1022HG","3023US","3027XB","3029SB","3030BE","3033BB","3035BB","3037TF","3037UB","3039GX","3043MR","60AJ88","60AZ51","60BA11","60BB42","60BBB1","60BBB3","60BBB7","60BBBB","60BBBD","60BBBJ","JB001K","JB001L","JB001P","JB001R","JB001V","JB001X","JB002B");
val rand = new java.util.Random(System.currentTimeMillis());
// URL endpoint
val count = http("count").get("/count");
val count_marks = http("count-marks").get("/count/marks");
val count_mark = http("count-mark").get("/count/mark/" + marks(rand.nextInt(marks.length)));
val vrp = http("vrp").get("/vrp/" + vrps(rand.nextInt(vrps.length)));
val vrp_notes_set = http("vrp-notes-set").
put("/vrp/" + vrps(rand.nextInt(vrps.length)) + "/notes").
body(StringBody("""{ "note": "Note about a car."}""")).asJSON;
val vrp_notes_clear = http("vrp-notes-clear").
put("/vrp/" + vrps(rand.nextInt(vrps.length)) + "/notes/clear");
}
APIManager.scala
package com.yenlo.cars;
import io.gatling.core.Predef._
import io.gatling.http.Predef._
class TestAPIManager extends BaseSimulation {
// headers assigned to request
val headers = Map(
"Content-Type"->"application/json",
"Accept"->"application/json",
// change to valid token
"Authorization"->"Bearer 94d7de858eba4b8f3f564153f6a4f103");
// scenario
val scn = scenario("cars").
exec(count.headers(headers)).pause(0,3). // request endpoint /count and pause 0..3 seconds
exec(count_marks.headers(headers)).pause(0,5). // request endpoint /count/marks and pause 0..5 seconds
exec(count_mark.headers(headers)).pause(0,5). // request endpoint /mark/{mark} and pause 0..5 seconds
exec(vrp.headers(headers)). // request endpoint /vrp/{vehicleLicencePlate}
exec(vrp_notes_set.headers(headers)). // request endpoint /vrp/{vehicleLicencePlate}/notes
exec(vrp_notes_clear.headers(headers)); // request endpoint /vrp/{vehicleLicencePlate}/notes/clear
// URL
val httpConf = http.baseURL("http://localhost:8280/cars/0.0.1");
setUp(
scn.inject(
atOnceUsers(25), // create 25 users at
once
rampUsers(25) over (30)) // create 25 during
30 seconds
).protocols(httpConf);
}
API.scala
package com.yenlo.cars;
import io.gatling.core.Predef._
import io.gatling.http.Predef._
abstract class API extends BaseSimulation {
val headers = Map(
"Content-Type"->"application/json",
"Accept"->"application/json");
val scn = scenario("cars").
exec(count.basicAuth("yenlo-cars", "yenlo-cars-demo").headers(headers)).pause(0,3).
exec(count_marks.basicAuth("yenlo-cars", "yenlo-cars-demo").headers(headers)).pause(0,5). exec(count_mark.basicAuth("yenlo-cars", "yenlo-cars-demo").headers(headers)).pause(0,5).
exec(vrp.basicAuth("yenlo-cars", "yenlo-cars-demo").headers(headers)).
exec(vrp_notes_set.basicAuth("yenlo-cars", "yenlo-cars-demo").headers(headers)).
exec(vrp_notes_clear.basicAuth("yenlo-cars", "yenlo-cars-demo").headers(headers));
}
LocalAPI.scala
package com.yenlo.cars;
import io.gatling.core.Predef._
import io.gatling.http.Predef._
class LocalAPI extends API {
val httpConf = http.baseURL(“http://localhost:8080/cars”);
setUp(
scn.inject(
atOnceUsers(25),
rampUsers(25) over (30))
).protocols(httpConf);
}
HerokuAPI.scala
package com.yenlo.cars;
import io.gatling.core.Predef._
import io.gatling.http.Predef._
class HerokuAPI extends API {
val httpConf = http.baseURL("https://yenlo-cars.herokuapp.com");
setUp(
scn.inject(
atOnceUsers(25),
rampUsers(25) over (30))
).protocols(httpConf);
}
Test run
# run test on WSO2 API Manager calling Yenlo Cars API |
Test results
The tests show how response times are distributed among standard ranges. Also informs about the number of OK (success)/KO (failed) requests.
The tests’ results are presented as short information on the display (console)
Simulation com.yenlo.cars.APIManager completed in 56 seconds
Parsing log file(s)...
Parsing log file(s) done
Generating reports...
=============================================================================
---- Global Information --------------------------------------------------------
> request count 300 (OK=300 KO=0 )
> min response time 3 (OK=3 KO=- )
> max response time 31441 (OK=31441 KO=- )
> mean response time 5854 (OK=5854 KO=- )
> std deviation 7778 (OK=7778 KO=- )
> response time 50th percentile 1100 (OK=1100 KO=- )
> response time 75th percentile 11329 (OK=11329 KO=- )
> response time 95th percentile 21986 (OK=21986 KO=- )
> response time 99th percentile 28585 (OK=28585 KO=- )
> mean requests/sec 5.263 (OK=5.263 KO=- )
---- Response Time Distribution ---------------------------------------------
> t < 800 ms 147 ( 49%)
> 800 ms < t < 1200 ms 4 ( 1%)
> t > 1200 ms 149 ( 50%)
> failed 0 ( 0%)
=============================================================================
Reports generated in 0s.
Please open the following file: /Users/thomas/private/projects/gatling/yenlo-stress-test/target/gatling/apimanager-1474029323378/index.html
Simulation com.yenlo.cars.LocalAPI completed in 48 seconds
Parsing log file(s)...
Parsing log file(s) done
Generating reports...
=============================================================================
---- Global Information -----------------------------------------------------
> request count 300 (OK=300 KO=0 )
> min response time 3 (OK=3 KO=- )
> max response time 24511 (OK=24511 KO=- )
> mean response time 4567 (OK=4567 KO=- )
> std deviation 5923 (OK=5923 KO=- )
> response time 50th percentile 1041 (OK=1041 KO=- )
> response time 75th percentile 7396 (OK=7396 KO=- )
> response time 95th percentile 16211 (OK=16211 KO=- )
> response time 99th percentile 23112 (OK=23112 KO=- )
> mean requests/sec 6.122 (OK=6.122 KO=- )
---- Response Time Distribution ---------------------------------------------> t < 800 ms 149 ( 50%)
> 800 ms < t < 1200 ms 4 ( 1%)
> t > 1200 ms 147 ( 49%)
> failed 0 ( 0%)
=============================================================================
Reports generated in 0s.
Please open the following file: /Users/thomas/private/projects/gatling/yenlo-stress-test/target/gatling/localapi-1474029770028/index.html
and as HTML file with details information and charts
Conclusion
The tests build using Gatling tool give us the possibility to check our WSO2 installation. We can simply test endpoints (http, https, jms) using many virtual users for it. We can proof that installation is stable and can by highly load without any problems.
The code is created with use of the data from: https://opendata.rdw.nl/Voertuigen/Open-Data-RDW-Gekentekende_voertuigen/m9d7-ebf2
If you have any questions about this blogpost contact us via the comments section of this blog. View also our WSO2 Tutorials, webinars or white papers for more technical information. Need support? We do deliver WSO2 Product Support, WSO2 Development Support, WSO2 Operational Support and WSO2 Training Programs.
