Some people say that integration is an art form. I disagree! Saying that integration is an art form means that you need to be an artist in order to be able to integrate systems. I have no problem with artists because I believe a true artist is an exceptional person in her or his field, but I do take issue to the idea that you need to be an artist in order to be able to integrate.
Because not everyone can be an artist, it is only for a very select group of people. Whereas anyone can integrate. Trust me as a trainer who has trained over a thousand people, I know not only artists can do system integration.
But there is a word with art in it that does apply to people who integrate systems. That word is artisan. If the word is unfamiliar to you when you look it up in a dictionary it will explain what they are, a craftsperson with a nice piece of explanation and I quote:
“ARTISANS AREN’T THE SAME AS ARTISTS, BUT IT CAN SOMETIMES BE HARD TO TELL THE DIFFERENCE. IN THE MIDDLE AGES, ARTISANS ORGANIZED THEMSELVES INTO GUILDS. IN EVERY CITY EACH GROUP OF ARTISANS
-WEAVERS, CARPENTERS, SHOEMAKERS AND SO ON – HAD ITS OWN GUILD, WHICH SET WAGES AND PRICES, KEPT STANDARDS HIGH, AND PROTECTED ITS MEMBERS FROM OUTSIDE COMPETITORS.”
I do believe that integration fits the definition of artisan. You can be an apprentice to becoming an integration artisan.
You might ask why do you start talking about art and integration? That’s a very good question and one that is very simple to answer.
I was actually integrating art. And by this, I mean that I was taking art that is available online at the Rijksmuseum in the Netherlands, downloading the art and submitting it to the digital canvas Meural 2 from Netgear.
Meural
For those of you who are unfamiliar with Meural, think of it as a digital canvas. Not unlike digital photo frames that were popular at the beginning of the 21st century but in this case bigger up to 16 x 24 ” (40 x 60 cm) of 19 x 29 ” (48 x 74 cm).
It has an Internet connection, a companion app and an API that will allow you to take art that you can download online and post it on your own canvas. Or even the pictures that you took while you were on holiday and display them as well. With the app in my mobile phone, I was already able to download pictures from the site of the Rijksmuseum and with the app upload them to the canvas to be displayed. However, that is something that I would like to do from a program that I control myself. In this blog I will explain how you do such an integration, taking resources from one side and putting them onto, in this case a digital canvas.
Another reason to do this is that I often find that people think that integration is limited to exchanging XML or JSON payloads. That is not true the Micro Integrator from WSO2 is able to handle all kinds of files, even binary ones as I will show you.
The set up
In order to explain the integration and for you to follow it on your own machine there are a number of things that you need:
- A version of the Micro Integrator (I am using version 4.2.0)
- Integration Studio in order to develop the artifacts that we’re going to deploy (I am using version 8.2)
- An API key from the Rijksmuseum in order to utilize their API
- A Meural canvas if you want to have the complete flow, if not you can store the image locally
WSO2
Setting up WSO2 is quite easy. Just download the micro integrator and integration studio from the website from WSO2. See the links above and make sure that you have the right version of Java installed (for instance OpenJDK 11) do some minor configurations like setting JAVA_HOME and make sure that Java is accessible from the path and that it is set up. I’ve written a number of blogs about doing such a setup on both windows and Linux and of course you can always join one of the trainings in which you will get the hands on experience in doing this.
Integration studio is an additional download from the same page where you downloaded the micro integrator but only slightly more to the bottom. Integration studio is a plugin on top of eclipse and for those of you that are familiar with the eclipse you will have no problem working with it. Even if you are a newbie the fact that integration studio allows you to create a high-level graphical structure of a proxy, or an API makes it a good tool to use if you’re new to integration programming. That is also the reason why I’m using it in our training micro integrator core as the development of environment for all of the labs that we do.
Rijksmuseum
Another thing that we need to do is that we need to get a key to the collection that is stored in the Rijksstudio. Create your own account using a screen name, e-mail address and a password and logging to the Rijksstudio environment. If you want, you can even log in with your Facebook account.
You do need to confirm your e-mail address, this can take a couple of minutes for the e-mail to arrive. When logged in, and you click on your icon (in our case a Y), you will see a cog, please click on that.
We scroll down and go to the Rijksmuseum API section. Fill in the intended usage and Click Request a key. This will be mailed to you at the supplied email address.
The key from the mail, is the api key we will use in the GET request to retrieve art. Contrary to what the website says you only get a key in the mail. However, there is also a link to the API documentation. This is handy if you want to know what other things you can do with the API, what kind of resources are available and so on. However, for this blog we are not going to look too deep into the documentation, but we will focus straight away on what we would like to do, namely retrieve a digitized masterpiece.
The API that the Rijksmuseum exposes is using a very well-known method to retrieve information namely doing a HTTP GET on a specific resource. Since we are adding the key to the URL as a parameter, we can actually do that from the browser which makes it easier to show you what kind of response we’re getting.
But first let’s take a look at the URL that we need to use. Digging through the API documentation this page shows us exactly what we need.
The Collections details API allows us to do a GET request with a number of parameters as you can see in the image below. Parameters are the key, the format, the culture and the object number. In the red rectangular box, you can see a sample request minus the API key to retrieve Rembrandt’s most famous masterpiece the Nightwatch
https://www.rijksmuseum.nl/api/nl/collection/SK-C-5?key=[api-key]
If you substitute the [api-key] with your API key, you will see the response.
Like mentioned before you can enter the url in a browser as well and you will get this response.
I have actually highlighted the URL but also parts of the JSON response that we’re interested in. The interest lies in the fact that the URL that is shown there actually is the place where you can download the image and save it to your local drive.
If you copy the URL and paste it in the browser, you will see the result.
In this blog I’m going to cut the integration up into little pieces and we will first look at how to do an integration to retrieve the information belonging the digitized image of the masterpiece.
Creating the API
I am going to use an API to make the connection, this might sound logical since APIs are the most common way of connecting but the beauty of an enterprise service bus solution like the micro integrator is that it is able to take various message types, like a soap message and send it to an API.
But I am going to stick with an API. In order to keep this blog shortish, I’m not going to explain every step that you need to take in order to work with integration studio. I am going to highlight the main steps but I’m not going to do all of the screen shots and all of the configuration that you would typically have to do when you create an integration project to deploy to your micro integrator.
The first step is to create an integration project and as you can see from the screenshot, I’m going to take the default settings but also add a connector sub project to it because we are going to need it later.
With regards to the API, I’m going to expose the API with the HTTP verb POST so it will require a payload where we can pass the ideas of the paintings that we would like to retrieve and use. When we call the API from the Rijksmuseum it will actually be a GET HTTP verb because that is what the museum allows us to do, retrieving information.
The API itself is quite simple and I deliberately also keep this as simple as possible. That means that some of the coding is in line rather than in a separate artifact like an endpoint. The first thing that happens in the object number property is that we take the value that is coming in and assign it.
Then we’re going to concatenate the end point where we simply sandwich the object number between the web address and the api key. We call the endpoint with the Uri template and as we can see can you screenshot from SoapUI we get a response back. In the final property we retrieved the URL where we can download the image and with that URL, we are going to download it at a later stage.
<?xml version="1.0" encoding="UTF-8"?>
<api context="/art" name="MeuralAPI" xmlns="http://ws.apache.org/ns/synapse">
<resource methods="POST">
<inSequence>
<property expression="json-eval($.object)" name="objectnr" scope="default" type="STRING"/>
<property expression="fn:concat('https://www.rijksmuseum.nl/api/en/collection/',//object,'?key=Z6sfcxEM')" name="uri.var.rmaddress" scope="default" type="STRING"/>
<call>
<endpoint name="RM">
<http method="get" uri-template="{uri.var.rmaddress}">
<suspendOnFailure>
<initialDuration>-1</initialDuration>
<progressionFactor>1</progressionFactor>
</suspendOnFailure>
<markForSuspension>
<retriesBeforeSuspension>0</retriesBeforeSuspension>
</markForSuspension>
</http>
</endpoint>
</call>
<property expression="json-eval($.artObject.webImage.url)" name="uri.var.imageurl" scope="default" type="STRING"/>
<log level="custom">
<property expression="get-property('uri.var.imageurl')" name="url"/>
</log>
<respond/>
</inSequence>
<outSequence/>
<faultSequence/>
</resource>
</api>
I’ve created JSON payload, {“object”:”SK-C-5”} that I am passing to the API.
That’s enough for this blog. In the next episode, called Part 2. I’m going to allow a JSON array to be submitted and processed and we’re actually going to download and save the images that we’re getting from the responses.