Multipart/form-data file uploads via WSO2 EI/ESB

Recently I’ve worked with Multipart/form-data for some integrations. During this time, I ran into some issues and found that all the related articles on the internet didn’t really answer my questions. So, in this blog I’ll describe the two main ways of working with multipart/form-data payloads in WSO2 Enterprise integrator.

Multipart/form-data

Multipart/form-data is one of the possible content types used in HTML forms when submitting data. When the form is submitted, a POST request is done where each of the fields are separate “parts” of the resulting payload. These parts a separated by a so called “Mime boundary”  

To show what this would look like I’ve constructed a 2-part example multipart/form-data message below. Part 1 includes an omitted binary payload, while Part 2 contains a metadata json object.

Option 1 – Build your own payload

To build a multipart/form-data message in WSO2 Enterprise integrator is relatively straightforward but has some nuances that can cause issues. To allow the product to build to message into xml format the correct builder and formatter need to be enabled inside the product but for multipart/form-data these should already be enabled by default. If you want to double check to be sure the axis2.xml file inside the <product_home>/conf/axis2 folder should contain the following two lines in the messageBuilders and messageFormatters sections of the file.

	<messageBuilder class="org.apache.axis2.builder.MultipartFormDataBuilder" contentType="multipart/form-data"/>
	<messageFormatter class="org.apache.axis2.transport.http.MultipartFormDataFormatter" contentType="multipart/form-data"/>

Now that the configuration has been verified, it is time to construct a payload.
For the incoming payload I will use the below dummy payload which contains a picture and a json object with metadata.

curl --location --request POST 'https://localhost:8243/myApi/uploadFile' \
--form 'fileContent=@"/C:/Users/Wouter/Pictures/picture.jpg"' \
--form 'fileMetadata="{\"FileName\": \" picture.jpg\", \"CorrelationId\": \"f145b832-2e5d-425d-a80b-05d5a1a5a129\", \"someNumber\": 7098813, \"Category\": \"AdditionalInfo\"}";type=application/json'

When this message is built into xml by the enterprise integrator it will look like this:

<mediate>
	<fileContent filename=”picture.jpg”>_binary_payload_</fileContent>
	<fileMetadata> {"FileName": "picture.jpg", "CorrelationId": " f145b832-2e5d-425d-a80b-05d5a1a5a129", "someNumber": 7098813, "Category": "AdditionalInfo"} </fileMetadata>
</mediate>

In the hypothetical example integration, I want to route messages that contain the “fileMetadata” alongside the “fileContent” element somewhere different than the messages that just contain the “fileContent” element. Additionally, I want to replace the fileMetadata element in the payload with my own json object.

Client Case TRUMPF Group: WSO2 middleware solution

Download now

After running a filter mediator to find the different messages, it’s possible to construct the payload in various ways. But for this example I will use the payloadFactory mediator.

<payloadFactory media-type="xml">
	<format>
		<root
			xmlns="">
			<myExtraJsonContent
				xmlns="http://org.apache.axis2/xsd/form-data" content-type="application/json" filename="$2">{“dummyData”:”thisIsDummyData”}
			</metadata>
			<file
				xmlns="http://org.apache.axis2/xsd/form-data"
filename="$2">$1
			</file>
		</root>
	</format>
	<args>
		<arg evaluator="xml" expression="$body/mediate/fileContent"/>
		<arg evaluator="xml" expression="$body/mediate/fileContent/@filename"/>
	</args>
</payloadFactory>

In the above payload factory, you can see we have two elements inside the <root> element. Each separate element with the “xmlns=”http://org.apache.axis2/xsd/form-data”” namespace is formatted into as a MIME boundary separate MIME part in the resulting outgoing message.

Each part can have its own content type using the “content-type” attributes on the xml elements as well, this can be very important depending on how the backend is configured to read the message.

After constructing the payload, the only remaining step is to set the correct message and content type, so the Enterprise Integrator knows to convert the payload back to multipart/form-data from xml.

<header name="Content-Type" scope="transport" value="multipart/form-data"/>
<property name="messageType" scope="axis2" value="multipart/form-data"/>

Once set the payload is ready to be sent off to a backend of your choosing.

Option 2 – Binary Passthrough

The first option is also the simplest, but not valid for as many use cases. As the name suggests, binary passthrough is used for passthrough integrations. For example, where a client calls an API, and the Enterprise integrator only forwards the data to the backend without building the message.

Binary passthrough must be enabled on a ‘per content’ type basis in the axis2.xml (or deployment.toml depending on the product version), it consists of two parts, the builder (which handles the incoming message) and the formatter (which handles the outgoing message).

Important: By default, the xml version of the multipart/form-data builder/formatter from Option 1 above are enabled, these need to be disabled for the binary passthrough builder/formatter to work.

Below I’ve written down what enabling this would look like in the configuration itself.

Deployment.toml – WSO2EI 6.6.0 and newer

[[custom_message_formatters]]
class = "org.wso2.carbon.relay.BinaryRelayBuilder"
content_type = "multipart/form-data"

[[custom_message_builders]]
class = "org.wso2.carbon.relay.BinaryRelayBuilder"
content_type = "multipart/form-data"

axis2.xml – WSO2EI 6.5.0 and older

<messageBuilders>
	<messageBuilder contentType="multipart/form-data" class="org.wso2.carbon.relay.BinaryRelayBuilder"/>
</messageBuilders>
<messageFormatters>
	<messageFormatter contentType="multipart/form-data" class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>
</messageFormatters>

Once this is done restart the product and any messages coming in with this content type will not be built into XML and will just pass through as pure binary.
The downside of this is that if there is another integration on the same integrator node that does need to have the multipart/form-data payload be built into xml, for example for a content-based router pattern it will not work.

If that sounds like your use case, or you just have a single integration that needs to mediate based on the contents of the multipart message, I recommend checking out Option 1 again.

I hope this blog helps you on your way with your integration project.

Whitepaper ESB selection guide

How do you choose?

Download now

Yenlo is the leading, global, multi-technology integration specialist in the field of API-management, Integration technology and Identity Management. Known for our strong focus on best-of-breed hybrid and cloud-based iPaaS technologies. Yenlo is the product leader and multi-award winner in WSO2, Boomi, Microsoft Azure and AWS technologies and offers best-of-breed solutions from multiple leading integration vendors.

With over 200 experts in the API, integration, and Identity Access Management domain and over $35 million in annual revenue, Yenlo is one of the largest and best API-first and Cloud-first integration specialists worldwide.