Discover our knowledge. Read our blogs!

Learn more

We build all our solutions with WSO2 and we are proud that we are WSO2 Premier Certified Integration Partner and Value-Added Reseller.

Learn more

Edit WSO2 XML configuration files with xmlstarlet

12 min read

configureMany times we need to perform small edits on XML files from within a shell script. For example, during software deployment, some XML element or attribute values need to change, or XML elements have to be added, removed, modified. This is a different use-case than installing/deploying WSO2 on a server for which we typically would use Ansible or Packer. In our case we need a lightweight solution that takes little time to create but saves us a lot of time. In a previous blog (Doing a simple scripted setup of WSO2 Enterprise Integrator) we used the stream editor SED. But there are alternatives as we will see in this blog.

Use Cases

A typical example is the installation of multiple WSO2 products, which can co-exist in the same machine; and, to avoid TCP Port duplication, we need to change the value of:

<Offset>0</Offset>

from the default (zero) to another small number.

Text editing in a script is traditionally a realm of the utility SED. An implementation with SED would look like this:

# applying offset to WSO2-AM
OFFSET=1
echo "Applying <Offset>"$OFFSET"</Offset>"
SEDFILE=/opt/wso2/wso2am-2.5.0/repository/conf/carbon.xml 
sed -i 's:<Offset>[0-9]</Offset>:<Offset>'$OFFSET'</Offset>:' $SEDFILE

SED can be used to replace the text in green, with the text in yellow. There are plentiful of SED examples on the World Wide Web.

It doesn’t easily work however for more complex situations. Take for example the change to enable Analytics in WSO2 API Manager:

<Analytics>
  <Enabled>false</Enabled>
  […]
</Analytics>
 <WorkflowConfigurations>
  <Enabled>false</Enabled>
  […]
</WorkflowConfigurations>
# Note: with SED we enable by mistake both <Analytics> and <WorkflowConfigurations>
sed -i 's:<Enabled>false</Enabled>:<Enabled>true</ Enabled >:' $SEDFILE
 

If we change the value of <Enabled> elements to “true”, we change many more than a single element, and end up with a wrong set-up. We may activate the WorkflowConfigurations, which we don’t want. This uncontrolled change is not what we want. We can actually make it work with SED, but that would take a lot of coding.

Luckily, for XML editing there is an equivalent of SED: the powerful and versatile utility XMLSTARLET.

With a single line of script the correct <Enabled> element, child of <Analytics>, can be changed to any value, for example to the value “true”:

# applying configuration changes to WSO2 AM-ANALYTICS
XMLFILE=/opt/wso2/wso2am/repository/conf/api-manager.xml
echo "Applying api-manager.xml <Analytics><Enabled>true</Enabled></Analytics>"
echo "the previous value was:" `xmlstarlet sel -t -v //Analytics/Enabled <$XMLFILE`
xmlstarlet ed --inplace -P -u //Analytics/Enabled -v true $XMLFILE

The previous XMLSTARLET invocation can be paraphrased as: search for the <Enabled> sub-element of <Analytics>, change the value to true, replacing (--inplace) the original file. The result is:

Applying api-manager.xml <Analytics><Enabled>true</Enabled></Analytics>
the previous value was: false 

Adding XML

The same technique can be used to add a new <datasource> at the end of a master-datasources.xml configuration file. We demonstrate here the addition of complex elements, simple elements and attributes. We concatenate several XMLSTARLET usages with the Unix pipe (|), so that the output of the previous command is input to the subsequent one. We use the trick to add a new complex element with an easy identifiable (unique) name, such as “<mydatasource>”, so that it can be manipulated easily with XMLSTARLET. Once finished with that element, we rename it to the correct tag (“<datasource>”):

#!/bin/sh
cd /opt/wso2/api-m/2.5.0/repository/conf/datasources
XMLFILE=master-datasources.xml
xmlstarlet ed --append //datasources/datasource[last\(\)] --type elem -n mydatasource $XMLFILE | \
xmlstarlet ed --subnode //mydatasource --type elem -n name -v SAMPLE_DATA_SOURCE | \
xmlstarlet ed --subnode //mydatasource --type elem -n jndiConfig | \
xmlstarlet ed --subnode //mydatasource/jndiConfig --type elem -n name -v SAMPLE_JNDI_CONFIG | \
xmlstarlet ed --subnode //mydatasource/jndiConfig --type elem -n properties | \
xmlstarlet ed --subnode //mydatasource/jndiConfig/properties --type elem -n myproperty | \
xmlstarlet ed --insert //mydatasource/jndiConfig/properties/myproperty --type attr -n name -v WSO2_SAMPLE_NAME | \
xmlstarlet ed --rename //mydatasource/jndiConfig/properties/myproperty -v property | \
xmlstarlet ed --subnode //mydatasource --type elem -n mydefinition | \
xmlstarlet ed --insert //mydatasource/mydefinition --type attr -n type -v RDBMS | \
xmlstarlet ed --subnode //mydatasource/mydefinition --type elem -n configuration | \
xmlstarlet ed --subnode //mydatasource/mydefinition/configuration --type elem -n url -v "jdbc:h2:repository/database/WSO2MB_DB;DB_CLOSE_ON_EXIT=FALSE;LOCK_TIMEOUT=60000" | \
xmlstarlet ed --subnode //mydatasource/mydefinition/configuration --type elem -n username -v wso2carbon | \
xmlstarlet ed --subnode //mydatasource/mydefinition/configuration --type elem -n password -v wso2carbon | \
xmlstarlet ed --subnode //mydatasource/mydefinition/configuration --type elem -n driverClassName -v "org.h2.Driver" | \
xmlstarlet ed --subnode //mydatasource/mydefinition/configuration --type elem -n maxActive -v 50 | \
xmlstarlet ed --subnode //mydatasource/mydefinition/configuration --type elem -n maxWait -v 60000 | \
xmlstarlet ed --subnode //mydatasource/mydefinition/configuration --type elem -n testOnBorrow -v true | \
xmlstarlet ed --subnode //mydatasource/mydefinition/configuration --type elem -n validationQuery -v "SELECT 1" | \
xmlstarlet ed --subnode //mydatasource/mydefinition/configuration --type elem -n validationInterval -v 30000 | \
xmlstarlet ed --subnode //mydatasource/mydefinition/configuration --type elem -n defaultAutoCommit -v false | \
xmlstarlet ed --rename //mydatasource/mydefinition -v definition | \
xmlstarlet ed --rename //mydatasource -v datasource >$XMLFILE

The resulting <datasource> has been added to the end of the <datasources> complex element:

    <datasource>
      <name>SAMPLE_DATA_SOURCE</name>
      <jndiConfig>
        <name>SAMPLE_JNDI_CONFIG</name>
        <properties>
          <property name="WSO2_SAMPLE_NAME"/>
        </properties>
      </jndiConfig>
      <definition type="RDBMS">
        <configuration>
          <url>jdbc:h2:repository/database/WSO2MB_DB;
                    DB_CLOSE_ON_EXIT=FALSE;
                    LOCK_TIMEOUT=60000
          </url>
          <username>wso2carbon</username>
          <password>wso2carbon</password>
          <driverClassName>org.h2.Driver</driverClassName>
          <maxActive>50</maxActive>
          <maxWait>60000</maxWait>
          <testOnBorrow>true</testOnBorrow>
          <validationQuery>SELECT 1</validationQuery>
          <validationInterval>30000</validationInterval>
          <defaultAutoCommit>false</defaultAutoCommit>
        </configuration>
      </definition>
    </datasource>

Installation

How do install the XMLSTARLET utility? The installation depends on the operating system you are using. Here are a couple of script fragments.

Installation instructions for CentOS:

  # xmlstarlet for CentOS
  sudo wget -q http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
  sudo rpm -Uvh epel-release*rpm
  sudo yum install -y xmlstarlet

For Ubuntu:

  # xmlstarlet for Ubuntu
  sudo apt install xmlstarlet

For other environments, please refer to the home page: http://xmlstar.sourceforge.net/

Conclusion

XMLSTARLET is a nifty tool that will make life easier. It is not the tool of choice for a serious deployment (use Ansible or Packer for that) but for simple changes to the configuration of a WSO2 product it is more than great.

Please let me know if you have any questions regarding this blog by leaving a comment in the comment section.

Care to share?
   
Picture of Luca Strozzi
Published October 25, 2018

Luca Strozzi

Luca Strozzi is an IT expert with experience since 1994 in enterprise service bus, service oriented architectures, messaging and event-based systems, real-time embedded applications and software realization. Within the design and development of complex systems, Luca covered roles such as Software Architect, Project Manager, Team Leader, Functional and Technical Designer, Coach and Trainer, Developer, Consultant.

Responses

Stay up to date with the latest articles