fb
Your challenge

Data Integration

Data Integration is one of the biggest challenges for many organizations nowadays. It all comes down to the complexity of data exchange and the multitude of factors that need to be considered for an effective and efficient setup.

IMG 4583 600x350 1

Non-aligned information systems

Every company uses several information systems to host its business operations. In the ideal world, these systems are aligned by Data Integration. In most cases, these information systems, like a CMS, ERP, and a webshop, are not optimally aligned with one another. For instance, systems are not completely standardized, or not all standardized at the same level, or information connections aren’t even in place between systems.

In order to align these information systems, the choice to use a Canonical Data Model (CDM) is commonplace. This results in an immediate question regarding to what extent and to what level of detail a CDM needs to be defined. An overly detailed CDM isn’t manageable, but a CDM that is too abstract and superficial isn’t practical either. The real challenge lies in creating a balanced, well thought-through CDM, in which exactly the right amount of detail is included.

Technical Incompatibility

As soon as a well-balanced CDM is defined, regardless of the level of abstractness and depth of the defined details, the immediate next question will be: how would you like your information systems to exchange data with one another? There are dozens of hurdles to overcome, each of them comprising a challenge for your IT-department.

Import and export formats are an example of such a challenge. Major differences can be observed between cases concerning modern information systems supporting RestServices, compared with somewhat old-fashioned systems which purely rely on SOAP formats or systems that only recognize one file interface.

Beyond the differences in format exchange, differences in versions of information systems could form the next hurdle. One example is found in the consequences of updates and upgrades for information systems, as well as the ongoing improvements within organizations leading to replacement of certain systems. After an update, upgrade, or replacement of a system, the new set-up needs to remain compatible with the way data management is done within the IT landscape; which presents the next challenge.

Real Time or Batch Oriented

Another challenge IT-Managers face is the gap they need to bridge between real-time and batch-oriented data processing. Say an organization requests real time information, regardless of the fact that the underlying information system only supports batch oriented data processing, the gap will need to be bridged from a technical perspective.

To make sure that a real-time information system is able to exchange data with a batch-oriented information system, a mechanism to guarantee the bidirectional data exchange is needed. Of course, there are technical solutions on the market, like message queuing of event streams via Kafka technology, but they inevitably lead to the next challenge of finding a suitable integration pattern to implement these solutions.

1D2A1654

Data Integration with a Single or Multiple Source of Truth 

In the past, we learned to work towards a single source of truth by using data warehouses. It turned out to be more complex in practice; making this classical approach of data warehouses not the optimal solution for every situation. We will have to deal with the fact that there may exist more parallel truths in different information systems within an organization. Therefore, it is a challenge to make each of these parallel sources of truth available at the right time to an information system, or person, within the organization looking for the respective information.

The most effective solution to this challenge is an integration pattern based on APIs, underlying Micro Services and, for example, Kafka Streams. Within this pattern, several information sources will need to be queried in real time, and received results will have to be consolidated in a so-called composite service. By use of this architectural principle, each underlying information system exists in its own right, and will not undermine the integrity of any higher source of truth.

More on Data Integration?

Are you interested in learning more about Data Integration and the solutions we offer? Schedule a consultancy session with one of our integration experts.

Plan a meeting
0K3A5966

How can we help you?

WSO2 Installation Configuration

Enterprise and Solution Architecture

A blueprint is always the start of any good build. And with Yenlo’s Enterprise and Solution Architecture services you have clarity on the shape of your technology stack and the best fit-for-purpose IT solutions to go with.
Read more
Healthcare icon

Consultancy & Health Checks

We help you plan, assess and shape your digital transformation.
Read more
WSO2 Migration Services

Migration Services

The smoothest way to transfer from on-premise to hybrid and cloud-first integration. You decide the pace and we’ll offer the right technology.
Read more
yenlo-image-8261

Migration Services

The smoothest way to transfer from on-premise to hybrid and cloud-first integration.
Read more
yenlo-image-8261

Installation & Configuration

We can help you implement and configure the right technology based on your integration needs.
Read more
WSO2 Software Development

Software Development

Our 180+ industry experts are here to guide you and optimize the implementation of your middleware technology.
Read more
WSO2 API Security

API Security via 42Crunch

To protect your APIs in the best way possible, Yenlo offers you additional API security with 42Crunch.
Read more
WSO2 Managed Services Support

Managed Services & Support

With Yenlo Managed Services & Support you have access to the best support in the industry worldwide.
Read more
WSO2 Training Certification

Training & Certification

We provide customized and private training courses
Read more

Get in touch, contact us!

Get in touch and contact us by submitting the contact form below.
Article notification
Privacy Policy*
This field is for validation purposes and should be left unchanged.

Full API lifecycle Management Selection Guide

WHITEPAPER

smartmockups l0qqucke