Despite the clear benefits of data-driven demand forecasting, many supply chain partners struggle to implement these systems.
In this blog, we’ll explore why organizations in supply chains must prioritize data driven demand forecasting.
As we’ll see, by letting advanced algorithms generate better forecasts, you can build leaner, more agile, and more profitable supply chains and all stakeholders do better as a result.
What is data-driven demand forecasting?
Data-driven demand forecasting uses both real-time and historical data from multiple sources to create highly accurate demand forecasts for a range of horizons.
If we compare it to traditional manual forecasting, data-driven demand forecasting is like an advanced GPS guiding you to your destination, while traditional forecasting is like trying to drive a car while only looking in the rear-view mirror and a map.
Good demand forecasting is essential for the health of any supply chain. The problem is that manual demand forecasting is often far from ‘good’. In fact, in 2023 poor demand forecasting resulted in additional costs of US$1.77 trillion globally.
And these are just the costs attributed to stockouts and overstock. Inaccuracies in supply chain planning also undermine trust, increase storage and transport costs, and affect key metrics like inventory turns, carrying cost, and cash flow use.
Unlike manual demand forecasting, data-driven demand forecasting uses advanced algorithms and AI to analyze various data sources to generate forecasts. These are more accurate, can adapt in real-time, and enable instantaneous coordination with supply chain partners.
To gain these benefits however, supply chain partners must integrate a wide variety of data sources. Algorithms also need access to good quality data if they are to generate reliable forecasts. This can be a significant challenge for organizations in a supply chain, as they must find a solution that can reliably combine data sources from all their legacy systems, ERPs, data lakes, real-time streams, and chain partners.
Whether an organization chooses individual APIs or an Integration Platform as a Service (iPaaS), the solution must be able to both exchange the right data and ensure its quality as it moves between different systems. However, when this is done correctly, the results are worth the effort.
Unpacking the advantages of data-driven demand forecasting
For decades, supply chain management has relied on manual forecasts that combine historical sales data with industry expertise and a pinch of ‘educated guesswork’.
Today, more supply chains are discovering that real-time insights, predictive models, and AI forecasting can deliver better results across the entire chain.
Data-driven demand forecasting has many benefits, including:
No more guesswork – Industry experience and historical sales data still have value, but these are now just one part of the picture. By using advanced data models that combine these with multiple data sources, you can take the human guesswork out of the equation.
Less time on spreadsheets – Instead of spending hours poring over spreadsheets, data-driven forecasting can generate more accurate demand forecasts for both long- and short-term perspectives. This can lead to substantial IT efficiency gains. As manual data entry is reduced, this is less error prone as well.
Holistic IT architecture – Data-driven forecasting requires a highly integrated IT architecture. By making this transition, the organization can reduce costs associated with manual integrations, consolidate their IT landscape, and achieve end-to-end visibility across all systems.
More accurate stock levels – Stock distortion is a problem for every stakeholder in a supply chain. With real-time data streams leading to increased accuracy, you can distribute stock more effectively across the chain. New synergies become possible for logistics hubs, warehouses, and suppliers.
Just in time – Demand forecasting made by advanced algorithms enables highly efficient business models such as ‘Just in Time’ (JIT) delivery. You can reduce inventory at every node and increase sell-through rates and profitability. Logistics planning is also more effective.
Integrating demand trends – Integrating a wide range of internal and external data sources enables a wider strategy that uses AI to anticipate customer demands by tracking trends on the macro and micro level. This can help accommodate surges in demand, as well as inform ideas for new products and services.
Avoid disruption – We’ve seen significant and unpredictable disruptions in recent years. These will continue to happen, but by incorporating real-time insights into your supply chain planning you can respond more quickly. For example, if GPS tracking detects that a shipment of raw supplies is stuck at one port, you can re-route or re-source from an alternative supplier. Or, if a sudden event means demand for a product surges or disappears, you can send a signal through the entire chain to adjust production and stock movements.
Contextual data is the key to successful predictive models
Each purchase or order is driven by a specific context. Contextual data is the basis for any forecasting model, as it helps to predict buying behaviors. Obvious examples include the increased demand for World Cup merchandise during a World Cup tournament, or for electric fans during hot weather.
Just as traditional demand forecasting uses contextual data, so do predictive models that leverage AI and advanced algorithms. These can predict trends that are much less obvious, but without access to the right contextual data the predictive models will fail to perform.
Predictive models must be able to combine your historical data with real-time information across multiple contexts. And this means you must make all the right contextual data available, so your AI forecasting can generate accurate results.
The difficulty however, is that connecting to these sources of data and consolidating them into a cohesive architecture can be very challenging. Your data may come from a range of sources including CRMs, legacy systems, ERPs, data lakes, third-parties, and real-time data streams.
To give algorithms access to all these sources of contextual data requires seamless connections that can instantly exchange and transform data as needed, regardless of the source.
Which data is needed for predictive algorithms?
Tracking real-time contextual data like the weather or sporting events can certainly assist with short-term planning. However, medium- and long-term forecasting also benefit from rich contextual data when it is fed into the right data analysis model.
For example, contextual data about trade agreements, population data, demographics, tariffs, supply costs, fuel prices, national holidays, new legal requirements, and other external contexts are all relevant to supply chain planning.
Equally, your internal contexts are also a crucial factor. You need to have clear insights from the top of the supply chain all the way downstream to ensure that capacity is efficiently utilized for production, storage, logistics, and workforce. This means that IT leaders must take the right steps to ensure that internal data is available when needed, from their WMS to the CRM and everything in between.
There are three categories of data you need to integrate into your supply chain forecasting model:
Internal data – This gives insights over internal variables like inventory, production, workforce and finances.
External data – This adds context from the wider market, such as transportation networks or suppliers and macro changes like geopolitics or weather. This category includes data from supply chain partners and stakeholders.
Real-time data – This uses data streams from devices, sensors, RFID, IoT, and GPS to enable real-time tracking of upstream and downstream inventories, shipments, and warehouse locations. Real-time data can be both internal and external.
Why most organizations fail to implement demand forecasting
The future supply chain will rely heavily on digital capabilities to improve predictive and operational performance. Organizations that can use advanced algorithms and AI will gain a huge advantage in the market.
According to McKinsey, AI-driven demand forecasting can reduce inventory by between 20% and 30%, increase warehouse capacity, and reduce workforce costs. But this capability requires an open IT infrastructure that allows secure and reliable data sharing between different systems and partners.
Several reasons make it hard to implement an advanced data-driven supply chain:
Partner alignment – Many organizations that operate within a supply chain tend to focus on their own business priorities (efficiency) instead of aligning with their partners to generate the best results for the whole chain (effectiveness). Aligning operations can be difficult, especially for downstream partners who are still enticed by volume-based price breaks and other ‘push’ behaviors that encourage excess inventory. Inertia can be hard to overcome, but someone must take the first step before others can follow.
IT complexity – Organizations may have a highly complex, fragmented IT landscape that includes legacy applications, databases, and highly customized ERPs. Integrating legacy systems presents a special challenge, and the right expertise is often scarce.
Perceived difficulty – Legacy systems are often too inflexible to accommodate new capabilities at speed. Organizations rely on their ERPs and have often invested a lot of time and money on customization. This can lead to a false impression that integrating more systems will be just as hard, when it actually requires a different approach.
Lack of vision – Without leadership, innovative transformation projects cannot move forward.
Data silos – This is the biggest reason by far, because disconnected systems cannot share data. It’s also a complex problem as data may be unstructured, held in spreadsheets or other documents, and systems may lack the ability to ‘talk to’ new applications or AI tools.
Breaking through data silos to unlock the power of data
Without taking steps to unify your fragmented IT landscape, data silos will present a significant obstacle for advanced demand forecasting and supply chain planning.
When important contextual information is locked away in a scattered landscape of databases, spreadsheets, and legacy systems, it’s impossible to achieve the end-to-end visibility needed to orchestrate your supply chain.
Conversely, when IT leaders implement a forward-looking integration strategy, they can create a unified view of their entire enterprise. Through extensive integration, the true potential value of data can be unleashed, and the benefits can be very tangible.
Here are some examples of what’s possible when organizations take the bold move to break down their silos and unleash the full power of their data across the supply chain:
Sunsweet Growers implemented a supply chain planning solution that enabled highly accurate algorithmic forecasting. The result was a 20% increase in forecast accuracy, and a 30% reduction in goods spoilage.
Johnson & Johnson gained 15% increased forecast accuracy, which relied on improved data flows between partners and 25 algorithms leveraging a range of statistical models.
Albert Heijn harvests real-time sales data to accurately predict demand on multiple horizons and drive a ‘Just in Time’ replenishment system. It orchestrates deliveries from suppliers, co-manufacturers, and multiple warehouses. This system has improved accuracy by 5%, reduced stock shrinkage by 50%, and reduced deliveries from an average of 60 per week to just 20 per week.
Procter & Gamble uses AI-powered demand forecasting which analyzes historical and real-time global contextual data to accurately predict demand on the micro level in different regions, provision inventory accordingly, and optimize logistics.
Free your data: next steps
The first step is to start internal discussions to identify which areas of your operations can gain the most from enhanced predictive capabilities. This will depend a lot on your position in the chain and relationship with supply chain partners, but it’s important to start with a clear-cut use case.
Next, you should analyze which data is needed and create a vision for how to make this data available. It is likely that many current processes will need to change, so change management will be a factor too.
Then, you need to determine which integration technology is most appropriate for your existing technology stack. There a several options, including custom-built connections and Application Programming Interfaces (APIs) or Enterprise Service Buses (ESBs), or API management platforms like WSO2 and Integration Platform as a Service (iPaaS) platforms like Boomi.
You will need a variety of different connectors to enable reliable data exchanges, so you must consider whether you want to build and maintain these yourself or use an iPaas which already has all the connectors you need.
Good advice can go a long way with a project like this and help avoid common issues that add cost and slow down delivery. Yenlo has significant experience in helping organizations to break down their data silos to create a modern IT architectures that’s more agile and scalable. So, if you want to talk about what might be possible, just get in touch.
