Blog – Future Processing
Home Blog AI/ML AI implementation without IT modernisation? A high-risk path to underperformance
AI/ML

AI implementation without IT modernisation? A high-risk path to underperformance

An AI implementation that works in a pilot can stall once it meets legacy IT. In IBM’s 2025 CEO Study, only 25% of AI initiatives delivered the expected ROI. Here is how outdated systems limit AI scaling, reliability, cost and return, and where to modernise first.

Share on:

Table of Contents

Share on:

An AI implementation that worked well in a pilot on a clean data extract can stall once it has to run on live data from existing systems: batch jobs, siloed databases and brittle point-to-point integrations. The AI model is the same one that passed the pilot, so the cause lies in the IT around it.

Below, we look at how legacy IT limits AI implementation: how far AI solutions scale, how reliable their outputs are, what they cost to run and what return they deliver against business objectives. Then we cover where to prioritise modernisation for the fastest value.

Why does skipping IT modernisation sink AI implementation ROI?

Skipping IT modernisation limits AI capabilities, because legacy systems and outdated technologies lack the speed, scalability and integration that AI algorithms, data pipelines and current AI tools need.

Organisations then meet poor data quality, bottlenecks, unreliable outputs and inflated costs, which eat into the return on investment AI was meant to deliver. Efficient systems and high-quality data are the foundation AI runs on.

What do the numbers say about AI on legacy IT?

Most AI initiatives still fall short of what was expected of them. In IBM’s 2025 CEO Study, run by the IBM Institute for Business Value with Oxford Economics among 2,000 CEOs in 33 countries, only 25% of AI initiatives had delivered the expected ROI over the last few years, and 16% had been scaled across the enterprise.

Half of the same CEOs said fast investment had left them with disconnected, piecemeal technology. Zapier’s 2025 survey of US enterprise leaders found the same problem at the level of integration: 78% struggle to connect AI tools with existing systems.

Data is the other half of the gap. A Gartner survey of 1,203 data management leaders, run in July 2024, found that 63% of organisations do not have, or are unsure they have, the right data management practices for AI, and Gartner predicts that through 2026 organisations will abandon 60% of AI projects unsupported by AI-ready data.

What counts as 'IT modernisation' for AI?

IT modernisation for AI means building a digital backbone that lets AI systems scale, run reliably and support business operations every day. Upgraded infrastructure is one part of it; the data platforms and delivery practices below are the others.

Enterprise data platforms

DevOps practices bring automation, collaboration and short release cycles, so AI systems can change as business needs change. They extend to MLOps (machine learning operations), which standardises how models are deployed, monitored and retrained. That continuous improvement keeps models accurate and cuts the errors that slip in through manual steps.

DevOps practices

DevOps practices introduce automation, collaboration, and rapid iteration, ensuring AI systems evolve in line with business needs and modern technologies. It also covers MLOps (Machine Learning Operations) which standardise model deployment, monitoring, and continuous improvement, maintaining accuracy, reliability, and reducing the risk of errors from human oversight.

Modernisation also prepares the organisation itself: once AI is part of daily work, teams and processes run in a different operational reality.

Legacy systems stack problem: latency, data silos and brittle integrations

Legacy IT stacks slow AI down at every step. Outdated servers and rigid databases add latency, so model training, analytics and real-time decisions take longer. Data silos keep AI away from a unified enterprise dataset, and outputs built on part of the data come out skewed or unreliable.

Brittle integrations mean every new AI tool or update can break existing workflows, and each break brings costly fixes and delays. Until these issues are fixed, AI systems are hard to scale, and both agility and ROI suffer.

Stuck between an AI pilot and production?

Find out which parts of your IT need modernising before your first AI project can scale.

Where does legacy IT hit AI implementation hardest?

Legacy IT hits AI implementation at the points that decide whether an AI project pays off: how far it scales, how reliable its outputs are, what it costs to run and what it returns.

Scaling: why AI pilots stall before production

A pilot runs on a curated extract that a data science team prepared by hand, so it never meets the batch jobs and point-to-point integrations of the production estate. Scaling means connecting the AI model to existing data in every system it needs, and that is where the cost of each integration shows.

Cisco’s 2025 AI Readiness Index puts figures on the gap. 54% of companies said their infrastructure cannot scale for AI workloads, and 18% had moved AI use cases into production, against 77% of the Pacesetters, the roughly 13% of companies Cisco rates as fully prepared.

Daria Polończyk, Head of Delivery at Future Processing, sees the same pattern in what clients ask for:

"More and more clients come to us with a working proof of concept of their own. It is the new way of saying what you want. Somebody still has to take it to production safely, and that is what they now ask us about."

Daria Polończyk
Head of Delivery

Reliability: model performance depends on input data

An AI model is only as reliable as its training data and input data, and legacy systems supply both late, duplicated or inconsistent between silos. Duplicated records also distort the data patterns a model learns from.

A supplier’s address updated in the ERP but not in the procurement system until the nightly batch gives two answers to the same question, and neither the model nor any data analysis built on it can tell which one is right.

Generative AI adds variability of its own: the same prompt can produce different outputs, and a model can hallucinate, stating a wrong answer with full confidence and no explanation.

On top of unreliable data, that makes errors hard to trace. Continuous monitoring of model performance, and data lineage tracking to follow a wrong answer back to its source, belong in the AI implementation strategy from the first release.

Cost: what legacy IT adds to every AI project

Legacy IT makes an AI project pay for the AI solution and again for the integration work and fixes around it.

Running costs grow as well. Generative AI services built on large language models are billed per token or per call, so a workflow that has to pass whole documents to the model, because the legacy system behind it cannot return only the relevant fields, costs more with every request.

The upfront bill is high too. Gartner estimated in 2024 that GenAI deployments aimed at business model innovation cost between US$5 million and US$20 million. For smaller businesses, any modernisation the AI needs comes on top of a bill of that size.

ROI: why the returns stay out of reach

The return on AI depends on whole business processes moving faster, and legacy IT, where it remains, sits in the middle of them. When an AI assistant drafts a reply in seconds but the order still waits for an overnight batch in the ERP, process automation stops at the legacy system and the business outcome does not change.

Dominik Samociuk, Head of Security & Compliance at Future Processing, saw this during our own AI transformation:

"When AI saves one person time, the next person in the process still expects the data two hours later. That is why adding AI to one step is not enough: the whole process has to be redesigned around it."

Dominik Samociuk
Head of Security & Compliance at Future Processing

The hidden cost of technical debt on AI performance and reliability

Technical debt has a direct cost: McKinsey’s 2023 analysis of technical debt, by Aamer Baig, Sven Blumberg, Arun Gundurao and Basel Kayyali, found that companies pay an additional 10 to 20% on top of a project’s cost to address it. Beyond that, it has strategic consequences: AI loses credibility, momentum stalls and organisations risk falling behind competitors in business transformation.

That is why technical debt is worth reducing early. AI systems can then scale reliably, keep output quality high and integrate with legacy systems and existing processes, and development teams spend their time on new work instead of firefighting.

Reducing debt early also makes systems more stable, data cleaner and governance stronger, which protects the return on investment from AI initiatives. Once AI tools write code and pipelines of their own, they add to that debt as well, and our article on AI technical debt explains how to stop it scaling.

Building the right ecosystem for artificial intelligence

A common question from business leaders is which AI they should implement. The more useful question is how to build an ecosystem in which AI can work for the business.

Preparing for AI starts with the foundations. Before introducing AI agents, organisations need to put their house in order: organise data, processes and infrastructure, so that agents do not multiply existing mistakes.

AI technology is not deterministic, so scaling it on weak foundations multiplies the variation in its outputs. Solid foundations also keep integration manageable when departments adopt AI at different speeds.

Measurement needs the same attention, through a total metrics approach. Traditional KPIs stop fitting once AI replaces or changes human tasks, so organisations need low-level metrics for process readiness, data literacy and integration health alongside business outcomes.

Tracking both progress and safeguards, and agreeing in advance what to change if results fall short, keeps AI adoption on course through early failures.

Read more about artificial intelligence on our blog:

Data readiness first: governance, quality and lineage for trustworthy AI adoption

AI needs high-quality, well-governed data. To get it, organisations need to:

  • set up data governance frameworks that define ownership, security and compliance;
  • check data quality and fix the duplicates, inconsistencies and gaps that distort model outputs;
  • track data lineage: where data comes from, how it is transformed and where it flows, so that every output can be traced back to its source.

Without these foundations, even advanced AI systems can produce unreliable or biased results, and people stop trusting and using them.

Security, privacy and compliance: model risk management in regulated sectors

In regulated industries such as finance, healthcare and government, AI adoption depends on security, privacy and compliance. Every other industry still has sensitive data to protect and privacy rules to follow.

Organisations also need model risk management: monitoring for bias, drift or unintended behaviour that could lead to regulatory penalties or reputational damage. Controls built into the AI lifecycle let teams try new uses of AI while handling the ethical and legal implications of each one.

Benefits of AI in digital transformation
Benefits of AI in digital transformation

How can you check whether your IT is ready for AI implementation?

Start from the business problem rather than the technology, and check readiness against the one AI project your AI strategy puts first, because an audit of the whole estate takes longer and answers questions nobody has asked yet.

The questions below cover data, integration, infrastructure, security and measurement.

Data: can the AI model reach the data it needs?

List the existing data the use case depends on, where it lives and how fresh it is. If a source updates once a night and the use case needs answers within minutes, closing that gap is a modernisation task, and it belongs in the plan before AI development starts.

Integration: does the system offer APIs or only batch files?

Count the systems the AI solution has to read from and write to, and how each one connects. A system that only exchanges batch files, or has no documented API, adds integration work and a point of failure to every AI service built on top of it, and slows solution development for each new use case.

Infrastructure: can it carry production load?

A pilot in a sandbox says little about production. Check whether the cloud infrastructure or on-premise servers can handle the expected volume of requests, and what that volume will cost each month.

Security: are access controls ready for AI?

An AI assistant that searches company documents will show people anything the underlying permissions let it reach. Review access controls, data security and the data protection laws that apply, such as GDPR, before the assistant goes live, because over-shared files become visible on the day it does.

Measurement: is there a baseline?

Set a clear, measurable objective for the AI project and record how long the process takes and what it costs today. Without that baseline, nobody can show the model’s effectiveness or measurable business outcomes later, and Cisco’s 2025 index found that only 32% of companies have a process to measure the success or failure of AI investments.

Future Processing’s AI Readiness Assessment covers the same ground across four domains (data and infrastructure, security and governance, legal and compliance, organisational readiness). First decision-ready findings come within four to eight weeks, with a go or no-go recommendation for the chosen initiative.

Operating model shift: platform teams, product thinking and FinOps for AI

Scaling AI also changes the operating model: how teams, processes and budgets are organised around AI.

  • Platform teams provide shared, reusable infrastructure and services, so business units can deliver AI faster. Centralised capabilities reduce duplication and let product teams build AI systems instead of rebuilding the underlying stack each time.
  • Product thinking treats AI initiatives as products with outcomes to reach, instead of one-off projects. Teams keep iterating, so the AI stays aligned with business goals as development cycles and priorities change.
  • FinOps practices bring financial accountability to cloud, AI and platform spending and tie costs to business value. They cover legacy systems, AI tools and the high-performance infrastructure that scaling AI capabilities requires.

Together, these changes keep complexity, cost and integration with existing processes under control as AI grows. That is how AI moves from a pilot experiment to a dependable part of business transformation.

For generative AI services billed per token, FinOps for AI applies the same cost discipline to model usage.

Buy, build or partner? Where to prioritise modernisation for fastest value

Modernising for AI comes down to a buy, build or partner decision for each part of the stack.

Quick wins, such as adopting cloud services or pre-built AI tools, deliver value fast and show the business that AI works. Strategic bets, such as bespoke data platforms, custom MLOps pipelines or a private AI implementation, build the foundation for a long-term advantage. Balancing the two lets early results fund the larger initiatives.

AI multiplies whatever it runs on, strengths and weaknesses alike. Without modernised IT, clean data and aligned operations, it magnifies existing inefficiencies.

Investing in IT modernisation, governance, security and the operating model prepares organisations to adopt AI models and to run their business on them.

Planning AI on top of legacy systems?

We will check your data, integrations and infrastructure against the AI project you want to scale first, and tell you what to modernise.

AI implementation and IT modernisation: frequently asked questions

Can you implement AI without modernising legacy systems?

You can run a pilot on legacy IT, but to successfully implement AI across business operations you need data and integrations the AI can reach in production. The pilot proves the AI model works; modernisation decides whether the business can use it every day.

Data silos, batch-only integrations, infrastructure that cannot scale and accumulated technical debt all trace back to legacy IT. Cultural and skills challenges, such as training employees to use the tools, slow AI projects too, and they need a plan of their own, because new systems do not fix them.

Modernisation comes after choosing the use case and before scaling it. Assess the data, integrations and infrastructure the first AI project needs, modernise those parts, then implement AI solutions on top, so the modernisation budget stays tied to one business problem.

Not always. A legacy system can be wrapped with APIs and its data moved into a governed data platform while the core system keeps running; replacing it makes sense once keeping it costs more than changing it.

A cross-functional team: business teams who own the outcome, technical teams who own the systems, an AI team of data scientists and machine learning engineers who build AI solutions, security and compliance specialists who own data protection, and the people who will use the tool every day, who should test it before rollout.

Define success metrics before the pilot and compare them with measurable outcomes after it: time and cost per process, model performance in production, and how long a new AI use case takes to move from pilot to production.

Let’s talk

Contact us and transform your business with our comprehensive services.