Why most AI fails to scale, and how strong data foundations will change that

Why building data foundations is critical to scaling AI and delivering measurable value

AI investment is accelerating, and expectations for measurable value are rising just as fast. Yet many technology leaders now find themselves defending the spend and struggling to explain why impact is harder to prove than the promise suggested.

We’re seeing AI tied to transformation targets. But despite the hype, most organisations still can’t scale beyond pilots. In fact, 95% of gen AI pilots fail to demonstrate a return on investment.¹ Too often, promising use cases stall, value remains elusive and capability erodes. The uncomfortable truth? AI isn’t the constraint, your data is.

When AI accelerates, and when it disrupts

Our Transformation Index research shows a clear divide. In lagging organisations, AI reshapes the technology strategy itself. Teams react to tools rather than designing around outcomes. In leading organisations, we found that AI is doing something very different. It accelerates transformation that’s already underway. It builds on stable foundations and compounds momentum, rather than disrupting it. The difference is the way your organisation is structured.

What do we really mean by data foundations?

Data foundations aren’t a compliance exercise or hygiene task. They’re the structural conditions that determine whether AI can scale in your organisation. At the core, there’s three key parts:

1. Data governance and quality – the integrity layer

Ownership, standards, lineage and active quality management determine whether outputs can be trusted. When definitions vary, lineage is unclear or upstream changes go unmanaged, those weaknesses flow directly into model performance and into decision-making.

2. Data infrastructure and architecture – the environment layer

Where data is stored, processed and integrated matters. Without intentional design across storage tiers, compute patterns, cost controls and integration models, costs escalate quickly and pipelines become fragile. What works in development fails under real-world demand.

3. Data driven measurement – the ‘enterprise truth’ layer

Without a consistent way to link operational performance to financial outcomes, value remains anecdotal. Teams report different numbers, decisions slow and investment becomes difficult to defend or adjust.

The impact of weak data foundations

Most will recognise the familiar challenges:

Automation creates bottlenecks as teams manually correct outputs
Storage and compute spend snowballs without a clear value story
Leaders debate different versions of the truth, delaying decisions
Solutions that worked in testing stumble under real-world load

This is where organisations start to over-promise and under-deliver. Leaders push for innovation, while the foundations beneath them crack under pressure.

The dilemma is: how do you strengthen your data foundations, without risking loss of momentum, credibility and organisational appetite?

How to rebuild data foundations, without hitting pause

The good news is you don’t need to stop your AI agenda to fix this. But you do need to change how you approach data foundations.

Focus on the data that actually matters, not everything

Resist the urge to fix everything. Many organisations attempt broad data-cleansing programmes and get overwhelmed. Instead, take a sharper approach.

Identify the handful of critical data paths that underpin priority AI use cases.
Prioritise quality and resilience along those paths first.
Embed quality checks directly into workflows, not as downstream controls.
Use automated profiling to surface anomalies early, before they affect outputs.
Formalise critical data as products

Shared data that anyone can change is a risk. To reduce fragility:

Treat high-value datasets as data products, not shared assets.
Assign clear ownership and accountability.
Define explicit data contracts covering quality, availability and change
Prevent breakages up front, rather than fixing issues downstream.
Shift accountability to the source

Central data teams can’t police quality across your organisation. Instead:

Make teams closest to data creation accountable for its quality.
Clarify expectations around stewardship, not just consumption.
Resolve issues where they originate from, not where they surface.

Infrastructure is where scale is quietly lost

Most organisations block their own ability to scale AI, often without realising it. The answer isn’t in ripping everything out or buying more tech. It’s about exposing where complexity actually sits and removing friction in a simpler step-by-step way.

Leading organisations:

Introduce discipline in how capacity is requested, allocated and monitored
Develop a unified view of demand rather than overprovisioning “just in case”
Ruthlessly remove duplication along critical data paths

Turn AI into a decision-making asset

If leaders see how AI connects operational reality to business outcomes, the organisation will quietly conclude it didn’t work.

Good measurement is a connected system. It tells a single story across three layers:

1. Leading indicators that show system health and adoption

Usage patterns
Latency
Output quality

2. Operational outcomes that link AI to workflow performance

Time saved
Cost per credit
Accuracy gains

3. Business impact that shows where value is created, and where it isn’t

ROI
Cost avoidance
Revenue lift

Why this matters now

CIOs are operating in a paradox of infinite expectation and finite capacity.

Over the next two years, the defining technology capability won’t be choosing the right tools. It will be the ability to stabilise, systemise and scale what sits underneath.

AI will keep advancing. The question is, are your data foundations ready to support it?

¹ MIT – The Gen AI Divide: State of AI in Business 2025

Was this helpful?

Cassie Clark

Senior consultant

Contact

George Burton