AI’s Billion Dollar Bottleneck

“The best and the brightest AI models have all passed a point of diminishing returns,” Berk said. “If you double the amount of required GPU power, the model only gets 5–10% better at answering your question.” 

The Power Problem Reshaping Computing

One of the challenges in building AI infrastructure is the amount of energy LLMs demand. The emerging solution is new, high-performance AI data centers built close to power sources, called “neoclouds.” 

But this creates a new set of problems that require government intervention, according to Karl Havard, Chief Commercial Officer of Nscale, a neocloud that claims to be the UK’s only full-stack sovereign AI infrastructure provider.

“Without [government involvement], you will have all sorts of chaos emerging,” Havard said. “People building up data centers wherever they want to, taking energy from the grid — which wouldn’t be great for any population. You then create this Wild West scenario where anybody who has money can start to build data centers and serve AI.”

This is why Havard is in favor of the UK Compute Roadmap, which aims to build an ecosystem of supercomputers around the British Isles. Everett Thompson, the Founder and Chief Executive of Las Vegas-based WiredRE, an independent AI data center development firm, also sees the roadmap as an opportunity. 

“This is a historic chance to modernize the UK’s energy and digital backbone, just as the National Grid did in the 1930s or civil nuclear did in the 1950s,” he said. 

“The difference now is that this transformation must happen through public–private collaboration, not central planning alone. The capital exists — in the hands of global tech firms racing to deploy AI — but it needs a clear, coordinated and confidence-inspiring framework to flow.”

AI is supposed to make decision-making faster and more efficient, but the infrastructure needed to support it demands the kind of long-term, coordinated planning that governments have struggled with for decades. Without that collaboration, we risk recreating the same fragmentation we’ve seen with previous technologies — a prime example being the cloud.  

The Cloud Bottleneck

Digital transformation experts Megan Starkey and Benjamin Hermann say there’s an even bigger problem than energy consumption: organizations and governments are buying shiny new technology without understanding how to integrate it into existing systems.

Hermann, Managing Director of German international IT consultancy Zoi, has seen the inner workings of many enterprise IT infrastructures firsthand. He believes some enterprises have taken a “lazy approach” to hybrid cloud deployment, moving only “the easy parts of their data” to the cloud while leaving their core data in legacy data centers. 

“That was fine for 10 years, but now with the generative AI wave, they can’t adopt the technology as quickly as their competitors,” he said.

Starkey agrees. She believes “close to 40% of cloud spend was wasted [over the past decade] because companies scrambled to rent GPUs before their software was even adapted for that environment,” she said. “What’s happening with AI now is almost identical.”

The Overlooked Infrastructure Problem

The infrastructure puzzle gets even more complicated when you consider where all this data should live. Organizations face two paths: keep their most sensitive information close to home or send it to external clouds.

AI and cybersecurity investor Alex Lanstein sees the future tilting toward the first option. He predicts that many organizations will eventually choose to process their most critical data on-premise, using free LLMs such as DeepSeek or Meta’s Llama, which can be downloaded to their own cloud.

But that approach requires significant technical expertise and infrastructure investment that not every organization can manage. There will still be plenty of data flowing to providers for processing.

Many organizations will face a fundamental challenge: how do you gather all your data — which may be scattered across various clouds worldwide — and feed it into an AI system that can actually use it?

David Flynn, the Chief Executive of US software firm Hammerspace, said this is the most overlooked problem facing AI infrastructure today. He put it bluntly: “AI is forcing the industry to do geographically dispersed work… You can have GPUs and frameworks, but if you can’t house the data and feed the data at pace, it’s not going to work.

“Think of the AI agents and models as a destination. Right now, we are using city streets to get the data to it. We need a superhighway.”

The Global Race That Misses the Real Challenge

Ask these experts about the UK’s Compute Roadmap announcement and you’ll get a straightforward assessment: it’s about building supercomputers and stockpiling GPUs for the future, driven by a global shortage. The more pressing question is whether this infrastructure will be used effectively.

“At least the UK is doing something,” Hermann said, pointing to the stark lack of data center capacity across Europe. Germany, for example, is currently two years behind the UK and five years behind the US in infrastructure development, he noted.

But Starkey recognizes the theme from previous technology waves. “In every case, it’s the same root pattern,” Starkey explained. “The US goes after speed and scale; the EU prioritizes governance and control; the UK goes for balance and splits the difference. But everywhere you look, the private sector keeps struggling to integrate and operationalize what it builds.”

The experts agree on one thing: success in AI won’t come solely from having the most powerful hardware. Whether it’s Berk’s warnings about diminishing returns, Flynn’s observations about data transport bottlenecks or Hermann’s critique of lazy cloud deployments, the message is consistent: for all the billions being spent on AI infrastructure, the industry may be missing the point entirely. As Starkey put it: “Different rhetoric, same outcome.”

Whether organizations learn from the mistakes made in cloud computing — or repeat them at scale — will determine who has an advantage in the AI race.