Leaders should design systems around flexibility, cost efficiency and open standards rather than trying to predict future AI technology winners: opinion
No one in this industry can tell you what AI will look like five years from now. No one knows the next dominant model architecture. No one knows where the balance between training and inference will settle. And no one knows how much AI will run in an enterprise data center, in a cloud provider’s data center or on a local device.
Yet, infrastructure leaders must still make decisions and investments today that may shape their organizations’ IT foundations for years. The greater risk may not be choosing the wrong technology today. It may be building infrastructure that cannot adapt as workloads evolve.
At a recent industry summit, I had argued that AI infrastructure should be designed around a fundamental question: How easily can the system adapt when requirements change?
Adaptability as a design principle
A few years ago, “AI infrastructure” largely meant large training runs creating predictable, batch-shaped demand on a single large cluster.
Today, that workload still matters. However, it now coexists with inference, agentic AI and an increasingly distributed set of workloads with very different requirements.
- Inference is often always on, and is sensitive to latency and cost per request.
- Agentic AI can compound that demand: one request may become a sequence of steps involving data retrieval, tool calls, code execution, coordination among multiple agents, and state that persists beyond a single request. That changes more than the amount of compute required. It changes the shape of the system.
Infrastructure optimized for yesterday’s training clusters may not simply be short of capacity. Its balance of compute, memory, networking and storage may no longer match where the work is actually done.
The key is to optimize the system, not only individual components. Compute, memory, networking, storage, software, power and reliability must work together.
Scaling a familiar architecture can be a budgeting exercise. Absorbing a new workload is an architectural and operational challenge.
Three properties that preserve choice
If requirements will keep changing, the goal is not to predict every technological winner. It is to preserve the ability to move when the market or workload changes.
In practice, that comes down to three properties:
- Adaptable: Infrastructure should allow organizations to match compute and other resources to the job at hand, rather than forcing every workload into the same architecture.
- Economical: This does not mean choosing the cheapest option. It means paying for the right tool for the job. Training may be a capital decision made periodically; inference can be an operating decision repeated at enormous scale. For agentic workloads, the more meaningful measure may be the cost of completing a task, rather than the cost of an individual token.
- Open: Broadly adopted standards can give customers more choice and reduce dependence on any single vendor’s product roadmap. Open interconnects and open networking can be particularly important here.
Build for change
Over the next five years, organizations may not be able to predict every turn. Those with the flexibility to make the turn when it arrives are well equipped to thrive and survive.
The following axioms still apply:
- ✓ Build adaptable — so the future, whatever shape it takes, still fits.
- ✓ Build economically — so you pay for the right tool rather than simply the biggest one.
- ✓ Build openly, so you preserve choice.
What we can reasonably expect is that AI will keep changing. Infrastructure should be designed with that assumption from the start.