This writer argues that enterprises should design systems around flexibility, economics and openness rather than trying to forecast every technological shift
No one in this industry can confidently predict what AI will look like five years from now.
The next dominant model architecture remains uncertain… so does the eventual balance between training and inference, and how AI workloads will be distributed across enterprise data centers, cloud providers and endpoint devices.
Yet infrastructure leaders still have to make decisions and investments today that may form the IT foundation for their businesses for five to seven years.
That is the challenge. For many organizations, a major risk is not simply choosing the wrong technology today. It is building infrastructure that cannot adapt as workloads and requirements evolve.
Alexey Navolokin, General Manager (APAC),AMD
At the AI Infra Summit this week, I had argued that we need to rethink how AI infrastructure is designed. Rather than starting with a particular chip or fabric, the starting point should be a more fundamental question: How easily can the system adapt when requirements change?
Adaptability as a design principle
A few years ago, “AI infrastructure” largely meant large training runs creating relatively predictable, batch-shaped demand on a single large cluster. That workload still matters, but it increasingly coexists with inference, agentic AI and a more distributed set of workloads with very different requirements.
Inference is often always on, with particular sensitivity to latency and cost per request. Agentic AI can add further complexity because a single request may become a sequence of steps: retrieving and shaping inputs, calling tools, executing code, coordinating sub-agents and maintaining state beyond any individual request. That changes more than the amount of compute required; it can change the shape of the system.
Infrastructure designed primarily around large training clusters may not be well suited to these evolving patterns. The balance of compute, memory and networking may no longer match where the work is actually done. The key is to optimise the system, not only individual components.
Compute, memory, networking, storage, software, power and reliability need to work together. Scaling a familiar architecture is largely a budgeting exercise; absorbing a new class of workload is often an architectural and operational challenge.
Three properties that preserve choice
If requirements continue to change, the design goal is not to predict every winner. It is to preserve the ability to move to what comes next. In practice, that comes down to three auditable properties:
- Adaptable: Organisations need enough breadth in their infrastructure options that, when workloads shift, they can change configuration rather than necessarily change suppliers. Matching compute to the job, supported by a practical software foundation, makes those choices easier.
- Economical: This does not mean choosing the cheapest option. It means paying for the right tool for the job. Training has often been treated as a capital decision, while inference can be an ongoing operating cost. For agentic workloads, where one request may involve a chain of inference calls, cost per completed task can be a more useful measure than cost per token.
- Open: Open ecosystems can provide choice through broadly adopted standards, potentially reducing dependence on any single vendor’s product roadmap. Open interconnects and open networking can support that objective, although the degree of portability and operational maturity will vary by workload and software stack.
Where this leads us
Breadth of technology matters because AI requirements span a wide range: general-purpose compute for inference, tokenization and orchestration; smaller models and local AI; large-language-model inference and fine-tuning; and distributed training and inference at larger scale.
However, breadth alone is not enough. Without a coherent software story, a broad portfolio is simply a catalog. With one in place, it can offer greater flexibility — provided buyers assess compatibility, support, performance and migration effort for their own workloads.
Open standards can help preserve choice across suppliers. They do not eliminate integration work, nor do they guarantee that every workload will move easily between platforms. But they can give organisations more room to choose the components best suited to a given requirement.
Enterprises do not need to predict every turn over the next five years to be successful: they just need to be those with the flexibility to navigate the turn well when it arrives. What seems safe to assume is that AI will keep changing. Infrastructure should be designed with that assumption in mind.
