As businesses move AI projects into production, the cost of running them is becoming a concern. Choosing a powerful model is only part of the equation; organisations also need infrastructure that can support their workloads and provide access to current, reliable data.
In conversation with iTNews Asia, Thorsten Walther, MD, CXO Advisory Asia at MongoDB, shares why legacy architecture can drive up AI costs, how organisations can measure returns and what changes as they deploy AI agents at scale.
While many organisations initially looked to the latest AI models to solve business problems, Walther said the focus is now shifting towards how models use operational data and whether their capabilities match the task. “The model layer itself is now a commodity,” he said.
Using the most capable model for every task can increase costs without delivering corresponding benefits. Organisations need to select models based on the work they are expected to perform and the value they generate.
Legacy infrastructure presents a separate challenge. Systems built around structured data, batch processing and static workflows may struggle to support AI applications that need to work with live and unstructured information.
“AI doesn't create technical debt; it just sends you the invoice,” Walther said.
As AI exposes the limitations of existing systems, organisations may face additional integration work, duplicated data and manual processes that increase the cost of deployment.
Outdated data can undermine AI results
The age of the data feeding an AI application can directly affect its usefulness.
Walther cited online grocery shopping as an example. If a system recommends substitutes based on inventory processed the previous day, it could suggest products that are no longer available. Live inventory data would allow it to make more relevant recommendations.
Organisations should assess whether their data is current enough for the decisions their AI applications need to make. They also need infrastructure that can scale as AI agents generate and process more information.
Walther identified these as two important indicators of AI readiness: data freshness and the ability to scale horizontally. Without these capabilities, businesses risk investing in AI applications that cannot deliver their intended results.
Modernisation becomes harder to postpone
Updating legacy systems has traditionally involved lengthy projects, high costs and the risk of disruption. Walther said AI is changing the equation by helping with parts of the migration process.
He described a phased approach in which organisations test each step and leave core systems untouched until the replacement path works.
The board question has flipped. It used to be, can we afford to modernise? Now it's more on can we afford to keep waiting.
- Thorsten Walther, MD, CXO Advisory Asia at MongoDB
However, modernisation still needs a business case. Walther said organisations are becoming more selective about AI projects, concentrating on use cases where the benefits can be demonstrated.
Revenue growth, cost reduction and return on investment are among the measures he recommended tracking. While some investments take longer to pay off, business leaders need a clear understanding of what they are funding and why.
Scaling agents brings governance into focus
For regulated industries such as banking and insurance, model selection also depends on where workloads can run. Walther said organisations must determine whether cloud-based models meet their requirements or whether they need open-weight models deployed in their own data centres.
He expects enterprises to move towards deployments involving thousands of agents, shifting attention towards how those agents interact, how their work is governed and how their outputs are checked.
Agents could, for example, collect policy details, accident photographs and police reports to help insurance claims officers assess a case. The agent prepares the information, while the employee makes the decision.
Walther said automation does not remove human responsibility for the outcome. “A human must be accountable for things the agent is doing,” he said.
He added that monitoring requirements may change as organisations gain confidence in their systems. Early deployments need closer checks, with the level of human oversight evolving as processes mature.
For businesses looking to scale AI, the challenge is to combine appropriate models with data infrastructure that can support them, while keeping costs measurable and accountability clear.
.jpeg&h=271&w=480&c=1&s=1)
