Companies are running into production of AI agents – and many of them will fail. But the reason for this is nothing to do with his AI model.

Two of two VB Transform 2025Industry leaders shared the hard-won lessons by deploying AI agents on a scale. A panel operated by Joan Chen, General Partner AT Foundation Capital, Sean Malhotra includes CTO Rocket companiesWhich uses agents in home ownership travels from hostage underwriting to customer chat; Shailesh Nalwadi, Head of Product SandbirdWhich creates agent customer service experiences for companies in many verticals; And These Vendors, SVP of AI Transformation CognateThe platform of which automates the customer experiences for large enterprise connectivity centers.

Their shared discovery: Companies that produce the first evaluation and orchestation infrastructure are successful, while people producing with powerful models fail on the scale.

,See all our transforms 2025 coverage,

ROI reality: beyond simple cost cuts

An important part of engineering AI agent Understanding the return (ROI) on investment for success. The deployment of the initial AI agent focuses on the reduction in cost. While it is a major component, enterprise leaders now report more complex ROI patterns that demand various technical architecture.

Cost reduction victory

Malhotra shared the most dramatic cost example from rocket companies. “We had an engineer [who] In about two days, the work was capable of building an ordinary agent, which was to handle a very high problem called ‘transfer tax calculation’ in the hostage part of the process. And that two -day effort saved us one million dollars per year, ”he said.

For Cognigy, Waanders stated that the cost per call is a major metric. He said that if AI agent Used to automate parts of those calls, it is possible to reduce the average handling time per call.

Revenue creation methods

Savings are one thing; Creating more revenue is another. Malhotra said that his team has seen improvement in conversion: As customers get answers to their questions rapidly and they have a good experience, they are converting to high rates.

Active revenue opportunity

Nalwadi highlighted completely new revenue capabilities through proactive outreach. His team enables active customer service, also customers know that they have any problem.

A food distribution example shows it perfectly. He said, “They already know that when an order is going to be late, and instead of harassing the customer and waiting to call them, he realized that there was an opportunity to get ahead of it,” he said.

Why AI agents break down in production

While there are concrete ROI opportunities for enterprises deploying agent AI, there are also some challenges in production and production.

Nalwadi identified the main technical failure: companies manufacture AI agents without evaluation infrastructure.

“Before you start making it too, you should have an eval infrastructure,” Nalwadi said. “We used to be all software engineers. No unit deploys for production without conducting tests. And I think a very simple way of thinking about Eval is that it is a unit test for your AI agent system.”

Traditional software test approaches do not work for AI agents. He said that it is not possible to predict all possible inputs or to write extensive testing cases for natural language interaction. Nalwadi team learned this through retail, food distribution and deployment of customer service in financial services. The standard quality assurance approach recalled cases of growth emerged in production.

AI Testing AI: New Quality Assurance paradigm

What should organizations do, given the complexity of AI testing? Waanders solved the problem of testing through simulation.

“We have a feature that we are releasing soon that is about following the possible conversation,” Vanders explained. “So it essentially tests AI agents AI agents.”

Testing is not just a conversation quality testing, it is a scale behavior analysis. Can it help to understand how an agent responds to angry customers? How does it handle many languages? What happens when customers use slang?

“The biggest challenge is that you don’t know what you don’t know,” Vanders said. “How does it react to anything that can come with someone?

The approach tests demographic variations, emotional stages and edge cases that human QA teams cannot cover mass covers.

Incoming complexity explosion

Current AI agents handle single tasks independently. Entrepreneurous leaders need to prepare for a different reality: hundreds of agents from each other learning organization.

The implications of the infrastructure are largely. When agents share data and cooperate, the failure mode multiplies rapidly. The traditional surveillance system cannot track these interactions.

Companies should now be architect for this complexity. The infrastructure for multi-agent systems costs significantly higher than that from the beginning to make retrofitting correctly.

Chen said, “If you proceed theoretically, one of them may have hundreds of them, and perhaps they are learning from each other,” Chen said. “The number of things that may occur can only explode. The complexity bursts.”