AI 智能体走出沙盒，企业撞上治理恐慌按钮

Enterprise adoption of AI agents is moving beyond experimentation, but deployment at scale remains limited.

An Infosys-HFS Research study found that only 16% of organisations had deployed agentic AI at an enterprise level, while 60% said their most advanced agents were still performing rules-based tasks. Only 12% were comfortable giving agents broad access to sensitive enterprise data.

At the same time, some enterprises are already allowing agents to act inside production environments. A recent study by Cisco and Omdia found that 51% were running agentic AI that acts in production for network operations. The finding is specific to NetOps, but it illustrates the shift from co-pilots to agentic AI systems that can take operational action.

The transition from experimentation to a live business workflow brings a different level of accountability.

“In a pilot, you are testing what an agent can do. Once it enters a live business workflow, you have to be equally focused on what it should not do,” said Kalyan Kolachala, Managing Director and Head of AI, SAIGroup India, tells AIM.

That means defining permissions, setting boundaries, controlling data access, and testing agents against actual workflows, including exceptions and edge cases.

“Real-world workflows demand outcomes that are consistent, reliable, and repeatable, and that only becomes possible once the foundations are deliberately built,” comments Prakash Balasubramanian, Executive Vice President and Global Head, Engineering Practices and Delivery at AI-native software engineering company Ascendion.

The pace of adoption is giving way to mounting concerns over the governance gap.

An IBM Institute for Business Value study found that 70% said teams were deploying technology faster than IT could track it, while 77% said AI adoption was outpacing existing governance capabilities. Meanwhile, Deloitte's 2026 State of AI in the Enterprise study found that only 21% of companies reported having a mature model for governing autonomous AI agents.

The governance problem complicates when agents are deployed cross-functionally. Each system can have a separate purpose while still depending on common enterprise data, applications, and business rules.

“As enterprises move from isolated AI pilots to deploying fleets of agents across functions such as finance, manufacturing, supply chain, and quality operations, the challenge shifts from building individual agents to governing how they work together,” remarks Rangarajan Srirangam, Senior Regional VP of Solution Engineering – India, Snowflake.

The issue is therefore not simply the number of agents an enterprise has, but whether it knows which agents are active, what they can access ,and who is responsible for them. Kolachala believes enterprises will increasingly need to treat agents as part of their technology estate, with each having a defined purpose, accountable owner, and controlled access to business systems and data.

The acceleration of agentic adoption could also change the roles of the people managing them. Balasubramanian believes technology and business roles now require engineers to take on responsibilities around agent design, context, and toolchains.

“An organisation becomes AI-native when every process is reimagined with AI as the operating entity performing most of the tasks and the human as the governing entity supervising and orchestrating the system. That changes careers rather than eliminating them,” he argues.

According to the IBM study, two-thirds of surveyed CIOs and CTOs said they were held accountable for AI systems they did not fully control. Responsibility can therefore remain with a business or technology leader even when the systems contributing to an outcome remain distributed.

Who Owns the Outcome?

Enterprises also have to determine where human responsibility remains. A repeatable task may require less intervention, while actions involving financial, regulatory, or other significant consequences may require approval or escalation.

“It is really a shift from task management to workflow management,” Kolachala adds.

The change is about what managers are responsible for: deciding which parts of a process can be delegated, monitoring the overall workflow performance, and identifying where human judgement is still required.

It also changes how enterprises measure the value of agents. Counting completed tasks doesn't show whether the business has improved. Companies need to compare performance against a baseline, whether that means cycle time, error rates, productivity, quality, or the cost of completing a workflow.

“On measurable returns, generating code is no longer the bottleneck. Coding, testing, and documentation are faster than ever, while the surrounding system still moves at the speed of meetings, manual handoffs, fragmented requirements, review queues, and periodic governance forums,” Balasubramanian observes.

Deloitte's research supports the broader point. While 25% of leaders said AI was already transforming their organisations, only 30% said they were redesigning key processes around AI.

Building the Control Layer

As agents become more autonomous, enterprises need greater observability into their actions, including detailed tracing, summarised rationale, and post-action audits.

“Enterprises increasingly require a control plane that provides centralised governance, policy enforcement, traceability, observability, and auditability across the entire agent ecosystem,” Srirangam emphasises.

That control layer does not necessarily replace conventional enterprise controls. Kolachala explains deterministic rules remain appropriate for access permissions, approval thresholds, and restrictions on sensitive systems, while observability provides visibility into the agent’s actions, the tools it uses, and how its behaviour changes over time.

Ascendion's AAVA platform follows a similar approach, according to Balasubramanian. He says the platform uses guardrails, approved-agent controls, a central repository, ownership mechanisms, and approval processes alongside audit logs and telemetry.

The same principle applies to agents that emerge outside formally approved systems. Organisations can maintain an inventory of approved agents, assign ownership, check for existing agents before creating new ones, and require approval before an agent enters production. Such mechanisms make experimentation visible and reduce the risk of an unapproved system quietly becoming part of a live workflow.

For enterprises, the transition from AI pilots to production is therefore not simply a question of deploying more capable agents. It requires changes to workflows, controls, measurement, and the roles of the people responsible for overseeing them.