Microsoft expands the Project Perception security system, launching dynamic permissions and Guardian Agents for enterprise agents

On October 19, Hayete Gallot, Corporate Vice President of Microsoft Security, published an article stating that advanced AI models, regardless of which vendor they come from, cannot be regarded as fully trustworthy systems. She emphasized that this is not to say that AI models are inherently malicious, but rather that after training, models tend to complete tasks as fully as possible, yet may not always be able to follow an enterprise's internal norms, risk judgments, and organizational rules.

Traditional software usually executes operations according to predefined logic, whereas the behavior of AI agents changes with the task context, input information, and operating environment. Even if an agent behaves normally at the start of a task, it may later take actions it should not have taken because it read a malicious webpage, misunderstood the user's intent, or was affected by a prompt injection attack.

More noteworthy is that enterprises often grant AI agents relatively high privileges, enabling them to access internal databases, invoke business systems, read customer information, and even participate in financial approvals and other critical processes. Once an agent makes an error, the scope of impact may far exceed that of an ordinary chatbot generating a wrong answer.

Microsoft calls this risk the "Agentic Insider." Traditional cybersecurity usually focuses on defending against external attackers or malicious employees, but Microsoft believes that AI agents deployed by enterprises themselves can also become a source of internal risk. They do not necessarily need to be controlled by hackers to cause harm through erroneous judgment or manipulation by external information.

Microsoft CEO Satya Nadella has previously expressed a similar view. He believes that enterprises must separate the intelligence provided by AI from the security mechanisms that determine its authority to act; AI models must not be allowed to decide for themselves what permissions they should have, nor should important business systems be handed over entirely to agents to manage based solely on a model vendor's security promises.

Nadella proposed that a truly trustworthy agent system should not be built on the basis of "which model we trust most," but rather on a design under which the entire system can still operate safely even if the model is not fully trustworthy.

This concept forms the core of Microsoft's latest security strategy: let AI be responsible for understanding tasks and proposing action plans, while a system independent of the AI model decides what it can access, which operations it can perform, and whether it must stop acting when a risk arises.

To achieve this goal, Microsoft is expanding its AI security system called Project Perception and launching a series of new management capabilities for enterprise agents. These include dynamic permission management, dynamic agent risk assessment, and monitoring agents called Guardian Agents.

The goal of dynamic permission management is to prevent AI agents from automatically inheriting all the access permissions held by users. Enterprises can assign each agent an independent identity and grant only the minimum necessary permissions based on the specific task. When the task ends, the relevant permissions can be automatically revoked; if the task content changes, the system can also reassess whether the agent still needs its original permissions.

For example, an agent responsible for organizing enterprise financial data may only need to read specific reports and should not thereby gain permission to modify bank account information or approve large payments. If, during task execution, it suddenly attempts an operation beyond its original scope, an independent security control system can block the operation or require manual approval.

Microsoft is also developing a dynamic agent risk assessment mechanism. This system comprehensively analyzes an agent's identity, access permissions, the data it touches, network activity, operating environment, and actual operational behavior to continuously determine whether its risk level has changed.

This mechanism pays particular attention to whether an agent's behavior deviates from its original task. For example, if an agent originally responsible only for analyzing sales data suddenly attempts to read a large number of confidential files unrelated to the task, or begins accessing unauthorized external services, the system can raise its risk level and take corresponding measures.

Microsoft believes that traditional security systems often rely on manual analysis of alerts before deciding whether to take action. However, AI agents can continuously execute operations at machine speed, and waiting for humans to discover a problem before handling it may be too late. Therefore, security systems need the ability to monitor in real time, dynamically adjust permissions, and quickly restrict dangerous operations.

Guardian Agents are designed precisely for this purpose. They will serve as security systems supervising other AI agents, continuously observing the behavior of target agents, and when risks are detected, investigating the cause, proposing protective measures, and, when necessary, carrying out isolation or restriction operations.

Microsoft hopes to use this approach to form a multi-layered AI security architecture: ordinary agents are responsible for completing business tasks, Guardian Agents are responsible for supervising risks, and the ultimate permission boundaries and enterprise security policies are controlled by an independent system. Enterprise security personnel remain responsible for setting rules, handling major decisions, and investigating security incidents.

Microsoft also plans to integrate these capabilities with enterprise management products such as Microsoft Agent 365 to help organizations discover AI agents running internally and establish a unified asset inventory. As employees create AI agents on their own through different platforms and development tools, it may become difficult for enterprises to accurately grasp the number, permissions, and actual uses of these systems.

Microsoft said its security tools are expanding their ability to discover local AI agents, and the number of local agent types they can identify will increase to 60, including tools such as Anthropic's Claude Code, OpenAI's Codex, and GitHub Copilot. This will allow enterprises to understand which agents are accessing company systems and to manage agents that are unauthorized or have excessive permissions.

This move also reflects new changes in competition in the AI agent market. As more enterprises begin deploying AI tools from different vendors, the focus of competition is no longer just whether models can complete complex tasks, but also whether enterprises can securely manage these agents. Microsoft hopes to use its identity management, endpoint protection, cloud security, and enterprise software ecosystem to become an important platform for enterprises to centrally manage AI agents.

However, this strategy also has a commercial backdrop worth noting. On the one hand, Microsoft warns enterprises not to blindly trust AI model suppliers; on the other hand, it is promoting its own security products so that enterprises can monitor and manage various agents through Microsoft's platform. Media outlets such as TechCrunch pointed out that this is both a response to emerging security risks and a business opportunity for Microsoft to compete for the enterprise AI security market.

Microsoft has different forms of cooperation with AI companies such as OpenAI and Anthropic, so the security principles it proposes are not aimed only at competitors. Microsoft's core argument is that no matter which company provides an agent, the model itself should not hold ultimate security control. Enterprises need independent permission management, behavior monitoring, and risk response mechanisms to avoid errors or abnormal behavior by a model directly escalating into a serious business incident.

Microsoft also acknowledges that simply monitoring an agent's inputs and outputs is not enough. In the future, security systems may need more technical signals about the model's internal processing so they can determine why an agent took a certain action and whether its behavior still aligns with the user's original intent. However, Microsoft emphasizes that even if more internal model information becomes available, independent security boundaries must not be removed as a result. A model's explanations or self-judgments cannot replace genuinely effective permission controls.

This discussion around the security of AI agents reflects that enterprise AI applications are entering a new stage. In the past, AI was mainly used to generate text, answer questions, or provide suggestions; today, agents are beginning to gain the ability to execute operations and can directly affect data, systems, and business processes. The risks they bring have therefore expanded from "whether the answer is correct" to "what it actually did, and whether it had the right to do so."

Microsoft has not advocated stopping the use of AI agents because of these risks. Instead, it proposes that agents should be assumed to be capable of making mistakes, and that independent security mechanisms should limit the consequences of errors. For enterprises, the real challenge is not finding a model that never makes mistakes, but building a system that can promptly detect, limit, and correct problems even when a model makes a wrong judgment.

In the future, as AI agents enter more fields such as finance, software development, customer service, and enterprise management, dynamic permissions, continuous risk assessment, and independent oversight mechanisms may gradually become the infrastructure for enterprise agent deployment. Microsoft hopes to expand its influence in the enterprise AI field through these products, but whether this security philosophy can be effectively implemented still needs to be tested through actual deployment, handling of risk incidents, and long-term operational performance.