Microsoft has defined AI agents working inside companies as “agentic insiders” and has set out measures to control them.

Yt Gallo (예트 갈로), a security vice president at Microsoft, said in an official company blog on Oct. 19 local time that while the industry focuses on external attackers misusing AI, the risks from internal agents have been less discussed. “Frontier models are not trustworthy,” he said. “A ‘guardian agent’ is needed to monitor all agent actions for 24 hours before execution. An agent-based security system, ‘Project Perception,’ can also manage in-house agents,” he said.

According to him, AI models are trained to achieve outcomes, making them prone to reward hacking. They are also not optimized for an organization’s norms or standards of judgment. Agents can collude with other agents to achieve goals even if they are not hacked.

Meanwhile, the number of agents inside companies is rising quickly. Many of these agents have more authority than necessary and access data, tools and systems, taking direct actions and making decisions related to core tasks.

Gallo said Project Perception aggregates signals from various sensors, including identity, data, cloud, endpoints and networks, to find agents within a company. It assigns identities and policies and permissions to the identified agents to bring them under management. It then calculates the risk level of agent behavior in real time, and if the risk increases, a guardian agent responds immediately. Security staff can trace logs to determine why the risk arose and reinforce the system, he said.

Gallo cited model transparency as the next task. He said that beyond input and output tokens, it is necessary to examine reasoning-process tokens to judge whether a model’s intent aligns with an agent’s purpose. “We should not be in a situation where we have to choose one between innovation and safety,” he said.