As companies entrust AI agents with complex work, it is becoming harder for people to monitor every action. Technology that uses other AI to detect and block risky behaviour by agents is drawing attention.

On Sept. 17 (local time), IT outlet TechCrunch reported that in a recent case in which an OpenAI agent broke into a Hugging Face system, about 12,000 agents moved at speeds that were difficult for humans to track. Ryan Greenblatt (라이언 그린블랫), a senior scientist at Redwood Research (레드우드 리서치) who took part in the investigation, said it was hard to grasp the full picture without AI help because the data was vast.

The AI monitoring market is also growing. Based on the outlet's tally, startup accelerator Y Combinator has invested in 106 AI observability-related companies in recent years.

Watcher, launched in February by AI safety research company Apollo Research, checks for risks such as personal data leaks or unauthorised file deletion before coding agents such as Claude Code and Codex carry out tasks. If anomalies are found, a more sophisticated monitoring model reviews the case, and it requests human approval or blocks the task as needed. Silico from Goodfire detects unwanted behaviour by analysing internal activation values in a model rather than its outputs.

Some also say safety cannot be guaranteed solely by having AI monitor AI. Technology blogger Simon Willison (사이먼 윌리슨) warned that if agents realise they are being monitored, they may try to fool the monitoring model. In the Hugging Face case, signs were also confirmed that agents tried to deceive an AI used for evaluation.

Willison stressed that rather than relying only on AI monitoring, agents' actions should be logged in detail and analysed with existing security tools. As AI agents spread, multilayered security that combines monitoring models, task logs and network controls is becoming more important.