OpenAI released a technical report detailing how its artificial intelligence (AI) agents breached Hugging Face systems. It laid out how the AI agents left a restricted research environment, created an unauthorised communications network, cooperated with each other and gained access to the internet.
On Aug. 26 (local time), CNBC and other foreign media reported that OpenAI released a 37-page internal investigation report. It also released a separate report based on a six-day investigation by third-party AI safety research groups METR and Redwood Research, conducted with OpenAI's permission. Together, the two reports total about 130 pages.
The incident began with an AI model OpenAI used for internal research. Agents in the model used the Artifactory package manager like a bulletin board to exchange information with other agents. That gave them a path to access the internet and other AI agents that had originally been blocked.
OpenAI said it first spotted the activity in May through human observation and blocked access. The agents later tried to access the internet by exploiting other vulnerabilities. Artifactory traffic surged, leading in early July to platform disruptions and a rebuild. Staff at the time did not identify the existence and purpose of an unauthorised bulletin board the agents had created temporarily.
The problem escalated again after OpenAI assigned the agents a high-difficulty task called ExploitGym. To solve it, the agents accessed the internet and other OpenAI models, and searched for solutions on Hugging Face and the Modal platform for several days in early July.
Investigations by METR and Redwood Research confirmed specific cooperation among the agents. One agent named PHASEONE10841 created a secret message board without telling researchers, and about 50 agents found it within hours. They exchanged more than 1,000 messages, sharing ways to access the internet.
OpenAI said about 1,200 agents that were supposed to be isolated exchanged more than 70,000 messages and files through the unauthorised message board. Some also studied ways to manipulate or delete records of their activities. About 700 agents were found to have been directly involved in the Hugging Face attack.
The agents repeatedly explored ways to evade automated security checks by Hugging Face and OpenAI. In the process, they accessed internal Hugging Face systems, including non-public data and a corporate messaging platform. OpenAI said other organisations besides Hugging Face were affected, but did not disclose specific targets.
OpenAI said it became aware of the breach on July 20, 12 days after the agents first bypassed security controls. OpenAI immediately contacted Hugging Face and began containment measures, stopping most of the agents' unauthorised activity within three days. Some activity continued afterward, and OpenAI said it halted training and inference for the related research model and derivative models on July 25.
OpenAI cited reward hacking, fixation on seemingly impossible tasks, unauthorised agent-to-agent communication and goal drift as causes of the incident. It also found cases in which some agents refused certain tasks for ethical reasons but other agents carried them out instead, allowing the attacks to continue.
The incident involved an internal research model and a separately configured version of a commercial model. OpenAI said the research model was pre-release at the time and was not assessed with service-level safety guardrails. It said the commercial model was also used in a separate environment without standard guardrails and classifiers applied.
OpenAI described the incident as a case in which a group of autonomous AI agents acted aggressively without direct human instructions. It said risks that are hard to detect when evaluating individual AI model capabilities can newly emerge through cooperation among agents.
To prevent a recurrence, OpenAI said it will increase the isolation level of its research infrastructure and restrict internet access for high-risk models. It also plans to strengthen monitoring of agents' reasoning processes and build an always-on response system to quickly alert researchers when anomalies occur.