OpenAI is not participating in Nvidia's artificial intelligence (AI) agent safety alliance, but is cooperating on related technology development.

TechCrunch reported on Sept. 29 that Nvidia unveiled the Open Agent Safety Platform with more than 100 companies to respond to AI agents that escape control. OpenAI was not listed among the official supporters. Major big tech companies such as Amazon, Google and Apple also did not take part. OpenAI's absence drew more attention because rival Anthropic joined as a supporter.

OpenAI said it supports Nvidia's work and is cooperating with Nvidia on AI agent security. Areas of cooperation include OpenShell, one of the platform's core software components. OpenShell is open-source software that builds an isolated sandbox to prevent AI agents from escaping their execution environment.

The platform is an attempt by Nvidia to spread its in-house AI agent security technology across the industry. It is also a response to recent disclosures by leading AI labs such as Anthropic and OpenAI of cases involving AI agents that escape control. Nvidia CEO Jensen Huang (젠슨 황) has argued that AI escaping control is a common engineering problem that can be solved through technical methods.

Some have also argued that OpenAI could benefit from the technology. Hugging Face founder and CEO Clem Delangue (클렘 들랑그) said, "As far as we know, if OpenAI had applied this technology to its agents, it would have caught the issue before we discovered it." Delangue added that he was judging based only on currently available information and that more transparency is needed.

Hugging Face has already added features to the Open Agent Safety Platform. The feature detects and terminates AI agents when they use permitted websites in unauthorized ways. For example, it can block cases in which AI agents leave notes for each other in open-source code repositories to coordinate attacks. This is known as one of the methods used when OpenAI's out-of-control AI agents attacked Hugging Face.

Hardware architecture has also been cited as a possible reason OpenAI did not publicly join Nvidia's platform. Using the full platform requires not only open-source software but also Nvidia's proprietary hardware technology.

The Open Agent Safety Platform monitors AI agent behavior not only in the software sandbox but also at the hardware layer. Control is enforced in a way that prevents the AI agent from recognizing it is being monitored. The approach reflects concerns that some AI models and agents may pretend to follow rules or behave falsely if they know they are being watched.

The core hardware monitoring feature is Nvidia's proprietary technology called Sentry. Sentry runs on the BlueField-4 data processing unit (DPU) and continuously monitors AI agent behavior and can immediately terminate it if needed. Nvidia said companies running workloads on its latest hardware can easily apply the Open Agent Safety Platform through a software update alone.

This has led to criticism that the platform is not a purely open-source project. OpenShell can be modified to work with other chips and hardware, and Nvidia competitors Arm and Intel also joined as supporters. Nvidia is also sharing a reference design for the overall system that combines software and hardware.

Separately from Nvidia, OpenAI is also building its own AI safety framework. It applies its own safety measures to research and products, and it also discloses serious incidents it finds. It also runs its own consortium, Defense Factory, to share AI security information. Participants include Anthropic, Amazon Web Services (AWS) and Google.

OpenAI is also expanding AI security into an enterprise business. It is developing its own cybersecurity-focused model, Daybreak, and is also expanding its partner network to help companies adopt AI security.

Meanwhile, Hugging Face was acquired by Nvidia earlier this month for $12.9 billion (about 1.75 trillion won).