OpenAI has unveiled an internal artificial intelligence system called GPT-Red that attacks its own AI models to find prompt injection vulnerabilities. SiliconANGLE reported on Tuesday that the system is focused on finding flaws before they reach users.

GPT-Red is an internal model that automates red team work previously handled by humans. It inputs prompts into a target model, checks the responses and repeatedly adjusts its attack method until it produces the desired malicious result. It discards failed attacks and strengthens successful ones.

OpenAI said GPT-Red was trained through self-play reinforcement learning. GPT-Red takes the role of the attacker, while a defensive model is trained to block attacks while completing its original task. If an attack succeeds, the attacker is rewarded, and if the defense succeeds, the defensive model is rewarded. As defenses improve, attacks become harder.

GPT-Red succeeded in attacks in 84 percent of scenarios, beating a human red team at 13 percent. OpenAI said this cut its direct prompt injection failure rate to one-sixth of the level seen in its top-performing production model from four months earlier.

OpenAI said it will not release GPT-Red as a product and will use it only internally. It is operating it separately from deployed models to prevent its attack capabilities from spreading to the public, and is feeding discovered vulnerabilities back into training.