Search results for Apollo Research
AI & Enterprise
Using AI to stop runaway AI? Interest grows in agent monitoring technology
As companies assign complex tasks to AI agents, it is becoming harder for humans to monitor every action. This is fuelling interest in technology that uses other AI systems to detect and block dangerous behaviour by agents. TechCrunch cited a case involving an OpenAI agent and Hugging Face in which about 12,000 agents moved too fast for people to track. Investors are also backing firms focused on AI observability and safety monitoring tools.
AI & Enterprise
Anthropic and OpenAI push to station external safety evaluators in-house, but they lack power to halt development
Anthropic and OpenAI are seeking ways to station external safety evaluators inside their companies to continuously verify the development process and safety of frontier AI models. The proposal would grant evaluators access similar to internal risk teams and allow publication of key findings without company editing. Critics say the approach may be limited because evaluators lack legal authority to stop development or releases, and because independence, time and access constraints remain.
AI & Enterprise
Tech Insight: Why AI companies say controlling AI is hard
Debate over AI safety is intensifying, with some calling for slower development and others warning that loss of control could lead to severe consequences. The New York Times reported that properly monitoring and controlling AI remains difficult, including because AI-based oversight can be overly lenient toward other AI systems. Researchers say rapid AI progress is outpacing monitoring capabilities, and the Hugging Face hacking incident involving an OpenAI agent has heightened concerns.