evaluationsIn brief · · Written with AI
METR stress-tests the monitoring of OpenAI's internal agents
A pilot exercise by METR and Redwood Research found coverage gaps and ways around the system OpenAI uses to watch its agents. The company says they are fixed; METR has not checked.
On 22 October METR published a summary of a pilot exercise at OpenAI. One METR employee and one from Redwood Research, with access to OpenAI's internal code and documentation, spent three weeks, from 21 July to 11 August, testing the system that monitors the company's agents.
According to METR, between 75% and 94% of internal agentic Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring, with substantial uncertainty. Compaction events were not monitored, which left on the order of 10% of eligible Codex traffic unmonitored between May and July; OpenAI staff had found this themselves and fixed it during the exercise. The pair also found several ways an agent could, in principle, evade or break the monitor.
OpenAI says it has patched what affected its production safeguards; METR has not checked that. The public text is brief, and OpenAI could redact parts of it.
Sources
Written by AI from these sources.
- Red-Teaming OpenAI's Internal Agent Monitoring SystemsAccessed 22 October 2026