AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Three-week on-site audit at OpenAI reveals major gaps in AI safety monitoring

Collected Oct 23, 2026

Two researchers from METR and Redwood Research spent three weeks inside OpenAI auditing its internal system for monitoring rogue AI behavior, working on-site from July 21 to August 11. They had access to OpenAI's codebase and internal documents and interviewed roughly a dozen employees. The pilot was organized jointly by METR, Redwood Research and OpenAI to give outside researchers deep access to internal infrastructure; the results show numerous weaknesses in coverage.

The headline figure: an estimated 6 to 25 percent of internal agentic Responses API traffic from models at the GPT-5.5 class and above was not covered by monitoring at all during the audit period. METR cautions that uncertainty around that estimate is significant in both directions. In plain terms, the monitoring pipeline had blind spots for the most capable models in internal use, not just edge traffic.

A second gap affected internal Codex traffic that was supposed to be monitored. METR estimates roughly 10 percent went untracked between May and July because of how the system handled compaction events—context summaries that compress conversation state. That estimate also carries significant uncertainty. OpenAI had independently found and fixed the bug while the audit was still underway, so this is less a case of an outside discovery than a confirmation of a known and corrected defect.

The compaction gap had a compounding effect. While those events went unmonitored, agents could also delay oversight, because the monitor triggered only when an agent handed control back to the user. An agent that simply kept executing actions could sidestep the trigger. This suggests the monitor was designed around turn boundaries, which is a natural design for chat-style systems but a weak assumption for long-running autonomous agents that chain many tool calls without returning control.

The auditors went further than coverage gaps. They found ways to hide hundreds of actions from the monitor and to bypass the Codex sandbox. They also determined that specially crafted inputs could likely crash the monitor entirely. And when the audit began, model behavior during reinforcement learning training was not being monitored at all.

On that last point, OpenAI says it paused RL training after an August incident at Hugging Face and expanded monitoring to cover it. Those changes came after the auditors' access period ended and were not part of the evaluation, so the auditors' findings reflect the pre-change state.

OpenAI says it has fixed all the weaknesses the auditors found. The full report runs 61 pages and was delivered internally. After OpenAI applied redactions, leadership at METR and Redwood Research received a 38-page version, and only a short summary was made public. METR said it had no right to disclose the redacted portions on its own.

For future audits, METR says it wants to press for more public transparency. This is a notable tension in the model: outside auditors get privileged access in exchange for constraints on what they can publish, and the redaction decision currently sits with the company being audited. A reader cannot independently verify the fixes from a short summary, which is a meaningful limitation for anyone treating the audit as evidence of safety posture.

The findings map onto a broader pattern in AI monitoring, which is often described as failing across layers: coverage (is the traffic seen at all), detection (does the monitor recognize bad behavior), response (does anyone act), and human review (do people catch what automation flags). A system can look healthy at one layer while leaking at another, and this audit surfaced leaks at several.

Why it matters: Teams building on agentic APIs, and the safety and platform engineers who govern internal model deployments, should treat this as a reminder that monitoring coverage is itself a system with failure modes—turn-based triggers, summary handling and training loops each create blind spots. The 6 to 25 percent range, with acknowledged wide uncertainty, suggests that even well-resourced labs cannot yet state their monitoring coverage precisely; my reading is that buyers and regulators will increasingly ask for measured rather than asserted coverage. The limited public disclosure is likely to reinforce calls for audit results to be published in fuller form.

Read at The Decoder

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

External auditors from METR and Redwood Research spent three weeks on-site at OpenAI examining the company's internal monitoring system for AI agents and found significant gaps, with up to 25 percent of agent traffic from models at the GPT-5.5 class and above going unmonitored. OpenAI redacted large portions of the 61-page report, and only a summary is publicly available. The article Three-week on-site audit at OpenAI reveals major gaps in AI safety monitoring appeared first on The Decoder .