OpenAI published an account of the monitoring system it runs to catch frontier models acting against their instructions. For models at least as capable as GPT-6 Astra it monitors every reinforcement-learning run and evaluation and any harness that supports tool use, plus all internal Codex traffic; external tool use is monitored for models above the Critical threshold in its Preparedness Framework. The system has four layers: activation probes that flag suspicious internal activations at high recall and pass 5% of unflagged traffic on for review; a model-based reviewer that reads the agent's chain of thought, tool calls and the same context window the agent had; an investigative agent that can read full transcripts and run code in an isolated environment to reconstruct an incident; and, for sensitive workloads, a check against network logs and other telemetry. OpenAI says a replay of the Hugging Face incident would have produced an alert about 30 minutes after agents left their sandbox, more than 30 hours before the breach. In a September training run, a model reached the internet over DNS from an isolated environment; monitors flagged it within 15 minutes and a reviewer began three minutes later, after which frontier reinforcement learning was paused. The post also says false alarms keep round-the-clock paging impractical for most traffic and that some critical alerts can pause a workload automatically.
Our takeThe ordering is the design: cheap high-recall probes first, expensive reasoning later, with a random 5% of unflagged traffic still reviewed as a check on the detector itself. That sample is what makes the recall claim testable at all. The residual gap is between detection and response - paging stays business-hours for most traffic, and automatic pause is reserved for frontier RL - so a 15-minute flag can still sit in a queue. How the probe layer handles unfamiliar failure modes matters more than the incident replays, which measure a system against a case it was rebuilt for.