OpenAI published a research note on Monday introducing Deployment Simulation, a method for estimating how a candidate model is likely to misbehave once it actually ships. The setup is simple in concept. Take a large pool of real anonymized historical conversations, strip out the assistant responses, regenerate those responses using the new candidate model, and score the regenerated outputs against the same safety taxonomy that production traffic gets graded on. The paper reports a retrospective study across the entire GPT-5 Thinking family from August 2025 through March of this year, covering roughly 1.3 million de-identified conversations, plus a pre-registered prediction for 20 categories of undesirable behavior on GPT-5.4 Thinking.
The headline number is a median multiplicative error of 1.5x on predicted failure frequency. If the true rate of a given misbehavior is ten incidents per hundred thousand messages, the simulation typically estimates somewhere between roughly 6.7 and 15. That is a useful range for product safety decisions about non-rare behavior. It is not a useful range for the rare-and-catastrophic tail, which the paper concedes can drift to a 10x multiplicative error and which the method cannot detect at all below a frequency of about 1 in 200,000 messages. That ceiling is the part of the paper that should be quoted in every coverage piece and mostly is not.
The agentic coding piece is more interesting than the chat piece. OpenAI ran the same simulation methodology against 120,000 internal employee tool-call trajectories to forecast how a hypothetical GPT-5.5 coding agent would behave in deployment. That is the first published industry example of pre-release evaluation specifically designed for agentic systems that take real actions against real systems, which is the failure mode that every CISO currently spends meeting time worrying about. Whether the methodology generalizes off OpenAI’s internal corpus to, for example, a third-party coding agent running against a customer codebase is unclear and not directly addressed in the paper.
The political read on the release timing is hard to ignore. OpenAI shipped this two days before the G7 working lunch where Altman, Amodei, and Hassabis were all in the same room with European leaders. The Europeans have been asking for exactly this kind of structured pre-deployment evaluation for two years. OpenAI is the first US frontier lab to publish anything that looks like a defensible answer to that ask. That is not a coincidence, and OpenAI’s policy team did not pretend it was.