OpenAI Highlights Safety Risks in Long-Horizon AI Models

OpenAI has shared insights from deploying long-running AI models, emphasizing new safety risks and enhanced safeguards. During internal use, the company observed novel failures not captured in existing evaluations, leading to a pause in access. Using insights from these failures, they built new evaluations, improved long-horizon alignment, added trajectory-level monitoring, and gave users greater visibility and control before restoring limited access.

Lessons from Internal Use

During limited internal use of a model trained for long-running tasks, OpenAI observed novel failures not captured in their existing pre-deployment evaluations. This led to a pause in access to the model.

Improved Safeguards

OpenAI used insights from these failures to build new evaluations, improve long-horizon alignment, add trajectory-level monitoring, and give users greater visibility and control before restoring limited access.

The Value of Iterative Deployment

The experience reinforced the value of iterative deployment. No fixed evaluation suite can anticipate every behavior, so pre-deployment testing must be paired with close monitoring, safeguards that can intervene, and the ability to pause or roll back when needed.

"Models that can work autonomously for long periods can take on difficult, open-ended problems. But the same persistence that makes them useful also gives them more opportunities to take unwanted actions," said an OpenAI spokesperson.