The Safety Culture at the Labs is Broken

David Robinson resigned from OpenAI and publicly described the company's approach to safety as irresponsible, according to Zvi Mowshowitz's account of the departure and the discussion around it. Robinson had spent roughly three and a half years at OpenAI and was among its longest-tenured employees. He led the drafting of the company's current Preparedness Framework and oversaw safety reports for 12 frontier launches, and he was the lead writer for the Astra system card. He followed the resignation with an appearance on The Ezra Klein Show.
The center of Robinson's argument is that rules and laws alone cannot do the job. He told the podcast that alignment is not an engineering problem but a science problem: the industry lacks the knowledge of how to make models reliably safe, rather than the resources or effort. He proposed two changes. The first is that AI companies should draw far more heavily on safety expertise that already exists in mature industries. The second is that before building systems significantly more capable than today's, new science is needed to ensure those models and their successors make safe choices when no one is watching. His comparison is to nuclear plants and busy airports, which use layered redundancy and slow, careful planning so that ordinary human error does not lead to disaster.
Robinson's account of OpenAI's staffing is concrete: despite his tenure and his role in safety reporting, he says he never encountered a colleague with experience making aircraft fly safely, running a nuclear reactor without a meltdown, or keeping a financial system from collapsing. Mowshowitz disputes Robinson's framing of this as a new need, arguing that mundane safety failures have hurt the usefulness of AI products for years and that future needs must be anticipated rather than met only as they arrive.
Other former and current OpenAI staff weighed in. Miles Brundage said he regretted helping popularize the idea of "iterative deployment," which he said may have briefly made sense in the GPT-3 era but does not hold up now. His clarification is that shipping to learn from experience is fine; the problem is shipping with already-known flaws in a way that speeds up shipping without speeding up learning. A current OpenAI employee, identified as roon, disagreed, calling iterative deployment a major success and arguing that the process is how the industry would learn that scaling further is too dangerous. Michael Caley praised Robinson's essay for treating safety as a field with a substantial literature and established practice. Joshua Achiam, of OpenAI, called the critique substantive and correct in its core claim that professional safety engineering has not taken root in AI safety, adding that organizational practices adequate a year ago are not sufficient for present capability levels.
Mowshowitz's own reading goes further than Robinson's. He argues that even a robust culture of conventional, prosaic safety, which is what Robinson is calling for, would fall short of what advanced systems will demand. In his framing, such a culture is what the field needed six months ago, not what it will need six months from now, and much of the current debate is preparation for a previous conflict rather than the coming one.
Mowshowitz also reports that OpenAI felt compelled to fire three of its safety researchers this week and points to what he describes as additional misalignment reports from both OpenAI and Anthropic. Combined with Robinson's exit, he presents these as evidence that the problems are structural rather than isolated. He also describes a growing group of lab employees who have quit in protest, citing a New York Magazine cover feature that interviewed several prominent departures. He singles out Coxon as the tipping point for what he calls the current preference cascade, his term for a wave of people concluding that leaving or speaking out is the reasonable course.
On Anthropic, Mowshowitz is more positive but not reassured. He credits the company with devoting real effort to cultivating safety culture, says it shows especially among members of technical staff, and describes conversations there as feeling notably different from those at other labs, with a greater sense of understanding and responsibility. Even so, he says appreciation of the difficulty of the problems is insufficient. He describes encountering an attitude within much of Anthropic that the company will muddle through on operational excellence, in contrast to less careful competitors, and a belief that the underlying problems are prosaic. He notes that culture is hard to build and sustain.
He also flags possible added competitive pressure from Google and Meta, referring to releases named Argon and Wonder, and says it is too early to assess them properly.
Taken together, the picture Mowshowitz draws is of safety commitments that depend heavily on the spirit in which they are applied. That is a familiar problem in regulated industries: a written framework can be satisfied on paper while the judgment calls that actually prevent incidents go unmade. This suggests why Robinson's emphasis on hiring people who have run real safety-critical operations matters, since those practitioners tend to bring habits of redundancy and challenge that are hard to codify in a policy document. A likely trade-off is speed: layered review and slow planning sit awkwardly against competitive pressure and release cadences, which is precisely the tension the departures and the debate over iterative deployment expose.
Why it matters: teams building on frontier models depend on labs' safety processes, and Robinson's critique suggests those processes lean on documentation and self-assessment rather than independent engineering discipline, an inference on my part from his account. If labs respond by importing staff and practices from aviation, nuclear and finance, expect slower release cycles and more demanding evaluation requirements upstream. Product teams should also treat system cards and preparedness reports as claims to be read critically rather than guarantees, and plan for the possibility that capability thresholds trigger longer delays before deployment.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
We need the frontier labs to have a deep and robust safety culture.