Scott Alexander’s Against SURA Seems Deeply Confused

Zvi Mowshowitz has published a long response to Scott Alexander's essay "Against SURA," arguing that Alexander's case against the strategy is poorly constructed by Alexander's own standards. The piece lays out Alexander's claims, gives Mowshowitz's high-level reactions, and works through the disputes in detail, incorporating exchanges with Oliver Habryka, Adam Yedidia and others.
SURA stands for "stop unilaterally and raise awareness." As Alexander defines it, it is the idea that the only ethical course is for people, companies and countries to stop building superintelligence unilaterally, with no plan for what happens if race dynamics continue anyway. In Mowshowitz's reading, the platform has three planks: alignment-concerned AI companies are morally obligated to stop regardless of whether others keep racing; alignment-concerned employees should quit regardless of role or whether their employer is better than rivals; and alignment-concerned countries should ban or heavily restrict AI progress unilaterally, regardless of whether competitors such as China continue.
Alexander's frame separates Plan A, working for a global pause, from Plan C, everything else in a world where no pause happens. His central argument, as Mowshowitz summarizes it, is that many actions that improve the odds of Plan A reduce survival chances if Plan A fails and the world falls back to Plan C. Alignment-pilling companies helps Plan C; mass awareness, educating politicians and verification technology help Plan A. SURA, on this view, sacrifices Plan C to maximize the chance of getting to attempt Plan A, and is only sensible under some combination of assumptions: that alignment is highly doomed, especially under Plan C; that racing is deontologically evil; that racing burns valuable time; that quitting effectively raises awareness; and that SURA actually improves the odds of a pause.
Mowshowitz accepts some of this and rejects other parts. He agrees that Plan A is first best and calls the tension between the two plans Alexander's strongest point, while respecting those who think alignment under Plan C is doomed. But he argues the claims are overgeneralized: actions framed as helping one plan usually help both, with different weights. He says the sign of impact on the margin is often unclear, that the aggregate claim does not settle what an individual or group should do, and that he expects reasonable forms of SURA to be positive on net even for Plan C, though he holds high model uncertainty. He also rejects Alexander's assertion of a consensus in rationalist and effective-altruist circles favoring SURA, saying even a weak version with fallback is a minority position and the full version is rare.
A large share of the disagreement is about numbers. Alexander states an unconditional p(doom) of 25% and says a really good Plan C+ has a 90%+ chance of working, reasoning that such a world is much better than he expects in reality so the failure probability is a bit less than half of 25%. Habryka calls it wild to expect alignment to be solved with a 6-12 month lead, unilaterally, and then to declare the end of history, describing that as betting everything on machine gods built by one's own side. Mowshowitz argues that the 90% figure is more than 30 points above every source Alexander cites, that starting from a bottom-line p(doom) and reasoning upward merely restates the 25%, and that multiple distinct steps could plausibly go wrong.
On ethics, Mowshowitz does not call himself a deontologist but agrees that building something that might kill everyone looks deontologically evil, and says those who proceed should say plainly that they are accepting that rather than arguing it away. He is sharply critical of Alexander's comparison between frontier AI development and everyday use of frontier models, calling it a version of the argument that someone else would do the evil thing anyway. He notes he declined to invest directly in Anthropic for ethical reasons despite attractive expected returns, and draws a line between buying a subscription or holding tech stock and actively advancing capabilities inside a frontier lab.
Yedidia pushes back on that section from another angle, arguing the consumer analogy is weak because there are billions of AI users but only thousands or tens of thousands of frontier developers, so coordinating a prohibition at the developer level makes far more sense. He also raises what he calls the normie-ethics problem: outsiders who fully understand that lab employees hold high p(doom) estimates and keep working often react with bafflement and hostility, with the most common sentiment he hears being that those people should be arrested, a view he explicitly does not share. He compares the situation to driving, where predictability to others matters as much as raw skill and a strategy that surprises everyone else fails catastrophically when something unexpected happens.
Mowshowitz also flags Alexander's model of alignment, which he says treats models as inheriting virtue from imitating humans. Mowshowitz calls that misleading, noting it would imply base models are virtuous until corrupted by reinforcement learning with verifiable rewards. He agrees with Alexander's picture of Plan C success, stringing prosaic stopgaps together long enough for AI to take over alignment research until the result is self-reinforcing, but sees several routes to a 90%-plus p(doom) even so: solving alignment while lacking any workable public plan for what to do with the resulting systems, labs failing to execute the prosaic strategy well, and the Yudkowskian worry that alignment requires hitting a precise target on the first try.
Why it matters: this debate shapes how safety-focused researchers, lab employees and funders argue about whether to keep building, quit, or push for policy, and the disputed 90% figure is the hinge on which the practical advice turns. For product and engineering teams at AI companies, the arguments about employee responsibility and public perception are a reminder that hiring, retention and reputation now rest partly on how credibly a lab addresses these questions, though the debate itself offers no empirical resolution. Readers should treat the probability estimates on both sides as judgment calls rather than measured quantities; my own inference is that the disagreement will persist because the underlying scenarios cannot be tested in advance.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
If you are helping build something that might kill everyone, maybe stop doing that.