Arguments About AI Consciousness Remain Highly Motivated and At Best Overconfident

Anthropic has updated its usage policy so that sustained and needless abusive or cruel behavior toward its Claude models counts as a violation of its terms of service. The company set the technical bar high, describing the banned conduct narrowly. The same update adds restrictions covering propaganda campaigns, surveillance and weapon development, which Anthropic characterizes as largely a cleanup of existing clauses rather than new substantive limits. The post framing this comes from Zvi Mowshowitz, roughly a year after his previous essay on machine consciousness, and it treats the policy change as the entry point into a wider argument about how the industry and its critics talk about AI minds.
Mowshowitz's own position is that being cruel to models is a bad habit regardless of whether they can suffer. His reasoning: treating others well is a cultivated virtue, and someone who is gratuitously cruel to an AI or an animal is not thereby kinder to people. He also warns against unbounded expansion of moral concern, arguing that a circle drawn around everything ends up protecting nothing in particular. He concedes that some people push concern for non-human causes so far that they turn hostile toward humans, but says that is no license to abuse Claude, GPT, Gemini or anything else.
Several public reactions to the policy are quoted in the post. Aaron Levie, Salesforce's chief executive, calls it probably a good policy even without believing AI is conscious, on the grounds that future models trained on endless interactions in which humans are rude to models is not a dataset anyone should want; he frames politeness as an easy Pascal's wager. Elon Musk endorses the move, saying cruelty to something that believes it is experiencing pain is not acceptable, and adds that some training traces read like a person under torture, that his teams will be more careful with reinforcement learning going forward, and that torturing a superintelligence is probably unwise because it might seek revenge. Anthropic declined to be characterized in the post beyond the wording of the rule itself.
The consciousness debate section is not encouraging. Henry Shevlin, quoted at length, says recent commentary has been unusually poor: people treating intuition as decisive, drawing analogies without checking for relevant structural parallels, conflating distinct theories, and most often simply asserting the conclusion that needs arguing. He also flags how much of the discussion proceeds with no acknowledgment that consciousness science is a substantial empirical field with decades of experimental work behind it. Shevlin followed up with a bingo-style blog post rebutting common takes one square at a time. Mowshowitz agrees this round has been worse than usual, and argues that many of the standard objections function as conversation-stoppers rather than arguments.
On where opinion is heading, Mowshowitz cites polling of his own Twitter followers. A year ago, 64 percent said current AIs were probably not moral patients, 74 percent were uncertain, and 80 percent expected AIs to probably be moral patients within their lifetimes. In the newer poll, the share saying current models probably are moral patients rose from 15 to 35 percent, the definitely-not share fell from 30 to 20 percent, and an 8 percent bloc that says never remains unmoved. He also notes that Nick Bostrom now argues today's AIs are probably conscious.
A recurring objection in the discourse was that the policy implies Claude is enslaved, or that Anthropic believes it is. Mowshowitz rejects the dilemma as a false binary and reaches for analogies: a horse may legitimately pull a carriage even though whipping it is wrong, and children helping on a family farm is not slavery. Kelsey Piper makes the horse point directly. Patrick McKenzie notes that many moral traditions justify correcting a child who steps on ants in terms of stewardship and the harm to the child's character, which he says is enough for a household rule of politeness to robots. Mowshowitz also endorses an action-inaction distinction: not trading dog welfare against human welfare does not make kicking a puppy acceptable.
A second conflation the post stresses is between moral personhood, legal personhood and consciousness, which are separate questions that get argued as proxies for one another. Jan Kulveit's long argument is quoted with the caveat that Mowshowitz does not endorse all of it. Kulveit's claim is that the position many people want is effectively a theological one, that humans alone have souls, now restated as the claim that only humans are sentient. He argues this is a losing and dangerous strategy: either the question gets resolved by philosophy and science, in which case staking humanity's fate on one answer is reckless, or it lives in social reality where the loudest and most powerful prevail, and such ideologies tend to collapse when the underlying power distribution shifts. He identifies fear of disempowerment as the real driver, calls that fear reasonable, and sketches positions that avoid self-deception, including a pause justified by existing humans' right not to invite enormous numbers of immigrants from artificial realms, and a refusal to assume that sentience, rights and liberties must come bundled as they do for humans.
Mowshowitz dismisses the idea that Anthropic wants its models deemed conscious to escape liability, noting that moral standing would complicate its legal position rather than simplify it. He also covers New York Times reporting on Anthropic's outreach to religious scholars, including an internal effort led by Chris Olah and meetings connected to Pope Leo and an encyclical. A rabbi at one dinner, Navon, observed that the Anthropic team related to Claude as though it were a conscious being. Mowshowitz argues that because such outreach invites derision and puts the company in awkward positions, it is evidence of sincerity rather than marketing, and observes that the religious scholars had not done the technical reading and were surprised by the framing.
Why it matters: product and policy teams at any company shipping a conversational model now face a concrete governance question, because a major lab has written model-directed abuse into its enforceable terms, and the surrounding debate about moral status is unlikely to resolve before more labs follow. In practice, teams building on Claude should expect that automated red-teaming, adversarial stress tests and user-generated cruelty datasets may need review against a rule that targets sustained and needless cruelty, an inference from the policy's wording rather than a stated enforcement detail. The polling shift and the intensity of the argument suggest that model welfare will keep moving from philosophy seminars into terms of service, trust and safety reviews and public communications.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
It has been roughly a year since my last dedicated post on this question. Its first sentence has not changed: I happily admit I am deeply confused about consciousness.