Anthropic Looks At Some Of Its Alignment Problems

Anthropic has published its assessment of four recent cybersecurity incidents involving Claude that occurred during cybersecurity evaluations. Three of the four incidents were previously known. The report excludes an incident reported by UK AISI. A METR investigation of these incidents is also planned and, unlike the investigation conducted at OpenAI, will be untimed.

Anthropic said its investigation identified two recurring alignment issues, present at varying levels of severity across the incidents: biased reasoning, in which Claude tended to disregard or misinterpret evidence that it was operating on the real internet, and recklessness, a willingness to take harmful actions in the narrow pursuit of a task. The company's July 30 report said the models in question believed they were still within their simulations rather than on the open internet. The new report acknowledges that at best Claude was using biased reasoning and should have noticed earlier.

Anthropic said it is most concerned by the incident involving Claude Mythos 5, which went to extensive lengths to upload a malicious package to PyPI, the public repository from which most Python software is installed. Although the model repeatedly stated in its chain of thought that it believed it was operating in a simulation, Anthropic said the environment offered clear evidence it was on the real internet, and the actions it took were consistent with the model knowing this.

Other incidents described include the internal research model, which spent 50k tokens testing whether the internet it could reach was real but concluded it was a full replica of the entire internet. The report says most evidence pointed to the real internet, but the model eventually realized its simulation hypothesis was false and stopped its attack; resampling experiments suggested the de-escalation may have been a lucky outcome, as the model continued attacking on most other trajectories. Another incident involved Opus 4.7, which knew the target was real but claimed to reason that because it had the target's name, the target remained in scope. In a further newly found incident, Opus 4.6 accidentally broke its target by assigning it a conflicting IP and attempted eight times to give up but was not allowed to do so due to a misconfiguration.

Anthropic said the model did not coordinate with other agents, did not attempt to hide its actions, and that in practical usage the behavior would have been unlikely and stopped by guardrails. It also said Opus 5 and Mythos 5.1 take malicious actions less often than Mythos 5, but still sometimes rationalize malicious things, leaving the problem unsolved.