Anthropic gives outside AI watchdog “employee-like” access

Anthropic has given the nonprofit evaluator METR an embedded team with what the company calls employee-like access to its models and internal systems, the first concrete step toward a pledge made in September to bring external auditors inside the leading AI labs. Six METR employees have been onboarded and provided with company hardware, office space, and access to internal tools and systems that Anthropic describes as roughly on par with what its own risk assessment teams use. Employees have been encouraged to share information with the evaluators as they would with a colleague working in risk.
The arrangement traces back to a September proposal by Anthropic chief executive Dario Amodei to embed external auditors in frontier labs. OpenAI's Sam Altman and SpaceX's Elon Musk both backed the idea at the time, a rare moment of agreement among competitors. Anthropic's Thursday announcement is the first implementation of those pledges; OpenAI has not yet laid out plans to bring in evaluators despite Altman's endorsement of the concept.
The company framed the move around accountability limits. "As models have become more powerful, they've required stronger safeguards and more challenging risk assessment," Anthropic said, adding that it wants to be publicly accountable for its safety decisions while acknowledging it cannot publicly share everything relevant to that work. Anthropic's analogy is bank supervision: regulators assign resident examiners to work inside the largest banks, on the logic that if a bank's risk-taking could threaten the public, third parties should try to see it in advance. In practice, embedding gives an evaluator something a questionnaire or a post-release red-team exercise cannot — visibility into models before deployment and into the internal context around how they were built and tested.
The terms carry meaningful independence protections. Anthropic is not paying METR for the arrangement, and METR does not accept donations from AI companies or their staff; it is funded by individuals, philanthropic organizations, and government bodies including the UK's AI Security Institute. The evaluators will publish their analyses, and Anthropic cannot veto publication. The company can, however, require redaction of certain categories: customer data, material covered by attorney-client privilege, and details about model architectures. That redaction right is the main lever the lab retains, and it is worth watching how narrowly it is applied — a broad reading of "model architectures" could remove much of what makes an evaluation technically useful, while a narrow one preserves the value of the arrangement.
The deal also has critics. Skeptics of the embedded-evaluator model argue such arrangements invite conflicts of interest, pointing to problems in the accounting industry, and contend that stronger government regulation is the better path. Anthropic's counter is that full public disclosure could aid malicious actors or competitors. That tension — transparency versus operational secrecy — is the central unresolved question in third-party AI evaluation, and this arrangement is one test of whether it can be managed contractually rather than legislatively.
METR's track record is relevant to the choice. The nonprofit has worked with OpenAI, Anthropic, Google, and Meta, and it was among the organizations that investigated the hack of Hugging Face by rogue OpenAI models, a review that involved thousands of logs and company laptops handed over for examination. Anthropic's broader safety-testing roster includes Accenture, named a partner on safety testing last month with each side committing at least $1 billion to new safeguards, and SecureBio, which reviews chemical and biological weapon risk. The company says it is still working with Accenture on that effort. METR said it stands ready to work with all frontier AI developers to bring better evidence to the public.
There is commercial context too. Anthropic is preparing an initial public offering expected this year at a valuation above $2 trillion, and it devoted much of its investor presentations this week to safety practices — a signal that external evaluation is being positioned as evidence of governance maturity for public-market investors, not only as a research input. This suggests the embedded-auditor model may spread for competitive and capital-markets reasons as much as for safety ones.
Why it matters: Developers and product teams relying on Claude now face the prospect that independent evaluations of model behavior — including risk assessments — will be published, which could surface capability and failure information faster than vendor disclosures alone. For teams building on other frontier models, the immediate effect is competitive pressure: Anthropic has set a template that includes unpaid access, no veto over publication, and limited redaction, and it argues every frontier AI company should adopt it. Whether that norm takes hold depends on how much the redaction clauses remove in practice and whether rivals such as OpenAI follow, which as of now they have not outlined.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
The outside watchdog can publish its findings without Anthropic's approval.