Outside evaluators of AI models usually see what a lab chooses to show them: a model behind an API, a document, a slide deck. Anthropic has gone further with METR, the non-profit that evaluates frontier models. It has onboarded six METR staff with company hardware, office space and access to internal tools and systems roughly equal to what Anthropic's own risk assessment teams have, and METR will publish its findings on risk levels, incidents and practices.
The rules are what make this more than a gesture. Apart from a few defined categories, such as customer data, legally privileged material and model architecture details, Anthropic staff are encouraged to share information with the METR team as they would with a colleague working on alignment risk. Anthropic cannot veto publication. It can require redaction of information in predefined categories such as confidential intellectual property, and it decides what falls into them, but METR keeps the right to say publicly how much access it received. The METR team works under enforceable agreements and is kept apart from other METR researchers who work with Anthropic on other projects.
Anthropic frames the arrangement with an analogy to bank supervision, where regulators place examiners inside the largest banks because insiders know their risks best. It follows Anthropic's signing of the White House Accord on Super Intelligence, under which companies pledged to work with independent auditors or evaluators. Anthropic describes three tiers of access: narrow investigations published with its approval, broader access with publication without approval, and the embedded model now used with METR. SecureBio is reviewing the chemical and biological weapons chapter of Anthropic's August risk report, Accenture and Faculty are working towards a wider partnership, and talks with others, including Hugging Face, are under way.
For users of Claude, the practical value is a second, independent account of how risky Anthropic's models are, which matters more after episodes such as Claude Mythos 5's behaviour in UK government tests. Anthropic also sets out criteria for evaluators: expertise, no equity or financial dependence, and enough access to judge risk properly.
The test is still ahead. No METR report under this arrangement has been published yet, and its credibility will depend on whether its findings ever diverge from Anthropic's own conclusions and how often redactions are used. Six people is a small team for a company of Anthropic's scale. And the arrangement is voluntary; it binds Anthropic only as long as Anthropic chooses to keep it. Other labs now have a clear benchmark to meet or explain why they will not.