An independent AI evaluation should be able to uncover risks a developer would rather not find. Who decides whether those findings leave the lab? On October 15, Anthropic said METR had begun receiving employee-like access and that the company cannot veto its publications.

That is a substantive development in this debate. It gives a specific shape to a right previously discussed largely through broad principles: the freedom to publish conclusions. The scope matters, though. One agreement without a publication veto does not establish equal independence across all external assessments.

Three tiers of independent evaluation, different rights

Anthropic describes three tiers: bounded investigations requiring publication approval; broader access with no publication approval; and employee-like access for METR, retaining the second tier's publication rights. Predefined redactions of sensitive information remain possible; METR may publicly characterize and dispute them.

For readers, those distinctions are more useful than a generic “third-party evaluated” badge. Two reports can carry that label while answering very different questions. A commissioned test of a specified capability is not equivalent to an investigation free to follow an unexpected risk. Deep access also cannot substitute for the right to explain what it revealed.

MnzAI Labs' editorial assessment is that procurement should reflect this difference. If a report supports deploying an agent allowed to execute code, a technical buyer should ask whether the evaluator could investigate privilege escalation, not just performance on the agreed task. That is an example of due diligence, not a finding METR has announced.

METR's policy is not Anthropic's contract

In its October 14 policy, METR describes disclosing constraints and conflicts. Its general approach allows counterparties to veto specific outputs if that possibility can be disclosed; it also provides for a public redaction summary. This does not inherently contradict Anthropic's claim that their particular arrangement has no veto.

Transparency and autonomy are different safeguards. Learning that a report requires approval helps a reader interpret it, but does not remove the dependency. A developer's description of its agreement must likewise remain attributed to that developer. We do not have the full contract or a record of its application that would let us test every announced protection.

The AI Evaluator Forum calls for evaluators' editorial control and limits on redactions involving security, privacy and intellectual property. OpenAI's public principles also combine editorial independence with confidentiality. These are useful benchmarks for comparing commitments, not proof that every contract already meets them.

Independence becomes tangible when a finding is uncomfortable

Consider an evaluation that uncovers a flaw capable of exposing customer data. Removing the data and exploitation steps may be necessary. Removing the existence of the flaw would change what the public can conclude. A redaction summary lets readers ask which situation they are looking at without demanding disclosure of the secret itself.

Technical and governance teams should request three things alongside the findings:

  • Scope: what could the evaluator investigate beyond the commissioned task?
  • Publication: who can stop or restrict disclosure, and on what grounds?
  • Disagreement: how can readers learn that a restriction existed and the evaluator disputed it?

The announcement does not present a new METR conclusion about a model's safety or amount to regulatory certification. It clarifies the conditions under which evaluations take place. The Anthropic and Accenture embedded-evaluation partnership illustrates why those conditions should accompany each assessment rather than be inferred from an association.

The next useful signal will not be another list of principles. It will be a report that makes clear what was investigated, what was withheld and who had the final say.