Anthropic’s METR Arrangement Turns Bank-Style AI Oversight Into a Contract, and Its Publication Terms Changed the Next Day

On Thursday, October 15, Anthropic announced that it had started giving a team from the nonprofit METR “employee-like access” to its systems. Six METR employees now have company hardware, office space and “access to internal tools and systems that are roughly on par with what our own risk assessment teams have access to.” METR will publish its analysis, Anthropic “can’t veto publication,” and Anthropic can require redactions in predefined categories. On Sunday, October 18, Ars Technica carried the Financial Times’s report of the arrangement with the subhead “The outside watchdog can publish its findings without Anthropic’s approval.”

The page has changed since it went up. Archived copies show that between the Internet Archive’s captures at 10:59 and 23:24 UTC on Friday, October 16, the paragraph on METR’s publication rights lost a sentence saying METR was “free to characterize our redactions publicly, and say whether it believes the redactions were justified.” It also lost the word “narrow” from its description of the redaction categories, and it gained a parenthetical: “Anthropic ultimately decides what information does and does not fall into these categories.” A footnote says the page was “Updated on October 16 to clarify METR’s current project terms with Anthropic.” It does not say what changed, and I found no statement from Anthropic or METR explaining the edit.

I held the current terms up against what Dario Amodei’s September essay promised, what the field’s own standard for evaluator independence asks for, and the bank-supervision model Anthropic invokes for the whole arrangement. The result is narrower than a verdict. This is a company-designed contract, and the features that keep a bank examiner independent of the bank (a statute that gives examiners power over the whole of a bank’s affairs and the ability to question its officers under oath, a rule that leaves the disclosure of examination findings to the regulator, and an assessment the regulator sets and the bank pays) are the features it lacks. Anthropic itself says the analogy “isn’t exact.” What the page does not do is tell readers which terms are settled.

What Anthropic announced

The October 15 page lists what an evaluator needs: expertise, independence (“they shouldn’t hold equity in the company they’re evaluating, or be financially or operationally dependent on it”), enough access, and “freedom to publish observations and assessments of a model or a process.” Anthropic says it is “embedding evaluators voluntarily,” that METR’s engagement is “limited to a small team operating under enforceable agreements,” and that other METR staff are siloed from that team’s work. Outside “a small number of clearly defined categories (e.g., data about customers, attorney-client privileged information, or information about our model architectures),” employees are encouraged to share information with the team “just as they would with a fellow employee working on alignment risk.”

The page describes three tiers. In the first, evaluators run “narrowly scoped risk investigations,” Anthropic decides what access they get, and they can publish “with our approval.” In the second they “can publish without our approval” and see more, though Anthropic “can still require removal of specific categories of information.” The third tier is the METR arrangement, with “employee-like access and publication rights similar to those of the second tier.” Anthropic says it is also working with SecureBio and Accenture as evaluators and is “in early discussions with several other organizations, including Hugging Face.” The post ties the announcement to the White House Accord on Super Intelligence, in which, it says, “companies pledged to partner with independent external auditors or evaluators.” Our earlier post found that the accord sets no deadlines, names no auditor, does not say whether audit results would be published and attaches no penalty.

Money is described in earlier statements. Anthropic is not paying METR: the FT reports that Anthropic “is not paying METR for this arrangement,” and METR’s own About page says “METR has not accepted funding from AI companies,” while noting that it makes “use of significant free tokens.” Accenture is paid. In a September 18 post, Anthropic said it “will fund Accenture’s work directly” and also that “Long-term, we think funding should come from pooled or government sources.” Anthropic’s post and Accenture’s release both say each company expects to invest at least $1 billion over five years; neither says what Anthropic pays Accenture.

How the publication terms changed

The Internet Archive holds eight captures of the page between 17:21 UTC on October 15 and 08:08 UTC on October 18, and they contain four distinct texts. The first two differ only in small wording and hyphenation changes. The third, first seen at 05:32 UTC on October 16, changed the SecureBio paragraph. It had said SecureBio “is currently in the first tier” and that Anthropic would consider “projects with increased access and publication rights.” It now says only “projects with increased access,” with a footnote that the paragraph was updated on October 15 “to clarify SecureBio’s current project terms.” The fourth, first seen at 23:24 UTC on October 16, changed the METR paragraph. The page I fetched on October 18 matches it.

How to read the edit

The edit can be read more than one way, and the page supports more than one. The footnote’s own verb is “clarify.” One reading (mine, not Anthropic’s) is that the new sentence moves the page closer to METR’s own wording, discussed below: “the level of access and information it received” sounds more like METR’s stated freedom “to say whether we ultimately received adequate access and whether redactions were adequately disclosed” than like a right to call a redaction justified. If so, the edit narrowed what the page promises without necessarily narrowing what METR can say. But the same edit added the sentence that gives Anthropic the last word on which information falls into a category, and no earlier version said who decided. And the page still carries the older wording elsewhere: the second-tier paragraph says evaluators “can publish their views on whether our redactions were justified,” and METR’s publication rights are described as “similar to those of the second tier.” A reader cannot tell from the page alone whether METR has that right.

Neither of the two accounts of the announcement that I read in full mentions the edit. A write-up dated October 16 from the UK site Resultsense paraphrases the first version’s sentence (“METR can say publicly that something was redacted and whether it thinks the redaction was fair, provided it does not reveal the content”), and the FT’s account as carried by Ars gives the no-veto rule and the redaction categories without the parenthetical. The September essay is the clearest statement of what Anthropic first promised, because it is the only place I found the promise about unfavorable findings: no archived version of the page repeats it.

What the field’s own standard asks for

Two documents set out what credible embedded evaluation should look like. AEF-1, “Minimum Operating Conditions for Independent Third Party AI Evaluations,” is a standard and checklist whose authors include staff of Transluce, METR, SecureBio and other groups. The Minimum Conditions for Embedding Evaluators letter, published September 18 with 200 or more signatories including Geoffrey Hinton, Stuart Russell and Arvind Narayanan, applies similar ideas to embedded evaluators and points to AEF-1 as an example of standardized terms. The table sets their conditions against what Anthropic’s page says.

“Not stated” means not stated publicly. The contracts are not public, and the page is an announcement, so its silence is not proof of absence. The conditions themselves are specific. AEF-1 says “The system provider did not have authority to redact results to conceal concerning findings,” and that “The evaluator should also retain the ability to comment on the appropriateness of redactions and how material they are to understanding the evaluation.” The letter asks for “a time-limited redaction process restricted to protecting critical interests in intellectual property, customers’ sensitive information, individual privacy, security, and public safety.” It names one exception to access: “to protect sensitive data belonging to the company’s customers and other third parties.”

METR’s own policy, published the day before Anthropic’s page, sets a floor that overlaps with these. Its best practices include “Freedom for us to say whether we ultimately received adequate access and whether redactions were adequately disclosed” and “Time limits on steps in our process (for example, on how long companies have for redaction).” It also says that “Not all of our previous or in-flight work meets this bar,” that companies “want the final say on what we can and cannot publish,” and that “it is possible for a company to use its rights to withhold access or redact to exert control over what evidence the public receives from us.” Anthropic’s page does not say whether this engagement meets METR’s bar.

Testing the bank-supervision analogy

Anthropic’s page says: “Although the analogy isn’t exact, bank supervision operates on similar logic. Banks know more about their own risk positions than outsiders do, so regulators assign resident examiners to work inside the largest banks.” The essay, which we covered when it appeared, says embedded evaluators have “precedent in the banking industry, which sometimes involves regulatory ‘supervisors’ embedded along with employees.” On presence, the comparison holds. The OCC’s Large Bank Supervision booklet says “the OCC assigns examiners to work full time at the largest and most complex banks,” with an examiner-in-charge assigned full time to each midsize and large bank. The differences are in what backs the examiner.

Most rows describe how a supervisor stays independent of what it supervises. The power to compel, the control of disclosure and the funding rest on statute or regulation rather than negotiation. In the Anthropic arrangement the access, the disclosure rules and the money are all matters of agreement, and METR says plainly that this leaves it exposed: “We remain dependent on the companies we work with to voluntarily engage, provide access, and permit us to publish.” The design also goes the other way on one point. Bank examination reports are non-public OCC information, while METR is meant to publish its analysis. That gives the public more to read, but how much more depends on the redaction categories, and the page now says Anthropic decides what falls into them. Our post on MIT Technology Review’s liability analysis described the same gap from the legal side: reporting rules only work if someone can verify what gets reported, and that job is voluntary today.

The independence question

Anthropic’s page defines independence as holding no equity and not being “financially or operationally dependent.” The letter asks for more: embedded evaluators “should not have other significant commercial business with” the company and “should not accept any form of payment or other reward contingent on the evaluator’s findings.” Accenture’s own newsroom shows the relationship the first condition is about. Its December 2025 release announced an Accenture Anthropic Business Group, about 30,000 Accenture professionals to be trained on Claude, Claude Code for tens of thousands of its developers (Amodei called it “our largest ever deployment”) and a Claude Center of Excellence inside Accenture that the companies will co-invest in. The Faculty unit that will lead the evaluation, as Accenture’s September 18 release puts it, “is now part of Accenture,” which reported about $70 billion in revenue for fiscal 2025.

Those facts cut two ways. A company with about $70 billion in annual revenue is unlikely to be financially dependent on one engagement, which is the test Anthropic’s page states. The letter’s test is a different one, and an Accenture that is also Anthropic’s go-to-market partner and a large Claude Code customer is in tension with it. Neither Anthropic’s post nor Accenture’s release says whether any payment to Accenture depends on what the evaluators find, which is the AEF-1 condition. METR sits on the other side of the ledger. It takes no payment and says so, but it also uses free model access from AI companies, and its October 14 post says “we can afford to pay for any free tokens we receive without materially affecting our operations.”

The FT adds that Anthropic is under pressure to reassure investors ahead of an expected public offering and “devoted a large part of investor presentations this week to laying out its safety practices.” That gives the publication terms an audience beyond safety researchers.

What to watch

1. METR’s first report from the engagement. METR says it prefers to “retain the right to unilaterally and prominently attach a ‘redaction summary’ to our reports.” Its report on the OpenAI and Hugging Face incident, which two METR staff and a Redwood Research staff member carried out on premises at OpenAI, opens with one: “Except where explicitly noted in this post, OpenAI redacted no additional information that was important to our conclusions.” The same report says “OpenAI was able to redact any non-public information from this post,” a broader power than Anthropic’s categories. I found no METR report from the Anthropic engagement yet. Whether it carries a redaction summary, and what it says, will answer more than the page does.
2. Whether Anthropic says what the October 16 edit changed, and whether the page makes its tier descriptions and its METR paragraph agree.
3. Terms for Accenture and Faculty: their tier, publication rights, what Anthropic pays them and whether any payment depends on findings.
4. Duration, rotation and time limits. The page gives none of the three.
5. OpenAI. The FT says OpenAI “has yet to lay out plans to bring in evaluators after Altman said Amodei’s proposals were a good idea.” METR was also one of the groups called in after the Hugging Face breach we covered.

Embedded evaluation may still prove useful. METR publishes its policies and says where its independence is thin. But the page is the public’s only description of the terms, it changed twice in two days, and the second change made Anthropic’s last word on the categories explicit.

Sources and method

I read Anthropic’s October 15 page live and through eight Internet Archive captures, the first at 17:21:19 UTC on October 15 and the last at 08:08:40 UTC on October 18. I reduced each capture to its article text and compared them sentence by sentence; the live page I fetched on October 18 matched the last capture. The captures are a third-party archive, while the footnote dates are Anthropic’s. I also read Dario Amodei’s essay at darioamodei.com, Anthropic’s September 18 post, Accenture’s releases of September 18, 2026 and December 9, 2025, METR’s October 14 post on its approach, its About page, its August 14 funding update and its August 26 report, the AEF-1 PDF and the September 18 letter, the OCC’s Large Bank Supervision booklet (version 1.2), the text of 12 U.S.C. 481, 12 U.S.C. 16 and 12 CFR 4.37 as published by Cornell’s Legal Information Institute, Ars Technica’s carriage of the Financial Times report (I did not open the FT’s own page) and Resultsense’s write-up. Quotations were checked against saved copies of each source.

I did not see the contracts, did not contact Anthropic, METR or Accenture, and did not test how any of these terms work in practice. I read the banking statutes only for the OCC and national banks; the FDIC and the Federal Reserve have separate but parallel powers. METR’s site showed no report from the Anthropic engagement as of October 18.