Claude graded the index that credits it with leading 26% of Anthropic's R&D
Anthropic has put a number on how much of its own AI research Claude now runs. The number comes out of a pipeline in which a Claude agent catalogued the tasks and a separate Claude judge scored them.
The Board Room · Leadership desk
What happened
- Anthropic said Claude "leads" 26% of its AI research and development work as of August 2026, measured on an automation scale developed by Epoch AI that runs from no AI involvement to no human in the loop.
- The ratings came from Anthropic's own model: a Claude research agent assessed each category and a separate Claude judge assigned the automation level, with employees scoring their own work areas blind to the model's evidence.
- For the week of July 13 to 20, Anthropic classified about 6% of its AI R&D compute as safety work, rising to about 12% of the compute used in AI-driven AI R&D.
Why it matters
- What it limits constraint The 542-node basket was frozen in July, so every later reading scores a fixed task list. An index that climbs on that basket cannot tell a manager whether the underlying work changed shape.
- Who has to decide decision AL4 keeps the deployment call with an engineer, so the number is not evidence that senior sign-off has moved. A leader citing 26% to justify headcount decisions is extrapolating well past what was measured.
- Who's exposed exposure The figures are Anthropic's own, produced largely by Claude, and the report does not say any outside party has checked them. Anyone quoting 26% to a board or a regulator inherits that provenance.
- What doesn't add up contradiction implicator.ai dates the sub-1% baseline to February 2026 and Business Insider to March, so the slope everyone will quote is uncertain by a month at the point where it is steepest.
Next story · Leadership · Oct 3
Altman warns against surrendering judgment to AI while OpenAI loses a safety leaderConfirmed3 publishers
Anthropic published two agreement rates, and the distance between them is what a reader has to judge. Its model's ratings matched staff ratings exactly 59% of the time; pairs of employees matched each other 35% of the time [3]. Model and human ratings landed within one level of each other 97% of the time [4]. One reading is that the model applies the scale more consistently than the people whose work it scores. The other is that 59% is about 1.7 times a rate that is itself barely one in three [16], which says the scale is hard to apply. Anthropic acknowledged that "the 'judge' model could make the same kinds of errors as the model it is checking" [15].
The basket being scored was built in July and then locked. For each week of July 2026, Anthropic randomly sampled 20% of staff in the departments that make up its model R&D loop, and a Claude research agent reviewed each sampled person's week using Slack and internal documentation [5]. That produced about 15,000 granular tasks, which Claude arranged into 542 nodes, roughly 28 tasks per node [6][17]. Anthropic froze the basket for later measurements [6]. So the August reading describes July's work.
AL4 describes supervised work. Anthropic's own example is a failed data pipeline: Claude inspects the logs, finds and tests a fix, handles surprises and compares the repaired output with the last good run, and an engineer still decides whether to deploy it [9]. "Claude is not operating fully autonomously for any measured subset of AI R&D work," Anthropic said [8]. The company also said the share of work at or above "AI collaborates" is above 90% [18], which puts more than 64% of the measured work at collaboration levels below "leads" [26].
The oversight numbers are large in one direction and small in the other. About 30,000 agents ran at once on Anthropic's most-used internal research and engineering platform in August, every action passing through an online monitor, typically within seconds [10]. Blocking about one in 47,000 of more than a billion decisions works out to roughly 21,000 blocked actions in the month, each reviewed by a person within a week [11][19]. A separate offline system flags about 100,000 transcripts a week, and about 50 high-priority cases reach people weekly, near 215 across a 30-day month [12][20]. Anthropic wrote, "We have been doing this kind of monitoring for months, not years" [13].
A skeptic would say a lab grading its own automation with its own model will arrive at a flattering number. Two things in the report cut against that. Anthropic classified about 6% of its AI R&D compute as safety work in the week of July 13 to 20, and about 12% of the compute used in AI-driven AI R&D, calling both estimates conservative because mixed-purpose work counted as capabilities research [21]. It also said one week is enough to show the measurement can be made and not enough to show a trend [22]. Anthropic said it plans to embed third-party evaluators and is setting them up [7].
The surrounding argument shapes how the figure travels. An Anthropic researcher resigned last week and accused leading AI companies of "gambling with our lives", and chief executive Dario Amodei has since called for frontier AI companies to coordinate on slowing development, according to Business Insider [14]. Anthropic said other frontier labs could publish the same figures and allow third parties to verify them [23]. "We are reporting these measurements because they give the public, third parties, and governments better visibility into the pace of AI development inside frontier labs," the company said [24].
What to watch
- A second reading against the same 542-node basket, which would show whether 26% is a level or a slope.
- Whether third-party evaluators score the same nodes and land at lower automation levels than Claude's judge did.
- Any redefinition of the 542 nodes, which would reset the series before it has two comparable points.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence45
- Adoption60
- Hype gap+25
- Incentives60
- Confidence55
Perspective Coverage
3 publishers- Builder
- Builder 27%
- Operator
- Operator 33%
- Investor
- Investor 40%
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
Anthropic said Claude "leads" 26% of its AI research and development work as of August 2026, on an automation scale developed by Epoch AI.
- [2]
The Anthropic R&D Automation Index rates tasks from AL0, with no AI involvement, to AL5, where AI works without a human in the loop. AL3 means Claude completes large portions under close direction. At AL4, "leads," Claude can complete most of a task end-to-end from a high-level prompt while a human supervises.
- [3]
A Claude research agent assessed each category and a separate Claude judge assigned an automation level; Anthropic employees rated their own work areas without seeing the model's evidence. The model and employees matched exactly 59% of the time, while pairs of employees matched exactly 35% of the time.
- [4]
Model and human ratings fell within one level of each other 97% of the time.
- [5]
For each week of July 2026, Anthropic randomly sampled 20% of staff in the departments that make up its model R&D loop, and a Claude research agent reviewed each sampled person's week using Slack and internal documentation.
- [6]
The review produced about 15,000 granular tasks; Claude arranged them into 542 nodes, and Anthropic froze that basket for later measurements.
- [7]
The report does not say any outside party has checked the figures, which are Anthropic's own measurements, largely produced by Claude. Anthropic says it plans to embed third-party evaluators and is setting them up.
- [8]
"Claude is not operating fully autonomously for any measured subset of AI R&D work," Anthropic said.
- [9]
For a failed data pipeline, an AL4 Claude could inspect logs, find and test a fix, handle surprises and compare the repaired output with the last good run. An engineer would still decide whether to deploy it.
- [10]
About 30,000 agents were working simultaneously on Anthropic's most-used internal research and engineering platform in August 2026; the figures cover that platform only, and every action passed through an online monitor, typically within seconds.
- [11]
Of more than one billion agent decisions during August 2026, the monitor blocked about one in 47,000, or 0.002%. Humans review blocked actions within one week.
- [12]
A separate offline system flags about 100,000 transcripts each week, a rate Anthropic gave as roughly one or two transcripts in every thousand, and about 50 high-priority cases reach people weekly.
- [13]
Anthropic wrote, "We have been doing this kind of monitoring for months, not years."
- [14]
An Anthropic researcher resigned last week and accused leading AI companies of "gambling with our lives" by pushing ahead with increasingly capable models. Anthropic CEO Dario Amodei has since called for frontier AI companies to coordinate on slowing development to give safety measures more time to catch up.
- [15]
Anthropic acknowledged that "the 'judge' model could make the same kinds of errors as the model it is checking."
- [16]
The model-staff exact agreement rate is about 1.7 times the employee-employee rate.
- [17]
The 542 nodes average roughly 28 granular tasks each.
- [18]
"The share of work at or above 'AI collaborates' is above 90%," Anthropic said.
- [19]
About 21,000 agent actions were blocked during August 2026.
- [20]
About 50 high-priority escalations a week is near 215 over a 30-day month.
- [21]
From July 13 through July 20, 2026, Anthropic classified about 6% of its AI R&D compute as safety work, and about 12% for compute used in AI-driven AI R&D. It called both estimates conservative because mixed-purpose work was counted as capabilities research, while safeguards classifiers were excluded.
- [22]
Anthropic said one week is enough to show that the measurement can be made, but not enough to show a meaningful trend, and that compute measures spending rather than effort.
- [23]
Anthropic said other frontier labs could publish the same figures and allow third parties to verify them.
- [24]
"We are reporting these measurements because they give the public, third parties, and governments better visibility into the pace of AI development inside frontier labs," the company said.
- [25]
Anthropic described recursive self-improvement as "a model fully autonomously building its successor".
- [26]
More than 64% of the measured AI R&D work sits at or above "AI collaborates" but below "leads".
- [27]
Anthropic said the figure was under 1% in February 2026.
- [28]
Business Insider reported that Anthropic said the figure was below 1% in March.
Sources
3 independent publishers whose own reporting we read for this story.
- businessinsider.comAnthropic says Claude is now leading more than a quarter of its AI research
1 article · September 17, 2026
- cryptobriefing.comAnthropic relies on AI for over 25% of R&D, raising oversight concerns
1 article · October 7, 2026
- implicator.aiAnthropic Says Claude Leads 26% of Its AI R&D Work
2 articles · September 18, 2026
Topics and entities
Entities
- OpenAI
- Claude
- Anthropic R&D Automation Index
- Epoch AI
- Dario Amodei
- Anthropic