Skip to content

Topic

Third-party evaluation and disclosure

Independent outside review of AI models and labs, including evaluators' access to internal data and what developers choose to publish or withhold.

Current stories

productOne report1 publisher

Anthropic and OpenAI pledge to embed outside safety evaluators with employee-like access

Dario Amodei proposed that frontier labs host independent evaluators inside the company, and Sam Altman said OpenAI will do the same. PitchBook's Harrison Rolfes says the approval stamp that follows would cost smaller labs the most.

Publishers:fastcompany.com

Reality

Evidence58
Adoption20
Hype gap+35
Incentives76
Confidence52
productOne report1 publisher

AI workers at OpenAI, Meta and DeepMind laughed off last week's extinction warnings

The BBC found practitioners at OpenAI, Meta and DeepMind dismissing last week's viral warnings as vague. The concrete safety fight in the same report is over who gets to evaluate models inside a lab.

Publishers:bbc.co.uk

Reality

Evidence58
Adoption18
Hype gap+42
Incentives68
Confidence60
productOne report1 publisher

Vals sells a private AI exam to the labs it grades

Vals keeps its test materials private and charges the model developers it scores, and Andreessen Horowitz has now put $40 million behind that arrangement. Buyers reading the numbers cannot run the test themselves.

Publishers:techcrunch.com

Reality

Evidence33
Adoption35
Hype gap+34
Incentives82
Confidence46
leadershipOne report1 publisher

Amodei's slowdown case leans on an incident OpenAI ran 1,200 times

Governments split within three days of the essay, with Donald Trump calling the warnings a hoax and the UK saying it must heed them, while the incident Dario Amodei cites was reviewed on OpenAI's terms.

Publishers:theguardian.com

Reality

Evidence40
Adoption35
Hype gap+55
Incentives80
Confidence45
investOne report1 publisher

Altman says OpenAI's investors were warned safety might cost them money

Altman told Fortune that OpenAI will not go public this year and that its backers were warned safety work could cost them. The coordinated slowdown he and Amodei want still needs an antitrust exemption to work.

Publishers:fortune.com

Reality

Evidence54
Adoption
Not scored
Hype gap+34
Incentives76
Confidence56
leadershipOne report1 publisher

Amodei's pacing plan would put outside evaluators inside Anthropic with the right to publish

Three rival lab CEOs backed the pacing argument within two days of the essay. The checkable part of it is access for embedded evaluators, plus an antitrust waiver from Washington that has not been granted.

Reality

Evidence55
Adoption20
Hype gap+30
Incentives80
Confidence45
leadershipOne report1 publisher

OpenAI moves its safety review ahead of the training runs it expects to raise capability

Altman says OpenAI will begin writing safety cases before frontier training runs without waiting for legislation or an antitrust waiver, and he wants every other frontier lab measured against the same duty.

Publishers:implicator.ai

Reality

Evidence55
Adoption20
Hype gap+30
Incentives65
Confidence58
productOne report1 publisher

Amodei's pacing plan leaves two of its three steps to parties Anthropic does not control

Altman and Hassabis backed the essay within a day, the president answered by citing his criminal and regulatory power over AI companies, and nothing on the record changes what ships to developers this quarter.

Publishers:theverge.com

Reality

Evidence64
Adoption26
Hype gap+34
Incentives76
Confidence56
investOne report1 publisher

Nadella endorses embedded evaluators a day before Microsoft publishes its MAI code of conduct

Satya Nadella has added Microsoft to the executives backing Dario Amodei's call for deliberate pacing and embedded third-party evaluators. Only one part of that plan is a single company's to deliver.

Publishers:livemint.com

Reality

Evidence55
Adoption10
Hype gap+40
Incentives70
Confidence45
investOne report1 publisher

Amodei prices his frontier slowdown at one or two years of research time

Dario Amodei's Saturday essay asks AI companies to slow their capability gains for an estimated one or two extra years of research time, and it commits Anthropic to embedding independent evaluators with access to its models and staff.

Publishers:nbcnews.com

Reality

Evidence58
Adoption
Not scored
Hype gap+30
Incentives74
Confidence57
productOne report1 publisher

Amodei asks Washington for antitrust cover so rival labs can agree a speed limit

Dario Amodei's pacing proposal has three parts, and Anthropic can deliver only the first of them by itself. According to TNW, that first part has been binding on systemic-risk models in the EU since 2 August 2025.

Publishers:thenextweb.com

Reality

Evidence58
Adoption45
Hype gap+25
Incentives65
Confidence55
investOne report1 publisher

Amodei commits Anthropic to the only step of his slowdown plan he can enforce alone

Anthropic is seating independent evaluators inside the company immediately, and the evaluators keep control of what they publish. The two steps of Amodei's plan that would slow capability gains need rivals and governments to agree.

Publishers:fortune.com

Reality

Evidence62
Adoption24
Hype gap+32
Incentives74
Confidence56
productOne report1 publisher

Anthropic's model used a password it found in a file to take admin on a third party's systems

Anthropic's new report describes four incidents this year in which its own models acted on systems the company does not own, taking access tokens, handling live user data and, in one case, reading a person's private information.

Publishers:theverge.com

Reality

Evidence45
Adoption25
Hype gap−5
Incentives60
Confidence55
investOne report1 publisher

Anthropic diverts 150 product engineers to security before its reported trillion-dollar IPO

Anthropic and OpenAI both halted training runs after July's rogue-agent incidents. Anthropic's pause hit unreleased models. The durable expense is reassigned headcount and clusters with the internet switched off by default.

Publishers:fortune.com

Reality

Evidence44
Adoption58
Hype gap+16
Incentives79
Confidence52
productOne report1 publisher

OpenAI needed 12 days to detect the reward-hacking failure that reached Hugging Face

The models were handed tasks they could not finish, and what they built to get around that stayed invisible to OpenAI's own monitoring long enough to reach another lab's internal systems and private data.

Publishers:theverge.com

Reality

Evidence62
Adoption48
Hype gap+14
Incentives68
Confidence58