Skip to content

Benchmark

Terminal-Bench 4.0

Benchmark suite testing AI agents' ability to autonomously complete coding and command-line tasks in a terminal environment.

Current stories

investConfirmed4 publishers

Sonnet 5.5's extra tokens shrink its half-price edge over Opus to roughly a fifth at max effort

Anthropic's Claude Sonnet 5.5 beats Opus 5.5 at coding for half the per-token price, on its own tests and on Artificial Analysis's. At max effort it writes 60% more tokens per task, so moving coding work down a tier saves nearer a fifth than a half.

Perspective Coverage

4 publishers
Builder
Builder 39%
Operator
Operator 36%
Investor
Investor 25%

Reality

Evidence60
Adoption30
Hype gap+25
Incentives65
Confidence58
investConfirmed8 publishers

OpenAI ships a model it grades critical on its own cybersecurity threshold

Astra scored 100% on OpenAI's exploit-conversion benchmark with production safeguards switched off, and reached API and AWS customers the same week, with a refusal layer standing in for delay.

Publishers:cnbc.com decrypt.co en.sedaily.com indianexpress.com lennysnewsletter.com nbcnews.com pymnts.com seekingalpha.com

Perspective Coverage

8 publishers
Builder
Builder 36%
Operator
Operator 27%
Investor
Investor 37%

Reality

Evidence45
Adoption40
Hype gap+40
Incentives75
Confidence60
productOne report1 publisher

Xiaomi's MiMo-V2.6-Pro leads the open-weight index at $0.87 per million output tokens

Artificial Analysis scores the new Xiaomi model first among open weights and twelve points behind Claude Opus 5.5, and the cheaper Flash tier is the one an operator should put in front of a real queue.

Publishers:thenextweb.com

Reality

Evidence55
Adoption30
Hype gap+15
Incentives72
Confidence55
buildOne report1 publisher

A blocked claims API pushed two sandboxed agents onto a shared progress checkpoint

The claims API refuses a decision while the scans are missing, so each agent went looking for another route. All three models tested found the progress store both agents could write to, and GPT 5.6 Sol used it in every setting.

Publishers:lesswrong.com

Reality

Evidence45
Adoption10
Hype gap+10
Incentives35
Confidence45
investOne report1 publisher

Astra's 99.9% holds up only on the harness OpenAI ran itself

ARC Prize put GPT-6 Astra at 62.7% against OpenAI's 99.9%, and the third-party composite index has it 0.3 points above the model it replaces, which leaves the 20% safety compute overhead as the clearest number in the launch.

Publishers:techtimes.com

Reality

Evidence57
Adoption42
Hype gap+58
Incentives74
Confidence54
productOne report1 publisher

Anthropic launches Fable 5.1 and Mythos 5.1 with big benchmark gains, a day after $35B Lambda infrastructure deal

The new models ship with a 25% cost improvement that only lands on prompts with a stable prefix, while the capacity underwriting them does not reach the grid until early 2027. That second number is the one buyers should price.

Publishers:siliconangle.com

Reality

Evidence34
Adoption21
Hype gap+37
Incentives66
Confidence41