Skip to content

Benchmark

Terminal-Bench-Science 0.1

Benchmark of agentic AI performance on scientific-research tasks run in a terminal environment, reporting task pass rates and per-task cost for frontier models.

Known aliases

  • Terminal-Bench Science 0.1
  • Terminal-Bench-Science 0.1

Current stories

productConfirmed5 publishers

OpenAI's 10-cent GPT-6.1 Sol price covers only cached input

OpenAI lists GPT-6.1 Sol at $2 per million input tokens and $10 per million output; the 10-cent figure in early coverage is its cached-input rate. Teams moving work off Astra should budget on the list rates and OpenAI's per-task costs.

Perspective Coverage

5 publishers
Builder
Builder 44%
Operator
Operator 34%
Investor
Investor 22%

Reality

Evidence50
Adoption
Not scored
Hype gap+35
Incentives70
Confidence60
leadershipOne report1 publisher

OpenAI's GPT-6.1 Sol price cut rewards teams whose agents resend the same long context

OpenAI's GPT-6.1 Sol halves cached-input pricing to $0.10 per million tokens and leaves standard rates at $2 and $10. Agents that resend long context collect the saving, while other buyers weigh gains shown mostly in OpenAI's own tests.

Publishers:implicator.ai

Reality

Evidence55
Adoption30
Hype gap+20
Incentives65
Confidence55
investOne report1 publisher

Musk grades his unreleased Grok 4.7 against a superseded Anthropic model

Musk said the delayed Grok 4.7 should be roughly on par with Opus 5.0, not 5.1. Parity with what Anthropic and OpenAI already sell he put two versions further up the ladder, at Grok 4.9.

Reality

Evidence26
Adoption12
Hype gap+42
Incentives76
Confidence36
buildOne report1 publisher

Re-running five Terminal-Bench-Science tasks at $12 each leaves Fable 5.1 passing one

The published doubling was measured with a per-task allowance most teams will never grant, under a harness Anthropic has not described. The number that transfers is the one you get at your own spend cap.

Publishers:thenewstack.io

Reality

Evidence58
Adoption22
Hype gap+30
Incentives60
Confidence55
buildOne report1 publisher

Fable 5.1 doubles science benchmark score, cuts bug-hunt task time by 3.6 seconds

Anthropic charges the same for 5.1 as it does for Fable 5, so the upgrade itself is free. Whether the Terminal-Bench-Science jump from 24.7% to 52.6% reaches your queue depends on how much work Fable 5 was already failing.

Publishers:thenewstack.io

Reality

Evidence44
Adoption30
Hype gap+40
Incentives68
Confidence55