Skip to content

Benchmark

ExploitBench

Benchmark measuring whether AI models can build working end-to-end exploits for known vulnerabilities in Chrome's V8 JavaScript engine.

Known aliases

  • Exploitbench

Current stories

buildConfirmed17 publishers

Astra's Critical cyber rating ships a real-time pause switch inside the Bedrock service boundary

Greg Brockman said AGI arrived with GPT-6 Astra on September 3. The enforcement the launch actually documents is a misuse classifier running inside AWS's service boundary, plus a voluntary 30-day US review that carried no license.

Perspective Coverage

17 publishers
Builder
Builder 39%
Operator
Operator 37%
Investor
Investor 24%

Reality

Evidence58
Adoption42
Hype gap+45
Incentives72
Confidence60
buildConfirmed4 publishers

Post-training alone took GLM-5.3 from 4.6 to 28.3 on Terminal-Bench 3.0

Z.ai says the base model did not change between GLM-5.2 and GLM-5.3, so the coding jump and the doubled exploitation score come out of the same post-training run. Security teams inherit the second half.

Perspective Coverage

4 publishers
Builder
Builder 49%
Operator
Operator 32%
Investor
Investor 19%

Reality

Evidence50
Adoption30
Hype gap+30
Incentives70
Confidence60
buildConfirmed4 publishers

Anthropic says a freely downloadable model builds exploits nearly as well as its restricted tool

Anthropic says Zhipu's freely downloadable GLM-5.3 built working V8 exploits in 50 of 410 tries, against 56 for its own restricted Claude Mythos Preview. With the weights public, its safeguards come off cheaply, so a lab that restricts its own model no longer keeps the capability out of reach.

Perspective Coverage

4 publishers
Builder
Builder 41%
Operator
Operator 38%
Investor
Investor 21%

Reality

Evidence62
Adoption30
Hype gap+15
Incentives72
Confidence62
leadershipOne report1 publisher

Anthropic finds attackers can bypass GLM-5.3's exploit safeguards up to 100% of the time

Anthropic says Zhipu AI's downloadable GLM-5.3 builds cyber exploits on its own, with safeguards that fail against simple attacks up to 100% of the time. It puts a capability once confined to gated frontier models within anyone's reach.

Publishers:anthropic.com

Reality

Evidence55
Adoption
Not scored
Hype gap+20
Incentives75
Confidence50
buildConfirmed5 publishers

GLM-5.3 keeps GLM-5.2's base model and claims 50% more on coding: plan for shorter eval cycles

Z.ai says every gain in GLM-5.3 came from post-training on an unchanged base. If that holds, refresh cadence for self-hosted weights is set by RL runs, not pretraining runs.

Perspective Coverage

5 publishers
Builder
Builder 58%
Operator
Operator 33%
Investor
Investor 9%

Reality

Evidence40
Adoption30
Hype gap+35
Incentives70
Confidence55
investConfirmed5 publishers

OpenAI grades its own unreleased Astra model Critical for autonomous zero-day discovery

The top rung of OpenAI's Preparedness Framework has now been reached by OpenAI, on a model it has not shipped, which moves AI-assisted exploitation out of argument and into a named vendor's published paperwork.

Perspective Coverage

5 publishers
Builder
Builder 28%
Operator
Operator 42%
Investor
Investor 30%

Reality

Evidence35
Adoption3
Hype gap+30
Incentives70
Confidence55
securityConfirmed4 publishers

OpenAI gates a 100% ExploitBench model behind refusals it plans to loosen in weeks

Astra scored a perfect 100% on OpenAI's own exploit-development benchmark, against 78.5% for GPT-5.6 Sol, and the shipped model's refusal to write proof-of-concept code is a policy the company has already said it will relax.

Perspective Coverage

4 publishers
Builder
Builder 30%
Operator
Operator 42%
Investor
Investor 28%

Reality

Evidence35
Adoption20
Hype gap+45
Incentives70
Confidence55
investOne report1 publisher

OpenAI's post-launch edits doubled Astra's math lead over Anthropic's Fable

Vendor benchmark tables are dated snapshots, and the GPT-6 Astra launch shows how much can move inside one afternoon without the headline score changing, which matters for anyone scoring a purchase off one.

Publishers:fortune.com

Reality

Evidence60
Adoption
Not scored
Hype gap+40
Incentives70
Confidence58
productConfirmed18 publishers

OpenAI's GPT-6 Astra pairs harder-to-monitor reasoning with a pledge to pause scaling if oversight slips

OpenAI's new model thinks repeatedly before it acts, and according to Manifold Security's CTO it usually does so without leaving the reasoning trace that agent audits read. Oversight moves to the buyer.

Perspective Coverage

18 publishers
Builder
Builder 32%
Operator
Operator 38%
Investor
Investor 30%

Reality

Evidence52
Adoption25
Hype gap+45
Incentives72
Confidence60
investConfirmed8 publishers

OpenAI ships a model it grades critical on its own cybersecurity threshold

Astra scored 100% on OpenAI's exploit-conversion benchmark with production safeguards switched off, and reached API and AWS customers the same week, with a refusal layer standing in for delay.

Publishers:cnbc.com decrypt.co en.sedaily.com indianexpress.com lennysnewsletter.com nbcnews.com pymnts.com seekingalpha.com

Perspective Coverage

8 publishers
Builder
Builder 36%
Operator
Operator 27%
Investor
Investor 37%

Reality

Evidence45
Adoption40
Hype gap+40
Incentives75
Confidence60
buildOne report1 publisher

Astra bills at long-context rates once a request passes 30 percent of its input window

OpenAI's spec lets gpt-6-astra take 922,000 input tokens, but requests above 272,000 move to higher long-context rates. Provisioning a model that drives a desktop, a shell and MCP servers starts with that threshold.

Publishers:dev.to

Reality

Evidence40
Adoption
Not scored
Hype gap+25
Incentives55
Confidence40
buildOne report1 publisher

Stripped GLM-5.3-Flash weights show what Z.ai's MIT license permits

DeAlignAI's downloadable FP8 build is the license working exactly as written, while its self-reported 320-of-320 HarmBench run remains unchecked by any outside researcher and measures compliance, not capability.

Publishers:runtimewire.com

Reality

Evidence46
Adoption20
Hype gap+18
Incentives72
Confidence55
productOne report1 publisher

ARC Prize's own harness scores GPT-6 Astra 37 points below OpenAI's adapter

ARC Prize ran GPT-6 Astra under two scaffolds and published both, 62.7% through its own minimal interface and 99.9% through OpenAI's Provider Adapter, which cost less. An eval that leaves the harness loose is scoring plumbing.

Publishers:thenextweb.com

Reality

Evidence68
Adoption55
Hype gap+64
Incentives72
Confidence62
productOne report1 publisher

OpenAI ships Astra into enterprise workspaces switched off by default

Reaching GPT-6 Astra costs a few days of patience rather than a $200 seat, so the gate worth budgeting against is Astra Pro, which Plus plans do not get, plus the credits sold on top of existing allowances.

Publishers:9to5mac.com

Reality

Evidence40
Adoption25
Hype gap+35
Incentives70
Confidence55
investOne report1 publisher

Astra's 99.9% holds up only on the harness OpenAI ran itself

ARC Prize put GPT-6 Astra at 62.7% against OpenAI's 99.9%, and the third-party composite index has it 0.3 points above the model it replaces, which leaves the 20% safety compute overhead as the clearest number in the launch.

Publishers:techtimes.com

Reality

Evidence57
Adoption42
Hype gap+58
Incentives74
Confidence54
investOne report1 publisher

OpenAI rates GPT-6 Astra capable of hacking hardened systems without human guidance

The 100% ExploitBench score and two fresh V8 zero-days are OpenAI's own numbers, one of them still unverified, but the "critical" designation is a dated document that every agent deployer's controls will now be read against.

Publishers:decrypt.co

Reality

Evidence28
Adoption22
Hype gap+58
Incentives82
Confidence34
leadershipOne report1 publisher

Every notable AI release today arrived with a grade written by its own vendor

OpenAI says Astra is the first model in any risk domain to reach Critical, on a score no outsider has seen. Buyers weighing today's releases have to grade the grader before they grade the model itself.

Publishers:implicator.ai

Reality

Evidence46
Adoption38
Hype gap+30
Incentives78
Confidence50
investOne report1 publisher

OpenAI holds two unpatched zero-days its own benchmark run produced

Astra found the flaws while working through a twenty-vulnerability internal test, and OpenAI's answer is to ration offensive capability to vetted defenders rather than sell it by the token. That reprices exploit labour.

Publishers:techtimes.com

Reality

Evidence33
Adoption28
Hype gap+34
Incentives76
Confidence40
scienceOne report1 publisher

OpenAI declares Astra the first model to reach its Critical cyber threshold

OpenAI says Astra can find unknown flaws and chain them into working exploits without a human guiding each step. The evidence published so far is one saturated public benchmark plus a 20-vulnerability internal set, in which the model found two zero-days of its own.

Publishers:openai.com

Reality

Evidence30
Adoption10
Hype gap+35
Incentives78
Confidence45

Earlier coverage

  1. Z.ai held back its own GLM-5.3 weights, and open-weight roadmaps have a new failure mode

    Leadership · August 17, 2026 · Confirmed3 publishers

  2. GLM-5.3 kept the base model and bought ten times the environments instead

    Build · August 14, 2026 · Confirmed2 publishers