Topic
Standardized tests and scoring suites used to measure and compare AI model capabilities—coding, reasoning, safety—across vendors and versions.
Mercor's Benchmark Map finds only 208 of 923 US occupations have a task with five or more AI benchmark prompts mapped to it. The report says about 90% of paid hours fall in tasks benchmarks barely cover, so leaderboard scores speak to a thin slice of work.
Reality
Google DeepMind's Gemini 4 Argon scored 97.5% on ARC-AGI-3 at a $4,800 run cost in ARC Prize testing, second to GPT-6 Astra's 99.9%. The score depends on vendor-specific harness features, so it says less about a team's own workload than the headline implies.
Reality
Meta launched Muse Wonder and Muse Wonder Full and says they rival the best models from OpenAI and Anthropic. Both reports rest on Meta's own benchmark figures, so teams choosing a vendor have one more candidate to test on their own work.
Perspective Coverage
26 publishersReality
xAI's Grok Voice 3.0 tops Artificial Analysis' speech-to-speech index at $0.08 a minute, the same price as Grok Voice 2.0. The cost case is a per-task comparison with two rivals, and runtimewire reports text input is billed separately.
Perspective Coverage
7 publishersReality
Yale's John Sous says correcting errors in physics benchmark problems lifted cutting-edge AI scores from as low as 30% to as high as 94%. Well-defined problems may be within reach, but the case that physics will follow maths is still open.
Reality
Meta released Muse Wonder and Muse Wonder Full on Oct. 12, with the Full version limited to paying subscribers. CNBC reported monthly plans between $20 and $200, so Meta's rebuilt AI team now has to earn subscriptions as well as benchmark scores.
Perspective Coverage
25 publishersReality
GPT-6-Astra, running in OpenAI's Codex, solved 49 of 50 unpublished theoretical-physics problems on the TheoPhysEval benchmark. The problems are routine research calculations, and the authors say AI still lacks the "taste" for breakthroughs.
Reality
Google says Gemini 4 Argon turned one photo into more than 4,500 training episodes that raised a real robot's success rate from 56 to 75 percent. The Decoder reports the demos still need detailed human prompting, and access is through paid developer programs.
Reality
GPT-4.1 scored 93.49% on BrowseComp-Plus with documents supplied and 14.58% when BM25 searched, according to The New Stack. That 78.9-point gap sizes the room for retrieval work on one fixed corpus. Production failure rates need their own measurement.
Reality
MIT researchers found the best models they tested solved about 35% of theory problems in mid-2026, up from 2% in mid-2024. The study covers only theory fields that need no lab work, and one author says answers still need verifying.
Reality
Google made Gemini 4 Argon widely available and says it ranks #1 on the Vals Index at 68.90%. On the CWE-bench v1 security benchmark it ties two other models at 68%.
Reality
Odyssey opened a free public research preview of Odyssey-3, a world model that generates interactive environments from text prompts in real time. The demo runs the Flash variant, while the 66.1 physics score Odyssey cites came from its Pro model on a best-of-eight run.
Reality
4DCodeBench, a 200-scene benchmark from Stanford's Ziga Kovacic and collaborators, finds the top coding agent scores 0.91 on static metrics but 0.67 on motion. Teams using agents to turn video into simulation code should test the motion on its own.
Reality
Google DeepMind's Gemini 4 Argon, priced at half of Claude Opus 5.5, goes first to vetted cyber defenders, and paid API customers have no date. Until Google sets one, teams can only check its benchmark claims on paper.
Reality
GitHub's ReviewBench grades AI code reviewers on 219 pull requests from 187 open-source repositories, counting real issues found and false alarms raised. Security teams get a shared, repeatable check before letting a bot gate a release, on a test GitHub also uses to tune its own Copilot reviewer.
Reality
Hone raised a $60 million seed round led by Benchmark and Index Ventures for AI agents that pursue business goals over weeks or months. Buyers have only Hone's own account of its safeguards to judge them by, because Hone has not disclosed revenue or customer results.
Reality
Google announced Gemini 4 Argon, its first Gemini 4 model, with early access limited to internal teams and vetted cyber defenders in its Fairwind program. The published API price doubles once an introductory window closes. Teams cannot yet test the coding and long-output claims they would budget around, because there is no public access.
Reality
Mistral launched a trillion-parameter Large 4 preview Tuesday and is selling its open weights as insurance against a closed vendor cutting off a capability. Buyers get a model no supplier can retire and pay for it with a coding score about twelve points behind the leading closed systems.
Perspective Coverage
3 publishersReality
TII's Falcon-ASR averaged a 20.92% word error rate on six Arabic test sets, under the 23.17% best published result. Its lead on Emirati speech, the dialect it was built around, comes from a test set TII assembled and scored itself.
Perspective Coverage
3 publishersReality
Scale AI and Elorian's Humanity's Sixth Sense benchmark scores the best multimodal model at 53.6% of tasks, against 93.1% for human participants. Teams building on multimodal models should test spatial and social inference on their own footage before relying on it.
Reality
Invest · October 2, 2026 · Confirmed12 publishers
Build · September 10, 2026 · Confirmed17 publishers
Build · October 6, 2026 · One report1 publisher
Product · October 6, 2026 · Confirmed5 publishers
Product · October 1, 2026 · Confirmed14 publishers
Build · September 30, 2026 · Confirmed10 publishers
Leadership · September 30, 2026 · Confirmed6 publishers
Build · October 3, 2026 · One report1 publisher
Security · October 1, 2026 · Confirmed8 publishers
Build · October 2, 2026 · One report1 publisher
Invest · October 1, 2026 · Confirmed4 publishers
Science · October 2, 2026 · One report1 publisher
Build · September 30, 2026 · One report1 publisher
Build · September 30, 2026 · One report1 publisher
Product · September 30, 2026 · Confirmed5 publishers
Leadership · September 29, 2026 · One report1 publisher
Invest · September 27, 2026 · Confirmed2 publishers
Invest · September 1, 2026 · Confirmed2 publishers
Product · September 3, 2026 · Confirmed3 publishers
Build · September 3, 2026 · One report1 publisher
Security · September 4, 2026 · Confirmed4 publishers
Science · September 3, 2026 · Confirmed2 publishers
Build · September 4, 2026 · Confirmed2 publishers
Product · September 15, 2026 · Confirmed3 publishers
Build · September 18, 2026 · Confirmed2 publishers
Leadership · September 22, 2026 · Confirmed2 publishers
Product · September 22, 2026 · Confirmed8 publishers
Leadership · September 23, 2026 · One report1 publisher
Build · September 23, 2026 · One report1 publisher
Product · September 21, 2026 · One report1 publisher
Build · September 21, 2026 · One report1 publisher
Build · September 20, 2026 · One report1 publisher
Invest · September 19, 2026 · One report1 publisher
Build · September 19, 2026 · One report1 publisher
Build · September 18, 2026 · One report1 publisher
Build · September 17, 2026 · One report1 publisher
Product · September 15, 2026 · One report1 publisher
Build · September 12, 2026 · Confirmed3 publishers
Build · September 10, 2026 · One report1 publisher
Build · September 9, 2026 · One report1 publisher