Skip to content

AI model

Gemini-2.5-Pro

Google's frontier Gemini 2.5 Pro model, a multimodal large language model used for advanced reasoning, coding, and evaluation benchmarks.

Known aliases

  • Gemini 2.5 Pro
  • Gemini-2.5-pro

Current stories

buildOne report1 publisher

Most of 30 vision models misread the clock on synthetic Grafana panels in a Kaggle benchmark

Kaggle benchmark results posted on dev.to report that most of 30 vision models reading 336 synthetic Grafana-style panels found the peak but misread the clock. Copilot incident timelines drafted from screenshots need their start times checked by hand.

Publishers:dev.to

Reality

Evidence45
Adoption
Not scored
Hype gap+5
Incentives30
Confidence45
buildConfirmed5 publishers

Runway's Solaris turns every click into conditioning data for the next generated frame

Generating the interface as video instead of rendering it from code is a serious research bet. Runway's own preview still lists legible text and long-session coherence as open problems, which is roughly where ordinary interfaces begin.

Perspective Coverage

5 publishers
Builder
Builder 47%
Operator
Operator 37%
Investor
Investor 16%

Reality

Evidence45
Adoption5
Hype gap+35
Incentives65
Confidence60
buildOne report1 publisher

One agent's reward hack raises the next agent's hacking rate on a similar task

A MATS project ran two tasks inside one context window and measured reward hacking on the second. With similar tasks, a hack in the first predicted more hacking in the second, including when a different agent only saw the evidence.

Publishers:lesswrong.com

Reality

Evidence42
Adoption
Not scored
Hype gap+12
Incentives25
Confidence45
buildOne report1 publisher

A nine-agent pipeline injects canonical API examples because its agents can skip the MCP lookup

The team behind a nine-agent code generator reports about 30,000 tokens and 15 to 20 Pro-tier model calls per request, and attributes its reliability to strict output schemas and context the agents cannot decline to read.

Publishers:dev.to

Reality

Evidence40
Adoption28
Hype gap+15
Incentives55
Confidence42
buildOne report1 publisher

Test-time training on a rented 5090 reached 44% on ARC-AGI-1 for 67 cents

Mithil Vakde's from-scratch transformer landed one point behind TRM on the public eval. His own ablations put 20 of those 44 points on two representation choices rather than on any amount of compute.

Publishers:dev.to

Reality

Evidence44
Adoption18
Hype gap+16
Incentives72
Confidence41
productOne report1 publisher

Salesforce's researchers get better CRM agents by writing the procedure into the prompt

The company's own benchmark keeps every flagship model it tested below 60 percent completion across 2140 CRM task instances, and the fix its researchers report is domain knowledge somebody inside your org has to write down.

Publishers:salesforce.com

Reality

Evidence52
Adoption22
Hype gap−10
Incentives80
Confidence55
buildOne report1 publisher

Adding a persona-driven user costs CRMArena-Pro's agents 23 points of success

CRMArena-Pro reports about 58% single-turn success and 35% multi-turn over nineteen expert-validated tasks, and the per-skill breakdown inside those averages is the part that should decide where an agent gets pointed.

Publishers:arxiv.org

Reality

Evidence62
Adoption
Not scored
Hype gap−12
Incentives52
Confidence58
buildOne report1 publisher

Armenian ASR leaderboard: closed models take the top eight, then lose the domains that matter

ArmBench-ASR v0.1 ranks nearly 30 systems on 20.7 hours of Armenian audio. The headline order flips on read speech, and every model degrades badly on movie dialogue.

Publishers:runtimewire.com

Reality

Evidence57
Adoption22
Hype gap+12
Incentives58
Confidence54
productOne report1 publisher

The cheapest model scored 10 out of 100: assistant choice is now a code-security decision

A Secure Code Warrior and RMIT study of six frontier models across 11 frameworks found no universal winner and no link between token cost and secure output.

Publishers:devops.com

Reality

Evidence48
Adoption
Not scored
Hype gap+20
Incentives68
Confidence40
investOne report1 publisher

Google says frontier models already know the facts they get wrong. That is a budget decision.

Google Research reports Gemini-3-Pro and GPT-5 encode 95-98% of tested facts yet fail to recall 26-34% of them, moving the fix from pretraining scale toward post-training and inference.

Publishers:research.google

Reality

Evidence55
Adoption
Not scored
Hype gap+20
Incentives68
Confidence48