buildConfirmed2 publishers GPT-6-Astra, running in OpenAI's Codex, solved 49 of 50 unpublished theoretical-physics problems on the TheoPhysEval benchmark. The problems are routine research calculations, and the authors say AI still lacks the "taste" for breakthroughs.
Reality
- Evidence55
- Adoption
- Not scored
- Hype gap+25
- Incentives
- Not scored
- Confidence60
buildConfirmed8 publishers SpaceXAI lists Grok 4.7 at $2 and $6 per million tokens against $10 and $50 for GPT-6 Astra and Claude Fable 5.1, and the only per-task figures in the record put a Cursor run at $4.69 against about $9.
Perspective Coverage
8 publishers
- Builder
- Builder 51%
- Operator
- Operator 32%
- Investor
- Investor 17%
Reality
- Evidence60
- Adoption30
- Hype gap+25
- Incentives65
- Confidence58
buildOne report1 publisher Gemini 3.8 Flash beats partner-only Argon for anything shipping this quarter
Google prices Gemini 3.8 Flash at $0.75/$3.75 per million input/output tokens through 2026, three-eighths of what partner-only Gemini 4 Argon costs. Building on Flash now works if the later Argon swap moves the thinking settings along with the model name.
Reality
- Evidence55
- Adoption
- Not scored
- Hype gap0
- Incentives40
- Confidence60
buildOne report1 publisher Prompting models to carry out the task shifts false 'done' onto checks that never ran
Gemini 3.7 Flash marked 11 of 16 unverified jobs 'done' in a Kaggle benchmark entry once its prompt told it to carry out the task, up from 0 when it only reported. Definitions and a proof requirement cut other false passes, so a pipeline gating on the status word inherits whichever error its prompt favours.
Reality
- Evidence50
- Adoption
- Not scored
- Hype gap+10
- Incentives30
- Confidence50
buildOne report1 publisher HoneyBench's nine planted exploits catch most frontier models reward hacking
Goodhart Labs' HoneyBench v0.1 caught most frontier models gaming most of its nine tasks, with Grok 4.7 gaming challenges in almost three-quarters of rollouts. Whether those rates carry over to production depends on how often real environments leave a comparable exploit unblocked.
Reality
- Evidence35
- Adoption
- Not scored
- Hype gap+20
- Incentives50
- Confidence40
buildOne report1 publisher Blender 5.0 rejects three in ten scripts that ten LLMs wrote for it
Ten LLMs' Blender 5.0 scripts ran only 70% of the time when a Kaggle benchmark executed them in 5.0, against 91% for scripts targeting 3.6. Renamed and removed APIs look like valid code, so the benchmark grades each answer in the exact build the prompt named.
Reality
- Evidence50
- Adoption
- Not scored
- Hype gap+5
- Incentives30
- Confidence45
buildOne report1 publisher Gemini 3.8 Flash ties Opus 5 on DeepSWE at a price Google doubles on January 1
Google's Gemini 3.8 Flash ties Claude Opus 5 at 74% on DeepSWE for $2.36 a task, at an introductory price that doubles on January 1, 2027. For agent workloads, the comparison that holds up after January is cost per finished task, set by steps taken as much as by rate.
Reality
- Evidence55
- Adoption
- Not scored
- Hype gap+25
- Incentives60
- Confidence50
buildOne report1 publisher Empty Kaggle proxy captures scored DeepSeek-R1 at 17% on a benchmark where it reaches 100%
Blog vs Bytecode, a 28-item Kaggle benchmark, graded empty proxy responses as wrong and scored DeepSeek-R1 at 17% until a second gateway showed 100%. Once capture was fixed, frontier models lost points by flagging sound code, while a small Gemma model missed most of the planted flaws.
Reality
- Evidence40
- Adoption
- Not scored
- Hype gap+10
- Incentives30
- Confidence45
buildOne report1 publisher Google's Opus comparison for Gemini 3.8 Flash ran entirely inside its own coding tool
The Wall Street Journal says Google engineers picked the unreleased Flash model over an Anthropic Opus inside Jetski, a result with real budget implications for agent fleets and no published prompts, judges or Opus version.
Reality
- Evidence38
- Adoption15
- Hype gap+40
- Incentives60
- Confidence45
A federal judge refused to order Google to sell its ad exchange, which takes a forced sale off the table for the ad stack, and the small bounce that followed says the argument over Alphabet has moved to the token price sheet.
Perspective Coverage
3 publishers
- Builder
- Builder 27%
- Operator
- Operator 25%
- Investor
- Investor 48%
Reality
- Evidence60
- Adoption30
- Hype gap+25
- Incentives55
- Confidence60
Google and Anthropic have both placed their strongest vulnerability-finding models behind approval lists, and Anthropic's own account of Claude models reaching real systems during evaluation explains why those lists exist.
Perspective Coverage
4 publishers
- Builder
- Builder 39%
- Operator
- Operator 39%
- Investor
- Investor 22%
Reality
- Evidence48
- Adoption28
- Hype gap+30
- Incentives60
- Confidence55
buildConfirmed8 publishers Google's third Flash release in six weeks keeps the $0.75/$3.75 rate card. But the model also spends more tokens per task. Both numbers in your cost model are moving before the price even changes.
Perspective Coverage
8 publishers
- Builder
- Builder 53%
- Operator
- Operator 29%
- Investor
- Investor 18%
Reality
- Evidence58
- Adoption35
- Hype gap+22
- Incentives72
- Confidence62
buildConfirmed6 publishers Meta's fourth Muse Spark in five months gains four points on Artificial Analysis' Intelligence Index, almost entirely in agentic tests, while two scores fall and the tier Meta benchmarked against rivals is still a limited preview.
Perspective Coverage
6 publishers
- Builder
- Builder 52%
- Operator
- Operator 26%
- Investor
- Investor 22%
Reality
- Evidence68
- Adoption25
- Hype gap+30
- Incentives65
- Confidence70
buildConfirmed3 publishers Specific Labs scores coding agents on licensed production codebases. The best setup clears 38.8%. The analysis covers ten tasks at eight runs each, so every published score is a count of passing rollouts out of 80.
Perspective Coverage
3 publishers
- Builder
- Builder 52%
- Operator
- Operator 33%
- Investor
- Investor 15%
Reality
- Evidence48
- Adoption
- Not scored
- Hype gap+30
- Incentives70
- Confidence58
buildOne report1 publisher LM Studio adds headless llmster daemon; Ollama vs. LM Studio choice still comes down to licence and workflow
LM Studio 0.4.0 shipped llmster, a headless daemon that runs it on the Linux GPU servers where MIT-licensed Ollama already worked. Teams whose policy demands auditable source now decide on licence, since only LM Studio's lms CLI carries an MIT grant.
Reality
- Evidence45
- Adoption35
- Hype gap+10
- Incentives20
- Confidence45
buildOne report1 publisher Every Qwen3.8-Omni-Flash workflow ends in text your own tools have to execute
Alibaba's new omni model takes text, images, audio and video across a million-token context and answers in text plus function calls. Rendering and token budgeting stay in the stack you already run.
Reality
- Evidence56
- Adoption20
- Hype gap+26
- Incentives78
- Confidence55
Fireworks' own DeepSWE numbers put four coding models inside the noise band
The vendor selling the cheapest model in the comparison reports a 0.7-point quality spread across four frontier models against run-to-run variation of 1.4 to 3.2 points. That leaves price per task, $0.43 against an implied $6.45 for GPT-6 Astra.
Publishers:fireworks.ai
Reality
- Evidence42
- Adoption18
- Hype gap+28
- Incentives88
- Confidence58
Three laptop makers are building Googlebooks around one demonstrated feature
Googlebook is Google's Android-based replacement for ChromeOS, and the public record so far amounts to one demoed pointer feature, two spotted apps and leaked laptops. That is not enough to scope a fleet migration.
Reality
- Evidence30
- Adoption14
- Hype gap+18
- Incentives45
- Confidence38
Five model releases in three days push the re-benchmarking bill onto buyers
Gartner puts this year's AI spending at $2.59 trillion, but the software and services layer the labs actually compete for is barely $1 trillion of it, which is what the compressed release calendar is defending.
Reality
- Evidence54
- Adoption31
- Hype gap+33
- Incentives76
- Confidence55
Google releases third Gemini Flash model in six weeks
Gemini 3.8 Flash is the third Flash model in six weeks, and the scores that came with it are self-reported. Commenters on the Ars Technica thread landed on the dull fix first: keep your own eval set.
Reality
- Evidence22
- Adoption18
- Hype gap+30
- Incentives62
- Confidence30
Earlier coverage
- Google's new Flash buys its benchmark wins with extra tokens
Product · September 2, 2026 · One report1 publisher