Newer AI models caught less fraud in Coinbase's replay of 16,140 real transactions
Coinbase replayed 16,140 real transactions and found each newer version of three AI model families caught less fraud than the one it would replace. The result undercuts the idea that a model upgrade is a free improvement for a working fraud screener.
Reality
- Evidence50
- Adoption40
- Hype gap+15
- Incentives50
- Confidence50
buildOne report1 publisher Claude Code's plugin eval caught skills firing 55.6% of the time once 89 were loaded
Claude Code's new plugin eval showed one developer's skills firing in 5 of 9 relevant runs once all 89 were loaded, down from every run with one skill. The test covers three prompts in one project, but it gives teams that keep rules in skills a way to measure how often those rules get consulted.
Reality
- Evidence35
- Adoption
- Not scored
- Hype gap+20
- Incentives
- Not scored
- Confidence30
buildOne report1 publisher Screen-reader testing found a second list of OneFindMe bugs after axe-core reported zero
OneFindMe's developer took axe-core to zero findings, then a keyboard and VoiceOver pass turned up six kinds of failure the scanner never flagged. One site and one blind iPhone user make a strong case for a screen reader in the release test plan.
Reality
- Evidence45
- Adoption8
- Hype gap+5
- Incentives25
- Confidence55
Gambit Security says the ransomware crew used a commercial coding agent for hands-on post-compromise work between 8 April and 21 May, alongside a new Linux encryptor that force-kills running guests before it touches ESXi datastores.
Perspective Coverage
4 publishers
- Builder
- Builder 34%
- Operator
- Operator 59%
- Investor
- Investor 7%
Reality
- Evidence78
- Adoption60
- Hype gap+22
- Incentives58
- Confidence70
Prices and payments a customer acts on belong in a calculator the chatbot calls
Omni Calculator says any number a customer acts on should come from a deterministic tool, after its benchmark scored AI models at 48.4% to 70.4% on math. Its own Toronto BMW example went wrong at the input, so a calculation engine protects operators only when the right data reaches it.
Reality
- Evidence35
- Adoption
- Not scored
- Hype gap+25
- Incentives80
- Confidence40
buildOne report1 publisher Haiku labels three of eight patched twins as reachable vulnerabilities
A Kaggle-challenge benchmark called ART scores models on whether they still flag a function after the fix is applied. On eight synthetic pairs, the difference between price tiers showed up only on the patched half.
Reality
- Evidence47
- Adoption12
- Hype gap−5
- Incentives58
- Confidence44
buildOne report1 publisher An Obsidian vault pipeline re-validates JSON from a model stripped of write tools
Every 10 minutes an unattended job summarizes Claude Code and Codex logs into an Obsidian vault and pushes the commit, so the pipeline treats the summarizer's own JSON as text that may carry pasted keys or injected instructions.
Reality
- Evidence58
- Adoption6
- Hype gap+10
- Incentives20
- Confidence55
Amazon ran an AI project five months before catching an 860% overrun
The figures driving 2026's AI budget panic reach operators secondhand, through a vendor's column citing Fortune and TechSpot. The one case with dates attached puts the cost in the five months before anyone checked.
Reality
- Evidence22
- Adoption30
- Hype gap+40
- Incentives75
- Confidence40
buildOne report1 publisher Claude Code's built-in review command runs on whatever model the session is set to
A developer pinned a model into all 11 of his own subagent files, then counted 576 launches and found 63% were built-ins inheriting the session default. One implementation plus one review emptied his top model's limit.
Reality
- Evidence62
- Adoption32
- Hype gap+12
- Incentives25
- Confidence58
buildOne report1 publisher Three judges score every answer in a RAG sweep, including one from the generator's own vendor
A dev.to write-up fixes retrieval at 200-word chunks and cosine top-5, then swaps three embedders and five generators across 100 questions and four topics to find out whether its own evaluation conclusions hold.
Reality
- Evidence52
- Adoption15
- Hype gap−12
- Incentives35
- Confidence45
Ramp's heaviest AI spenders cut per-employee spend 9.7% to $7,205 in August
Ramp's September index has the top 1% of US AI buyers paying $7,205 a head. Its chief economist points to August vacations, cheaper tokens and a steady migration down from frontier models.
Reality
- Evidence52
- Adoption66
- Hype gap+10
- Incentives58
- Confidence50
buildOne report1 publisher A browser agent's cheapest Google Maps run was the one where it wrote a selector
Across 60 coffee-shop listings, an agent with browser tools and a plain Playwright script pulled the same fields. The agent's per-place token count fell by about a third once it started reading aria-label attributes.
Reality
- Evidence58
- Adoption12
- Hype gap−5
- Incentives18
- Confidence56
Salesforce exposes its platform to coding agents through more than 60 new MCP tools
Headless 360 makes capabilities that used to sit behind the Salesforce console callable as APIs, MCP tools and CLI commands, and more than 60 of the new tools point coding agents such as Claude Code and Cursor at live orgs.
Reality
- Evidence28
- Adoption26
- Hype gap+42
- Incentives92
- Confidence55
Harvey post-trained a 27B open-weight model into the frontier band on its own legal benchmark
The band it joined is one where the best closed models still finish under 10% of tasks end to end at roughly $50 and 20 minutes each, so what post-training buys a law firm is price and hosting rather than capability.
Publishers:harvey.ai
Reality
- Evidence41
- Adoption
- Not scored
- Hype gap+32
- Incentives76
- Confidence46
buildOne report1 publisher Nushell was slower on every timed task in a trial pre-registered for improvement, regression, or no difference
The three possible outcomes were written down before the 380 runs began, which is why a mostly unfavourable result still carries information: aggregation cut one summary by 83 percent while every timed task got slower.
Reality
- Evidence52
- Adoption12
- Hype gap−15
- Incentives30
- Confidence50
Anthropic and GitHub have moved AI costs from the seat to the meter
Enterprise AI pricing has separated the seat from the usage, which moves the cost driver from headcount to demand. The overruns now on the record suggest buyers tend to find out months after the fact.
Reality
- Evidence38
- Adoption47
- Hype gap+27
- Incentives80
- Confidence44
buildOne report1 publisher Jamf enforces per-engineer Bedrock budgets by rewriting an IAM policy every 15 minutes
The loop reads Bedrock invocation logs through Athena, prices them at published rates, and denies Claude Opus at 80 percent of an engineer's daily budget. It transfers only if every model call carries a federated user identity.
Reality
- Evidence58
- Adoption30
- Hype gap+25
- Incentives78
- Confidence54
buildOne report1 publisher 255 tools, 71,929 tokens: the standing charge hidden in your MCP config
One measurement puts a 50-server tool catalog at 56 percent of a 128K context window before the first prompt. The per-server arithmetic is the part worth budgeting.
Reality
- Evidence34
- Adoption
- Not scored
- Hype gap+44
- Incentives86
- Confidence38
buildOne report1 publisher Claude's limits are token meters on two clocks, and your open session is what drains them
Anthropic meters Pro and Max on a rolling 5-hour window and a weekly cap, both counted in tokens. What drains them is re-read context, not how many questions you ask.
Reality
- Evidence42
- Adoption48
- Hype gap+12
- Incentives58
- Confidence40
buildOne report1 publisher The 97% saving was an agent failing quietly: token metrics need a completion gate
A killed side project produced the useful result: 11 of 31 pilot pairs, $5.60, and zero Java fixes from the cheap model. A savings figure without a pass rate is not a number.
Reality
- Evidence46
- Adoption14
- Hype gap+42
- Incentives24
- Confidence55
Earlier coverage
- Councils keep a quarter of the ideas only one model had
Product · August 23, 2026 · One report1 publisher