Topic
How far buyers can trust vendor-published performance and cost benchmarks when methods go undisclosed and no independent party has reproduced the results.
The router picks a cheaper model for work that resembles work it has already seen, and falls back to the strongest one when nothing matches. The same traces that enable that call are the proposed training set.
Reality
Tau puts enterprise context on the desktop, but the load-bearing claim is a 5.2x per-query token-cost edge over Claude Cowork, produced by Glean's own testing and shipped without a method.
Reality
Its August 20 bake-off priced three agents on one Nemotron port. The claim that findings transfer to the next model-chip pair is the one with no number attached.
Reality
A vendor benchmark puts wake-word at 5.3% of one core on a $15 Pi Zero 2 W. That is the bar for on-device listening now, and the free-phrase-only catch travels with it.
Reality
The vendor says its indexed context engine cuts per-task cost 48 percent against Claude Code on the same Opus 4.8 model. The benchmark is its own, on one open-source repo.
Reality
A plasma-physics team is betting that aerospace and nuclear buyers will pay for machine-checkable proofs. The proof still stops where the specification stops.
Reality