Skip to content

Topic

Vendor Benchmark Verification

The gap between self-published model scores on private and public benchmarks and independent reproduction.

Current stories

buildOne report1 publisher

Folding 92 layers into one Pallas call put Kimi K3 at 709 tokens a second on TPU v7

Inferact measured 709 output tokens a second on 16 Ironwood chips against 452 on 16 GB200s, at low concurrency with speculative decoding. The engineering worth reading is the hand-written memory schedule underneath.

Publishers:runtimewire.com

Reality

Evidence45
Adoption22
Hype gap+20
Incentives78
Confidence58
buildOne report1 publisher

Redis 8.10 converts hashes to the compact encoding only on HIMPORT or an RDB reload

Redis's release blog advertises up to 50% lower hash memory. One tester found that existing keys keep the old encoding until you rewrite them through HIMPORT or reload them from disk with three parameters that ship at zero.

Publishers:dev.to

Reality

Evidence62
Adoption20
Hype gap+32
Incentives40
Confidence55
leadershipOne report1 publisher

Investors quintupled Positron's valuation months before Asimov's tapeout

The $875 million round prices an inference chip whose performance figures come from cycle-accurate simulations and whose production date has moved from early 2027 into the second half. The system Positron ships today runs on HBM.

Publishers:implicator.ai

Reality

Evidence42
Adoption38
Hype gap+45
Incentives80
Confidence52
buildOne report1 publisher

Unsloth's 10% quant claim is really about which machines can run a 27B model

Dynamic 3.0 ships Qwen3.8-27B GGUFs from 6.2GB up, with an unreproduced accuracy claim attached. The number that matters is the one that decides where the file fits.

Publishers:runtimewire.com

Reality

Evidence34
Adoption45
Hype gap+28
Incentives74
Confidence41
buildOne report1 publisher

WRITER's new flagship is a post-train of Z.ai's GLM-5.2, and that is the story

Palmyra X6 arrived on August 13 with a 52% cost-reduction claim from WRITER's own evaluations. The more durable fact is where the weights came from.

Reality

Evidence32
Adoption22
Hype gap+34
Incentives78
Confidence44