Benchmark
Benchmark of real-world questions that test general AI assistants on reasoning, web browsing, tool use and multimodal handling, introduced in 2023.
Anthropic's Claude Science helped Brice Menard build the first complete ultraviolet sky map, modeling roughly a third no UV telescope has observed. Agents ran the days-long pipeline, but the Johns Hopkins astrophysicist caught an artifact two agent reviews missed, runtimewire reported.
Reality
Webb watched Chariklo, the first small body found to have rings, cross in front of a star in 2022, and the ring opacities it measured do not match the ground-based ones from the previous decade. The paper offers three explanations.
Reality
Berkeley's RDI center attacked the step where each benchmark computes its score, and without solving a task its own scorecard reports 100% on five of the eight, about 98% on GAIA and 73% on OSWorld.
Reality
An arXiv tracing study of Claude Code agents on Gemma and Qwen measured prefix-cache hit rates between 84.6 and 99.5 percent, which moves the serving bottleneck to how long you can keep KV blocks resident between tool calls.
Reality
A new diagnostic benchmark treats the execution layer as something to vary rather than a fixed backdrop. Its conclusion is that agent capability belongs to a model-harness pair.
Reality