WMT23 QE Competition

Unbabel Wins the QE Shared Task at WMT23: Continuing Excellence in Translation Quality Assessment

September 7, 2023

We are excited to share that we have once again won the prestigious Quality Estimation (QE) shared task at WMT23! This honor further displays our commitment to being the world’s foremost translation quality provider and delivering the highest quality translation experience for our customers. The WMT23 competition, which showcases cutting-edge QE systems, saw Unbabel outperform its rivals, further solidifying its position as the leading translation-as-a-service provider for the second year running.

Celebrating Our Commitment to Excellence

Unbabel’s impressive performance was not limited to any one language, as it won first place in sentence-level quality estimation for Hebrew-English, English-Chinese, English-Hindi, and English-Gujarati. It also achieved the best overall performance on the multilingual track, which is determined by the average performance on all 8 tested language pairs.

In addition to first place in sentence-level QE, we dominated the word-level and Error Span Detection subtasks, achieving first place on overall performance and winning most of the tested language pairs. Competing against esteemed institutions such as Huawei Translation Services Center (HW-TSC), Bering Lab, Nanjing University, and others, Unbabel once again demonstrated its unparalleled expertise.

The 2023 WMT shared task on quality estimation presented unique challenges, as it required the assessment of translation quality at both word and sentence levels, as well as the detection and classification of the severity of translation errors. Unbabel’s winning strategy for WMT23 involved building an LLM QE system with up to 11B parameters. This is the largest QE system ever built, and it is a testament to Unbabel’s commitment to advancing the field of quality estimation.

We are proud to share our continued success in the WMT QE shared task, and as always, we are committed to further advancing the state-of-the-art in translation quality assessment by open-sourcing these models in a near future. At Unbabel, we believe QE is essential for ensuring the quality of machine translation, and as such, we are committed to providing our customers with the best possible translation experience.

Official results can be found here. To learn more about Unbabel’s QE capabilities and LangOps platform, visit https://unbabel.com/langops_platform/ and our UnbabelQi demo: https://qi.unbabel.com/

About the Author

Profile Photo of Phill Brougham
Phill Brougham

Director of Product Marketing at Unbabel, Phill Brougham spent the last five years working for SaaS businesses focused on applying artificial intelligence to solving real-world business and productivity problems. Throughout his roles, Phill’s focus has been on translating technological capability into clear, understandable value.