Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages

Xavier Garcia, Aditya Siddhant, Orhan Firat, Ankur Parikh


Abstract
Unsupervised translation has reached impressive performance on resource-rich language pairs such as English-French and English-German. However, early studies have shown that in more realistic settings involving low-resource, rare languages, unsupervised translation performs poorly, achieving less than 3.0 BLEU. In this work, we show that multilinguality is critical to making unsupervised systems practical for low-resource settings. In particular, we present a single model for 5 low-resource languages (Gujarati, Kazakh, Nepali, Sinhala, and Turkish) to and from English directions, which leverages monolingual and auxiliary parallel data from other high-resource language pairs via a three-stage training scheme. We outperform all current state-of-the-art unsupervised baselines for these languages, achieving gains of up to 14.4 BLEU. Additionally, we outperform strong supervised baselines for various language pairs as well as match the performance of the current state-of-the-art supervised model for Nepali-English. We conduct a series of ablation studies to establish the robustness of our model under different degrees of data quality, as well as to analyze the factors which led to the superior performance of the proposed approach over traditional unsupervised models.
Anthology ID:
2021.naacl-main.89
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
June
Year:
2021
Address:
Online
Editors:
Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, Yichao Zhou
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1126–1137
Language:
URL:
https://aclanthology.org/2021.naacl-main.89
DOI:
10.18653/v1/2021.naacl-main.89
Bibkey:
Cite (ACL):
Xavier Garcia, Aditya Siddhant, Orhan Firat, and Ankur Parikh. 2021. Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1126–1137, Online. Association for Computational Linguistics.
Cite (Informal):
Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages (Garcia et al., NAACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.naacl-main.89.pdf
Video:
 https://aclanthology.org/2021.naacl-main.89.mp4
Data
FLoRes