Robust Unsupervised Neural Machine Translation with Adversarial Denoising Training

Haipeng Sun, Rui Wang, Kehai Chen, Xugang Lu, Masao Utiyama, Eiichiro Sumita, Tiejun Zhao


Abstract
Unsupervised neural machine translation (UNMT) has recently attracted great interest in the machine translation community. The main advantage of the UNMT lies in its easy collection of required large training text sentences while with only a slightly worse performance than supervised neural machine translation which requires expensive annotated translation pairs on some translation tasks. In most studies, the UMNT is trained with clean data without considering its robustness to the noisy data. However, in real-world scenarios, there usually exists noise in the collected input sentences which degrades the performance of the translation system since the UNMT is sensitive to the small perturbations of the input sentences. In this paper, we first time explicitly take the noisy data into consideration to improve the robustness of the UNMT based systems. First of all, we clearly defined two types of noises in training sentences, i.e., word noise and word order noise, and empirically investigate its effect in the UNMT, then we propose adversarial training methods with denoising process in the UNMT. Experimental results on several language pairs show that our proposed methods substantially improved the robustness of the conventional UNMT systems in noisy scenarios.
Anthology ID:
2020.coling-main.374
Volume:
Proceedings of the 28th International Conference on Computational Linguistics
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Donia Scott, Nuria Bel, Chengqing Zong
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
4239–4250
Language:
URL:
https://aclanthology.org/2020.coling-main.374
DOI:
10.18653/v1/2020.coling-main.374
Bibkey:
Cite (ACL):
Haipeng Sun, Rui Wang, Kehai Chen, Xugang Lu, Masao Utiyama, Eiichiro Sumita, and Tiejun Zhao. 2020. Robust Unsupervised Neural Machine Translation with Adversarial Denoising Training. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4239–4250, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Cite (Informal):
Robust Unsupervised Neural Machine Translation with Adversarial Denoising Training (Sun et al., COLING 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.coling-main.374.pdf