Revisiting Modularized Multilingual NMT to Meet Industrial Demands

Sungwon Lyu, Bokyung Son, Kichang Yang, Jaekyoung Bae


Abstract
The complete sharing of parameters for multilingual translation (1-1) has been the mainstream approach in current research. However, degraded performance due to the capacity bottleneck and low maintainability hinders its extensive adoption in industries. In this study, we revisit the multilingual neural machine translation model that only share modules among the same languages (M2) as a practical alternative to 1-1 to satisfy industrial requirements. Through comprehensive experiments, we identify the benefits of multi-way training and demonstrate that the M2 can enjoy these benefits without suffering from the capacity bottleneck. Furthermore, the interlingual space of the M2 allows convenient modification of the model. By leveraging trained modules, we find that incrementally added modules exhibit better performance than singly trained models. The zero-shot performance of the added modules is even comparable to supervised models. Our findings suggest that the M2 can be a competent candidate for multilingual translation in industries.
Anthology ID:
2020.emnlp-main.476
Volume:
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Month:
November
Year:
2020
Address:
Online
Editors:
Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5905–5918
Language:
URL:
https://aclanthology.org/2020.emnlp-main.476
DOI:
10.18653/v1/2020.emnlp-main.476
Bibkey:
Cite (ACL):
Sungwon Lyu, Bokyung Son, Kichang Yang, and Jaekyoung Bae. 2020. Revisiting Modularized Multilingual NMT to Meet Industrial Demands. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5905–5918, Online. Association for Computational Linguistics.
Cite (Informal):
Revisiting Modularized Multilingual NMT to Meet Industrial Demands (Lyu et al., EMNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.emnlp-main.476.pdf
Video:
 https://slideslive.com/38939024