Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations

Arturo Oncevay, Barry Haddow, Alexandra Birch


Abstract
Sparse language vectors from linguistic typology databases and learned embeddings from tasks like multilingual machine translation have been investigated in isolation, without analysing how they could benefit from each other’s language characterisation. We propose to fuse both views using singular vector canonical correlation analysis and study what kind of information is induced from each source. By inferring typological features and language phylogenies, we observe that our representations embed typology and strengthen correlations with language relationships. We then take advantage of our multi-view language vector space for multilingual machine translation, where we achieve competitive overall translation accuracy in tasks that require information about language similarities, such as language clustering and ranking candidates for multilingual transfer. With our method, we can easily project and assess new languages without expensive retraining of massive multilingual or ranking models, which are major disadvantages of related approaches.
Anthology ID:
2020.emnlp-main.187
Volume:
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Month:
November
Year:
2020
Address:
Online
Editors:
Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2391–2406
Language:
URL:
https://aclanthology.org/2020.emnlp-main.187
DOI:
10.18653/v1/2020.emnlp-main.187
Bibkey:
Cite (ACL):
Arturo Oncevay, Barry Haddow, and Alexandra Birch. 2020. Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2391–2406, Online. Association for Computational Linguistics.
Cite (Informal):
Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations (Oncevay et al., EMNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.emnlp-main.187.pdf
Video:
 https://slideslive.com/38939178
Code
 aoncevay/multiview-langrep