Filtering and Mining Parallel Data in a Joint Multilingual Space

Holger Schwenk


Abstract
We learn a joint multilingual sentence embedding and use the distance between sentences in different languages to filter noisy parallel data and to mine for parallel data in large news collections. We are able to improve a competitive baseline on the WMT’14 English to German task by 0.3 BLEU by filtering out 25% of the training data. The same approach is used to mine additional bitexts for the WMT’14 system and to obtain competitive results on the BUCC shared task to identify parallel sentences in comparable corpora. The approach is generic, it can be applied to many language pairs and it is independent of the architecture of the machine translation system.
Anthology ID:
P18-2037
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Iryna Gurevych, Yusuke Miyao
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
228–234
Language:
URL:
https://aclanthology.org/P18-2037
DOI:
10.18653/v1/P18-2037
Bibkey:
Cite (ACL):
Holger Schwenk. 2018. Filtering and Mining Parallel Data in a Joint Multilingual Space. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 228–234, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Filtering and Mining Parallel Data in a Joint Multilingual Space (Schwenk, ACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/P18-2037.pdf
Note:
 P18-2037.Notes.pdf