Morphological Word Embeddings for Arabic Neural Machine Translation in Low-Resource Settings

Pamela Shapiro, Kevin Duh


Abstract
Neural machine translation has achieved impressive results in the last few years, but its success has been limited to settings with large amounts of parallel data. One way to improve NMT for lower-resource settings is to initialize a word-based NMT model with pretrained word embeddings. However, rare words still suffer from lower quality word embeddings when trained with standard word-level objectives. We introduce word embeddings that utilize morphological resources, and compare to purely unsupervised alternatives. We work with Arabic, a morphologically rich language with available linguistic resources, and perform Ar-to-En MT experiments on a small corpus of TED subtitles. We find that word embeddings utilizing subword information consistently outperform standard word embeddings on a word similarity task and as initialization of the source word embeddings in a low-resource NMT system.
Anthology ID:
W18-1201
Volume:
Proceedings of the Second Workshop on Subword/Character LEvel Models
Month:
June
Year:
2018
Address:
New Orleans
Editors:
Manaal Faruqui, Hinrich Schütze, Isabel Trancoso, Yulia Tsvetkov, Yadollah Yaghoobzadeh
Venue:
SCLeM
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–11
Language:
URL:
https://aclanthology.org/W18-1201
DOI:
10.18653/v1/W18-1201
Bibkey:
Cite (ACL):
Pamela Shapiro and Kevin Duh. 2018. Morphological Word Embeddings for Arabic Neural Machine Translation in Low-Resource Settings. In Proceedings of the Second Workshop on Subword/Character LEvel Models, pages 1–11, New Orleans. Association for Computational Linguistics.
Cite (Informal):
Morphological Word Embeddings for Arabic Neural Machine Translation in Low-Resource Settings (Shapiro & Duh, SCLeM 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-1201.pdf