Almost Free Semantic Draft for Neural Machine Translation

Xi Ai, Bin Fang


Abstract
Translation quality can be improved by global information from the required target sentence because the decoder can understand both past and future information. However, the model needs additional cost to produce and consider such global information. In this work, to inject global information but also save cost, we present an efficient method to sample and consider a semantic draft as global information from semantic space for decoding with almost free of cost. Unlike other successful adaptations, we do not have to perform an EM-like process that repeatedly samples a possible semantic from the semantic space. Empirical experiments show that the presented method can achieve competitive performance in common language pairs with a clear advantage in inference efficiency. We will open all our source code on GitHub.
Anthology ID:
2021.naacl-main.307
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
June
Year:
2021
Address:
Online
Editors:
Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, Yichao Zhou
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3931–3941
Language:
URL:
https://aclanthology.org/2021.naacl-main.307
DOI:
10.18653/v1/2021.naacl-main.307
Bibkey:
Cite (ACL):
Xi Ai and Bin Fang. 2021. Almost Free Semantic Draft for Neural Machine Translation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3931–3941, Online. Association for Computational Linguistics.
Cite (Informal):
Almost Free Semantic Draft for Neural Machine Translation (Ai & Fang, NAACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.naacl-main.307.pdf
Video:
 https://aclanthology.org/2021.naacl-main.307.mp4
Data
WMT 2016