Greedy Search with Probabilistic N-gram Matching for Neural Machine Translation

Chenze Shao, Xilin Chen, Yang Feng


Abstract
Neural machine translation (NMT) models are usually trained with the word-level loss using the teacher forcing algorithm, which not only evaluates the translation improperly but also suffers from exposure bias. Sequence-level training under the reinforcement framework can mitigate the problems of the word-level loss, but its performance is unstable due to the high variance of the gradient estimation. On these grounds, we present a method with a differentiable sequence-level training objective based on probabilistic n-gram matching which can avoid the reinforcement framework. In addition, this method performs greedy search in the training which uses the predicted words as context just as at inference to alleviate the problem of exposure bias. Experiment results on the NIST Chinese-to-English translation tasks show that our method significantly outperforms the reinforcement-based algorithms and achieves an improvement of 1.5 BLEU points on average over a strong baseline system.
Anthology ID:
D18-1510
Volume:
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Month:
October-November
Year:
2018
Address:
Brussels, Belgium
Editors:
Ellen Riloff, David Chiang, Julia Hockenmaier, Jun’ichi Tsujii
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
4778–4784
Language:
URL:
https://aclanthology.org/D18-1510
DOI:
10.18653/v1/D18-1510
Bibkey:
Cite (ACL):
Chenze Shao, Xilin Chen, and Yang Feng. 2018. Greedy Search with Probabilistic N-gram Matching for Neural Machine Translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4778–4784, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Greedy Search with Probabilistic N-gram Matching for Neural Machine Translation (Shao et al., EMNLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/D18-1510.pdf
Video:
 https://aclanthology.org/D18-1510.mp4
Code
 ictnlp/GS4NMT