Auto-Encoding Variational Neural Machine Translation

Bryan Eikema, Wilker Aziz


Abstract
We present a deep generative model of bilingual sentence pairs for machine translation. The model generates source and target sentences jointly from a shared latent representation and is parameterised by neural networks. We perform efficient training using amortised variational inference and reparameterised gradients. Additionally, we discuss the statistical implications of joint modelling and propose an efficient approximation to maximum a posteriori decoding for fast test-time predictions. We demonstrate the effectiveness of our model in three machine translation scenarios: in-domain training, mixed-domain training, and learning from a mix of gold-standard and synthetic data. Our experiments show consistently that our joint formulation outperforms conditional modelling (i.e. standard neural machine translation) in all such scenarios.
Anthology ID:
W19-4315
Volume:
Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)
Month:
August
Year:
2019
Address:
Florence, Italy
Editors:
Isabelle Augenstein, Spandana Gella, Sebastian Ruder, Katharina Kann, Burcu Can, Johannes Welbl, Alexis Conneau, Xiang Ren, Marek Rei
Venue:
RepL4NLP
SIG:
SIGREP
Publisher:
Association for Computational Linguistics
Note:
Pages:
124–141
Language:
URL:
https://aclanthology.org/W19-4315
DOI:
10.18653/v1/W19-4315
Bibkey:
Cite (ACL):
Bryan Eikema and Wilker Aziz. 2019. Auto-Encoding Variational Neural Machine Translation. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 124–141, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Auto-Encoding Variational Neural Machine Translation (Eikema & Aziz, RepL4NLP 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-4315.pdf
Code
 Roxot/AEVNMT
Data
WMT 2016