Unsupervised Neural Machine Translation with Weight Sharing

Zhen Yang, Wei Chen, Feng Wang, Bo Xu


Abstract
Unsupervised neural machine translation (NMT) is a recently proposed approach for machine translation which aims to train the model without using any labeled data. The models proposed for unsupervised NMT often use only one shared encoder to map the pairs of sentences from different languages to a shared-latent space, which is weak in keeping the unique and internal characteristics of each language, such as the style, terminology, and sentence structure. To address this issue, we introduce an extension by utilizing two independent encoders but sharing some partial weights which are responsible for extracting high-level representations of the input sentences. Besides, two different generative adversarial networks (GANs), namely the local GAN and global GAN, are proposed to enhance the cross-language translation. With this new approach, we achieve significant improvements on English-German, English-French and Chinese-to-English translation tasks.
Anthology ID:
P18-1005
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Iryna Gurevych, Yusuke Miyao
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
46–55
Language:
URL:
https://aclanthology.org/P18-1005
DOI:
10.18653/v1/P18-1005
Bibkey:
Cite (ACL):
Zhen Yang, Wei Chen, Feng Wang, and Bo Xu. 2018. Unsupervised Neural Machine Translation with Weight Sharing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 46–55, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Unsupervised Neural Machine Translation with Weight Sharing (Yang et al., ACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/P18-1005.pdf
Note:
 P18-1005.Notes.pdf
Presentation:
 P18-1005.Presentation.pdf
Video:
 https://aclanthology.org/P18-1005.mp4
Code
 ZhenYangIACAS/unsupervised-NMT
Data
WMT 2016