Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation

Vladimir Karpukhin, Omer Levy, Jacob Eisenstein, Marjan Ghazvininejad


Abstract
Contemporary machine translation systems achieve greater coverage by applying subword models such as BPE and character-level CNNs, but these methods are highly sensitive to orthographical variations such as spelling mistakes. We show how training on a mild amount of random synthetic noise can dramatically improve robustness to these variations, without diminishing performance on clean text. We focus on translation performance on natural typos, and show that robustness to such noise can be achieved using a balanced diet of simple synthetic noises at training time, without access to the natural noise data or distribution.
Anthology ID:
D19-5506
Volume:
Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Wei Xu, Alan Ritter, Tim Baldwin, Afshin Rahimi
Venue:
WNUT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
42–47
Language:
URL:
https://aclanthology.org/D19-5506
DOI:
10.18653/v1/D19-5506
Bibkey:
Cite (ACL):
Vladimir Karpukhin, Omer Levy, Jacob Eisenstein, and Marjan Ghazvininejad. 2019. Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation. In Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019), pages 42–47, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation (Karpukhin et al., WNUT 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-5506.pdf
Data
MTNT