WMDO: Fluency-based Word Mover’s Distance for Machine Translation Evaluation

Julian Chow, Lucia Specia, Pranava Madhyastha


Abstract
We propose WMDO, a metric based on distance between distributions in the semantic vector space. Matching in the semantic space has been investigated for translation evaluation, but the constraints of a translation’s word order have not been fully explored. Building on the Word Mover’s Distance metric and various word embeddings, we introduce a fragmentation penalty to account for fluency of a translation. This word order extension is shown to perform better than standard WMD, with promising results against other types of metrics.
Anthology ID:
W19-5356
Volume:
Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)
Month:
August
Year:
2019
Address:
Florence, Italy
Editors:
Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, André Martins, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Matt Post, Marco Turchi, Karin Verspoor
Venue:
WMT
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
494–500
Language:
URL:
https://aclanthology.org/W19-5356
DOI:
10.18653/v1/W19-5356
Bibkey:
Cite (ACL):
Julian Chow, Lucia Specia, and Pranava Madhyastha. 2019. WMDO: Fluency-based Word Mover’s Distance for Machine Translation Evaluation. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 494–500, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
WMDO: Fluency-based Word Mover’s Distance for Machine Translation Evaluation (Chow et al., WMT 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-5356.pdf