Manual and Automatic Paraphrases for MT Evaluation

Aleš Tamchyna, Petra Barančíková


Abstract
Paraphrasing of reference translations has been shown to improve the correlation with human judgements in automatic evaluation of machine translation (MT) outputs. In this work, we present a new dataset for evaluating English-Czech translation based on automatic paraphrases. We compare this dataset with an existing set of manually created paraphrases and find that even automatic paraphrases can improve MT evaluation. We have also propose and evaluate several criteria for selecting suitable reference translations from a larger set.
Anthology ID:
L16-1563
Volume:
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
Month:
May
Year:
2016
Address:
Portorož, Slovenia
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
3543–3548
Language:
URL:
https://aclanthology.org/L16-1563
DOI:
Bibkey:
Cite (ACL):
Aleš Tamchyna and Petra Barančíková. 2016. Manual and Automatic Paraphrases for MT Evaluation. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 3543–3548, Portorož, Slovenia. European Language Resources Association (ELRA).
Cite (Informal):
Manual and Automatic Paraphrases for MT Evaluation (Tamchyna & Barančíková, LREC 2016)
Copy Citation:
PDF:
https://aclanthology.org/L16-1563.pdf
Data
WMT 2014