Vis-Eval Metric Viewer: A Visualisation Tool for Inspecting and Evaluating Metric Scores of Machine Translation Output

David Steele, Lucia Specia


Abstract
Machine Translation systems are usually evaluated and compared using automated evaluation metrics such as BLEU and METEOR to score the generated translations against human translations. However, the interaction with the output from the metrics is relatively limited and results are commonly a single score along with a few additional statistics. Whilst this may be enough for system comparison it does not provide much useful feedback or a means for inspecting translations and their respective scores. VisEval Metric Viewer VEMV is a tool designed to provide visualisation of multiple evaluation scores so they can be easily interpreted by a user. VEMV takes in the source, reference, and hypothesis files as parameters, and scores the hypotheses using several popular evaluation metrics simultaneously. Scores are produced at both the sentence and dataset level and results are written locally to a series of HTML files that can be viewed on a web browser. The individual scored sentences can easily be inspected using powerful search and selection functions and results can be visualised with graphical representations of the scores and distributions.
Anthology ID:
N18-5015
Volume:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations
Month:
June
Year:
2018
Address:
New Orleans, Louisiana
Editors:
Yang Liu, Tim Paek, Manasi Patwardhan
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
71–75
Language:
URL:
https://aclanthology.org/N18-5015
DOI:
10.18653/v1/N18-5015
Bibkey:
Cite (ACL):
David Steele and Lucia Specia. 2018. Vis-Eval Metric Viewer: A Visualisation Tool for Inspecting and Evaluating Metric Scores of Machine Translation Output. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 71–75, New Orleans, Louisiana. Association for Computational Linguistics.
Cite (Informal):
Vis-Eval Metric Viewer: A Visualisation Tool for Inspecting and Evaluating Metric Scores of Machine Translation Output (Steele & Specia, NAACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/N18-5015.pdf