The Second Multilingual Surface Realisation Shared Task (SR’19): Overview and Evaluation Results

Simon Mille, Anja Belz, Bernd Bohnet, Yvette Graham, Leo Wanner


Abstract
We report results from the SR’19 Shared Task, the second edition of a multilingual surface realisation task organised as part of the EMNLP’19 Workshop on Multilingual Surface Realisation. As in SR’18, the shared task comprised two tracks with different levels of complexity: (a) a shallow track where the inputs were full UD structures with word order information removed and tokens lemmatised; and (b) a deep track where additionally, functional words and morphological information were removed. The shallow track was offered in eleven, and the deep track in three languages. Systems were evaluated (a) automatically, using a range of intrinsic metrics, and (b) by human judges in terms of readability and meaning similarity. This report presents the evaluation results, along with descriptions of the SR’19 tracks, data and evaluation methods. For full descriptions of the participating systems, please see the separate system reports elsewhere in this volume.
Anthology ID:
D19-6301
Original:
D19-6301v1
Version 2:
D19-6301v2
Volume:
Proceedings of the 2nd Workshop on Multilingual Surface Realisation (MSR 2019)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Simon Mille, Anja Belz, Bernd Bohnet, Yvette Graham, Leo Wanner
Venue:
WS
SIG:
SIGGEN
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–17
Language:
URL:
https://aclanthology.org/D19-6301
DOI:
10.18653/v1/D19-6301
Bibkey:
Cite (ACL):
Simon Mille, Anja Belz, Bernd Bohnet, Yvette Graham, and Leo Wanner. 2019. The Second Multilingual Surface Realisation Shared Task (SR’19): Overview and Evaluation Results. In Proceedings of the 2nd Workshop on Multilingual Surface Realisation (MSR 2019), pages 1–17, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
The Second Multilingual Surface Realisation Shared Task (SR’19): Overview and Evaluation Results (Mille et al., 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-6301.pdf