A Web-based Framework for Collecting and Assessing Highlighted Sentences in a Document

Sasha Spala, Franck Dernoncourt, Walter Chang, Carl Dockhorn


Abstract
Automatically highlighting a text aims at identifying key portions that are the most important to a reader. In this paper, we present a web-based framework designed to efficiently and scalably crowdsource two independent but related tasks: collecting highlight annotations, and comparing the performance of automated highlighting systems. The first task is necessary to understand human preferences and train supervised automated highlighting systems. The second task yields a more accurate and fine-grained evaluation than existing automated performance metrics.
Anthology ID:
C18-2017
Volume:
Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations
Month:
August
Year:
2018
Address:
Santa Fe, New Mexico
Editor:
Dongyan Zhao
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
78–81
Language:
URL:
https://aclanthology.org/C18-2017
DOI:
Bibkey:
Cite (ACL):
Sasha Spala, Franck Dernoncourt, Walter Chang, and Carl Dockhorn. 2018. A Web-based Framework for Collecting and Assessing Highlighted Sentences in a Document. In Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations, pages 78–81, Santa Fe, New Mexico. Association for Computational Linguistics.
Cite (Informal):
A Web-based Framework for Collecting and Assessing Highlighted Sentences in a Document (Spala et al., COLING 2018)
Copy Citation:
PDF:
https://aclanthology.org/C18-2017.pdf