An Exploratory Study into Automated Précis Grading

Orphee De Clercq, Senne Van Hoecke


Abstract
Automated writing evaluation is a popular research field, but the main focus has been on evaluating argumentative essays. In this paper, we consider a different genre, namely précis texts. A précis is a written text that provides a coherent summary of main points of a spoken or written text. We present a corpus of English précis texts which all received a grade assigned by a highly-experienced English language teacher and were subsequently annotated following an exhaustive error typology. With this corpus we trained a machine learning model which relies on a number of linguistic, automatic summarization and AWE features. Our results reveal that this model is able to predict the grade of précis texts with only a moderate error margin.
Anthology ID:
2020.lrec-1.50
Volume:
Proceedings of the Twelfth Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
397–404
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.50
DOI:
Bibkey:
Cite (ACL):
Orphee De Clercq and Senne Van Hoecke. 2020. An Exploratory Study into Automated Précis Grading. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 397–404, Marseille, France. European Language Resources Association.
Cite (Informal):
An Exploratory Study into Automated Précis Grading (De Clercq & Van Hoecke, LREC 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.lrec-1.50.pdf