Annotation and Classification of Evidence and Reasoning Revisions in Argumentative Writing

Tazin Afrin, Elaine Lin Wang, Diane Litman, Lindsay Clare Matsumura, Richard Correnti


Abstract
Automated writing evaluation systems can improve students’ writing insofar as students attend to the feedback provided and revise their essay drafts in ways aligned with such feedback. Existing research on revision of argumentative writing in such systems, however, has focused on the types of revisions students make (e.g., surface vs. content) rather than the extent to which revisions actually respond to the feedback provided and improve the essay. We introduce an annotation scheme to capture the nature of sentence-level revisions of evidence use and reasoning (the ‘RER’ scheme) and apply it to 5th- and 6th-grade students’ argumentative essays. We show that reliable manual annotation can be achieved and that revision annotations correlate with a holistic assessment of essay improvement in line with the feedback provided. Furthermore, we explore the feasibility of automatically classifying revisions according to our scheme.
Anthology ID:
2020.bea-1.7
Volume:
Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications
Month:
July
Year:
2020
Address:
Seattle, WA, USA → Online
Editors:
Jill Burstein, Ekaterina Kochmar, Claudia Leacock, Nitin Madnani, Ildikó Pilán, Helen Yannakoudakis, Torsten Zesch
Venue:
BEA
SIG:
SIGEDU
Publisher:
Association for Computational Linguistics
Note:
Pages:
75–84
Language:
URL:
https://aclanthology.org/2020.bea-1.7
DOI:
10.18653/v1/2020.bea-1.7
Bibkey:
Cite (ACL):
Tazin Afrin, Elaine Lin Wang, Diane Litman, Lindsay Clare Matsumura, and Richard Correnti. 2020. Annotation and Classification of Evidence and Reasoning Revisions in Argumentative Writing. In Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 75–84, Seattle, WA, USA → Online. Association for Computational Linguistics.
Cite (Informal):
Annotation and Classification of Evidence and Reasoning Revisions in Argumentative Writing (Afrin et al., BEA 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.bea-1.7.pdf
Video:
 http://slideslive.com/38929856