Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input

Youmna Farag, Helen Yannakoudakis, Ted Briscoe


Abstract
We demonstrate that current state-of-the-art approaches to Automated Essay Scoring (AES) are not well-suited to capturing adversarially crafted input of grammatical but incoherent sequences of sentences. We develop a neural model of local coherence that can effectively learn connectedness features between sentences, and propose a framework for integrating and jointly training the local coherence model with a state-of-the-art AES model. We evaluate our approach against a number of baselines and experimentally demonstrate its effectiveness on both the AES task and the task of flagging adversarial input, further contributing to the development of an approach that strengthens the validity of neural essay scoring models.
Anthology ID:
N18-1024
Volume:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)
Month:
June
Year:
2018
Address:
New Orleans, Louisiana
Editors:
Marilyn Walker, Heng Ji, Amanda Stent
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
263–271
Language:
URL:
https://aclanthology.org/N18-1024
DOI:
10.18653/v1/N18-1024
Bibkey:
Cite (ACL):
Youmna Farag, Helen Yannakoudakis, and Ted Briscoe. 2018. Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 263–271, New Orleans, Louisiana. Association for Computational Linguistics.
Cite (Informal):
Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input (Farag et al., NAACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/N18-1024.pdf
Code
 Youmna-H/Coherence_AES