Multi-headed Architecture Based on BERT for Grammatical Errors Correction

Bohdan Didenko, Julia Shaptala


Abstract
In this paper, we describe our approach to GEC using the BERT model for creation of encoded representation and some of our enhancements, namely, “Heads” are fully-connected networks which are used for finding the errors and later receive recommendation from the networks on dealing with a highlighted part of the sentence only. Among the main advantages of our solution is increasing the system productivity and lowering the time of processing while keeping the high accuracy of GEC results.
Anthology ID:
W19-4426
Volume:
Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications
Month:
August
Year:
2019
Address:
Florence, Italy
Editors:
Helen Yannakoudakis, Ekaterina Kochmar, Claudia Leacock, Nitin Madnani, Ildikó Pilán, Torsten Zesch
Venue:
BEA
SIG:
SIGEDU
Publisher:
Association for Computational Linguistics
Note:
Pages:
246–251
Language:
URL:
https://aclanthology.org/W19-4426
DOI:
10.18653/v1/W19-4426
Bibkey:
Cite (ACL):
Bohdan Didenko and Julia Shaptala. 2019. Multi-headed Architecture Based on BERT for Grammatical Errors Correction. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 246–251, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Multi-headed Architecture Based on BERT for Grammatical Errors Correction (Didenko & Shaptala, BEA 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-4426.pdf