An LSTM Adaptation Study of (Un)grammaticality

Shammur Absar Chowdhury, Roberto Zamparelli


Abstract
We propose a novel approach to the study of how artificial neural network perceive the distinction between grammatical and ungrammatical sentences, a crucial task in the growing field of synthetic linguistics. The method is based on performance measures of language models trained on corpora and fine-tuned with either grammatical or ungrammatical sentences, then applied to (different types of) grammatical or ungrammatical sentences. The results show that both in the difficult and highly symmetrical task of detecting subject islands and in the more open CoLA dataset, grammatical sentences give rise to better scores than ungrammatical ones, possibly because they can be better integrated within the body of linguistic structural knowledge that the language model has accumulated.
Anthology ID:
W19-4821
Volume:
Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP
Month:
August
Year:
2019
Address:
Florence, Italy
Editors:
Tal Linzen, Grzegorz Chrupała, Yonatan Belinkov, Dieuwke Hupkes
Venue:
BlackboxNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
204–212
Language:
URL:
https://aclanthology.org/W19-4821
DOI:
10.18653/v1/W19-4821
Bibkey:
Cite (ACL):
Shammur Absar Chowdhury and Roberto Zamparelli. 2019. An LSTM Adaptation Study of (Un)grammaticality. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 204–212, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
An LSTM Adaptation Study of (Un)grammaticality (Chowdhury & Zamparelli, BlackboxNLP 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-4821.pdf
Code
 LiCo-TREiL/Computational-Ungrammaticality
Data
CoLA