Self-Attention Architectures for Answer-Agnostic Neural Question Generation

Thomas Scialom, Benjamin Piwowarski, Jacopo Staiano


Abstract
Neural architectures based on self-attention, such as Transformers, recently attracted interest from the research community, and obtained significant improvements over the state of the art in several tasks. We explore how Transformers can be adapted to the task of Neural Question Generation without constraining the model to focus on a specific answer passage. We study the effect of several strategies to deal with out-of-vocabulary words such as copy mechanisms, placeholders, and contextual word embeddings. We report improvements obtained over the state-of-the-art on the SQuAD dataset according to automated metrics (BLEU, ROUGE), as well as qualitative human assessments of the system outputs.
Anthology ID:
P19-1604
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6027–6032
Language:
URL:
https://aclanthology.org/P19-1604
DOI:
10.18653/v1/P19-1604
Bibkey:
Cite (ACL):
Thomas Scialom, Benjamin Piwowarski, and Jacopo Staiano. 2019. Self-Attention Architectures for Answer-Agnostic Neural Question Generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6027–6032, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Self-Attention Architectures for Answer-Agnostic Neural Question Generation (Scialom et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1604.pdf
Poster:
 P19-1604.Poster.pdf
Data
SQuAD