Book QA: Stories of Challenges and Opportunities

Stefanos Angelidis, Lea Frermann, Diego Marcheggiani, Roi Blanco, Lluís Màrquez


Abstract
We present a system for answering questions based on the full text of books (BookQA), which first selects book passages given a question at hand, and then uses a memory network to reason and predict an answer. To improve generalization, we pretrain our memory network using artificial questions generated from book sentences. We experiment with the recently published NarrativeQA corpus, on the subset of Who questions, which expect book characters as answers. We experimentally show that BERT-based retrieval and pretraining improve over baseline results significantly. At the same time, we confirm that NarrativeQA is a highly challenging data set, and that there is need for novel research in order to achieve high-precision BookQA results. We analyze some of the bottlenecks of the current approach, and we argue that more research is needed on text representation, retrieval of relevant passages, and reasoning, including commonsense knowledge.
Anthology ID:
D19-5811
Volume:
Proceedings of the 2nd Workshop on Machine Reading for Question Answering
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo, Eunsol Choi, Danqi Chen
Venue:
WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
78–85
Language:
URL:
https://aclanthology.org/D19-5811
DOI:
10.18653/v1/D19-5811
Bibkey:
Cite (ACL):
Stefanos Angelidis, Lea Frermann, Diego Marcheggiani, Roi Blanco, and Lluís Màrquez. 2019. Book QA: Stories of Challenges and Opportunities. In Proceedings of the 2nd Workshop on Machine Reading for Question Answering, pages 78–85, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Book QA: Stories of Challenges and Opportunities (Angelidis et al., 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-5811.pdf