Read and Comprehend by Gated-Attention Reader with More Belief

Haohui Deng, Yik-Cheung Tam


Abstract
Gated-Attention (GA) Reader has been effective for reading comprehension. GA Reader makes two assumptions: (1) a uni-directional attention that uses an input query to gate token encodings of a document; (2) encoding at the cloze position of an input query is considered for answer prediction. In this paper, we propose Collaborative Gating (CG) and Self-Belief Aggregation (SBA) to address the above assumptions respectively. In CG, we first use an input document to gate token encodings of an input query so that the influence of irrelevant query tokens may be reduced. Then the filtered query is used to gate token encodings of an document in a collaborative fashion. In SBA, we conjecture that query tokens other than the cloze token may be informative for answer prediction. We apply self-attention to link the cloze token with other tokens in a query so that the importance of query tokens with respect to the cloze position are weighted. Then their evidences are weighted, propagated and aggregated for better reading comprehension. Experiments show that our approaches advance the state-of-theart results in CNN, Daily Mail, and Who Did What public test sets.
Anthology ID:
N18-4012
Volume:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop
Month:
June
Year:
2018
Address:
New Orleans, Louisiana, USA
Editors:
Silvio Ricardo Cordeiro, Shereen Oraby, Umashanthi Pavalanathan, Kyeongmin Rim
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
83–91
Language:
URL:
https://aclanthology.org/N18-4012
DOI:
10.18653/v1/N18-4012
Bibkey:
Cite (ACL):
Haohui Deng and Yik-Cheung Tam. 2018. Read and Comprehend by Gated-Attention Reader with More Belief. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 83–91, New Orleans, Louisiana, USA. Association for Computational Linguistics.
Cite (Informal):
Read and Comprehend by Gated-Attention Reader with More Belief (Deng & Tam, NAACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/N18-4012.pdf