Document Modeling with External Attention for Sentence Extraction

Shashi Narayan, Ronald Cardenas, Nikos Papasarantopoulos, Shay B. Cohen, Mirella Lapata, Jiangsheng Yu, Yi Chang


Abstract
Document modeling is essential to a variety of natural language understanding tasks. We propose to use external information to improve document modeling for problems that can be framed as sentence extraction. We develop a framework composed of a hierarchical document encoder and an attention-based extractor with attention over external information. We evaluate our model on extractive document summarization (where the external information is image captions and the title of the document) and answer selection (where the external information is a question). We show that our model consistently outperforms strong baselines, in terms of both informativeness and fluency (for CNN document summarization) and achieves state-of-the-art results for answer selection on WikiQA and NewsQA.
Anthology ID:
P18-1188
Erratum e1:
P18-1188e1
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Iryna Gurevych, Yusuke Miyao
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2020–2030
Language:
URL:
https://aclanthology.org/P18-1188
DOI:
10.18653/v1/P18-1188
Bibkey:
Cite (ACL):
Shashi Narayan, Ronald Cardenas, Nikos Papasarantopoulos, Shay B. Cohen, Mirella Lapata, Jiangsheng Yu, and Yi Chang. 2018. Document Modeling with External Attention for Sentence Extraction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2020–2030, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Document Modeling with External Attention for Sentence Extraction (Narayan et al., ACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/P18-1188.pdf
Note:
 P18-1188.Notes.pdf
Presentation:
 P18-1188.Presentation.pdf
Video:
 https://aclanthology.org/P18-1188.mp4
Code
 shashiongithub/Document-Models-with-Ext-Information
Data
MS MARCONewsQASQuADWikiQA