Sliding Selector Network with Dynamic Memory for Extractive Summarization of Long Documents

Peng Cui, Le Hu


Abstract
Neural-based summarization models suffer from the length limitation of text encoder. Long documents have to been truncated before they are sent to the model, which results in huge loss of summary-relevant contents. To address this issue, we propose the sliding selector network with dynamic memory for extractive summarization of long-form documents, which employs a sliding window to extract summary sentences segment by segment. Moreover, we adopt memory mechanism to preserve and update the history information dynamically, allowing the semantic flow across different windows. Experimental results on two large-scale datasets that consist of scientific papers demonstrate that our model substantially outperforms previous state-of-the-art models. Besides, we perform qualitative and quantitative investigations on how our model works and where the performance gain comes from.
Anthology ID:
2021.naacl-main.470
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
June
Year:
2021
Address:
Online
Editors:
Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, Yichao Zhou
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5881–5891
Language:
URL:
https://aclanthology.org/2021.naacl-main.470
DOI:
10.18653/v1/2021.naacl-main.470
Bibkey:
Cite (ACL):
Peng Cui and Le Hu. 2021. Sliding Selector Network with Dynamic Memory for Extractive Summarization of Long Documents. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5881–5891, Online. Association for Computational Linguistics.
Cite (Informal):
Sliding Selector Network with Dynamic Memory for Extractive Summarization of Long Documents (Cui & Hu, NAACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.naacl-main.470.pdf
Video:
 https://aclanthology.org/2021.naacl-main.470.mp4
Code
 pcui-nlp/ssn_dm