Challenges in Data-to-Document Generation

Sam Wiseman, Stuart Shieber, Alexander Rush


Abstract
Recent neural models have shown significant progress on the problem of generating short descriptive texts conditioned on a small number of database records. In this work, we suggest a slightly more difficult data-to-text generation task, and investigate how effective current approaches are on this task. In particular, we introduce a new, large-scale corpus of data records paired with descriptive documents, propose a series of extractive evaluation methods for analyzing performance, and obtain baseline results using current neural generation methods. Experiments show that these models produce fluent text, but fail to convincingly approximate human-generated documents. Moreover, even templated baselines exceed the performance of these neural models on some metrics, though copy- and reconstruction-based extensions lead to noticeable improvements.
Anthology ID:
D17-1239
Volume:
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
Month:
September
Year:
2017
Address:
Copenhagen, Denmark
Editors:
Martha Palmer, Rebecca Hwa, Sebastian Riedel
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
2253–2263
Language:
URL:
https://aclanthology.org/D17-1239
DOI:
10.18653/v1/D17-1239
Bibkey:
Cite (ACL):
Sam Wiseman, Stuart Shieber, and Alexander Rush. 2017. Challenges in Data-to-Document Generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2253–2263, Copenhagen, Denmark. Association for Computational Linguistics.
Cite (Informal):
Challenges in Data-to-Document Generation (Wiseman et al., EMNLP 2017)
Copy Citation:
PDF:
https://aclanthology.org/D17-1239.pdf
Attachment:
 D17-1239.Attachment.pdf
Code
 harvardnlp/data2text +  additional community code
Data
RotoWireRoboCup