Aspect-based summarization of pros and cons in unstructured product reviews

Florian Kunneman, Sander Wubben, Antal van den Bosch, Emiel Krahmer


Abstract
We developed three systems for generating pros and cons summaries of product reviews. Automating this task eases the writing of product reviews, and offers readers quick access to the most important information. We compared SynPat, a system based on syntactic phrases selected on the basis of valence scores, against a neural-network-based system trained to map bag-of-words representations of reviews directly to pros and cons, and the same neural system trained on clusters of word-embedding encodings of similar pros and cons. We evaluated the systems in two ways: first on held-out reviews with gold-standard pros and cons, and second by asking human annotators to rate the systems’ output on relevance and completeness. In the second evaluation, the gold-standard pros and cons were assessed along with the system output. We find that the human-generated summaries are not deemed as significantly more relevant or complete than the SynPat systems; the latter are scored higher than the human-generated summaries on a precision metric. The neural approaches yield a lower performance in the human assessment, and are outperformed by the baseline.
Anthology ID:
C18-1188
Volume:
Proceedings of the 27th International Conference on Computational Linguistics
Month:
August
Year:
2018
Address:
Santa Fe, New Mexico, USA
Editors:
Emily M. Bender, Leon Derczynski, Pierre Isabelle
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2219–2229
Language:
URL:
https://aclanthology.org/C18-1188
DOI:
Bibkey:
Cite (ACL):
Florian Kunneman, Sander Wubben, Antal van den Bosch, and Emiel Krahmer. 2018. Aspect-based summarization of pros and cons in unstructured product reviews. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2219–2229, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Cite (Informal):
Aspect-based summarization of pros and cons in unstructured product reviews (Kunneman et al., COLING 2018)
Copy Citation:
PDF:
https://aclanthology.org/C18-1188.pdf
Code
 fkunneman/Product_review_summary