tBERT: Topic Models and BERT Joining Forces for Semantic Similarity Detection

Nicole Peinelt, Dong Nguyen, Maria Liakata


Abstract
Semantic similarity detection is a fundamental task in natural language understanding. Adding topic information has been useful for previous feature-engineered semantic similarity models as well as neural models for other tasks. There is currently no standard way of combining topics with pretrained contextual representations such as BERT. We propose a novel topic-informed BERT-based architecture for pairwise semantic similarity detection and show that our model improves performance over strong neural baselines across a variety of English language datasets. We find that the addition of topics to BERT helps particularly with resolving domain-specific cases.
Anthology ID:
2020.acl-main.630
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Editors:
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7047–7055
Language:
URL:
https://aclanthology.org/2020.acl-main.630
DOI:
10.18653/v1/2020.acl-main.630
Bibkey:
Cite (ACL):
Nicole Peinelt, Dong Nguyen, and Maria Liakata. 2020. tBERT: Topic Models and BERT Joining Forces for Semantic Similarity Detection. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7047–7055, Online. Association for Computational Linguistics.
Cite (Informal):
tBERT: Topic Models and BERT Joining Forces for Semantic Similarity Detection (Peinelt et al., ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.630.pdf
Video:
 http://slideslive.com/38928734
Code
 wuningxi/tBERT
Data
100DOH