To Attend or not to Attend: A Case Study on Syntactic Structures for Semantic Relatedness

Amulya Gupta, Zhu Zhang


Abstract
With the recent success of Recurrent Neural Networks (RNNs) in Machine Translation (MT), attention mechanisms have become increasingly popular. The purpose of this paper is two-fold; firstly, we propose a novel attention model on Tree Long Short-Term Memory Networks (Tree-LSTMs), a tree-structured generalization of standard LSTM. Secondly, we study the interaction between attention and syntactic structures, by experimenting with three LSTM variants: bidirectional-LSTMs, Constituency Tree-LSTMs, and Dependency Tree-LSTMs. Our models are evaluated on two semantic relatedness tasks: semantic relatedness scoring for sentence pairs (SemEval 2012, Task 6 and SemEval 2014, Task 1) and paraphrase detection for question pairs (Quora, 2017).
Anthology ID:
P18-1197
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Iryna Gurevych, Yusuke Miyao
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2116–2125
Language:
URL:
https://aclanthology.org/P18-1197
DOI:
10.18653/v1/P18-1197
Bibkey:
Cite (ACL):
Amulya Gupta and Zhu Zhang. 2018. To Attend or not to Attend: A Case Study on Syntactic Structures for Semantic Relatedness. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2116–2125, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
To Attend or not to Attend: A Case Study on Syntactic Structures for Semantic Relatedness (Gupta & Zhang, ACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/P18-1197.pdf
Presentation:
 P18-1197.Presentation.pdf
Video:
 https://aclanthology.org/P18-1197.mp4
Data
SICK