Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport

Kyle Swanson, Lili Yu, Tao Lei


Abstract
Selecting input features of top relevance has become a popular method for building self-explaining models. In this work, we extend this selective rationalization approach to text matching, where the goal is to jointly select and align text pieces, such as tokens or sentences, as a justification for the downstream prediction. Our approach employs optimal transport (OT) to find a minimal cost alignment between the inputs. However, directly applying OT often produces dense and therefore uninterpretable alignments. To overcome this limitation, we introduce novel constrained variants of the OT problem that result in highly sparse alignments with controllable sparsity. Our model is end-to-end differentiable using the Sinkhorn algorithm for OT and can be trained without any alignment annotations. We evaluate our model on the StackExchange, MultiNews, e-SNLI, and MultiRC datasets. Our model achieves very sparse rationale selections with high fidelity while preserving prediction accuracy compared to strong attention baseline models.
Anthology ID:
2020.acl-main.496
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Editors:
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5609–5626
Language:
URL:
https://aclanthology.org/2020.acl-main.496
DOI:
10.18653/v1/2020.acl-main.496
Bibkey:
Cite (ACL):
Kyle Swanson, Lili Yu, and Tao Lei. 2020. Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5609–5626, Online. Association for Computational Linguistics.
Cite (Informal):
Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport (Swanson et al., ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.496.pdf
Dataset:
 2020.acl-main.496.Dataset.pdf
Video:
 http://slideslive.com/38929403
Code
 asappresearch/rationale-alignment
Data
Multi-NewsMultiRCSNLIe-SNLI