Annotating and analyzing the interactions between meaning relations

Darina Gold, Venelin Kovatchev, Torsten Zesch


Abstract
Pairs of sentences, phrases, or other text pieces can hold semantic relations such as paraphrasing, textual entailment, contradiction, specificity, and semantic similarity. These relations are usually studied in isolation and no dataset exists where they can be compared empirically. Here we present a corpus annotated with these relations and the analysis of these results. The corpus contains 520 sentence pairs, annotated with these relations. We measure the annotation reliability of each individual relation and we examine their interactions and correlations. Among the unexpected results revealed by our analysis is that the traditionally considered direct relationship between paraphrasing and bi-directional entailment does not hold in our data.
Anthology ID:
W19-4004
Volume:
Proceedings of the 13th Linguistic Annotation Workshop
Month:
August
Year:
2019
Address:
Florence, Italy
Editors:
Annemarie Friedrich, Deniz Zeyrek, Jet Hoek
Venue:
LAW
SIG:
SIGANN
Publisher:
Association for Computational Linguistics
Note:
Pages:
26–36
Language:
URL:
https://aclanthology.org/W19-4004
DOI:
10.18653/v1/W19-4004
Bibkey:
Cite (ACL):
Darina Gold, Venelin Kovatchev, and Torsten Zesch. 2019. Annotating and analyzing the interactions between meaning relations. In Proceedings of the 13th Linguistic Annotation Workshop, pages 26–36, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Annotating and analyzing the interactions between meaning relations (Gold et al., LAW 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-4004.pdf
Code
 MeDarina/meaning_relations_interaction
Data
SICKSNLI