Investigating Word-Class Distributions in Word Vector Spaces

Ryohei Sasano, Anna Korhonen


Abstract
This paper presents an investigation on the distribution of word vectors belonging to a certain word class in a pre-trained word vector space. To this end, we made several assumptions about the distribution, modeled the distribution accordingly, and validated each assumption by comparing the goodness of each model. Specifically, we considered two types of word classes – the semantic class of direct objects of a verb and the semantic class in a thesaurus – and tried to build models that properly estimate how likely it is that a word in the vector space is a member of a given word class. Our results on selectional preference and WordNet datasets show that the centroid-based model will fail to achieve good enough performance, the geometry of the distribution and the existence of subgroups will have limited impact, and also the negative instances need to be considered for adequate modeling of the distribution. We further investigated the relationship between the scores calculated by each model and the degree of membership and found that discriminative learning-based models are best in finding the boundaries of a class, while models based on the offset between positive and negative instances perform best in determining the degree of membership.
Anthology ID:
2020.acl-main.337
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Editors:
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3657–3666
Language:
URL:
https://aclanthology.org/2020.acl-main.337
DOI:
10.18653/v1/2020.acl-main.337
Bibkey:
Cite (ACL):
Ryohei Sasano and Anna Korhonen. 2020. Investigating Word-Class Distributions in Word Vector Spaces. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3657–3666, Online. Association for Computational Linguistics.
Cite (Informal):
Investigating Word-Class Distributions in Word Vector Spaces (Sasano & Korhonen, ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.337.pdf
Video:
 http://slideslive.com/38928833