Disambiguated skip-gram model

Karol Grzegorczyk, Marcin Kurdziel


Abstract
We present disambiguated skip-gram: a neural-probabilistic model for learning multi-sense distributed representations of words. Disambiguated skip-gram jointly estimates a skip-gram-like context word prediction model and a word sense disambiguation model. Unlike previous probabilistic models for learning multi-sense word embeddings, disambiguated skip-gram is end-to-end differentiable and can be interpreted as a simple feed-forward neural network. We also introduce an effective pruning strategy for the embeddings learned by disambiguated skip-gram. This allows us to control the granularity of representations learned by our model. In experimental evaluation disambiguated skip-gram improves state-of-the are results in several word sense induction benchmarks.
Anthology ID:
D18-1174
Volume:
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Month:
October-November
Year:
2018
Address:
Brussels, Belgium
Editors:
Ellen Riloff, David Chiang, Julia Hockenmaier, Jun’ichi Tsujii
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1445–1454
Language:
URL:
https://aclanthology.org/D18-1174
DOI:
10.18653/v1/D18-1174
Bibkey:
Cite (ACL):
Karol Grzegorczyk and Marcin Kurdziel. 2018. Disambiguated skip-gram model. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1445–1454, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Disambiguated skip-gram model (Grzegorczyk & Kurdziel, EMNLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/D18-1174.pdf
Attachment:
 D18-1174.Attachment.zip