D-GloVe: A Feasible Least Squares Model for Estimating Word Embedding Densities

Shoaib Jameel, Steven Schockaert


Abstract
We propose a new word embedding model, inspired by GloVe, which is formulated as a feasible least squares optimization problem. In contrast to existing models, we explicitly represent the uncertainty about the exact definition of each word vector. To this end, we estimate the error that results from using noisy co-occurrence counts in the formulation of the model, and we model the imprecision that results from including uninformative context words. Our experimental results demonstrate that this model compares favourably with existing word embedding models.
Anthology ID:
C16-1174
Volume:
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
Month:
December
Year:
2016
Address:
Osaka, Japan
Editors:
Yuji Matsumoto, Rashmi Prasad
Venue:
COLING
SIG:
Publisher:
The COLING 2016 Organizing Committee
Note:
Pages:
1849–1860
Language:
URL:
https://aclanthology.org/C16-1174
DOI:
Bibkey:
Cite (ACL):
Shoaib Jameel and Steven Schockaert. 2016. D-GloVe: A Feasible Least Squares Model for Estimating Word Embedding Densities. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1849–1860, Osaka, Japan. The COLING 2016 Organizing Committee.
Cite (Informal):
D-GloVe: A Feasible Least Squares Model for Estimating Word Embedding Densities (Jameel & Schockaert, COLING 2016)
Copy Citation:
PDF:
https://aclanthology.org/C16-1174.pdf