Projection Aléatoire Non-Négative pour le Calcul de Word Embedding / Non-Negative Randomized Word Embedding

Behrang Qasemizadeh, Laura Kallmeyer, Aurelie Herbelot


Abstract
Non-Negative Randomized Word Embedding We propose a word embedding method which is based on a novel random projection technique. We show that weighting methods such as positive pointwise mutual information (PPMI) can be applied to our models after their construction and at a reduced dimensionality. Hence, the proposed technique can efficiently transfer words onto semantically discriminative spaces while demonstrating high computational performance, besides benefits such as ease of update and a simple mechanism for interoperability. We report the performance of our method on several tasks and show that it yields competitive results compared to neural embedding methods in monolingual corpus-based setups.
Anthology ID:
2017.jeptalnrecital-long.8
Volume:
Actes des 24ème Conférence sur le Traitement Automatique des Langues Naturelles. Volume 1 - Articles longs
Month:
6
Year:
2017
Address:
Orléans, France
Editors:
Iris Eshkol-Taravella, Jean-Yves Antoine
Venue:
JEP/TALN/RECITAL
SIG:
Publisher:
ATALA
Note:
Pages:
109–122
Language:
URL:
https://aclanthology.org/2017.jeptalnrecital-long.8
DOI:
Bibkey:
Cite (ACL):
Behrang Qasemizadeh, Laura Kallmeyer, and Aurelie Herbelot. 2017. Projection Aléatoire Non-Négative pour le Calcul de Word Embedding / Non-Negative Randomized Word Embedding. In Actes des 24ème Conférence sur le Traitement Automatique des Langues Naturelles. Volume 1 - Articles longs, pages 109–122, Orléans, France. ATALA.
Cite (Informal):
Projection Aléatoire Non-Négative pour le Calcul de Word Embedding / Non-Negative Randomized Word Embedding (Qasemizadeh et al., JEP/TALN/RECITAL 2017)
Copy Citation:
PDF:
https://aclanthology.org/2017.jeptalnrecital-long.8.pdf