Learning Geometric Word Meta-Embeddings

Pratik Jawanpuria, Satya Dev N T V, Anoop Kunchukuttan, Bamdev Mishra


Abstract
We propose a geometric framework for learning meta-embeddings of words from different embedding sources. Our framework transforms the embeddings into a common latent space, where, for example, simple averaging or concatenation of different embeddings (of a given word) is more amenable. The proposed latent space arises from two particular geometric transformations - source embedding specific orthogonal rotations and a common Mahalanobis metric scaling. Empirical results on several word similarity and word analogy benchmarks illustrate the efficacy of the proposed framework.
Anthology ID:
2020.repl4nlp-1.6
Volume:
Proceedings of the 5th Workshop on Representation Learning for NLP
Month:
July
Year:
2020
Address:
Online
Venues:
ACL | RepL4NLP | WS
SIG:
SIGREP
Publisher:
Association for Computational Linguistics
Note:
Pages:
39–44
URL:
https://www.aclweb.org/anthology/2020.repl4nlp-1.6
DOI:
10.18653/v1/2020.repl4nlp-1.6
Bib Export formats:
BibTeX MODS XML EndNote
PDF:
https://www.aclweb.org/anthology/2020.repl4nlp-1.6.pdf
Software:
 2020.repl4nlp-1.6.Software.zip
Video:
 http://slideslive.com/38929772