Parallax: Visualizing and Understanding the Semantics of Embedding Spaces via Algebraic Formulae

Piero Molino, Yang Wang, Jiawei Zhang


Abstract
Embeddings are a fundamental component of many modern machine learning and natural language processing models. Understanding them and visualizing them is essential for gathering insights about the information they capture and the behavior of the models. In this paper, we introduce Parallax, a tool explicitly designed for this task. Parallax allows the user to use both state-of-the-art embedding analysis methods (PCA and t-SNE) and a simple yet effective task-oriented approach where users can explicitly define the axes of the projection through algebraic formulae. %consists in projecting them in two-dimensional planes without any interpretable semantics associated to the axes of the projection, which makes detailed analyses and comparison among multiple sets of embeddings challenging. In this approach, embeddings are projected into a semantically meaningful subspace, which enhances interpretability and allows for more fine-grained analysis. We demonstrate the power of the tool and the proposed methodology through a series of case studies and a user study.
Anthology ID:
P19-3028
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Marta R. Costa-jussà, Enrique Alfonseca
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
165–180
Language:
URL:
https://aclanthology.org/P19-3028
DOI:
10.18653/v1/P19-3028
Bibkey:
Cite (ACL):
Piero Molino, Yang Wang, and Jiawei Zhang. 2019. Parallax: Visualizing and Understanding the Semantics of Embedding Spaces via Algebraic Formulae. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 165–180, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Parallax: Visualizing and Understanding the Semantics of Embedding Spaces via Algebraic Formulae (Molino et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-3028.pdf
Code
 uber-research/parallax +  additional community code