The Secret is in the Spectra: Predicting Cross-lingual Task Performance with Spectral Similarity Measures

Haim Dubossarsky, Ivan Vulić, Roi Reichart, Anna Korhonen


Abstract
Performance in cross-lingual NLP tasks is impacted by the (dis)similarity of languages at hand: e.g., previous work has suggested there is a connection between the expected success of bilingual lexicon induction (BLI) and the assumption of (approximate) isomorphism between monolingual embedding spaces. In this work we present a large-scale study focused on the correlations between monolingual embedding space similarity and task performance, covering thousands of language pairs and four different tasks: BLI, parsing, POS tagging and MT. We hypothesize that statistics of the spectrum of each monolingual embedding space indicate how well they can be aligned. We then introduce several isomorphism measures between two embedding spaces, based on the relevant statistics of their individual spectra. We empirically show that (1) language similarity scores derived from such spectral isomorphism measures are strongly associated with performance observed in different cross-lingual tasks, and (2) our spectral-based measures consistently outperform previous standard isomorphism measures, while being computationally more tractable and easier to interpret. Finally, our measures capture complementary information to typologically driven language distance measures, and the combination of measures from the two families yields even higher task performance correlations.
Anthology ID:
2020.emnlp-main.186
Volume:
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Month:
November
Year:
2020
Address:
Online
Editors:
Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2377–2390
Language:
URL:
https://aclanthology.org/2020.emnlp-main.186
DOI:
10.18653/v1/2020.emnlp-main.186
Bibkey:
Cite (ACL):
Haim Dubossarsky, Ivan Vulić, Roi Reichart, and Anna Korhonen. 2020. The Secret is in the Spectra: Predicting Cross-lingual Task Performance with Spectral Similarity Measures. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2377–2390, Online. Association for Computational Linguistics.
Cite (Informal):
The Secret is in the Spectra: Predicting Cross-lingual Task Performance with Spectral Similarity Measures (Dubossarsky et al., EMNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.emnlp-main.186.pdf
Optional supplementary material:
 2020.emnlp-main.186.OptionalSupplementaryMaterial.zip
Video:
 https://slideslive.com/38939057
Data
Panlex