Cross-lingual Information Retrieval with BERT

Zhuolin Jiang, Amro El-Jaroudi, William Hartmann, Damianos Karakos, Lingjun Zhao


Abstract
Multiple neural language models have been developed recently, e.g., BERT and XLNet, and achieved impressive results in various NLP tasks including sentence classification, question answering and document ranking. In this paper, we explore the use of the popular bidirectional language model, BERT, to model and learn the relevance between English queries and foreign-language documents in the task of cross-lingual information retrieval. A deep relevance matching model based on BERT is introduced and trained by finetuning a pretrained multilingual BERT model with weak supervision, using home-made CLIR training data derived from parallel corpora. Experimental results of the retrieval of Lithuanian documents against short English queries show that our model is effective and outperforms the competitive baseline approaches.
Anthology ID:
2020.clssts-1.5
Volume:
Proceedings of the workshop on Cross-Language Search and Summarization of Text and Speech (CLSSTS2020)
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Kathy McKeown, Douglas W. Oard, Elizabeth, Richard Schwartz
Venue:
CLSSTS
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
26–31
Language:
English
URL:
https://aclanthology.org/2020.clssts-1.5
DOI:
Bibkey:
Cite (ACL):
Zhuolin Jiang, Amro El-Jaroudi, William Hartmann, Damianos Karakos, and Lingjun Zhao. 2020. Cross-lingual Information Retrieval with BERT. In Proceedings of the workshop on Cross-Language Search and Summarization of Text and Speech (CLSSTS2020), pages 26–31, Marseille, France. European Language Resources Association.
Cite (Informal):
Cross-lingual Information Retrieval with BERT (Jiang et al., CLSSTS 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.clssts-1.5.pdf