Distantly-Supervised Dense Retrieval Enables Open-Domain Question Answering without Evidence Annotation

Chen Zhao, Chenyan Xiong, Jordan Boyd-Graber, Hal Daumé III


Abstract
Open-domain question answering answers a question based on evidence retrieved from a large corpus. State-of-the-art neural approaches require intermediate evidence annotations for training. However, such intermediate annotations are expensive, and methods that rely on them cannot transfer to the more common setting, where only question–answer pairs are available. This paper investigates whether models can learn to find evidence from a large corpus, with only distant supervision from answer labels for model training, thereby generating no additional annotation cost. We introduce a novel approach (DistDR) that iteratively improves over a weak retriever by alternately finding evidence from the up-to-date model and encouraging the model to learn the most likely evidence. Without using any evidence labels, DistDR is on par with fully-supervised state-of-the-art methods on both multi-hop and single-hop QA benchmarks. Our analysis confirms that DistDR finds more accurate evidence over iterations, which leads to model improvements. The code is available at https://github.com/henryzhao5852/DistDR.
Anthology ID:
2021.emnlp-main.756
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
9612–9622
Language:
URL:
https://aclanthology.org/2021.emnlp-main.756
DOI:
10.18653/v1/2021.emnlp-main.756
Bibkey:
Cite (ACL):
Chen Zhao, Chenyan Xiong, Jordan Boyd-Graber, and Hal Daumé III. 2021. Distantly-Supervised Dense Retrieval Enables Open-Domain Question Answering without Evidence Annotation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9612–9622, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Distantly-Supervised Dense Retrieval Enables Open-Domain Question Answering without Evidence Annotation (Zhao et al., EMNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.emnlp-main.756.pdf
Video:
 https://aclanthology.org/2021.emnlp-main.756.mp4
Code
 henryzhao5852/distdr