Generative Context Pair Selection for Multi-hop Question Answering

Dheeru Dua, Cicero Nogueira dos Santos, Patrick Ng, Ben Athiwaratkun, Bing Xiang, Matt Gardner, Sameer Singh


Abstract
Compositional reasoning tasks such as multi-hop question answering require models to learn how to make latent decisions using only weak supervision from the final answer. Crowdsourced datasets gathered for these tasks, however, often contain only a slice of the underlying task distribution, which can induce unanticipated biases such as shallow word overlap between the question and context. Recent works have shown that discriminative training results in models that exploit these underlying biases to achieve a better held-out performance, without learning the right way to reason. We propose a generative context selection model for multi-hop QA that reasons about how the given question could have been generated given a context pair and not just independent contexts. We show that on HotpotQA, while being comparable to the state-of-the-art answering performance, our proposed generative passage selection model has a better performance (4.9% higher than baseline) on adversarial held-out set which tests robustness of model’s multi-hop reasoning capabilities.
Anthology ID:
2021.emnlp-main.561
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7009–7015
Language:
URL:
https://aclanthology.org/2021.emnlp-main.561
DOI:
10.18653/v1/2021.emnlp-main.561
Bibkey:
Cite (ACL):
Dheeru Dua, Cicero Nogueira dos Santos, Patrick Ng, Ben Athiwaratkun, Bing Xiang, Matt Gardner, and Sameer Singh. 2021. Generative Context Pair Selection for Multi-hop Question Answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7009–7015, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Generative Context Pair Selection for Multi-hop Question Answering (Dua et al., EMNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.emnlp-main.561.pdf
Video:
 https://aclanthology.org/2021.emnlp-main.561.mp4
Data
HotpotQAWikiHop