TextGraphs 2020 Shared Task on Multi-Hop Inference for Explanation Regeneration

Peter Jansen, Dmitry Ustalov


Abstract
The 2020 Shared Task on Multi-Hop Inference for Explanation Regeneration tasks participants with regenerating large detailed multi-fact explanations for standardized science exam questions. Given a question, correct answer, and knowledge base, models must rank each fact in the knowledge base such that facts most likely to appear in the explanation are ranked highest. Explanations consist of an average of 6 (and as many as 16) facts that span both core scientific knowledge and world knowledge, and form an explicit lexically-connected “explanation graph” describing how the facts interrelate. In this second iteration of the explanation regeneration shared task, participants are supplied with more than double the training and evaluation data of the first shared task, as well as a knowledge base nearly double in size, both of which expand into more challenging scientific topics that increase the difficulty of the task. In total 10 teams participated, and 5 teams submitted system description papers. The best-performing teams significantly increased state-of-the-art performance both in terms of ranking (mean average precision) and inference speed on this challenge task.
Anthology ID:
2020.textgraphs-1.10
Volume:
Proceedings of the Graph-based Methods for Natural Language Processing (TextGraphs)
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Dmitry Ustalov, Swapna Somasundaran, Alexander Panchenko, Fragkiskos D. Malliaros, Ioana Hulpuș, Peter Jansen, Abhik Jana
Venue:
TextGraphs
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
85–97
Language:
URL:
https://aclanthology.org/2020.textgraphs-1.10
DOI:
10.18653/v1/2020.textgraphs-1.10
Bibkey:
Cite (ACL):
Peter Jansen and Dmitry Ustalov. 2020. TextGraphs 2020 Shared Task on Multi-Hop Inference for Explanation Regeneration. In Proceedings of the Graph-based Methods for Natural Language Processing (TextGraphs), pages 85–97, Barcelona, Spain (Online). Association for Computational Linguistics.
Cite (Informal):
TextGraphs 2020 Shared Task on Multi-Hop Inference for Explanation Regeneration (Jansen & Ustalov, TextGraphs 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.textgraphs-1.10.pdf
Data
GLUEQASCWorldtree