A Knowledge Hunting Framework for Common Sense Reasoning

Ali Emami, Noelia De La Cruz, Adam Trischler, Kaheer Suleman, Jackie Chi Kit Cheung


Abstract
We introduce an automatic system that achieves state-of-the-art results on the Winograd Schema Challenge (WSC), a common sense reasoning task that requires diverse, complex forms of inference and knowledge. Our method uses a knowledge hunting module to gather text from the web, which serves as evidence for candidate problem resolutions. Given an input problem, our system generates relevant queries to send to a search engine, then extracts and classifies knowledge from the returned results and weighs them to make a resolution. Our approach improves F1 performance on the full WSC by 0.21 over the previous best and represents the first system to exceed 0.5 F1. We further demonstrate that the approach is competitive on the Choice of Plausible Alternatives (COPA) task, which suggests that it is generally applicable.
Anthology ID:
D18-1220
Volume:
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Month:
October-November
Year:
2018
Address:
Brussels, Belgium
Editors:
Ellen Riloff, David Chiang, Julia Hockenmaier, Jun’ichi Tsujii
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1949–1958
Language:
URL:
https://aclanthology.org/D18-1220
DOI:
10.18653/v1/D18-1220
Bibkey:
Cite (ACL):
Ali Emami, Noelia De La Cruz, Adam Trischler, Kaheer Suleman, and Jackie Chi Kit Cheung. 2018. A Knowledge Hunting Framework for Common Sense Reasoning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1949–1958, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
A Knowledge Hunting Framework for Common Sense Reasoning (Emami et al., EMNLP 2018)
Copy Citation:
PDF:
https://aclanthology.org/D18-1220.pdf
Attachment:
 D18-1220.Attachment.pdf
Data
COPAWSC