Learning and Knowledge Transfer with Memory Networks for Machine Comprehension

Mohit Yadav, Lovekesh Vig, Gautam Shroff


Abstract
Enabling machines to read and comprehend unstructured text remains an unfulfilled goal for NLP research. Recent research efforts on the “machine comprehension” task have managed to achieve close to ideal performance on simulated data. However, achieving similar levels of performance on small real world datasets has proved difficult; major challenges stem from the large vocabulary size, complex grammar, and, the frequent ambiguities in linguistic structure. On the other hand, the requirement of human generated annotations for training, in order to ensure a sufficiently diverse set of questions is prohibitively expensive. Motivated by these practical issues, we propose a novel curriculum inspired training procedure for Memory Networks to improve the performance for machine comprehension with relatively small volumes of training data. Additionally, we explore various training regimes for Memory Networks to allow knowledge transfer from a closely related domain having larger volumes of labelled data. We also suggest the use of a loss function to incorporate the asymmetric nature of knowledge transfer. Our experiments demonstrate improvements on Dailymail, CNN, and MCTest datasets.
Anthology ID:
E17-1080
Volume:
Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers
Month:
April
Year:
2017
Address:
Valencia, Spain
Editors:
Mirella Lapata, Phil Blunsom, Alexander Koller
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
850–859
Language:
URL:
https://aclanthology.org/E17-1080
DOI:
Bibkey:
Cite (ACL):
Mohit Yadav, Lovekesh Vig, and Gautam Shroff. 2017. Learning and Knowledge Transfer with Memory Networks for Machine Comprehension. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 850–859, Valencia, Spain. Association for Computational Linguistics.
Cite (Informal):
Learning and Knowledge Transfer with Memory Networks for Machine Comprehension (Yadav et al., EACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/E17-1080.pdf
Data
MCTest