DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning

Wenhan Xiong, Thien Hoang, William Yang Wang


Abstract
We study the problem of learning to reason in large scale knowledge graphs (KGs). More specifically, we describe a novel reinforcement learning framework for learning multi-hop relational paths: we use a policy-based agent with continuous states based on knowledge graph embeddings, which reasons in a KG vector-space by sampling the most promising relation to extend its path. In contrast to prior work, our approach includes a reward function that takes the accuracy, diversity, and efficiency into consideration. Experimentally, we show that our proposed method outperforms a path-ranking based algorithm and knowledge graph embedding methods on Freebase and Never-Ending Language Learning datasets.
Anthology ID:
D17-1060
Volume:
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
Month:
September
Year:
2017
Address:
Copenhagen, Denmark
Editors:
Martha Palmer, Rebecca Hwa, Sebastian Riedel
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
564–573
Language:
URL:
https://aclanthology.org/D17-1060
DOI:
10.18653/v1/D17-1060
Bibkey:
Cite (ACL):
Wenhan Xiong, Thien Hoang, and William Yang Wang. 2017. DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 564–573, Copenhagen, Denmark. Association for Computational Linguistics.
Cite (Informal):
DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning (Xiong et al., EMNLP 2017)
Copy Citation:
PDF:
https://aclanthology.org/D17-1060.pdf
Video:
 https://aclanthology.org/D17-1060.mp4
Code
 xwhan/DeepPath +  additional community code
Data
NELL-995NELL