Structured Dialogue Policy with Graph Neural Networks

Lu Chen, Bowen Tan, Sishan Long, Kai Yu


Abstract
Recently, deep reinforcement learning (DRL) has been used for dialogue policy optimization. However, many DRL-based policies are not sample-efficient. Most recent advances focus on improving DRL optimization algorithms to address this issue. Here, we take an alternative route of designing neural network structure that is better suited for DRL-based dialogue management. The proposed structured deep reinforcement learning is based on graph neural networks (GNN), which consists of some sub-networks, each one for a node on a directed graph. The graph is defined according to the domain ontology and each node can be considered as a sub-agent. During decision making, these sub-agents have internal message exchange between neighbors on the graph. We also propose an approach to jointly optimize the graph structure as well as the parameters of GNN. Experiments show that structured DRL significantly outperforms previous state-of-the-art approaches in almost all of the 18 tasks of the PyDial benchmark.
Anthology ID:
C18-1107
Volume:
Proceedings of the 27th International Conference on Computational Linguistics
Month:
August
Year:
2018
Address:
Santa Fe, New Mexico, USA
Editors:
Emily M. Bender, Leon Derczynski, Pierre Isabelle
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1257–1268
Language:
URL:
https://aclanthology.org/C18-1107
DOI:
Bibkey:
Cite (ACL):
Lu Chen, Bowen Tan, Sishan Long, and Kai Yu. 2018. Structured Dialogue Policy with Graph Neural Networks. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1257–1268, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Cite (Informal):
Structured Dialogue Policy with Graph Neural Networks (Chen et al., COLING 2018)
Copy Citation:
PDF:
https://aclanthology.org/C18-1107.pdf