A Neural Network Approach for Knowledge-Driven Response Generation

Pavlos Vougiouklis, Jonathon Hare, Elena Simperl


Abstract
We present a novel response generation system. The system assumes the hypothesis that participants in a conversation base their response not only on previous dialog utterances but also on their background knowledge. Our model is based on a Recurrent Neural Network (RNN) that is trained over concatenated sequences of comments, a Convolution Neural Network that is trained over Wikipedia sentences and a formulation that couples the two trained embeddings in a multimodal space. We create a dataset of aligned Wikipedia sentences and sequences of Reddit utterances, which we we use to train our model. Given a sequence of past utterances and a set of sentences that represent the background knowledge, our end-to-end learnable model is able to generate context-sensitive and knowledge-driven responses by leveraging the alignment of two different data sources. Our approach achieves up to 55% improvement in perplexity compared to purely sequential models based on RNNs that are trained only on sequences of utterances.
Anthology ID:
C16-1318
Volume:
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
Month:
December
Year:
2016
Address:
Osaka, Japan
Editors:
Yuji Matsumoto, Rashmi Prasad
Venue:
COLING
SIG:
Publisher:
The COLING 2016 Organizing Committee
Note:
Pages:
3370–3380
Language:
URL:
https://aclanthology.org/C16-1318
DOI:
Bibkey:
Cite (ACL):
Pavlos Vougiouklis, Jonathon Hare, and Elena Simperl. 2016. A Neural Network Approach for Knowledge-Driven Response Generation. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3370–3380, Osaka, Japan. The COLING 2016 Organizing Committee.
Cite (Informal):
A Neural Network Approach for Knowledge-Driven Response Generation (Vougiouklis et al., COLING 2016)
Copy Citation:
PDF:
https://aclanthology.org/C16-1318.pdf
Code
 pvougiou/Aligning-Reddit-and-Wikipedia