Dirichlet Latent Variable Hierarchical Recurrent Encoder-Decoder in Dialogue Generation

Min Zeng, Yisen Wang, Yuan Luo


Abstract
Variational encoder-decoders have achieved well-recognized performance in the dialogue generation task. Existing works simply assume the Gaussian priors of the latent variable, which are incapable of representing complex latent variables effectively. To address the issues, we propose to use the Dirichlet distribution with flexible structures to characterize the latent variables in place of the traditional Gaussian distribution, called Dirichlet Latent Variable Hierarchical Recurrent Encoder-Decoder model (Dir-VHRED). Based on which, we further find that there is redundancy among the dimensions of latent variable, and the lengths and sentence patterns of the responses can be strongly correlated to each dimension of the latent variable. Therefore, controllable responses can be generated through specifying the value of each dimension of the latent variable. Experimental results on benchmarks show that our proposed Dir-VHRED yields substantial improvements on negative log-likelihood, word-embedding-based and human evaluations.
Anthology ID:
D19-1124
Volume:
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
Venues:
EMNLP | IJCNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1267–1272
Language:
URL:
https://aclanthology.org/D19-1124
DOI:
10.18653/v1/D19-1124
Bibkey:
Cite (ACL):
Min Zeng, Yisen Wang, and Yuan Luo. 2019. Dirichlet Latent Variable Hierarchical Recurrent Encoder-Decoder in Dialogue Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1267–1272, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Dirichlet Latent Variable Hierarchical Recurrent Encoder-Decoder in Dialogue Generation (Zeng et al., EMNLP-IJCNLP 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-1124.pdf
Attachment:
 D19-1124.Attachment.zip