Enhancing Unsupervised Generative Dependency Parser with Contextual Information

Wenjuan Han, Yong Jiang, Kewei Tu


Abstract
Most of the unsupervised dependency parsers are based on probabilistic generative models that learn the joint distribution of the given sentence and its parse. Probabilistic generative models usually explicit decompose the desired dependency tree into factorized grammar rules, which lack the global features of the entire sentence. In this paper, we propose a novel probabilistic model called discriminative neural dependency model with valence (D-NDMV) that generates a sentence and its parse from a continuous latent representation, which encodes global contextual information of the generated sentence. We propose two approaches to model the latent representation: the first deterministically summarizes the representation from the sentence and the second probabilistically models the representation conditioned on the sentence. Our approach can be regarded as a new type of autoencoder model to unsupervised dependency parsing that combines the benefits of both generative and discriminative techniques. In particular, our approach breaks the context-free independence assumption in previous generative approaches and therefore becomes more expressive. Our extensive experimental results on seventeen datasets from various sources show that our approach achieves competitive accuracy compared with both generative and discriminative state-of-the-art unsupervised dependency parsers.
Anthology ID:
P19-1526
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5315–5325
Language:
URL:
https://aclanthology.org/P19-1526
DOI:
10.18653/v1/P19-1526
Bibkey:
Cite (ACL):
Wenjuan Han, Yong Jiang, and Kewei Tu. 2019. Enhancing Unsupervised Generative Dependency Parser with Contextual Information. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5315–5325, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Enhancing Unsupervised Generative Dependency Parser with Contextual Information (Han et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1526.pdf
Data
Universal Dependencies