Latent Structure Models for Natural Language Processing

André F. T. Martins, Tsvetomila Mihaylova, Nikita Nangia, Vlad Niculae


Abstract
Latent structure models are a powerful tool for modeling compositional data, discovering linguistic structure, and building NLP pipelines. They are appealing for two main reasons: they allow incorporating structural bias during training, leading to more accurate models; and they allow discovering hidden linguistic structure, which provides better interpretability. This tutorial will cover recent advances in discrete latent structure models. We discuss their motivation, potential, and limitations, then explore in detail three strategies for designing such models: gradient approximation, reinforcement learning, and end-to-end differentiable methods. We highlight connections among all these methods, enumerating their strengths and weaknesses. The models we present and analyze have been applied to a wide variety of NLP tasks, including sentiment analysis, natural language inference, language modeling, machine translation, and semantic parsing. Examples and evaluation will be covered throughout. After attending the tutorial, a practitioner will be better informed about which method is best suited for their problem.
Anthology ID:
P19-4001
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Preslav Nakov, Alexis Palmer
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–5
Language:
URL:
https://aclanthology.org/P19-4001
DOI:
10.18653/v1/P19-4001
Bibkey:
Cite (ACL):
André F. T. Martins, Tsvetomila Mihaylova, Nikita Nangia, and Vlad Niculae. 2019. Latent Structure Models for Natural Language Processing. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, pages 1–5, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Latent Structure Models for Natural Language Processing (Martins et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-4001.pdf
Presentation:
 P19-4001.Presentation.pdf