An Exploration of Three Lightly-supervised Representation Learning Approaches for Named Entity Classification

Ajay Nagesh, Mihai Surdeanu


Abstract
Several semi-supervised representation learning methods have been proposed recently that mitigate the drawbacks of traditional bootstrapping: they reduce the amount of semantic drift introduced by iterative approaches through one-shot learning; others address the sparsity of data through the learning of custom, dense representation for the information modeled. In this work, we are the first to adapt three of these methods, most of which have been originally proposed for image processing, to an information extraction task, specifically, named entity classification. Further, we perform a rigorous comparative analysis on two distinct datasets. Our analysis yields several important observations. First, all representation learning methods outperform state-of-the-art semi-supervised methods that do not rely on representation learning. To the best of our knowledge, we report the latest state-of-the-art results on the semi-supervised named entity classification task. Second, one-shot learning methods clearly outperform iterative representation learning approaches. Lastly, one of the best performers relies on the mean teacher framework (Tarvainen and Valpola, 2017), a simple teacher/student approach that is independent of the underlying task-specific model.
Anthology ID:
C18-1196
Volume:
Proceedings of the 27th International Conference on Computational Linguistics
Month:
August
Year:
2018
Address:
Santa Fe, New Mexico, USA
Editors:
Emily M. Bender, Leon Derczynski, Pierre Isabelle
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2312–2324
Language:
URL:
https://aclanthology.org/C18-1196
DOI:
Bibkey:
Cite (ACL):
Ajay Nagesh and Mihai Surdeanu. 2018. An Exploration of Three Lightly-supervised Representation Learning Approaches for Named Entity Classification. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2312–2324, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Cite (Informal):
An Exploration of Three Lightly-supervised Representation Learning Approaches for Named Entity Classification (Nagesh & Surdeanu, COLING 2018)
Copy Citation:
PDF:
https://aclanthology.org/C18-1196.pdf
Data
OntoNotes 5.0