Partial Or Complete, That’s The Question

Qiang Ning, Hangfeng He, Chuchu Fan, Dan Roth


Abstract
For many structured learning tasks, the data annotation process is complex and costly. Existing annotation schemes usually aim at acquiring completely annotated structures, under the common perception that partial structures are of low quality and could hurt the learning process. This paper questions this common perception, motivated by the fact that structures consist of interdependent sets of variables. Thus, given a fixed budget, partly annotating each structure may provide the same level of supervision, while allowing for more structures to be annotated. We provide an information theoretic formulation for this perspective and use it, in the context of three diverse structured learning tasks, to show that learning from partial structures can sometimes outperform learning from complete ones. Our findings may provide important insights into structured data annotation schemes and could support progress in learning protocols for structured tasks.
Anthology ID:
N19-1227
Volume:
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Jill Burstein, Christy Doran, Thamar Solorio
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2190–2200
Language:
URL:
https://aclanthology.org/N19-1227
DOI:
10.18653/v1/N19-1227
Bibkey:
Cite (ACL):
Qiang Ning, Hangfeng He, Chuchu Fan, and Dan Roth. 2019. Partial Or Complete, That’s The Question. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2190–2200, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Partial Or Complete, That’s The Question (Ning et al., NAACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/N19-1227.pdf
Supplementary:
 N19-1227.Supplementary.pdf