Semantic Annotation Aggregation with Conditional Crowdsourcing Models and Word Embeddings

Paul Felt, Eric Ringger, Kevin Seppi


Abstract
In modern text annotation projects, crowdsourced annotations are often aggregated using item response models or by majority vote. Recently, item response models enhanced with generative data models have been shown to yield substantial benefits over those with conditional or no data models. However, suitable generative data models do not exist for many tasks, such as semantic labeling tasks. When no generative data model exists, we demonstrate that similar benefits may be derived by conditionally modeling documents that have been previously embedded in a semantic space using recent work in vector space models. We use this approach to show state-of-the-art results on a variety of semantic annotation aggregation tasks.
Anthology ID:
C16-1168
Volume:
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
Month:
December
Year:
2016
Address:
Osaka, Japan
Editors:
Yuji Matsumoto, Rashmi Prasad
Venue:
COLING
SIG:
Publisher:
The COLING 2016 Organizing Committee
Note:
Pages:
1787–1796
Language:
URL:
https://aclanthology.org/C16-1168
DOI:
Bibkey:
Cite (ACL):
Paul Felt, Eric Ringger, and Kevin Seppi. 2016. Semantic Annotation Aggregation with Conditional Crowdsourcing Models and Word Embeddings. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1787–1796, Osaka, Japan. The COLING 2016 Organizing Committee.
Cite (Informal):
Semantic Annotation Aggregation with Conditional Crowdsourcing Models and Word Embeddings (Felt et al., COLING 2016)
Copy Citation:
PDF:
https://aclanthology.org/C16-1168.pdf