Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labeling

Ouyu Lan, Xiao Huang, Bill Yuchen Lin, He Jiang, Liyuan Liu, Xiang Ren


Abstract
Sequence labeling is a fundamental task for a range of natural language processing problems. When used in practice, its performance is largely influenced by the annotation quality and quantity, and meanwhile, obtaining ground truth labels is often costly. In many cases, ground truth labels do not exist, but noisy annotations or annotations from different domains are accessible. In this paper, we propose a novel framework Consensus Network (ConNet) that can be trained on annotations from multiple sources (e.g., crowd annotation, cross-domain data). It learns individual representation for every source and dynamically aggregates source-specific knowledge by a context-aware attention module. Finally, it leads to a model reflecting the agreement (consensus) among multiple sources. We evaluate the proposed framework in two practical settings of multi-source learning: learning with crowd annotations and unsupervised cross-domain model adaptation. Extensive experimental results show that our model achieves significant improvements over existing methods in both settings. We also demonstrate that the method can apply to various tasks and cope with different encoders.
Anthology ID:
2020.acl-main.193
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Editors:
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2134–2146
Language:
URL:
https://aclanthology.org/2020.acl-main.193
DOI:
10.18653/v1/2020.acl-main.193
Bibkey:
Cite (ACL):
Ouyu Lan, Xiao Huang, Bill Yuchen Lin, He Jiang, Liyuan Liu, and Xiang Ren. 2020. Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labeling. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2134–2146, Online. Association for Computational Linguistics.
Cite (Informal):
Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labeling (Lan et al., ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.193.pdf
Video:
 http://slideslive.com/38928969
Code
 INK-USC/ConNet
Data
CoNLL++GUMMulti-Domain Sentiment