SC-LSTM: Learning Task-Specific Representations in Multi-Task Learning for Sequence Labeling

Peng Lu, Ting Bai, Philippe Langlais


Abstract
Multi-task learning (MTL) has been studied recently for sequence labeling. Typically, auxiliary tasks are selected specifically in order to improve the performance of a target task. Jointly learning multiple tasks in a way that benefit all of them simultaneously can increase the utility of MTL. In order to do so, we propose a new LSTM cell which contains both shared parameters that can learn from all tasks, and task-specific parameters that can learn task-specific information. We name it a Shared-Cell Long-Short Term Memory SC-LSTM. Experimental results on three sequence labeling benchmarks (named-entity recognition, text chunking, and part-of-speech tagging) demonstrate the effectiveness of our SC-LSTM cell.
Anthology ID:
N19-1249
Volume:
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Jill Burstein, Christy Doran, Thamar Solorio
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2396–2406
Language:
URL:
https://aclanthology.org/N19-1249
DOI:
10.18653/v1/N19-1249
Bibkey:
Cite (ACL):
Peng Lu, Ting Bai, and Philippe Langlais. 2019. SC-LSTM: Learning Task-Specific Representations in Multi-Task Learning for Sequence Labeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2396–2406, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
SC-LSTM: Learning Task-Specific Representations in Multi-Task Learning for Sequence Labeling (Lu et al., NAACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/N19-1249.pdf
Data
CoNLL 2003Universal Dependencies