Low Resource Sequence Tagging using Sentence Reconstruction

Tal Perl, Sriram Chaudhury, Raja Giryes


Abstract
This work revisits the task of training sequence tagging models with limited resources using transfer learning. We investigate several proposed approaches introduced in recent works and suggest a new loss that relies on sentence reconstruction from normalized embeddings. Specifically, our method demonstrates how by adding a decoding layer for sentence reconstruction, we can improve the performance of various baselines. We show improved results on the CoNLL02 NER and UD 1.2 POS datasets and demonstrate the power of the method for transfer learning with low-resources achieving 0.6 F1 score in Dutch using only one sample from it.
Anthology ID:
2020.acl-main.239
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Editors:
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2692–2698
Language:
URL:
https://aclanthology.org/2020.acl-main.239
DOI:
10.18653/v1/2020.acl-main.239
Bibkey:
Cite (ACL):
Tal Perl, Sriram Chaudhury, and Raja Giryes. 2020. Low Resource Sequence Tagging using Sentence Reconstruction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2692–2698, Online. Association for Computational Linguistics.
Cite (Informal):
Low Resource Sequence Tagging using Sentence Reconstruction (Perl et al., ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.239.pdf
Video:
 http://slideslive.com/38928819