A BERT-based One-Pass Multi-Task Model for Clinical Temporal Relation Extraction

Chen Lin, Timothy Miller, Dmitriy Dligach, Farig Sadeque, Steven Bethard, Guergana Savova


Abstract
Recently BERT has achieved a state-of-the-art performance in temporal relation extraction from clinical Electronic Medical Records text. However, the current approach is inefficient as it requires multiple passes through each input sequence. We extend a recently-proposed one-pass model for relation classification to a one-pass model for relation extraction. We augment this framework by introducing global embeddings to help with long-distance relation inference, and by multi-task learning to increase model performance and generalizability. Our proposed model produces results on par with the state-of-the-art in temporal relation extraction on the THYME corpus and is much “greener” in computational cost.
Anthology ID:
2020.bionlp-1.7
Volume:
Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing
Month:
July
Year:
2020
Address:
Online
Editors:
Dina Demner-Fushman, Kevin Bretonnel Cohen, Sophia Ananiadou, Junichi Tsujii
Venue:
BioNLP
SIG:
SIGBIOMED
Publisher:
Association for Computational Linguistics
Note:
Pages:
70–75
Language:
URL:
https://aclanthology.org/2020.bionlp-1.7
DOI:
10.18653/v1/2020.bionlp-1.7
Bibkey:
Cite (ACL):
Chen Lin, Timothy Miller, Dmitriy Dligach, Farig Sadeque, Steven Bethard, and Guergana Savova. 2020. A BERT-based One-Pass Multi-Task Model for Clinical Temporal Relation Extraction. In Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing, pages 70–75, Online. Association for Computational Linguistics.
Cite (Informal):
A BERT-based One-Pass Multi-Task Model for Clinical Temporal Relation Extraction (Lin et al., BioNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.bionlp-1.7.pdf