Analysing Representations of Memory Impairment in a Clinical Notes Classification Model

Mark Ormerod, Jesús Martínez-del-Rincón, Neil Robertson, Bernadette McGuinness, Barry Devereux


Abstract
Despite recent advances in the application of deep neural networks to various kinds of medical data, extracting information from unstructured textual sources remains a challenging task. The challenges of training and interpreting document classification models are amplified when dealing with small and highly technical datasets, as are common in the clinical domain. Using a dataset of de-identified clinical letters gathered at a memory clinic, we construct several recurrent neural network models for letter classification, and evaluate them on their ability to build meaningful representations of the documents and predict patients’ diagnoses. Additionally, we probe sentence embedding models in order to build a human-interpretable representation of the neural network’s features, using a simple and intuitive technique based on perturbative approaches to sentence importance. In addition to showing which sentences in a document are most informative about the patient’s condition, this method reveals the types of sentences that lead the model to make incorrect diagnoses. Furthermore, we identify clusters of sentences in the embedding space that correlate strongly with importance scores for each clinical diagnosis class.
Anthology ID:
W19-5005
Volume:
Proceedings of the 18th BioNLP Workshop and Shared Task
Month:
August
Year:
2019
Address:
Florence, Italy
Editors:
Dina Demner-Fushman, Kevin Bretonnel Cohen, Sophia Ananiadou, Junichi Tsujii
Venue:
BioNLP
SIG:
SIGBIOMED
Publisher:
Association for Computational Linguistics
Note:
Pages:
48–57
Language:
URL:
https://aclanthology.org/W19-5005
DOI:
10.18653/v1/W19-5005
Bibkey:
Cite (ACL):
Mark Ormerod, Jesús Martínez-del-Rincón, Neil Robertson, Bernadette McGuinness, and Barry Devereux. 2019. Analysing Representations of Memory Impairment in a Clinical Notes Classification Model. In Proceedings of the 18th BioNLP Workshop and Shared Task, pages 48–57, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Analysing Representations of Memory Impairment in a Clinical Notes Classification Model (Ormerod et al., BioNLP 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-5005.pdf