Safeguarding against spurious AI-based predictions: The case of automated verbal memory assessment

Chelsea Chandler, Peter Foltz, Alex Cohen, Terje Holmlund, Brita Elvevåg


Abstract
A growing amount of psychiatric research incorporates machine learning and natural language processing methods, however findings have yet to be translated into actual clinical decision support systems. Many of these studies are based on relatively small datasets in homogeneous populations, which has the associated risk that the models may not perform adequately on new data in real clinical practice. The nature of serious mental illness is that it is hard to define, hard to capture, and requires frequent monitoring, which leads to imperfect data where attribute and class noise are common. With the goal of an effective AI-mediated clinical decision support system, there must be computational safeguards placed on the models used in order to avoid spurious predictions and thus allow humans to review data in the settings where models are unstable or bound not to generalize. This paper describes two approaches to implementing safeguards: (1) the determination of cases in which models are unstable by means of attribute and class based outlier detection and (2) finding the extent to which models show inductive bias. These safeguards are illustrated in the automated scoring of a story recall task via natural language processing methods. With the integration of human-in-the-loop machine learning in the clinical implementation process, incorporating safeguards such as these into the models will offer patients increased protection from spurious predictions.
Anthology ID:
2021.clpsych-1.20
Volume:
Proceedings of the Seventh Workshop on Computational Linguistics and Clinical Psychology: Improving Access
Month:
June
Year:
2021
Address:
Online
Editors:
Nazli Goharian, Philip Resnik, Andrew Yates, Molly Ireland, Kate Niederhoffer, Rebecca Resnik
Venue:
CLPsych
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
181–191
Language:
URL:
https://aclanthology.org/2021.clpsych-1.20
DOI:
10.18653/v1/2021.clpsych-1.20
Bibkey:
Cite (ACL):
Chelsea Chandler, Peter Foltz, Alex Cohen, Terje Holmlund, and Brita Elvevåg. 2021. Safeguarding against spurious AI-based predictions: The case of automated verbal memory assessment. In Proceedings of the Seventh Workshop on Computational Linguistics and Clinical Psychology: Improving Access, pages 181–191, Online. Association for Computational Linguistics.
Cite (Informal):
Safeguarding against spurious AI-based predictions: The case of automated verbal memory assessment (Chandler et al., CLPsych 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.clpsych-1.20.pdf