Reviving a psychometric measure: Classification and prediction of the Operant Motive Test

Dirk Johannßen, Chris Biemann, David Scheffer


Abstract
Implicit motives allow for the characterization of behavior, subsequent success and long-term development. While this has been operationalized in the operant motive test, research on motives has declined mainly due to labor-intensive and costly human annotation. In this study, we analyze over 200,000 labeled data items from 40,000 participants and utilize them for engineering features for training a logistic model tree machine learning model. It captures manually assigned motives well with an F-score of 80%, coming close to the pairwise annotator intraclass correlation coefficient of r = .85. In addition, we found a significant correlation of r = .2 between subsequent academic success and data automatically labeled with our model in an extrinsic evaluation.
Anthology ID:
W19-3014
Volume:
Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Kate Niederhoffer, Kristy Hollingshead, Philip Resnik, Rebecca Resnik, Kate Loveys
Venue:
CLPsych
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
121–125
Language:
URL:
https://aclanthology.org/W19-3014
DOI:
10.18653/v1/W19-3014
Bibkey:
Cite (ACL):
Dirk Johannßen, Chris Biemann, and David Scheffer. 2019. Reviving a psychometric measure: Classification and prediction of the Operant Motive Test. In Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology, pages 121–125, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Reviving a psychometric measure: Classification and prediction of the Operant Motive Test (Johannßen et al., CLPsych 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-3014.pdf