Situated Data, Situated Systems: A Methodology to Engage with Power Relations in Natural Language Processing Research

Lucy Havens, Melissa Terras, Benjamin Bach, Beatrice Alex


Abstract
We propose a bias-aware methodology to engage with power relations in natural language processing (NLP) research. NLP research rarely engages with bias in social contexts, limiting its ability to mitigate bias. While researchers have recommended actions, technical methods, and documentation practices, no methodology exists to integrate critical reflections on bias with technical NLP methods. In this paper, after an extensive and interdisciplinary literature review, we contribute a bias-aware methodology for NLP research. We also contribute a definition of biased text, a discussion of the implications of biased NLP systems, and a case study demonstrating how we are executing the bias-aware methodology in research on archival metadata descriptions.
Anthology ID:
2020.gebnlp-1.10
Volume:
Proceedings of the Second Workshop on Gender Bias in Natural Language Processing
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Marta R. Costa-jussà, Christian Hardmeier, Will Radford, Kellie Webster
Venue:
GeBNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
107–124
Language:
URL:
https://aclanthology.org/2020.gebnlp-1.10
DOI:
Bibkey:
Cite (ACL):
Lucy Havens, Melissa Terras, Benjamin Bach, and Beatrice Alex. 2020. Situated Data, Situated Systems: A Methodology to Engage with Power Relations in Natural Language Processing Research. In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing, pages 107–124, Barcelona, Spain (Online). Association for Computational Linguistics.
Cite (Informal):
Situated Data, Situated Systems: A Methodology to Engage with Power Relations in Natural Language Processing Research (Havens et al., GeBNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.gebnlp-1.10.pdf
Data
GAP Coreference Dataset