Finding Microaggressions in the Wild: A Case for Locating Elusive Phenomena in Social Media Posts

Microaggressions are subtle, often veiled, manifestations of human biases. These uncivil interactions can have a powerful negative impact on people by marginalizing minorities and disadvantaged groups. The linguistic subtlety of microaggressions in communication has made it difficult for researchers to analyze their exact nature, and to quantify and extract microaggressions automatically. Specifically, the lack of a corpus of real-world microaggressions and objective criteria for annotating them have prevented researchers from addressing these problems at scale. In this paper, we devise a general but nuanced, computationally operationalizable typology of microaggressions based on a small subset of data that we have. We then create two datasets: one with examples of diverse types of microaggressions recollected by their targets, and another with gender-based microaggressions in public conversations on social media. We introduce a new, more objective, criterion for annotation and an active-learning based procedure that increases the likelihood of surfacing posts containing microaggressions. Finally, we analyze the trends that emerge from these new datasets.


Introduction
Toxicity and offensiveness are not always expressed with toxic language. While a substantial community effort has rightfully focused on identifying, preventing, and mitigating overtly toxic, profane, and hateful language (Schmidt and Wiegand, 2017), offensiveness spans a far larger spectrum that includes comments with more implicit and subtle signals that are no less offensive (Jurgens et al., 2019). One significant class of subtle-but-offensive comments includes microaggressions (Sue et al., 2007, MAS), defined in Merriam-Webster as "a comment or action that Figure 1: Existing state-of-the-art tools for hate speech detection and sentiment analysis cannot identify the veiled offensiveness of microaggressions (MAS) like the one in this real comment, because in many cases, the framing of a MA includes stylistic markers of positive language. This motivates the need for a corpus and new tools for detecting MAS. subtly or often unconsciously expresses a prejudiced attitude toward a member of a marginalized group such as a racial minority." Though subtle, MAS have been shown to have lasting harmful impacts on their targets. Qualitative interviews suggest that the subtlety of MAS may cause even greater levels of situational stress than overt aggression (Sue, 2010;Nadal et al., 2014).
Despite a public effort to recognize and reduce-if not eliminate-their occurrence (Kim, 2013;Neff, 2015), there has been no computational work to detect and analyze MAS at scale. Instead, much of the recent work has focused on explicitly toxic language (e.g., Waseem et al., 2017), with surveys of the area also overlooking this important and challenging task of recognizing this subtle toxicity (van Aken et al., 2018;Salminen et al., 2018;Fortuna and Nunes, 2018). Indeed, as Figure 1 suggests, current popular tools for toxic language detection do not recognize the toxicity of MAS and further, sentiment tools can label these comments as being positive. As a result, applications using such techniques to promote inoffensive content may potentially promote MAS. Such biased online content is then used as training data in NLP tools-dialogue systems, question answering, and others-thereby perpetuating and amplifying biases (Zhao et al., 2017).
Computational modeling of MAS is challenging for many reasons. MAS are subjective, context-sensitive, and expressed subtly in language (as shown in Figure 1); there are no welldefined annotation guidelines, no corpus of MAS for training and evaluating a model, and prior computational approaches to detecting overtly offensive speech are likely not suitable for classifying MAS. Moreover, they are diluted in the ocean of social media content, and it would be infeasible to create a corpus of diverse types of MAS just by crowdsourcing annotations of randomly sampled social media posts. This paper makes a first step towards closing this gap, providing (i) a typology of MAS, (ii) a corpus, and (iii) a first computational model to surfacing MAS.
Here, we introduce the first computational analysis of MAS with three main contributions. First, we create a seed corpus through collecting selfreported accounts of MAS online; we analyze this data using prior research in social science on MAS in order to derive an operationalizable classification scheme ( §2.2), which we show can be used as training data to classify types of MAS ( §2.4). Second, we develop a new active-learning based crowdsourcing technique to identify candidate microaggression messages for annotation ( §3). In a large-scale study of gender-based MAS on Reddit, we show that our method is effective at surfacing MAS. Our novel approach surfaces MAS not found through simply looking for offensive posts ( §3.3). Finally, we compare the types of MAS from self-reported accounts and those observed in social media comments, and we draw new conclusions about how MAS behave and may be identified in online settings ( §4). 1

A Corpus of Microaggression Classes
Previous examinations of MAS have typically focused on small scale studies with interviews, surveys, or discussions with clinicians (e.g., Shelton and Delgado-Romero, 2013;Campbell and Manning, 2014;Forrest-Bank and Jenson, 2015), aiming at understanding the causes and effects of MAS on their targets . A cru-cial gap in existing MAS research is the lack of systematic analysis of reports (Lilienfeld, 2017), which has contributed to the difficulty of understanding what constitutes a microagression. Here, we aim to address this gap by introducing a new dataset of MAS and then use this data and prior qualitative research to construct a new typology of MAS. This new typology enables us to ground the notion of what is a microagression, which also addresses criticism that the concept is too slippery. The typology is critical for downstream analyses of determining who is affected and how they are being targeted. Further, we use the typology as a practical heuristic for identifying which types of MAS are the highest impact for computational approaches. As a test, we develop a classifier using the typology and show that different types of MAS are readily distinguished.

SELFMA: microaggressions.com Dataset
The Tumblr website www.microaggressions.com curates a collection of self-reported accounts of MAS. They were collected by asking people to fill out an online form with three questions: the story of the microaggression, the context, and how the microaggression made them feel. As such, the microaggression accounts in this corpus come from individuals who self-report the incident, and who, presumably, are at least familiar with the term "microaggressions." From this website, we collected 2,934 accounts of MAS (henceforth, posts). Most posts are manifestations of bias, targeting social groups frequently discriminated against, including gender (1,314 posts), race (1,278 posts), sexuality (461 posts), and religion (88 posts), among others. Some are formatted to show the conversation structure of the recollection of the incidents, while some others are shown as plain narratives. We refer to this dataset as SELFMA.

A New Typology of Microaggressions
As originally specified in Sue et al. (2007), MAS were categorized into 13 themes across specific axes of discrimination such as racism and sexism. These themes were developed qualitatively from transcripts with therapists. However, while they were comprehensive in the clinical domain, here, we aim to create a more general framework for classifying MAS in online interactions. Specifically, our new typology builds upon the work of Sue et al. (2007) with three goals in mind: the ty-pology should be (1) generalizable across different axes of discrimination, (2) sufficiently represented in our corpus, and (3) comprehensive over all distinct microaggression types in the corpus.
Four key themes were identified in our analysis of the MAS data: Attributive, Institutionalized, Teaming, and Othering (see Table 1). We discuss each of these next. 2 The Attributive theme covers instances where a microaggression attributes a stereotype to an individual based on their identity. These stereotypes may have inherently negative connotations ("lazy"), but may also be neutral ("liking pink") or positive ("strong"), which complements recent work on benevolent sexism (Jha and Mamidi, 2017). The Institutionalized theme reflects larger institutionalized biases, such as in employment or law enforcement. The Teaming theme is derived from the term forced teaming, coined by de Becker (1997) to describe a strategy of abuse where the abusers frames themselves as being on the same team as the victim. The Othering theme covers MAS which revolve around framing the target in relation to some "othered" group. This theme often co-exists with "Attribution of Stereotype," but is distinct in that its focus is on redefining the target's sense of identity.

Annotation Results
Three annotators familiar with the theoretical background and prior research on microagressions performed an open coding procedure after examining the SELFMA data to codify the typology and determine annotation guidelines. All three labeled 200 instances of the dataset to estimate agreement.
Despite the difficulty of the task, annotators had moderate agreement, as shown in Table 2. While lower than what is considered high agreement (Artstein and Poesio, 2008), given the potentiallysubjective nature of MAS and criticism for lack of objectivity (Lilienfeld, 2017), we view this result as a strongly positive sign that reliable annotation is possible despite the challenge. This moderate agreement is on-par with other difficult annotation tasks, such as those for connotation frames (Rashkin et al., 2016), which had 0.52 percentage agreement for rating the polarity of the sentence towards a target, dimensions of social relationships (Rashid and Blanco, 2017), which had κ val- The final dataset was determined by first retaining all posts where at least two annotators agreed (183 posts). And where there were no agreement (17 posts), the annotators determine the labels through a follow-up adjudication process. 3 After this process, another 1,100 posts were singly annotated, distributed across the three annotators, for a total of 1,300 posts. Relative percentages of each theme and examples comments are shown in Table 3. We also show the distribution of different axes of discrimination among the subthemes in Figure 2.

Classifying Types of Microaggressions
As a further examination of the typology, we perform a study where a model is trained to classify a comment into one of the four themes. Linear SVM classifiers were trained in a one-versus-rest setup using (i) word unigrams and bigrams on the tokenized version of the posts, 4 (ii) categories in the 2015 LIWC lexicon (Pennebaker et al., 2015), (iii) sentiment lexicons, and (iv) the topic distribution for a 40-topic LDA model (Blei et al., 2003). They were trained on 1,000 posts and evaluated using a held-out test set of 300 posts.
Classifier performance, measured as F1, was on Criminal Status Link a persons identity to criminality, danger, or illness.
"You look like a terrorist with that beard."

Second-Class Citizen
Marginalized individuals belong to low-status positions in society.
"Oh, you work at an office? I bet you're a secretary."

Forced Teaming
Myth of Meritocracy Differences in treatment are due to ones merit.
"They just cast actors who are best for it. Why does it matter if they're all white?" Denial of Lived Experience Minimize the experiences of a marginalized individuals.
"It was just a joke! You're too sensitive." Ownership Anyone can have some claim to a marginalized groups experiences.
"Why is it offensive for a white person to wear a bindi? It's just jewelry."

Monolith
All members of a marginalized group are identical.
"My gay friend doesn't have a problem with this show. I don't get why you're mad."

Erasure
Anyone can claim that an individual does not belong to that group.
"Your mom is white, so it's not like you're really black, though."

Locating Microaggressions in the Wild
The vast majority of microaggression data comes from self-reported recollections, including the MAS corpus that we collected. Self-reported data is crucial for studying the phenomenon and its effects, but is arguably not suitable for training classifiers, due to two critical limitations: (i) biases in how the offending comment was originally made versus how it was recalled, and (ii) biases in which kinds of microaggression comments are recalled. These make future computational attempts at detecting MAS more difficult by skewing a model away from linguistic signals used in real-worldrather than recalled-behavior. A natural solution is to collect and annotate examples of MAS, scaling this process by using crowdsourcing. There are, however, two critical obstacles to using the straightforward approach of asking crowdworkers to identify random social media posts as MAS: (1) untrained crowdworkers are unlikely to reliably identify these posts due to the potential subjectivity of MAS, and (2) a naive random sampling of social media posts is unlikely to uncover MAS due to their sparsity, and since MAS are veiled manifestations of biases, it is also not clear how to use lexicons as a sieve for a stratified selection of posts to annotate.
In this second part of the paper, we devise a novel methodology to address the two challenges to curate microaggression corpora from social media posts with a two-pronged approach: operationalizing and surfacing MAS. .07 Summer's coming! Gotta fit into that tight swimsuit.

Operationalizing Microaggressions
.23 Teaming 23 Most workplace harassment is an example of women being oversensitive to advances of men .27 So were you good enough to get in or did you get in through a women's quota?
.19 Othering 17 I don't want another woman boss. Already have one and she has me running around all the time .17 Table 3: The final microaggression typology after analyzing 1,300 self-reported microaggressions in SELFMA dataset. As shown by the percentages of instances covered each category, this typology effectively summarizes the broad themes (note that one instance may be labeled with multiple categories). For each category, the overall toxicity score (measured in [0,1] using the Google Perspective API) is low, indicating that currently none are reliably recognized as being offensive.
on gender-based MAS, the dominant group (male annotators) and the marginalized groups (women and non-binary annotators) 5 are easily elicited from the annotators through self-reporting. Since training crowdworkers to directly identify MAS is infeasible, we look for a more objective proxy that can identify, or at least help surface, posts containing MAS. A key property of MAS is that they often result from different lived experience between the individuals in the dominant group and those in the marginalized groups. Thus, our hypothesis is: there will be a discrepancy of perceived offensiveness between the dominant group and the marginalized groups for MAS. If true, we can then ask annotators to annotate the perceived offensiveness of a statement, and use this as a proxy to surface MAS from social media posts. Based on Figure 2, we pick gender as the discrimination axis for this work, since it covers a large number of our posts.

Surfacing Microaggressions
To alleviate the sparsity issue, we treat the annotation process as an active learning procedure, where batches of posts to be annotated are provided by a classifier that is trained to predict the discrepancy of perceived offensiveness of a post, which we have claimed above to be a proxy to MAS. This classifier can then be iteratively updated using the results of all previous batches. Assuming the classifier is good enough in predicting the proxy, the classifier will help surface the most likely candidates of posts that contain MAS. In what follows, we describe the annotation process in detail and the resulting crowdsourced dataset.

Crowdsourcing Microaggressions
The full data collection process is set-up using a multi-round annotation scheme, summarized as 5 Even this interpretation of gender is limited, as women and nonbinary individuals experience gendered discrimination in different ways, and transgender men also have a marginalized gender identity. follows. We first select a random sample of Reddit posts from relevant subreddits. These posts are then given to crowdworkers to be annotated for perceived offensiveness. This first round of annotations is then used to train a classifier for a MArelavent task. The classifier's ratings are used to pick the next batch of Reddit posts to be annotated, in order to surface more instances than random sampling.
Task Crowdworkers are shown comments in the context of an internet forum discussion. Crowdworkers are asked to imagine they have written the displayed post, and then after a button click, we reveal how someone (a replier) has commented to that post. They are then asked rate if they found the response offensive and if so, to what degree was it offensive on a seven-point Likert scale. This scale of offensiveness allows us to capture cases where a microaggression is perceived as offensive by both annotator groups (genders), but to different degrees. To test our hypothesis and quantify how certain posts could be perceived differently by annotators of different genders, each annotation task includes a demographic question. 6 Data To focus the data on interactions where gender may be a salient variable, Reddit data was drawn from RtGender (Voigt et al., 2018), which has a post-and-reply interactions where the gender of each author has been inferred with high confidence. After preliminary analyses, the initial data was randomly selected from 28 subreddits based on their focus around gender issues (e.g,. Relationships, AskWomen). 7 For the first round of annotation, we used these four subsets of posts to be annotated for of-6 Following best practices in gender elicitation (Jaroszewski et al., 2018), we include a third option of "nonbinary, genderqueer, or otherwise", which has a freeform text box. We opted not to ask about cis or trans status in the demographic survey. 7 See Supplementary Material §1 for the complete list.

(SELFMA) Posts from our MAS corpus ( §2) that contain gender-based MAS (n = 59); 2. (NON-OFFENSIVE) Posts manually picked
from the Reddit dataset that should quite clearly be perceived as not offensive by everyone (n = 36); 3. (OFFENSIVE) Posts manually picked from the Reddit dataset, filtering by profane lexicons targeting women, and a corpus of Twitter hate speech ) that should quite clearly be perceived as offensive (n = 21); 4. (RANDOM) Posts randomly selected from the 28 subreddits (n = 1007).
The first three sets of posts serve as controls both to verify the soundness of our hypothesis (SELFMA) and to filter out untrusted crowdworkers (NON-OFFENSIVE and OFFENSIVE).
Adaptive Data Selection As described previously, after the first round of annotations, we use a classifier trained to predict discrepancy in perceived offensiveness to pick the next batch of posts for the second round of annotation. The classifier was given, as positive examples, posts with ≥ 0.25 discrepancy 8 between the averaged perceived offensiveness of the dominant group and the marginalized groups. Other posts are considered negative examples. The feature sets that we used are motivated by prior work on gender bias and power dynamics. The following seven feature categories are used: (1) unigram and bigram features to capture lexical patterns, (2) two categories of formal and informal words derived from the formality corpus of Pavlick and Tetreault (2016), (3) the gendered occupations lexicon of Bolukbasi et al. (2016), grouped across definitional and stereotypical gender (male, female, neutral), (4) the gender stereotype lexicon of Fast et al. (2016), (5) the gender lexicon for social media from (Sap et al., 2014), (6) a manually-compiled corpus of gendered words, extended from seed list from https://www.hrc.org/resources/glossary-of-terms and (7) a manually-compiled sentiment lexicon, inspired by LIWC. To facilitate reproducibility, all lexicons will be available in the software release. 8 The threshold was chosen empirically to ensure enough number of positive examples.

Experimental Setup
We designed the annotation task with two purposes in mind: (1) We want to test our hypothesis that there is discrepancy in how annotators of different genders perceive MAS. By comparing the distribution of discrepancy of perceived offensiveness in SELFMA, which consists solely of MAS, and of that in OFFENSIVE, which consists solely of offensive posts, we can see whether the discrepancy test actually is an operationalizable criteria to distinguish MAS from offensive posts in general.
(2) We want to test our hypothesis that the discrepancy can be used to surface MAS, better than the alternatives. For the purpose of this experiment, we also trained a classifier to predict the offensiveness level of a post, and use that to provide a batch of posts to be annotated as well. By comparing the actual number of MAS by these two classifiers, we can test whether discrepancy can be used to surface more MAS compared to offensiveness in general.
Platform Setup Data was crowdsourced using Figure Eight with only Level 3 workers, a small "group of most experienced, highest accuracy contributors". Workers were shown 10 posts and replies per page, one of which was a test question. Workers whose ratings were substantially outside of the rating range of our ground truth annotations in more than 60% of cases were removed. Workers were paid $0.40 per page. To control for language and cultural norms, workers were restricted to being from the US, UK, Canada, and Australia.
Much like Mechanical Turk (Hara et al., 2019), Figure Eight has substantial representation of female-identifying individuals (Posch et al., 2018), with slightly more than 60% of users in the US identifying as female on both platforms (a higher rate than other countries). Studying genderbased MAS on these platforms means we are more likely to get a gender-balanced sample than compared with other social categories subject to microaggressions such as religion, age, or sexual orientation, which would likely require targeted recruiting of respondents or a massive collection of responses in order to achieve a balanced sample.

Results
For the first experiment, Figure 3 shows the distribution of perceived offensiveness discrepancy between the dominant group (men) and the marginalized group (women and non-binary) an- notator genders show that MAS in the SELFMA were perceived as substantially more offensive by annotators identifying as women (shown through a positive rating discrepancy on y-axis) than offensive comments in OFFENSIVE. Difference is significant under t-test (p < 0.001).
notators, which supports our hypothesis that MAS in SELFMA are perceived as substantially more offensive by annotators identifying as women and non-binary. For the second experiment, we asked crowdworkers to annotate 2,000 posts with the highest scores as predicted by the classifier, and then we took the top 6% of Reddit posts with the highest discrepancies scores and manually annotated them for MAS. We predicted that these posts contain a higher percentage of MA posts than those selected randomly, and this hypothesis is supported: Out of these aggregate 120 posts, 13 are MAS (10.8%), 9 compared to a manually annotated 200 random posts from the RANDOM batch, where only 6 of them (3%) were MAS. Examples are given in Table 4.
A potential next hypothesis suggested by our original hypothesis is that the classifiers trained on annotator discrepancy (with a higher level of offense for female and non-binary annotators) would be more successful in locating MAS than the classifiers trained on pure offensiveness. However, we did find equal performance or slight gains in microaggression location when using the offensiveness classifier. Based on analysis of the SELFMA data, we believe the cause may be that genderbased MAS were not always annotated as being more offensive to female annotators-many had strong discrepancies with male annotators being more offended. As our annotator discrepancy classifier was trained on samples more offensive to women, these types of MAS would be filtered out and decrease the model's overall effectiveness at surfacing MAS.

Reported vs Observed Themes
How do self-reported and passively-observed MAS differ in thematic frequencies? Here, we use our thematic classifier ( §2.4) to label all the microaggression data in Reddit. Analyzing the thematic differences reveals substantial differences between the two datasets. The Stereotype theme was by far the most common in our SELFMA data (53%), but it rarely occurs in our Reddit (3%). Instead, Objectification is the most common, as shown through the Institutionalized category (79%). Analyzing where the MAS occurred in Reddit, we observed that of the 19 instances of MAS identified by our annotators, many came from strongly male-focused subreddits (e.g., Men-sRights, Tinder, seduction, justneckbeardthings), where it would be expected that one male poster is talking to another. These online settings mirror offline highly-gendered settings where casually sexist language is known to happen, e.g., "locker room talk" (Katz, 1995). This is embodied in our Institutionalized category (specifically the subcategory "Objectification") having a much stronger presence in the Reddit MAS than the SELFMA ones in Table 5. While such language may not be perceived as offensive in the original dialog context, other readers who encounter it would likely feel offended and potentially marginalized.
Our work further points to two potential limitation of self-reported statements. First, the baseline rates of MAS may not be reliably estimated since the offensive comments are made elsewhere (outside of direct experience) and would only be discovered to through observational analysis. Second, the difference in baseline rates suggests that the relative differences in perceived offensiveness by theme may result in different reported quantities of microagressions; as not all comments and themes are equally offensive, people are more likely to recall the extremes of events (Tversky and Kahneman, 1973), potentially contributing to the difference in thematic rates in our two datasets.

Discussion and Limitations
As a first computational study of MAS, our work comes with some notable limitations that each provide clear avenues for future work. First, the current dataset contains just over 6.5K instances. While useful, this corpus does is likely not fully   representative of all types of MAS. However, our crowdsourcing approach does provide an effective way to surface MAS and our dual objective approach, which uses both annotator discrepancy and offensiveness, provides complementary views into what statements could be perceived as MAS. Additional iterations of this procedure are likely to improve microaggression recognition and substantially increase the seize of the corpus. We note that one option is to have workers generate examples, rather than rate (e.g., Xu et al., 2013;Su et al., 2016;Jiang et al., 2018); however, such a process raises ethical concerns of having crowdworkers generate toxic statements towards others. Second, our current focus is on gender-based MAS. This choice was motivated by the observation that gender-based MAS are the largest category in the SELFMA data and, given that prior studies have shown substantial gender disparity online, with women receiving more negative behaviors (Duggan, 2017), this choice has the potential for highest impact. Our work builds upon a growing body of literature focused on identifying and mitigating gender disparity through computational means (e.g., Magno and Weber, 2014;Garimella and Mihalcea, 2016;Li et al., 2018;. Further, our focus on gender also allows us to reliably recruit crowdworkers across the gender spectrum, whereas other social categories such as race or religion are more difficult to recruit in a balanced proportion through traditional mechanisms. However, despite this current focus, both the typology and annotation approach are designed for of perceived offensiveness between annotator gender. for the four sets of posts ( §3.1). Note that compared to clearly offensive data [OFFENSIVE] and clearly non-offensive data [NON-OFFENSIVE], the SELFMA data ( §2.1) identified as gendered MAS are annotated as somewhat offensive (indicated by the distribution around the middle of the y-axis), and having more discrepancy in perceived offensiveness between annotator genders (indicated by the higher degree of spread on the +x-axis). RANDOM data collected from Reddit sit all along the two axes of offensiveness and discrepancy. and readily amenable to other societal dimensions of MAS. Third, performance could still be improved even for the targeted case of gender-based MAS. For example, Figure 4 plots the distribution of posts according to the averaged perceived offensiveness (y-axis) and the discrepancy between the dominant group and the marginalized groups (x-axis). This shows that offensiveness and discrepancy in conjunction are better features than using just discrepancy in separating MAS from simply offensive and non-offensive posts. A possible future direction could be to train a multi-task classifier to predict both features in order to choose posts to annotate.
A final concern is that our approach may not truly be an "objective" replacement to current MAIdentification, as annotator bias is still present in this process. By choosing perceived offensiveness and discrepancies in perception as proxy measures for MAS, we hope to shift studies of MAS from a judgment call on an individual's intent to a description of how it affects its targets, which can be described as an objective measure of human behavior that can be validated through study. However, it is true that without that validation, it is difficult to prove the behavior of our crowdsourced annotators represents inherent, objective truths about how groups react to MAS.

Conclusion
Offensive language can take many forms: seemingly positive comments can in reality be dismissive of the experiences and validity of other persons. MAS, everyday comments that marginalize others on the basis of their identity, are a common form of this linguistically subtle offensive language. While prior work in NLP has largely focused on overtly offensive language, efforts to recognize this more insidious and equally impactful language has been stymied by the lack of data and methods for effectively annotating large corpora. In this work, we posit that covert toxicity can be detected through contextualized analysis of subtle linguistic cues: pronoun uses, adjectives, mentions of social groups, stylistic cues, etc. We thus view the task of detection of MAS as a yet unexplored, socially-relevant NLP task, somewhat similar to (but possibly harder than) sentiment analysis and detection of hate speech.
This work makes three key contributions. First, we introduce a new thematic classification of MAS and show that annotations can reach moderate agreement on a highly difficult task-countering claims that microaggressions are too subjective to be reliable-and also showing that computational methods can distinguish these themes. Second, we introduce a new active-learning crowdsourcing approach for surfacing potential MAS from social media using not only predictions of offensiveness but also predictions of rating discrepancy by gender within the annotators themselves. We show that both classifiers are effective at increasing rates of finding microaggression and that each finds different types. Finally, we show that selfreported and observed microaggression differ substantially in their types, suggesting computational mechanisms are needed to precisely estimate their true pervasiveness. Our work and corresponding data and models fill a key missing gap for expanding the range of offensive language detection to linguistically subtle comments.