Connotation Frames of Power and Agency in Modern Films

The framing of an action influences how we perceive its actor. We introduce connotation frames of power and agency, a pragmatic formalism organized using frame semantic representations, to model how different levels of power and agency are implicitly projected on actors through their actions. We use the new power and agency frames to measure the subtle, but prevalent, gender bias in the portrayal of modern film characters and provide insights that deviate from the well-known Bechdel test. Our contributions include an extended lexicon of connotation frames along with a web interface that provides a comprehensive analysis through the lens of connotation frames.


Introduction
A viewer's impression of a movie character is influenced by how they are written and portrayed, which can in turn influence how people form stereotypes on gender norms (Behm-Morawitz and Mastro, 2008). A character's actions can be projected with varying levels of power and agency, depending on the specific verbs used. For instance, somebody who "accepts" things is implied to be a passive decision-maker (or of lower agency) than somebody who "assesses" things. While not explicitly stated, these connotative meanings projected by different verbs can influence the assumptions the audience makes about the people being described. These assumptions can have negative consequences if they reinforce negative stereotypes (Walton and Spencer, 2009).
To formalize this implicit information about people projected by actions, we introduce power and agency connotation frames, two new types of Bolded words are the predicates, solid underlined phrases are the agent of the verb, and dash underlined words are the theme. The full example with additional nuanced discussion is available in Figure 6 in the appendix.
predicate-specific connotative relationships as an extension to Rashkin et al. (2016)'s connotation frame lexicon. For instance, in Figure 1, the verb "beckoning" implies that its theme (Irene) has less power than its agent (the man). In the third line, Irene displays strong agency when she "slices" in self-defense. In contrast, when the man "obeys", the man has low implied agency.
Using the new connotation lexicon, we present a quantitative study to reveal the subtle, but prevalent gender 1 bias in modern films. Going beyond the surface level analysis such as screen time or number of female characters (Google, 2017), our study aims for a more focused and precise analysis of power differentials between fictional men and women.
In summary, our major contributions include the creation and release of a lexicon with two new connotative dimensions: power and agency and an AGENT THEME power(AG < TH) VERB implore He implored the tribunal to show mercy.
The princess waited for her prince.
AGENT THEME agency(AG) = -VERB wait Figure 2: The formal notation of the connotation frames of power and agency. The first example shows the relative power differential implied by the verb "implored", i.e., the agent ("he") is in a position of less power than the theme ("the tribunal"). In contrast, "He demanded the tribunal show mercy" implies that the agent has authority over the theme. The second example shows the low level of agency implied by the verb "waited".
interactive demo website of our findings (see Figure 5 in the appendix for a screenshot). 2 Furthermore, as will be seen in Section 4.1, connotation frames offer new insights that complement and deviate from the well-known Bechdel test (Bechdel, 1986). In particular, we find that high-agency women through the lens of connotation frames are rare in modern films. It is, in part, because some movies (e.g., Snow White) accidentally pass the Bechdel test and also because even movies with strong female characters are not entirely free from the deeply ingrained biases in social norms.

Connotation Frames of Power and Agency
We create two new connotation relations, power and agency (examples in Figure 3), as an expansion of the existing connotation frame lexicons. 3 Three AMT crowdworkers annotated the verbs with placeholders to avoid gender bias in the context (e.g., X rescued Y; an example task is shown in the appendix in Figure 7). We define the annotated constructs as follows: Power Differentials Many verbs imply the authority levels of the agent and theme relative to power(AG<TH) power(AG>TH) agency(AG)=− agency(AG)=+ Figure 3: Sample verbs in the connotation frames with high annotator agreement. Size is indicative of verb frequency in our corpus (bigger = more frequent), color differences are only for legibility. one another. For example, if the agent "dominates" the theme (denoted as power(AG>TH)), then the agent is implied to have a level of control over the theme. Alternatively, if the agent "honors" the theme (denoted as power(AG<TH)), the writer implies that the theme is more important or authoritative. We used AMT crowdsourcing to label 1700 transitive verbs for power differentials.
Agency The agency attributed to the agent of the verb denotes whether the action being described implies that the agent is powerful, decisive, and capable of pushing forward their own storyline. For example, a person who is described as "experiencing" things does not seem as active and decisive as someone who is described as "determining" things. AMT workers labeled 2000 transitive verbs for implying high/moderate/low agency (inter-annotator agreement of 0.27). We denote high agency as agency(AG)=+, and low agency as agency(AG)=−.
Pairwise agreements on a hard constraint are 56% and 51% for power and agency, respectively. Despite this, agreements reach 96% and 94% when moderate labels are counted as agreeing with either high or low labels, showing that annotators rarely strongly disagree with one another. Some contributing factors in the lower KA scores include the subtlety of choosing between neutral  and positive/negative as well as the skews in the distributions of labels (i.e. more positive than negative labels, see Figure 4). Note that a similar difference between KA scores and soft percent agreement was found in our previous connotation frame work (Rashkin et al., 2016).

Bias in Movie Scripts
We use 772 movie scripts from (Gorinski and Lapata, 2015) as a test bed to validate our new connotation frames. Scripts have distinct structure, which allows us to easily parse narrations, dialogues and character names. We automatically extract 21K male/female characters, using a name-gender list 4 along with gender specific lexicons (e.g., "actor"/"actresses", "duke"/"duchess") to automatically assign gender based on their first three narrations. To identify verbs with characters as their agent, we dependency parse the narratives using the SpaCy 5 parser. Power and agency label distributions in our corpus are consistent with the annotation distribution (Figure 4), and there is little variance across movies (see Figure 8 in the appendix).
In our dataset, there are nearly twice as many men as there are women (34.6% women), in line with previous findings by Smith et al. (2015) and Radford and Gallé (2015). Women are also less present on screen and speak less in movies (Google, 2017). We control for that disparity in all subsequent analyses by including the number of words for each character (standardized) as a covariate. Findings in all the following sections hold when controlling for movie genre (as retrieved from IMDB.com), as well as when controlling for effects from individual movies.  Table 1: Power and agency connotation frames for male and female narratives, controlled for length of narrative text. β represents the change in logodds of a character being male/female were the corresponding frame to change by one unit. Significant results ( * * : p<.001) are in bold. "Male" was coded as 0, "Female" as 1.

Bias in Narratives
Narratives describe what characters are doing. We investigate how they vary in terms of power and agency, using our connotation frames. We measure how each standardized frame metric is associated with the gender of the character through a logistic regression, controlling for the total number of words that the character said, and correcting for multiple comparisons using Holm's correction (Holm, 1979).
Listed in Table 1, our results show that male characters are portrayed with higher level of agency compared to women. Men are also portrayed to have more authority than women as they are more often the agent of powerful verbs.
This suggests that screenwriters tend to have female characters contribute more to the aesthetic of the movie through low-agency verbs, rather than the plot, which is reminiscent of existing gender bias tests for movies (Yehl, 2013).

Bias in Character Expression
To further our validation of the new connotative dimensions, we look at how characters express themselves in movies. Using our connotation frames and LIWC (Tausczik and Pennebaker, 2016), we compile metrics for every character's dialogue. As in subsection 3.1, metrics were standardized for better β interpretability. LIWC results that are not discussed below can be found in the appendix (Tables 4 and 5).
From Table 2, it seems male characters display more power and authority through their speech than their female counterparts do. Specifically, women are written to use more hedges (# Hedges) whereas men are written to use more imperative sentences (# Imperative Sent.), a finding that re-  flects real-world dialogues (Prabhakaran et al., 2014). The usage of imperatives tends to convey power and dominance according to the findings of Bramsen et al. (2011). Along with the fact that female characters tend to agree (assent) more than male characters, this corroborates the finding in subsection 3.1 that male characters are generally given more power and agency. Furthermore, male characters use inhibitory language more (inhib), which contains words pertaining to blocking or allowing, suggesting that these characters are in positions of power. Further evidence of power imbalances is found through function words. Women tend to use I and you pronouns more, whereas men use we and they pronouns more, echoing real life (Schwartz et al., 2013). Kacewicz et al. (2014) found an association between using "I" pronouns and being lower status, and conversely between "we" pronouns and being higher status. This corroborates the theory that women in movies are generally portrayed with a lower status than men.
Men in movies tend to mention more physical actions (space category) whereas women tend talk about what "could" be but isn't (discrep; e.g.,"should", "would"). This evokes more commanding connotations for male characters and subordinate connotations for female ones, reinforcing gender stereotypes.
ing bias in how male and female characters display different levels of power and agency in their dialogue.

Power, Agency and the Bechdel test
A movie passes the Bechdel test (Bechdel, 1986) if it (1) has two (named) female characters, (2) who talk to each other, (3) about something other than a man. While this is a low bar, a surprising number of movies fail at least one of the three criteria. In particular, as many as 42% of the movies in our dataset fail the test according to an online database of the Bechdel scores. 6

Beyond the Bechdel Test
We provide comparative insights between the analysis based on connotation frames and the Bechdel test. First, we aggregate our connotation frames, both on dialogue and narration, into movie-level averages per gender. Then, we add features capturing presence of female/male characters (e.g., # F/M words, # F/M characters). Table 3 shows the correlation between passing the Bechdel test and our movie-level connotation frame features using a multivariate logistic regression.
As expected, a movie with more female speaking time is more likely to pass the Bechdel test since it mostly captures female representation. We also find that female characters using agentempowering verbs, which tend to be more assertive, slightly increases the odds of passing the Bechdel test. Female speakers who use empowering verbs, regardless of the verb's agent, tend to go against the gender-norms of hedging and being less assertive (as we showed in subsection 3.2).
Unexpectedly, movies where women talk with high agency are much less likely to pass the Bechdel test. Perhaps these movies typically only show scenes of women interacting in a male-dominated setting. Similarly, the use of more agent-empowering verbs in female narratives decrease the odds of passing the Bechdel test. Chances of two powerful women talking to each other might be lower because movies are less likely to have a lot of powerful women. 7 Power and Agency of Princesses We further provide a qualitative analysis using Wikipedia plot summaries for movies that are not in our script dataset. Bechdel-passing movies with female protagonists, such as Frozen (2013) or Cinderella (1950), still perpetuate negative female stereotypes. In Frozen, Elsa is portrayed as the only high agency, high power woman, as seen below. 8 Anna and Cinderella, despite also being protagonists, display significantly less power and agency. The Bechdel test is limited, either by being too inclusive of movies who portray women in nonauthoritative, passive positions or by excluding movies that have strong women with agency, who just happen not to talk to each other about something besides men. Our extensions to the connotation frame lexicon provide finer grained information about how women are portrayed through their expression and their actions, which can act complementary to measures of their presence.

Related work
There is much prior research focus on bias in social media (Garcia et al., 2014;Prabhakaran et al., 2014;Ratkiewicz et al., 2011;Yano et al., 2010;Srivastava and Sahami, 2009), complementing our investigation on movies. Fast et al. (2016) examine the stereotypes present in fan-fiction us-ing a lexicon-based strategy that focus on commonly gender-biased attributes (e.g., emotional for women) rather than the overall power dynamics of the story. In a similar vein, Ramakrishna et al. (2015) learn word-level "gender ladenness" features by looking at the neighbors of 925 manually annotated words.
There exist various sets of high-level criteria to assess gender bias of character portrayal in fiction (Yehl, 2013;Romano, 2013;Powers, 2016). Agarwal et al. (2015), in particular, automate the Bechdel test using social network features, finding that women are less central to the plot in movies that fail it. We compare our linguistic analysis of power and agency with the Bechdel test, demonstrating the need for more fine-grained analysis of how gender is depicted in movies.
Close in spirit to our investigation, Schofield and Mehr (2016) train a number of classifiers over movie scripts for determining the gender of individual (and pairs) of speakers as well as the expected length of their relationships. In contrast, we focus on understanding how the gender of a given character implicitly relates to features that track their control over their own path (agency) and the world around them (power).

Conclusion
We created and released new connotation frames of power and agency, allowing for more nuanced writing analysis than previously possible. We validate our new frames through a case study on movie scripts. Specifically, we analyze differences in power and agency for male and female characters, and compare these dimensions to the Bechdel test. Our connotation frames confirm evidence of imbalances in gender portrayal in movies.