Expressions of Anxiety in Political Texts

Anxiety has a special importance in politics since the emotion is tied to decision-making under uncertainty, a feature of democratic institutions. Yet, measuring specific emotions like anxiety in political settings remains a challenging task. The present study tackles this problem by making use of natural language processing (NLP) tools to detect anxiety in a corpus of digitized parliamentary debates from Canada. I rely upon a vector space model to rank parliamentary speeches based on the semantic similarity of their words and syntax with a set of common expressions of anxiety. After assessing the performance of this approach with annotated corpora, I use it to test an implementation of state-trait anxiety theory. The findings support the hypothesis that political issues with a lower degree of familiarity, such as foreign affairs and immigration , are more anxiogenic than average, a conclusion that appears robust to estimators accounting for unobserved individual traits.


Introduction
Of the variety of emotions experienced by humans, anxiety ranks arguably among the most relevant for politics. Put simply, anxiety means feeling worried or concerned in the face of uncertainty (LaBar, 2016). We inherited this emotion from evolutionary processes with the critical function of keeping us alert to external threats (Suomi and Harlow, 1976;LeDoux, 2012;LaBar, 2016). Scholars have tied the emotion not only to uncertainty, but also to one's lack of control over events (Mandler, 1972;Mineka and Kelly, 1989). Of all social contexts, democratic institutions appear to combine the full set of ingredients required to rouse anxiety. Indeed, parliaments gather individuals with the specific purpose of making decisions under uncertainty, sometimes in situations of emergency. The rules of the game are collective by nature, meaning that individual concerns are at times ignored by the majority, which can exacerbate the sense of helplessness. As a result, it appears natural to associate the domain of politics with anxiety. The emotion has already gained the attention of scholars interested in politics, although empirical evidence has focused mostly on actors outside political institutions, such as voters (Marcus et al., 2000, e.g.). This paper represents a systematic attempt to automatically measure the level of anxiety in an extensive corpus of parliamentary debates.
Previous studies seeking to measure anxiety in text using methods of NLP have been sporadic, yet influential (Gilbert and Karahalios, 2010;Bollen et al., 2011). Since the task of detecting anxiety involves the recognition of rather subtle forms of expression in language, the levels of accuracy reached using machine learning classifiers are typically lower than for other tasks. More than a decade ago, Mishne (2005) released a valuable resource for the study of emotions in text, namely a corpus of blog posts from the LiveJournal website where users were invited to self-report their current mood by selecting options from a list, including anxious. In the original study, classifiers trained with Bag-of-Words (BoW) features reached an accuracy rate of 54.25% for the detection of the anxious mood, in balanced class problems. In a more recent study using an unbalanced dataset, Gilbert and Karahalios (2010) reached a very high level of accuracy for the detection of non-anxious texts, but only around 30% of correct predictions for anxious ones. Papers focusing on the related concept of fear have reported F1-scores between 0.20 and 0.74 (Strapparava and Mihalcea, 2007;Roberts et al., 2012;Mohammad, 2012). 1 It should be noted, however, that trying to map specific emotions such as anxiety into binary categories may not be the optimal approach. Feelings can be of various intensities, and binary annotations amount to losing precious information. Several modern approaches have instead relied upon emotion-specific lexicons that can be used to create continuous indicators (Tumasjan et al., 2010;Mohammad et al., 2013;Zhao et al., 2014). Using such a methodology, for instance, Bollen et al. (2011) were able to show that anxiety in social media is related to movements in the stock market.
The objective of this study is twofold. First, I seek to develop a methodology for the automatic measurement of levels of anxiety in writings, in particular, one that is adaptive to domain-specific corpora. The third section of this report describes the method and evaluates its performance using two annotated datasets: a sample of sentences from the Canadian House of Commons debates (the Hansard), annotated for anxiety using a crowd-sourcing platform, and the above-mentioned LiveJournal mood corpus. Second, my goal is to explain the prevalence of anxiety in real-life political speeches using a large section of the Hansard corpus. 2 Based on state-trait anxiety theory, which I discuss next, I implement predictive models in which some of the topics debated are expected to induce anxiety among parliamentarians.

Theoretical Background
I derive my expectations about the presence of anxiety in politics from the seminal state-trait anxiety theory proposed by Spielberger (1966). The theory emphasizes the distinction between anxiety as a trait, a more permanent feature of one's personality, and anxiety as a state, a temporary reaction to stressful situations. In this model, state anxiety is caused by the combination of both external events and individual predispositions. Anxiety as a trait has been the object of a voluminous literature, and corresponds to one of the "Big Five" personality traits (neuroticism) (Eysenck, 1997). In fact, other popular theories in psychology emphasize the role of factors such as personal experiences and biases in explaining anxious responses to external stimuli, for instance appraisal theory (Smith and Lazarus, 1993). Transposed to the realm of politics, this means that individual members of parliament (MPs) should exhibit various levels of anxiety in response to a situational trigger, depending on their personalities. An advantage of the Hansard corpus is that it allows to observe the same individuals repeatedly, under various situations. Thus, it becomes possible to account for individual traits, even if they are unobserved.
Regarding external factors, a central idea guiding my empirical analysis is that anxiety is more likely to occur the less an issue is known to MPs. Such an expectation directly follows from the nature of anxiety, which entails uncertainty, and parallels other theories in political psychology arguing that unfamiliar situations make individuals more anxious . In the context of a parliament, I expect subject-matters that are external to the country, such as foreign affairs and immigration, to be less familiar, hence more anxiogenic. One could view this as an argument from analogy: for individuals, strangers can be a source of anxiety since their intentions are unknown; for representatives of a state, other countries and their nationals can be a similar source of unpredictability. Of course, national issues such as natural disasters or economic downturns can also be worrisome to politicians. However, a reasonable working hypothesis is that the more foreign the object of debate, the more likely MPs are to experience anxiety.
Unlike ordinary social settings, however, members of democratic institutions have functions that may constrain the manner in which they respond to ongoing events. In particular, members face different incentives depending on the role that their political formation occupies in a legislature. Just like personality traits may predispose to anxiety, one's partisan affiliation can affect the likelihood of expressing signs of that emotion. Parties forming the opposition are required by convention to challenge the government's decisions; in fact, a period in the daily business of the Canadian House of Commons, the Oral Question Period, is dedicated to that duty. It is natural to expect MPs from opposition parties to communicate more forcefully their worries about the problems faced by their constituents, as opposed to members of the party in power, who are expected to bring reassurance. Figure 1 is a simplified depiction of the model guiding the present study. The figure is loosely based on the state-trait anxiety model in Spielberger (1966), amended to account for the peculiarities of parliamentary life. Basically, political speeches are expected to become anxious due to external triggers (unfamiliar topics under debate), individual-level factors (mostly unobservable and deeply entrenched personality traits) and party-level functions (the government/opposition division).

Automated Detection of Anxiety in Text
A key challenge with the detection of anxiety in the Hansard corpus is to account for the specific language of a parliament. In contrast to the casual English used in social media, politicians in parliament may be less explicit about their mood. To illustrate, one message from the LiveJournal corpus annotated with the anxious mood starts with the following words: [I] have finals today :( I am very nervous..
In this case, the statement expresses an unfiltered feeling that can be easily detected from word usage by a computer. However, parliamentary debates are subject to a decorum and speeches are usually less personal. An efficient methodology needs to take into account the vocabulary used to communicate emotions in such a formal setting.
For this reason, I rely upon a methodology that makes use of a vector space representationthe transformation of the corpus under study into numerical vectors based on word-word cooccurrences-to identify the proximity of each new word with a small set of seed words closely related to anxiety. This approach has been used previously to perform related tasks where semantic targets can be organized as polar opposites (Turney and Littman, 2003). I have tested the approach with different lists of seed words. Although the selection of these initial seed words may be a sensitive task, accuracy results based on the annotated corpora were not severely affected by small changes in the seed list. For this analysis, I make use of 30 lemma/part-of-speech tuples associated with anxiety on the one hand, and with the absence thereof on the other hand. In the anxiety pole, seed lemmas include threat, worry, fear, danger, anxious and tension, for instance. These are general words selected recursively from a thesaurus for their association with the concept of anxiety. In the opposite pole, seed lemmas comprise ease, protect, confidence, safe, trust and calm. Considering pairs of lemmas and parts of speech (PoS) allows to take into account the distinction between the syntactic roles of each word, for instance when being used as a verb or as a noun.
Specifically, I start by preprocessing the full Hansard Corpus between 1980 and 2015. This stage includes word and sentence segmentation, the attribution of PoS tags and lemmatization, all performed using the Stanford CoreNLP library . This subset of the Hansard comprises above 245 million tokens, and the period was chosen to encompass that used in the rest of the empirical analysis. The vector space model is computed on a reconstructed corpus of combined lemma and PoS pairs using the Glove program (Pennington et al., 2014), which can be trained on the entire co-occurrence matrix. I create word vectors of 300 dimensions using a symmetric window size of 15 words.
Next, for each new lemma and PoS pair in the Hansard corpus, I compute the cosine similarity between their vectors and the vectors of each lemma and PoS pairing in the seed list. The sum of similarity scores with the no-anxiety seeds is then subtracted from the sum of similarity scores obtained for the anxiety pole, and scaled back into a [−1, 1] range, where 1 means the most anxious lemma and −1 the least anxious. The created lexicon comprises 14,703 lemmas occurring 200 times or more in the corpus, after excluding digits and proper nouns.
Finally, I match the numerical scores of anxiety to the lemma/PoS pairs in the original corpus and compute average values for each individual speech. The result is a continuous indicator of the level anxiety, adapted to the specific register and tone of pol-

Trait Anxiety
A politician's predisposition to anxiety.

External Threat
A stressful event is debated in parliament.

State Anxiety
A politician experiences the feeling of anxiety.

Anxiety in Speech
Signs of anxiety are expressed during the debates.

Party Function
Opposition parties must raise concerns about ongoing issues.
itics, which can also be linked back to metadata on individual speakers, their party and the topic under debate.
To assess the performance of this approach, I retained the services of contributors from the Crowd-Flower website. I randomly selected two series of 2,000 sentences from the Hansard, respectively from the Government Orders and Oral Question Period between 1994 and 2015. 3 The decision to use sentences as a unit of annotation is justified by the large variation in the length of speeches during parliamentary debates. Preliminary assessments made with the help of a research assistant, based on entire speeches as units of analysis, revealed the task to be impractical for human coders, since full speeches may introduce several questions about how to properly code a document expressing different intensities of emotions. Shorter units of text can be more readily coded as anxious or non-anxious, and they reduce the risk of contradictory signals. I imposed the length of sentences to be above 8 tokens and removed procedural interventions (e.g. the introduction of motions). The 4,000 sentences were annotated by three different contributors via the crowd-sourcing platform, with an average pairwise agreement of 79.3%. Contributors were pre-screened with questions from a test set and their judgments considered reliable only if they reached a 70% agreement or above on these test questions. Although reliability may not reach the levels of more traditional methods, a re-cent study suggests that crowd-sourcing yields annotations comparable to those obtained from expert surveys (Benoit et al., 2016).
The proportion of sentences coded as anxious reaches 10.7% in the 4,000 Hansard sentences, larger than in the LiveJournal corpus. To avoid an unbalanced class problem, I have matched anxious sentences with the same number of randomly selected non-anxious ones. Although this is an admittedly small sample (854 sentences), the methodology reaches appreciable levels of accuracy using support vector machines (SVMs) with a linear kernel and a penalty parameter set at 0.5. Using the anxiety indicator as a single feature, the level of accuracy is around 63% (see Table 1), already above the levels of some previously cited attempts to detect the same emotion. Performance is slightly improved when adding a BoW including unigrams and bigrams.
I achieve important gains after including a simple categorical variable identifying the speaker, as well as title annotations from the original Hansard. With these metadata as features, the rate of accuracy reaches 83.6% (with a F1-score of 0.83, after selecting the 700 best features based on chi-square tests). This result is consistent with psychology theory, which emphasizes the importance of accounting for individual-specific traits, and illustrates the role of non-textual features in classification tasks. Accuracy scores are computed with randomly assigned training and testing sets; using 3K-fold crossvalidation, the percentage of accuracy reaches 82% in the best model. Additionally, I have considered grammatical features such as verb tense, after identifying the verb phrase containing the root in the dependency grammar structure of each sentence. These features are significantly related to anxiety (the future tense being positively related to anxiety, and conversely for the past tense). However, they did not improve accuracy further once accounting for speaker identification. Finally, I also assess the methodology against the LiveJournal corpus. The latter comprises over 700,000 entries associated with several moods. Since the proposed methodology is adaptive, I replicated the same steps with the LiveJournal data, creating anxiety scores using word vectors of the entire corpus and the same seed lists as before. Thus, the measure is now tailored to the specific genre of casual online discussions. To select anxious posts, I combined semantically equivalent reported moods (namely anxious, stressed, worried and restless). Following the approach in Mishne (2005), I matched a random sample of 15,000 anxious posts to the same number of non-anxious ones (merging calm, relaxed, optimistic and relieved) and to a sample of all other moods except anxiety. 4 The two bottom panels of Table 1 report the results. Accuracy using the anxiety score alone is lower than before, around 57%, still using SVMs (C = 80). Yet, the fact that a univariate indicator is sufficient to achieve a performance similar to that reported in previous studies using BoW models brings some additional legitimacy to the methodology. When including unigrams and bigrams, the predictive model reaches a level of accuracy exceeding 70%. Despite the possible limitations of the two corpora discussed in this section, these results suggest that the proposed score is significantly related to anxiety, as perceived by human coders or self-reported by Internet users.

Application to Parliamentary Debates
To examine the prevalence of anxiety in political speeches, I rely upon the previously introduced Hansard of the Canadian House of Commons. Using the anxiety scores generated with the method described in the previous section, speeches in that corpus are now associated with a level of anxiety. The database is constructed at the speech level, each observation corresponding to one intervention, and each being associated with metadata. I removed non-substantive speeches, for instance introductory remarks, questions of procedure, and interventions from the Speaker of the House. 5 I have also removed speeches for which the MP was not identified and those comprising less than 8 tokens, to avoid a number of common brief utterances such as "Yes/No" responses.
A portion of the Canadian parliamentary debates had been manually annotated by other researchers using the Comparative Agendas Project (CAP) topic scheme, namely a set of over 40,000 questions asked during the Oral Question Period between 1983 and 2004 (Penner et al., 2006). Coders attributed a topic to each new question in a given exchange (an exchange may include a question, an answer, and sometimes follow-up questions), which means that each annotation can be matched to more than one speech in the Hansard corpus. Using variables common to both datasets, I was able to match most of those annotations back with their original text. 6 This leaves a dataset of 119,623 speeches annotated by topic, which I focus on for the empirical analysis below. The topic scheme used by the CAP team is Speech Score Mr. Speaker, the Prime Minister is joking about the serious and even catastrophic situation of Canada's public finances.

0.29
That this House condemn those policies of the government which, having bred a sense of insecurity, powerlessness and doubt among Canadians, are a cause of the recent increase in regional tensions and incidents of racism and bigotry in Canada.

0.27
In those capacities we have absolute confidence in the judgment of the Secretary of State for External Affairs, and we can offer that categoric assurance to the Leader of the New Democratic Party. -0.48 Mr. Speaker, as always we will strive to protect the best interests of all Canadians.
-0.50 Table 2: Examples of Anxious and Non-Anxious Speeches rather detailed, comprising 25 main topics and more than 200 subtopics. For the sake of this study, I have simplified the scheme to 10 categories by relabeling the original CAP annotations. These 10 topics should sound fairly intuitive to both political scientists and scholars from other disciplines: constitutional affairs, economy, education, environment, foreign affairs, health care, immigration, law and order, natural resources, and a residual category containing all other national topics such as land management and intergovernmental affairs.
Illustrating the methodology when applied to political debates, Table 2 reports two examples of speeches above the 99th percentile on the anxiety score, and two under the 1st percentile. As can be seen, the measure correctly ranks the first speech as an expression of anxiety, despite the presence of a word like joking, which could have been misinterpreted as indicative of a light tone. The examples may also serve to illustrate how political functions influence the emotional valence of speeches: the two anxious speeches come from members of the opposition parties, whereas the bottom two speeches of Table 2 come from the government. The large majority of speeches fall under 0 on the anxiety scale, which is consistent with the fact that the feeling remains infrequent. The average value in the full sample is -0.13. In what follows, I normalize the indicator by transforming it into a z-score with a mean of 0, which facilitates the interpretation of findings.
The main objective from now on is to test the hypotheses laid out earlier in the theoretical section. I start by looking at the bivariate relationship between the topic under debate and the level of anxiety of MPs. Figure 2 shows the average level of anxiety by topic category (I use a weighted average with token counts as weights to account for discrepancies in speech lengths). As can be seen, immigration and foreign affairs are the topics for which Canadian MPs displayed the highest average level of anxiety between 1983 and 2004 during oral questions, which is consistent with the starting hypothesis. It shall be noted that anxiety is not bound to external topics, however. The question of natural resources-historically important for the Canadian economy, and including debates on sectors such as oil production and nuclear energy-ends up closely behind, before environment and economy. Law and order (which includes crime) and health care range somewhere in the middle, being neither the most anxious nor the least. Education and constitutional affairs rank as the least anxiogenic topics during this period.
It may be helpful to consider topic averages broken down by year to understand why immigration turns out first in the ranking during that time span. The highest levels of anxiety on the topic of immigration are observed between 1993 and 2001, period during which one of the major parties-the Conservatives-had ruptured into factions. The new parties (Reform and Bloc) began to voice a more forceful opposition on that particular topic. The period culminated with a major reform of immigration legislation in 2001, year during which the average level of anxiety reaches 0.31.
The corpus also allows for an examination of anxiety by party function, another component of the model introduced above. Table 3 reveals that members from parties in the opposition have levels of anxiety larger than the global average (+0.26), while the converse holds for parties forming the government (-0.20). The specific parties exhibiting the highest levels of anxiety in Table 3 are also those that remained in the opposition the whole time. I have also examined the link between anxiety and available biographical information on MPs. Women, for instance, appear slightly less anxious than men, still using the 1983-2004 sample of oral questions, based on a one-tailed mean comparison t-test (p < 0.02). Age also appears to be a significant predictor of anx-       iety in speeches: older MPs are less likely to express anxiety than younger ones.
Bivariate relationships, however, ignore the fact that some of those speeches are uttered by the same individuals. As pointed out earlier in the theoretical section, individual characteristics (traits) may affect whether someone is likely to feel nervous at any given point (state). Some MPs may have a tendency to experience anxiety and express worries in the face of adversity, whereas others may have a more stoic personality. Such individual traits can represent a confounding factor, and the unexplained variance of the anxiety score may be clustered within individual MPs. For these reasons, I rely upon statistical methods accounting for individual characteristics. I first consider a "within" estimator that subtracts individual averages from each variable in the model (which amounts to including a specific intercept parameter for each MP). This is also known as a fixed effects estimator that can be expressed in the form where y ij is the measured anxiety of a speech i uttered by MP j, the a j are MP-specific intercepts, x is a vector of observed metadata for each speech, and ε ij is an error component. The within transformation precludes the inclusion of variables that do not vary within individuals, such as gender. On the other hand, this transformation implies that the estimator accounts for all unobserved MP characteristics, such as personality traits. Note that the governmentopposition status varies for some MPs, and can be included among observed covariates. The second estimator is a hierarchical model with random intercepts at the party and individual levels. The model corresponds to: where λ j and θ k are random intercepts for MP j and party k. This estimator allows for the inclusion of variables such as age and gender. Table 4 reports estimates from both these models. Starting with the fixed effects model in the left column, estimated by OLS (F = 40.04; p = 0.000), the results are consistent with several of the findings stressed earlier about the relation between topics and anxiety. Since the topic variable is categorical, estimates are to be interpreted with respect to a base category, in this case the residual category "other topics". As can be seen, the estimates suggest that foreign affairs and immigration are significantly more  anxiogenic than the base category, supporting the results presented so far. These estimates are statistically significant, with reported p-values computed using Arellano's heteroskedasticity and autocorrelation (HAC) consistent standard errors (Arellano, 1987). However, once accounting for individualspecific factors, the topic of national resources appears to produce the same effect on anxiety than immigration, and a larger effect than foreign affairs. Education and constitutional affairs remain among the least anxious topics. Members of the government appear significantly less likely to express anxiety during their interventions, in line with the observations made previously. The last column of Table 4 reports results from the three-level hierarchical model estimated by restricted maximum likelihood. This model includes age and gender as covariates, which now turn out as non-significant predictors. A few data points on the birth dates of MPs are missing, reducing the number of observations. However, the results are very close to those obtained with the first estimator, and again support some of the key findings discussed so far.

Conclusion
Overall, this study illustrates the potential of NLP methods for the detection of specific emotions such as anxiety in politics. The finding that some topics are a source of anxiety may help to inform future research on legislative politics, for instance to examine the role of emotions in the formation of preferences over issues. From a methodological standpoint, the study also illustrates the importance of theory and non-textual features in predictive tasks. In particular, the results suggest that machine learning models can be improved by accounting for author identification in corpora where documents are clustered by individuals. Although the findings bring support to the proposed model, a limitation of this study is that expressions of anxiety may not reflect the true emotional state of a speaker. For example, politicians may appeal to emotions strategically, in an attempt to persuade. Disentangling the motives behind the use of language would require additional research. Nonetheless, the framework proposed in these pages appears reliable enough to derive substantive results of interest. Looking forward, the methodology could serve to answer other important questions about the role of anxiety in politics, ones that fell beyond the scope of this study. Previous research on anxiety has shown that the emotion influences how people make decisions. The experience of anxiety may ultimately make some choices less appealing, or refrain individuals from action altogether (Raghunathan and Pham, 1999;Gray and McNaughton, 2000).  stressed the hypothesis that anxiety leads individuals to search for more information before making decisions. Whether this has positive effects on the quality of decisions made by anxious individuals, or whether the feeling clouds their judgment, remains a debated question, albeit an important one (Brader, 2011;Ladd and Lenz, 2008;Marcus et al., 2011). Moreover, improving computational tools for the detection of specific emotions in texts can have applications useful beyond the study of politics. Examples include the detection of health conditions such as anxiety disorders, stock market forecasting or, more generally, contributions to the development of an artificial intelligence able to accurately identify specific emotions from language.