To Do or Not to Do: the Role of Agendas for Action in Analyzing News Coverage of Violent Conflict

One critical function that news narratives perform is orienting action: Providing a selective, coherent account of events, they suggest what needs to be done, coordinating and motivating public agendas. The importance of news narratives’ agendas for action has been particularly salient in the coverage of conflict 1 (Wolfsfeld 1997, Robinson et al. 2010): Conflict spurns heated debates wherein advocated courses of action collide, while audiences rely heavily on various media to comprehend ongoing events. Keeping track of the cacophony of agendas advanced in print and online newspapers and magazines, social media, and other public discourse confronts news readers, journalists, decision makers, and scholars alike with a major challenge. Computer assisted analyses have the potential to help comprehending conflict news, distilling agendas for action and possibly predicting the mobilization of consensus and collective action (Snow and Benford 1988). This paper presents the INFOCORE consortium’s ongoing efforts at automatically capturing agendas in conflict discourse, employing NLP technology and statistical analysis. We demonstrate the utility and potential of our approach using coverage of the Syrian chemical weapons crisis in 2013.


Introduction
Frame analysis is long established as one mainstream approach to the analysis of news narratives in communications and links to numerous traditions also in the humanities (Tewksbury and Scheufele 2009, Lakoff 2004, Souders and Dillard 2014. Focusing on the variety of narratives constructed to interpret the same news reality, its power lies in highlighting how different narratives influence audiences' beliefs, attitudes and actions. Specifically, Entman's (1993) seminal definition of frames posits the recommendation of specific treatments as one of four frames' primary functions. However, while numerous scientists (Snow and Benford 1988, Gamson 1995, Giugni 2006, Sanfilippo et al. 2008) have underscored frames' motivational and mobilizing functions, there has been remarkably little research on how frames advance specific agendas. Despite ample evidence documenting the public agenda setting power of the news (McCombs 2005), its direct antecedentthe agendas for action embedded in the news narrativehave been operationalized crudely as broad topics, or captured laboriously in highly case-bound studies. Computational linguistics have approached the frame analysis of the news texts by identifying the sentiment of articles (cf. Godbole, Srinivasaiah, andSkiena 2007, Scholz andConrad 2013), and capturing places, people and events as frame elements using named entity recognition technique (Lloyd, Kechagias, andSkiena 2005, Best et al. 2006). Frames' mobilizing component, however, has thus far eluded systematic study in humanities, social sciences and computational linguistics. In this paper, we present the INFOCORE 1 consortium's computer-assisted strategy for detecting and classifying agendas for action, which overcomes this limitation.

INFOCORE's approach to computing news storylines
In the news, conflict is described on several levels of abstraction. On the lowest level, the actors, objectives, aims and other relevant ideas in the given conflict are positioned, and specific evidential claims about these are presented, informing readers what is reportedly the case. At the next level, interpretative frames contextualize these claims, suggesting how the reported facts are to be interpreted. Combined into complex narratives to meaningfully link sequences of events, these frames finally advance specific agendas for action required to bring the conflict narrative to closure (Baden 2014): Integrating the available information to make sense of the situation, frames' motivational function translates the specific understanding of the news narrative into concrete, applicable agendas. Extracting these agendas, in this paper, we thus focus on news storylines' ability to direct action, constituting one primary societal effect of the news.

Agendas for action
From a semantic point of view, agendas for action consist of three components: First, as amply documented by agenda setting research, the issue to be acted upon has to be identified. Second, there needs to be a specific expression of a need to act. Third, the mandated course of action needs to be specified. While the range of relevant issues is principally unbounded and must be determined for each studied context, research in both linguistics and communication has emphasized a finite list of common ways for expressing the need to act:  Commissive, directive and partially expressive speech acts as defined by Searle (1976), with an enhanced list of speech act verbs (Wierzbicka 1987);  Imperative sentences: "Fight them!";  Sentences containing modal verbs obliging someone to do something: "They must obey";  Sentences expressing the speaker's dissatisfaction: "[Sb.] condemned such a motion" or "We will not stand this aggression";  General expressions that something cannot stand: "Something should be done"  Rhetorical questions: "Can we accept such a treatment?";  Propositions about desirable, but absent states: "Peace is the only answer".
The course of action, again, can include virtually any kind of activities and inactivities. It is therefore useful to classify different kinds of actions more broadly. Owing to the focus on conflict-related news, INFOCORE's analysis aims to distinguish the following agenda types:  peaceful solution/de-escalationagendas for peace, a ceasefire, to stop fighting, etc.: "People need to understand that violence is not an acceptable way to solve disputes."  violent solution/escalationthe opposite of the above category includes calls for military action, violence, escalation, etc.: "…Fatah and Hamas, which Israel deems a terrorist organizations calling for its destruction"  involvement/dialogue/supportincluding calls for cooperation, negotiations as well as all sorts of help and support: "Eradication of poverty should be the main priority of humanitarian action."  punishment/sanctions/toughness the opposite of the previous category, calling for a tough stance and (non-violent) coercion: "UN official applauds sentencing of militia leader for war crimes."  general/rhetorical questionsagendas that do not call for something specific, but express dissatisfaction with the status quo that should be changed: "Now is the time to take action."  multiclasscomplex treatments wherein multiple clauses express different agenda: "The international community must break that habit, accept the Palestinian membership application, guarantee Palestinians a war crimes case, prioritize peace and end Israel's impunity -or see international law perverted further in ways that is certain to harm the entire world."  negativecalls for not doing something: "We must not lower our guard, at any time, Prime Minister Manuel Valls told Parliament, adding that "serious and very high risks remain". Such agendas may be also expressed without using negators: "The Department of State warns U.S. citizens of the risks of travel to eastern Ukraine". Sentences criticizing others for doing something also belong here: "We condemn these barbaric crimes."  othersentences that contain an agenda for action but are semantically ambiguous are classified here: "The militants who massacred schoolchildren, beheaded soldiers and attacked defense installations have surely committed war crimes and must be dealt with as such." As expressing agendas for action necessarily takes propositional form, the task may be formulated as sentence classification. We apply a two-step classification procedure: We identify those sentences expressing an agenda for action in the first step, and classify the expressed agenda by type in a second step. For the present paper, we apply a simplified classification of agenda types: peaceful and dialogic agendas are merged into "cooperative" treatments, while violent and punitive agendas constitute "restrictive" treatments, and the rest fall under "other".

Related work
Our approach builds upon recent advances in automated content analysis and extraction of frames, as well as in sentence classification. In communication research, automated approaches to frame analysis mostly rely on a detection of co-occurrence patterns: Following Entman's (1993) frame definition, Matthes and Kohring (2008) identify frame elements manually and collate frames based on the systematic joint appearance of these elements (see also Wettstein 2014, Hughes, Lancaster, andSpicer 2011). Kutter and Kantner (2011) rely on an operationalization of semantic fields (Gliozzo and Strapparava 2009) to perform a semiautomated, corpus-based analysis of semantic co-occurrences. Baden (2010) measures frames as "areas of heightened density in a semantic network" (page 90) of systematic, dyadic concept co-occurrences. Sanfilippo et al. (2008) depart from an extended frame definition and extract frame components (Promoter, Intention, Target, Issue, etc.) using different NLP techniques (grammar parsing, NER, coreference and temporal resolution). In order to measure Intention, they also capture intent verbs, which are directly related to our approach in the current paper.
Relevant work on sentence classification has focused on assessing different classification algorithms and fine-tuning their parameters and features (Kim 2014, Khoo, Marom, and Albrecht 2006, Revathi et al. 2012, for application to specific tasks and domains (Cohen, Carvalho, and Mitchell 2004, Qadir and Riloff 2011, Kim, Martinez, and Cavedon 2011, McKnight and Srinivasan 2003. Most applicable here are those studies classifying text segments as speech acts: Cohen, Carvalho, and Mitchell (2004) categorize whole email messages as requests, proposals, amends, commitments, deliveries, and other speech acts. Using TFIDF-weighted bag of words, bigrams, and POS-tags, they compare four classifiers -Voted Perceptron, AdaBoost, SVM, Decision Treethe latter two outperforming the rest.
Qadir and Riloff (2011) finally move the categorization task to sentence level, assigning speech act labels as defined by Searle (1976) to message board posts. Their approach also reflects grammatical structure of sentences in features (e.g., capturing imperative sentences and disambiguating them from interrogative ones), which were used to train a SVM classifier. While we are not interested here in representatives or expressives that do not advance specific agendas, their classification strategy is closely related to ours; however, we also included instances that merely imply an agenda for action, but fall short of forming classic directives. To our knowledge, ours presents the first study to comprehensively distil agendas, including also speech-act-like structures, in conflict news coverage to date.

Corpus
To train the classifier, we manually annotated a 1723-sentence corpus, labeling each sentence as "cooperative treatment" (287 items), "restrictive treatment" (204 items), "other" (249 items) or "none" (983 items). For the first round of classification the former three categories are combined. For the second round, the "none" category was excluded from the pool of sentences. The LexisNexis 1 database was used for crafting the corpus, retrieving all English language sources including the keywords "conflict", "war" or "violence" from 1 January to 1 March 2015. Articles were split into sentences and labeled accordingly.

Features
For both rounds of classification, n-gram features with n between 1 and 3 were used. For the second round the words were stemmed, as it improved the performance in the second round but not in the first: To classify a sentence as containing an agenda, not only lexical, but also grammatical information is important, which is contained in endings and suffixes. Also stopwords removala classical pre-processing step in NLPwas not performed as it reduces performance: stop-word lists usually include prepositions, particles, articles and auxiliary words which contain important grammatical information needed for such classification (Khoo, Marom, and Albrecht 2006). By contrast, recognizing the agenda type is a purely semantic classification task, for which stems are sufficient. Features extraction and model training was carried out using the Weka toolkit for data mining (Hall et al. 2009).

Classifiers
We compared the results of three classification algorithmsdecision tree (J48), Naïve Bayes Multinomial (NBM) and support vector machine (SVM) -to find out which one performs best. Cross validation with 5 and 3 folds was used for the first and the second rounds respectively. For the first round, SVM outperformed the rest, classifying 74% of instances correctly (see Table 1).  For the second round, SVM and NBM performed equally well: 52% of instances were classified correctly (see Table 2). Poor classification results may be explained by little amount of training data and high semantic ambiguity of the concept. Generally, our results support the findings of Khoo, Marom and Albrecht (2006) that SVM is the most powerful algorithm for sentence level classification.

The Story
Extracting agendas for action from the news coverage on violent conflict helps tracking the dynamics of a conflict, identifying and possibly predicting phases of escalation and deescalation: Dominantly expressed agendas not only prepare news audiences for impending policy moves and collective action in conflict, they also reflect preeminent interpretations of current conflict as accessible to or beyond peaceful resolution. Conflict events extend ongoing news narratives, update conflict perceptions, and shift conflict policy agendas. The Syrian chemical weapons (CW) crisis 2013 progressed from initial rumors, uncertainty and global hesitation in March, through rapid escalation following the large scale CW attacks in Ghouta in August, culminating in projected imminent US air strikes, to international disengagement following Syria's surrender of its CW arsenals to UN control in October, and their destruction by the OPCW. Widely diverse agendas were discussed at all times, but only few became dominant temporarily (for more in depth analysis of news coverage of the crises see Baden and Stalpouskaya 2015).

Material
For our analysis, we used news coverage of the Syrian CW crisis in the British Guardian and the American New York Times (NYT). Both media act as papers of record for the respective countries, key global players in the crisis, closely observing their foreign policy debates. The coverage was retrieved from the respective news archives based on a search for references to Syria (e.g. "Syria", "Damascus") and chemical weapons (e.g. "chemical weapon", "sarin", "WMD"). The dataset comprised 584 articles from the Guardian and 609 from NYT. Articles were grouped by month, split into sentences, and each sentence classified based on the two-step procedure and pre-trained SVM classifier described above.

Results
The analysis reveals a steady presence of agendas, which are detected in one third of all sentences throughout the entire time range (Table 3). Figure 1 shows that there are initially fewer agendas expressed in the UK, whose role in the conflict crystallizes only once premier Cameron calls for active intervention, building steadily as the debate picks up speed; however, when the UK's parliament votes against intervention, the need to discuss an active British role ceases, the administrative attention wanders on, and the share of agendas expressed in the Guardian wanes. By contrast, the NYT considers Syria a case for possible US intervention from the outset (reflecting Obama's "red line" 1 ), and discusses policy options long before the administration openly considers military action. The role of agendas in the coverage is high at all times. Interestingly, the share remains stable despite major changes in the amount of attention to the crisis, culminating in September as the Ghouta attacks initiate a hectic search for viable action (less so in the UK, where military action is now off the table).
Looking at the kinds of agendas distinguished in Figure 2, political, cooperative efforts dominate at all times over military options (the spike of restrictive agendas in the Guardian's coverage in July overrepresents few calls for arming rebels and cited militants during a month of very low coverage). In the US, support for military action builds slowly but steadily, while the White House's justification of its hesitant stance results in many calls for forging international agreement and alliances on Syria. Culminating in July, the US adopt a more unilateral stance after Ghouta, when cooperative agendas drop and calls for independent investigation (other) dominate. Following the US' adoption of Russia's plan to put Syria's CW under UN control, also restrictive agendas drop, while monitoring compliance becomes the primary concern. In the Guardian, many calls for military action exist at the outset, but are quickly diffused into a more general punitive agenda ("stopping Assad", "not standing by idly", classified as "other") as rumors solidify. While the UK waits for the US to take the lead, agreement that someone urgently needs to find some solution builds, but Cameron's military agenda remains at a share of one fifth as only one of many such proposals. In both papers, the resolve to escalate the crisis is perceptibly limited: Even after the Ghouta attacks, when patience with global diplomacy finally fails and the US are at the verge of ordering missile strikes, cooperative agendas remain more salient than restrictive ones. And while the tone is notably more hostile toward Assad in the Guardian, the vagueness of agendas correctly indicates that no specific action is imminent. st/consulate-said-to-support-claim-of-syrian-gasattack.html

Limitations and future work
The present study remains work in progress in important ways. First, the data presented uses a simplified classification of agendas and does not yet integrate with the analysis of frames and evidential claims underway within the INFOCORE project. Second, we do not yet capture certain implicit agendas for action, whose illocutionary force indirectly constitutes a perlocutionary act (Austin 1962) e.g., "we see these violations with grave concern". Such agendas rely on conventional values to unfold their directive qualities, and therefore require different operational strategies. Third, we only detect agendas for action contained within one sentence, yet, one sentence may contain several agendas, and agendas span multiple sentences. The former case is classified as "multiclass" in our fine classification, but allows further differentiation. To detect agendas spanning sentences, tools are needed that tie together sentences based on anaphora resolution, conjunctions such as "also", "as well", or identity chains of synonymous predicates.
Furthermore, with the current set-up, fine classification performs poorly: the best accuracy achieved was 36% using NBM. More training data is needed, but also utilizing lexical resources (e.g., WordNet) should boost precision and recall. Similarly, grammatical ambiguity is responsible for some misclassification of agendas vs. non-agendas (e.g., "they request" vs. "request is being processed") and requires resolution using additional NLP resources.
Finally, to get the full picture of the news storyline, agendas for action need to be linked to the speaker advancing and others commenting upon that agenda. While strategies exist for most of the named limitations, bringing them to bear on our analysis remains a task for future work.

Conclusion
Our goal was to highlight the role of "agendas for action" as one key, socially relevant conclusion arising from competing news storylines, and propose a strategy for detecting these algorithmically in a text corpus. Related, but not identical to directive speech acts, the approach pursued by the INFOCORE consortium combines grammatical and semantic resources to classify relevant statements in the news. Our focus on agendas bridges a gap in the study of news content and discourse, which often postulates the action coordinating and directing role of new narratives, but has focused much more on the descriptive semantic qualities of news frames than on their mobilizing capacities. Agendas for action present the critical link between discursive representations and social action, but have to date mostly evaded scholarly attention. Identifying common ways of suggesting specific courses of action, we applied machine learning technique to extract sentences expressing agendas for action. Currently standard n-rams with n between 1 and 3 were used as features. In a number of trials, we found large margin classifiers to perform best, and applied the trained model to analyze news coverage of Syrian CW crisis in 2013. Sentences were classified in two steps, first discriminating agendas for action from statements without directive force, and qualifying the nature of the advocated treatment in the second step. The procedure demonstrates the potential of combining grammatical and semantic information for sentence classification, and opens up avenues for further research. We found consistently high percentages of agendas expressed in the news, advocating different kinds of action at different moments during the crisis. The extracted agendas are informative about the quality of news debates about the conflict, and can be tied to actual policies and developments within the crisis. However, further work is needed to increase differentiation, accuracy, and critically, to integrate the detected agendas into the context of the news storyline.