Construction of a Chinese Corpus for the Analysis of the Emotionality of Metaphorical Expressions

Metaphors are frequently used to convey emotions. However, there is little research on the construction of metaphor corpora annotated with emotion for the analysis of emotionality of metaphorical expressions. Furthermore, most studies focus on English, and few in other languages, particularly Sino-Tibetan languages such as Chinese, for emotion analysis from metaphorical texts, although there are likely to be many differences in emotional expressions of metaphorical usages across different languages. We therefore construct a significant new corpus on metaphor, with 5,605 manually annotated sentences in Chinese. We present an annotation scheme that contains annotations of linguistic metaphors, emotional categories (joy, anger, sadness, fear, love, disgust and surprise), and intensity. The annotation agreement analyses for multiple annotators are described. We also use the corpus to explore and analyze the emotionality of metaphors. To the best of our knowledge, this is the first relatively large metaphor corpus with an annotation of emotions in Chinese.


Introduction
Metaphorical expressions are frequently used in human communication, and they occur on average in every third sentence of natural language, according to empirical studies (Cameron, 2003;Steen et al., 2010;Shutova and Teufel, 2010). Metaphor not only involves linguistic expressions, but also involves a cognitive process of conceptual knowledge (Lakoff and Johnson, 1980). According to Lakoff and Johnson (1980), humans use one concept in metaphors to describe another concept for reasoning and communication. For instance, in the metaphorical utterance: "experience is treasure," we use "treasure" to describe "experience" to emphasize that "experience" can be valuable. To take another metaphorical instance as an example: "he killed the engine." "An engine" is viewed as a living thing, and thus stopping its operation is related to the act of killing. Metaphor has been viewed as a mapping system that conceptualizes one domain (target) in terms of another (source).
Emotion, as an abstract and vague conception, is frequently described and conceptualized by metaphor (Goatly Musolff and Project LLE, 2007;Kövecses, 1995Kövecses, , 2000. There seem to be two main types of metaphors that evoke emotion. One is the metaphor in which the target domain is emotion. For example, in the instance "he was blazing at what she did," the angry, emotional self is conceptualized as "fire," and so is expressed metaphorically in terms of "blaze." The other type is metaphors that have emotional connotations. For example, in "The financial crisis has eaten up all my savings," the target domain is finances and the source domain, implied by the verb "eat up," is some sort of ravenous beast. This metaphorical sentence thus may express senses of anger and fear about a "financial crisis." From the above examples, we can see that metaphorical expressions often state or evoke emotions implicitly and indirectly. Neuroimaging studies have provided evidence that metaphorical language elicits more emotional activation of the human brain than literal language in the same context (Citron and Goldberg, 2014).
The interaction between emotion and metaphor has been studied from different perspectives by scholars in many fields such as psychology (Averill, 1990;Thibodeau and Boroditsky, 2011;Fetterman et al., 2016), linguistics (Fainsilber andOrtony, 1987;Kövecses, 2010), neuroscience (Az-iz-Zadeh and Damasio, 2008;Malinowski and Horton, 2015;Jabbi et al., 2008) and natural language processing (NLP) (Mohammad, Shutova and Turney, 2016). Many approaches for sentiment analysis of metaphorical texts have been proposed in the area of NLP (Smith et al., 2007;Veale, 2012;Reyes and Rosso, 2012;Kozareva, 2013;Strzalkowski et al., 2014). In particular, along with the rapid explosion of social media applications such as Twitter and Weibo, emotional texts containing metaphorical expressions have increased considerably. It seems to be very common for Internet users to use vivid and colorful metaphorical language to express emotions on social media.
Corpora are fundamental for sound analysis of emotionality in metaphor and for high-quality automatic emotion detection in metaphor. However, many resources cover sentiment analysis (Alm et al., 2005;Dong et al., 2014;Kiritchenko et al., 2014;Mohammad et al., 2013;Strapparava and Mihalcea, 2007;Ratnadeep et al., 2013) and metaphor detection (Lönneker, 2004;Martin, 2006;Pragglejaz Group, 2007;Steen et al., 2010) separately. Moreover, although NLP has proposed approaches for sentiment analysis of metaphor, as mentioned above, an overwhelming majority of studies focus on the annotation of only positive and negative emotions rather than a range of emotions. In addition, there is limited research in NLP in languages other than English analyzing emotions in metaphors. Nevertheless there are likely to be many differences in emotional expressions of metaphorical usages in different cultures, although multiple languages share similar conceptual metaphors based on the same human cognition and physical experience (Kövecses, 1995).
According to the above account, we propose a Chinese corpus with annotations of both linguistic metaphors and emotion. Unlike the widely applied annotation of only positive and negative, we have annotated a range of emotions (joy, anger, sadness, fear, love, disgust and surprise).Based on the analysis of the corpus, our results indicate that a significant proportion of Chinese metaphorical expressions in the corpus contain emotions and the most frequent emotion is love. We also suggest potentials of the corpus contributing to automatic emotion and metaphor detection as well as further investigating mechanisms underlying emotion in metaphor from the perspectives of different cultures for future work. To the best of our knowledge, this is the first relatively large metaphor corpus with an annotation of emotions in Chinese.

Data Collection
With the aim of constructing a corpus in the study of emotionality of metaphorical texts in real-world Chinese, data collection took place in accordance with two principles: (1) balance, and (2) relatively abundant emotional information. Specifically, to ensure the corpus is balanced in genre, theme, and style, we selected data from a wide range of sources including books, journals, movie scripts, and networks. In addition, we focused on sources with rich emotional information such as microblogs.

Annotation Model
We annotated metaphorical sentences with target and source domain vocabulary, emotion categories and intensity, metaphor categories (verb or noun metaphor : verb or noun used metaphorically),data sources, and metaphor devices such as 像 "like," 好似 "as," etc. as "indicators". "Indicators" can be null, while the other variables cannot. For example, if there are some terms without values, we need annotators to complete them.

Metaphor Annotation
Metaphor category. Based on our investigation of a wide range of texts, we focused on two main types of the most frequently appearing metaphorical sentences: verb metaphor, which contains a verb used metaphorically (e.g., 她怀揣着 美 好的梦 想 "She wove a good dream in her mind"); noun metaphor, which contains a noun used metaphorically. Noun metaphor includes a metaphor of "A is B" (e.g., 语言就是力量 "language is power") and metaphor with linguistic makers such as "as" and "like" (e.g., 他像箭似的 跑开了"he ran away like an arrow"), which is normally identified as "simile" from a linguistic perspective, but as "metaphor" in this paper, because it accords with metaphor as we define it: whenever one concept is used to describe another concept (Lakoff and Johnson, 1980). The decision to define both metaphors and similes as metaphors is based on the wish to give a fuller picture of metaphor in our study than one that does not include similes.
Literal or metaphorical. The metaphor annotation is at the relational level, which involves identification of metaphorical relations between source and target domain vocabulary. However, scholars have different opinions on the distinction between literal and metaphorical senses. Some only consider novel expressions (e.g.,她为办公室注 入新的活力"She breathed new energy into the office") as metaphorical, whereas others consider conventional expressions as metaphors (e.g.,他 们赢得了这场争论 "they won the argument"), where they are conventionalized and fixed in form, and they are used literally by native speakers, although they have the nature of metaphor (Nunberg, 1987). In this study, following Shutova (2017), we define metaphors as both novel and conventional, but we exclude "dead metaphors" (from which the literal sense has disappeared) from conventional metaphors. That is, conventional metaphors only include those for which the literal and metaphorical senses are clearly distinctive, and both are used contemporarily. This consideration is based on the potential application of our annotation for identification of metaphor, which focuses on word sense disambiguation rather than novel or conventional identification.

Emotion Annotation
Emotion categories. Scholars define basic emotions in numerous different ways despite research that has challenged the theories of basic emotions (Lindquist et al., 2012;De Leersnyder et al., 2015). Confucianism claims that there are seven basic human emotions (joy, anger, sadness, fear, love, disgust, and desire) (Ma et al., 2011). "七情" (Seven Emotions) is an idiomatic expression commonly used by Chinese people to describe human emotions. However, according to our study of the collected instances in the corpus, we found that "desire" does not appear widely and that "surprise" does. We therefore adopted "surprise" to replace "desire" in the Severn Emotions. The resulting classification is very close to Ekman (1992), with the only difference being the inclusion of "love" as a basic emotion. However, based on our study of the research and its wide appearance in the corpus, "love" is listed as a basic emotion in this paper.
The annotation of emotions takes place on the sentence level. The emotion contained in each metaphorical sentence was identified from one of the seven categories of emotion. We also categorized the intensity of emotion into one of five levels: 1, 3, 5, 7, or 9.

Annotation Process
Annotation setup. The annotator team comprised seven native Chinese annotators. Annotators were given standards and principles of annotation and detailed instruction for potential difficult and common problems with many annotated samples. Aside from giving them annotators detailed guidelines, we gave them a formal training lesson and a lab meeting to exchange ideas and to discuss problems about annotation once a month during the nine months of the annotation process.
The guidelines changed four times, as we added information on newly found annotation difficulties during the project period. The annotator team comprised seven students, who were not paid for their work.
These seven annotators were divided to three groups with two members for each group plus one group with one person. Using cross-validation methods for annotation, the two-member groups annotated, and the one-person group participated in the final decision when there was divergence. If there was no divergence between the members of the same group, the annotating work was complete. Otherwise another group annotated again, and the final group annotated if there was still divergence. Finally, if the three groups could not reach agreement on the annotation, everyone discussed and determined the annotation to ensure its accuracy and consistency.
Quality monitoring and control (QMC).We used a standardized operating method to achieve high-quality annotation as follows: (1) Entry interface. On the basis of multiple manual checks and controlled information updating, we provided an interface that allowed us to enter information precisely and quickly.
(2) Error correction. We used emotional lexicon ontology 1 as a support tool to correct human errors. When there was divergence of emotion in the annotation and the sentence/word, we did not enter the annotation. flag=WordConsistency(Memo,Wemo)SentConsiste ncy(Memo, Semo). (1) If the annotation result was the same as the word's emotion, we set WordConsistency(M emo , W emo ) to 1; otherwise, we set it to 0; SentConsistency(M emo , S emo ) followed the same logic. We entered the result when the flag was 1, while it needed checking when it was 0.

Annotation Agreement
Annotations of both metaphor and emotion were based on the annotators' intuition, which may be very subjective. The reliability of annotations needed to be verified, so three independent annotators annotated the same 811 sentences in the corpus to assess inter-annotator agreement. 1 http://ir.dlut.edu.cn/EmotionOntologyDownload The kappa score, κ, is widely adopted by computational linguistics to correct for agreement on the reliability of the annotation scheme by chance. We use the κ statistic to measure inter-annotator agreements (Siegel and Castellan, 1988) for emotion annotation. κ is calculated as below: The agreement on the identification of source and target domain words was κ=0.82, which means it is substantially reliable. Compared with noun metaphors, for verb metaphors it is relatively difficult to identify source and target words, because they are related to the assignment of levels of conventionality of metaphorical senses as discussed in 4.1.
The agreement on the choice of emotion category scored κ=0.68. For the agreement measure on emotion intensity, we classified emotion intensity into three: {1, 3}, {5, 7}, or {9}. The resulting agreement on classification of emotion intensity was κ=0.58.

Corpus Analysis
We annotated 4,600 sentences out of a total of 5,605 metaphorical sentences as containing emotions. That is, a significant proportion of Chinese metaphorical expressions in the corpus contain emotions. The most frequent emotion in the corpus is love. Figure 1 shows the number of sentences in the corpus of each emotion category. We also explored the interactions between emotions and source (or target) words. We analyzed every emotional category and related it to each source or target word; we also analyzed which source or target vocabulary conveyed which emotions most frequently. Our results indicate emotions are related to some particular source or target vocabulary. These associations frequently occur together. For example, the emotion love is related to the source words 海洋"sea", 花"flower"and 阳光"sunshine" etc.
Unlike the widely applied annotation of valence (positive-neutral-negative) in sentiment analysis, we annotated a wider range of emotions in metaphorical texts. The corpus proposed by  focused on containing or not containing emotion in metaphor. We have extended their study by providing evidence that metaphorical texts can convey specific emotions such as love and joy. A simple positive/negative emotion distinction does not seem very useful for anything beyond evaluating product reviews. In addition, Mohammad et al. (2016) focus on verb metaphors, whereas we collect both verb and noun metaphors from a variety of sources. Furthermore, since both metaphor and emotion annotations are very subjective, we propose a QMC method (see above 4.4) to achieve high-quality annotation.

Conclusions and Future Work
With 5,605 diverse instances and 101,616 Chinese characters of metaphor, our corpus provides an important resource with relatively fine-grained sorting and annotation with both metaphors and emotion. Our study involves the Chinese la nguage, which is very different from English, the focus of the vast majority of current research. This may encourage research into emotion analysis in other languages, particularly Sino-Tibetan languages, since there are differences in emotion between cultures (De Leersnyder, 2015).
Seven annotators spent nine months on the annotation. The manually annotated data is an important step towards automatic emotion analysis and detection of metaphorical texts, as well as metaphor detection. In addition, the application of the corpus to machine translation will be explored to improve the poor translation of metaphorical expressions (Shutova et al., 2013). Furthermore, we hope research using bilingual resources will be conducted on the datasets we have released. This may make contributions to some novel and interesting studies of the emotionality of metaphor from cross cultural perspectives as well as exploring the related, underlying mechanism.