Personality Profiling of Fictional Characters using Sense-Level Links between Lexical Resources

This study focuses on personality prediction of protagonists in novels based on the Five-Factor Model of personality. We present and publish a novel collaboratively built dataset of ﬁctional character personality and design our task as a text classiﬁ-cation problem. We incorporate a range of semantic features, including WordNet and VerbNet sense-level information and word vector representations. We evaluate three machine learning models based on the speech, actions and predicatives of the main characters, and show that especially the lexical-semantic features signiﬁ-cantly outperform the baselines. The most predictive features correspond to reported ﬁndings in personality psychology.


Introduction
Recent progress in NLP has given rise to the field of personality profiling -automated classification of personality traits based on written, verbal and multimodal behavior of an individual. This research builds upon findings from classical personality psychology and has applications in a wide range of areas from medicine (suicide prevention) across security (forensics, paedophile detection, cyberbullying) to marketing and sales (recommendation systems, target group profiles). The gold standard labels for an objective evaluation of personality are mostly obtained by means of personality tests of the Five Factor Model (FFM) (McCrae and Costa, 1987;Goldberg, 1990), which is wellknown and widely accepted in psychology and other research fields. The FFM defines personality along five bipolar scales: Extraversion (sociable vs. reserved), Emotional stability (secure vs. neurotic), Agreeableness (friendly vs. unsympathic), Conscientiousness (organized vs. careless) and Openness to experience (insightful vs. unimaginative). Psychologists have shown that these five personality traits are stable across individual lifespan, demographical and cultural differences (John and Srivastava, 1999) and affect many life aspects. (Terracciano et al., 2008;Rentfrow et al., 2011).
It has been shown that the personality traits of readers impact their literature preferences (Tirre and Dixit, 1995;Mar et al., 2009). Psychology researchers also found that perceived similarity is predictive of interpersonal attraction (Montoya et al., 2008;Byrne, 1961;Chartrand and Bargh, 1999). More explicitly, recent research (Kaufman and Libby, 2012) shows that readers of a narrative develop more favorable attitudes and less stereotype application towards a character, if his difference (e.g. racial) is revealed only later in the story. We therefore hypothesize that readers might have a preference for reading novels depicting fictional characters that are similar to themselves. Finding a direct link between reader's and protagonist's personality traits would advance the development of content-based recommendation systems. As a first step to explore this hypothesis further, it needs to be determined if we are able to construct a personality profile of a fictional character in a similar way as it is done for humans, and which aspects of personality profiling can be exploited to automatize such procedure.
In this paper, we open this research topic by presenting a novel collaboratively built dataset of fictional character personality in Section 3, which we make available on our website. 1 Framing the personality prediction as a text classification task, we incorporate features of both lexical-resource-based and vector space semantics, including WordNet and VerbNet sense-level information and vectorial word representations. We evaluate three machine learning models based on the speech (Section 4), actions (Section 5) and predicatives (Section 6) of the protagonists, and show that especially on the direct speech and action data the lexical-semantic features significantly outperform the baselines. Qualitative analysis reveals that the most predictive features correspond to reported findings in psychology and NLP.

Related work
Research in the the area of content-based recommendation systems have shown that incorporating semantic information is valuable for the user and leads to measurable improvements (Passant, 2010;Di Noia et al., 2012;Heitmann andHayes, 2010). De Clercq et al. (2014) incorporated semantic frames from FrameNet into the recommendation system for books. They represent the plot of each book with a sequence of ca. 200 semantic frames and has shown that the frame information (such as Killing -Revenge -Death) outperforms the bag-of-words approach.Recent NLP experiments begin to reveal the importance of entitycentric models in a variety of tasks. Chambers (2013) show improvement in event schema induction by learning entity-centric rules (e.g., a victim is likely to be a person). Bamman et al. (2014) and Smith et al. (2013) present latent variable models for unsupervised learning of latent character types in movie plot summaries and in English novels, taking authorial style into account. However, even the state-of-the-art NLP work rather describes personas of fictional characters by their role in the story -e.g., action hero, valley girl, best friend, villain etc. -or by their relations to other characters, such as mother or daughter (Elson et al., 2010;Kokkinakis and Malm, 2011), rather than by their inner preferences and motivations. It is important to note here that determining a personality of a character is a very different task from determining its role in the story. Psychological understanding of personality, in contrast to role attribution requires a certain detached objectivityeven outright villains may have traits considered desirable in real life. For example, the devil has in many tales a very high aspiration level, appearing highly conscientious and agreeable. We hypothesize that these deeper personality aspects are those which drive reader's affiliation to the character, thus deserve to be examined closer.
Also literary scholars formulate ad hoc personality descriptions for their experiments, for example to test hypotheses from evolutionary psychology (Johnson et al., 2011) or examine fictional portrayals of physicists (Dotson, 2009). These descriptions are usually adjusted to the experiment focus (e.g. emotions, relationships, ambitions). As McCrae et al. () point out, a standard set of personality traits, that encompass the full range of characteristics found in all characters in literature (p.77), is needed for a better comparison.
Hence we base our present study primarily on the previous NLP research on personality prediction of human individuals. Correlations between lexical and stylistic aspects of text and the five FFM personality traits of the author have been found in numerous experiments, with extraversion receiving the most attention (Pennebaker and King, 1999;Dewaele and Furnham, 1999;Gill and Oberlander, 2002;Mehl et al., 2006;Aran and Gatica-Perez, 2013;Lepri et al., 2010). The LIWC lexicon (Pennebaker et al., 2001) established its position as a powerful mean of such analysis.
The first machine learning experiments in this area were conducted by Argamon et al. (2005), Oberlander and Nowson (2006) and Mairesse et al. (2007). Researchers predicted the five personality traits of the authors of stream-ofconscientiousness essays, blog posts and recorded conversation snippets. Given balanced data sets, Mairesse et al. (2007) report binary classification accuracy of 50-56% on extraversion in text and 47-57% in speech, using word ngrams, LIWC, MRC psycholinguistic database (Coltheart, 1981) and prosodic features. Additional improvement is reported when the extraversion was labeled by external judges rather than by self-testing. Extended studies on larger datasets achieve accuracies around 55% (Nowson, 2007;Estival et al., 2007). More recent work in this area focuses on the personality prediction in social networks Kosinski et al., 2014) and multimodal personality prediction (Biel and Gatica-Perez, 2013;Aran and Gatica-Perez, 2013). These trends emphasized the correlation of network features and audiovisual features with extraversion, giving rise to the Workshop on Computational Personality Recognition (for an overview see (Celli et al., 2013;Celli et al., 2014).

Data set construction
Traditionally, the gold standard for this supervised classification task is obtained by the means of personality questionnaires, used for the Five-Factor Model, taken by each of the individuals assessed. This poses a challenge for fictional characters. However, strong correlations have been found between the self-reported and perceived personality traits (Mehl et al., 2006). Our gold standard benefits from the fact that readers enjoy discussing the personality of their favourite book character online. A popular layman instrument for personality classification is the Myers-Brigggs Type Indicator (Myers et al., 1985), shortly MBTI, which sorts personal preferences into four opposite pairs, or dichotomies, such as Thinking vs. Feeling or Judging vs. Perceiving. While the MBTI validity has been questioned by the research community (Pittenger, 2005), the Extraversion scale is showing rather strong validity and correlation to similar trait in the Five-Factor Model (McCrae and Costa, 1989;MacDonald et al., 1994). Our study hence focuses on the Extraversion scale.
Our data was collected from the collaboratively constructed Personality Databank 2 where the readers can vote if a book character is, among other aspects, introverted or extraverted. While the readers used codes based on the MBTI typology, they did not apply the MBTI assessment strategies. There was no explicit annotation guideline and the interpretation was left to readers' intuition and knowledge. 3 This approach of gold standard collection has several obvious drawbacks. First, the question is posed as dichotomic, while in reality the extraversion is a normally distributed trait in human population (Goldberg, 1990). Second, users can view the vote of previous participants, which may influence their decision. While we address both of these issues in our ongoing data collection project based on the Five-Factor Model, we consider them acceptable for this study due to the exploratory character of our pilot research.
We have collected extraversion ratings for 298 book characters, of which 129 (43%) are rather extraverted and 166 (56%) rather introverted. Rated characters come from a wide range of novels that the online users are familiar with, often covering classical literature which is part of the high school syllabus, as well as the most popular modern fiction, such as the Harry Potter series, Twilight, Star Wars or A Game of Thrones. A sample of the most rated introverts and extraverts is given in table 1. The rating distribution in our data is strongly Ushaped. The percentage agreement of voters in our data is 84.9%, calculated as: where k = 2 (introvert, extravert), N is the number of book characters and n the number of votes per character. Voters on the website were anonymous and cannot be uniquely identified for additional corrections. There is no correlation between the extraversion and the gender of the character. Our set of English e-books covered 220 of the characters from our gold standard. We have built three systems to assess the following: 1. Direct speech: Does the style and content of character's utterances predict his extraversion in a similar way as it was shown for living individuals? 2. Actions: Is the behavior, of which a character is an agent, predictive for extraversion? 3. Predicatives and adverbs: Are the explicit (John was an exhibitionist) or implicit (John shouted abruptly) descriptions of the character in the book predictive for extraversion?
In the next three sections we present the experimental settings and results for each of the systems.

Direct speech of fictional characters
The system for the direct speech resembles the most to the previous systems developed for author personality profiling, e.g. on stream of consciousness essays (Mairesse et al., 2007) or social media posts (Celli et al., 2013) and therefore provides the best opportunity for comparison between human individuals and fictional characters. On top of the comparison to previous research, we exploit the sense links between WordNet and VerbNet to extract additional features -an approach which is novel for this type of task.

Extraction and assignment of speech
We process the book text using freely available components of the DKPro framework (Gurevych et al., 2007). The most challenging task in building the direct speech data set is assigning to the direct speech utterance the correct speaker. We benefit from the epub format of the e-books which defines a paragraph structure in such a way, that only the indirect speech chunk immediately surrounding the direct speech can be considered: <p> John turned to Harry. "Let's go," he said.</p> Given the large amount of text available in the books we focus on precision and discard all utterances with no explicit speaker (i.e., 30-70% of the utterances, dependent on the book), as the performance of current systems on such utterance types is still fairly low (O'Keefe et al., 2012;He et al., 2013;Iosif and Mishra, 2014). Similarly, conventional coreference resolution systems did not perform well on this type of data and were therefore not used in the final setup. We adapt the Stanford Named Entity Recognizer (Finkel et al., 2005) to consider titles (Mr., Mrs., Sir...) as a part of the name and to treat the first person I as a named entity. However, identifying only the named entity PERSON in this way is not sufficient. On our evaluation sample consisting of A Game of Thrones and Pride and Prejudice books (the former annotated by us, the latter by He et al. (2013)), 20% of utterances with explicit named speaker were not recognized. Of those correctly identified as a Person in the adjacent indirect speech, 17% were not the speakers. Therefore we implemented a custom heuristics (Algorithm 1), which additionally benefits from the WordNet semantic classes of verbs, enriching the speaker detection by grabbing the nouns . With this method we retrieve 89% of known speakers, of which 92% is assigned correctly. Retrieved names are grouped based on string overlap (e.g. Ser Jaime and Jaime Lannister), excluding the match on last name, and corrected for non-obvious groupings (such as Margaret and Peggy). Algorithm 1 Assign speaker

Classification approach for direct speech
All speech utterances of one book character are represented as one instance in our system. We use the leave-one-out classification setup due to the relatively small dataset size, using the support vector machines (SVM-SMO) classifier, which performs well on comparable tasks (Celli et al., 2013). The classification is performed through the DKPro TC Framework (Daxenberger et al., 2014).
Lexical features As a bottom-up approach we use the 1000 most frequent word uni-, bi-and trigrams, 1000 dependency word pairs, 1000 character trigrams and 500 most frequent verbs, adverbs, adjectives and interjections as binary features.
Semantic features Since the top-down approach, i.e. not focusing on individual words, has been found more suitable for the personality profiling task on smaller data sets (Celli et al., 2013), we aim on capturing additional phenomena on a higher level of abstraction. The main part of our features is extracted on sense level. We use the most frequent sense of WordNet (Miller, 1995) to annotate all verbs in the direct speech (a simple but well performing approach for books). We then label the disambiguated verbs with their semantic field given in WordNet (WordNet defines 14 semantic classes of verbs which group verbs by their semantic field) and we measure frequency and occurence of each of these classes (e.g. cognition, communication, motion, perception) 5 . Additionally, we use the lexical-semantic resource UBY (Gurevych et al., 2012) to access the Word-Net and VerbNet information, and to exploit the VerbNet sense-level links which connects Word-Net senses with the corresponding 273 main Verb-Net classes (Kipper-Schuler, 2005). These are more fine-grained (e.g. pay, conspire, neglect, discover) than the WordNet semantic fields. WordNet covered 90% and VerbNet 86% of all the verb occurences.
On word level, we extract 81 additional features using the Linguistic Inquiry and Word Count (LIWC) tools (Pennebaker et al., 2001), which consists of lexicons related to psychological processes (cognitive, perceptual, social, biological, affective) and personal concerns (achievement, religion, death...) and other categories such as fillers, disfluencies or swear words 6 . Additionally, since emotion detection has been found predictive in previous personality work (Mohammad and Kiritchenko, 2013), we measure overall positive and negative sentiment expressed per character, using SentiWordNet (Esuli and Sebastiani, 2006) and NRC Emotion Lexicon (Mohammad and Turney, 2010) for the word lookup, inverting sentiment scores for negated dependency sub-tree given by the Stanford Parser.
Stylistic features Features of this group capture the syntactic and stylistic properties of the utterances of a character, disregarding the content. Starting from the surfacial properties, we measure the sentence, utterance and word length, including the proportion of words shorter than 4 or longer than 6 letters, frequency of each punctuation mark, 5 https://wordnet.princeton.edu/man/ lexnames.5WN.html 6 For complete overview refer to www.liwc.net and endings of each adjective as per Corney et al. (2002). On the syntax level we measure the frequency of each part of speech as well as the 500 most frequent part-of-speech bi-, tri-and quadrigrams, and the frequency of each dependency obtained from the Stanford Parser. We additionally capture the frequency of superlatives, comparatives and modal verbs, the proportion of verbs in present, past and future tense, and the formality of the language as per the part-of-speech-based formality coefficient (Heylighen and Dewaele, 2002), and measure the average depth of the parse trees.
Word embeddings as features Since vector space semantics has been beneficial for predicting author's personality in previous work (Neuman and Cohen, 2014), we use a pre-trained word vector model created by the GloVe algorithm (Pennington et al., 2014) on English Wikipedia. GloVe employs a global log-bilinear regression model that combines the advantages of the global matrix factorization and local context window methods. We assign the resulting 300-dimensional vectors to the words in character's direct speech, excluding stopwords, and calculate an average vector for each character. We calculate for each test character the cosine similarity to the mean vector of extravert, resp. introvert, in the training data, and to each character in the training set individually using the DL4J NLP package 7 . We consider both the final scalar outcome and the difference of each of the individual vector dimensions as features. .849  Similarly to previous research (Mairesse et al., 2007;Celli et al., 2013), the bottom-up word based approach is outperformed by top-down semantic approaches which employ a more abstract feature representation. As in previous work, LIWC features exhibit good performance. However, the highest performance is achieved employing the VerbNet verb classes with WordNet wordsense disambiguation. Also stylistic features contribute substantially to the classification despite the mixture of genres in our book corpus -especially frequencies of modal verbs and part-ofspeech ratios were particularly informative. The most predictive features from each group are listed in Table 3 together with their correlation merit (Hall, 1999), and compared with previous work in Table 4.  Pennebaker and King (1999) [2] Dewaele and Furnham (1999) [3] Mairesse et al. (2007)  In accordance with the experiments of Pennebaker and King (1999), we observe more frequent exclusions (e.g. without, but), hedging and negation expressed by introverts, and inclusion (e.g. with, and) by extraverts. Extraverts talk more in first person plural, use more back-channels and interjections, and talk more about aspects related to their body. Introverts show more rationalization through insight words and more factual speech using less pronouns.

Classification results on direct speech
Additionally, the semantic features in Table 3 confirm the broad psychological characteristics of both types in general, i.e., for introverts the rationalization, uncertainty and preference for individual or rather static activities, and for extraverts their spontaneity, talkativeness and preference for motion. Furthermore, we observe certain directness in extraverts' speech -note the predictive words fat and dirty and frequent descriptions of body functions.
Discussion Exploiting the links between lexicalsemantic resources (performing WordNet wordsense disambiguation and using VerbNet verb classes linked to the disambiguated senses) was particularly beneficial for this task. WordNet semantic fields for verbs alone are too coarsegrained to capture the nuances in direct speech, and experiments with fine-grained VerbNet classes without WSD resulted in noisy labels. We did not confirm the previously reported findings on emotional polarity -we observe that the genre of the books (e.g. love romance vs horror story) have blurred the subtle differences between individual characters, unfortunately the dataset size did not allow for genre distinctions. Furthermore, a perceived extravert in our case can be a pure villain (Draco Malfoy, Joffrey Baratheon...) as well as a friendly companion (Gimli, Ron Weasley...), while the evil extravert types are possibly rarer in the experiments on human writing, or are more likely to fit under the MBTI definition of extraversion than FFM facets. Another potential cause, based on the error analysis, is the different target of the same sentiment for extraverts and introverts. For example, the ngram "I fear" is highly predictive for an introvert in our data while extraverts would rather use formulations to imply that others should fear. Similarly to Nowson et al. (2005), we did not find any difference in the formality measure of Heylighen and Dewaele (2002). Neither we did in the complexity of sentences as per the parse tree depth and sentence length. It is probable that these aspects were also impacted by our broad variety of author style (F. Dostoyevsky vs J. K. Rowling). Our basic vector-based features carried no useful information in our case, in contrast to the personality research of Neuman and Cohen (2014). We observed that the factual content of the stories contributed to the character similarity measure more than the subtle personality differences.

Actions of fictional characters
While psycholinguists and consequenlty NLP researchers analyzed the relation between speech, resp. writing, and personality of an individual, psychologists often evaluate extraversion through behavioral personality questionnaries (Costa and McCrae, 1992;Goldberg et al., 2006). We hypothesize that similar behavior shall be predictive for extraversion of fictional characters as perceived by the readers.

Action extraction
For our purpose we define actions as the subject, verb and context of a sentence, where the subject is a named entity Person and the context is either a direct object in relation dobj to the verb or a first child of the adjacent verb phrase in a parse tree. After grouping the actions per character, the subject name is removed. For example, a sample of actions of the character Eddard Stark of Game of Thrones would be: X paused a moment, X studied his face, X changed his mind, X unrolled the paper, X said etc., visualized in Figure 1. We obtained 22 030 actions for 205 characters (102 E, 116 I), with on average 100 actions for E and 101 for I. Note that also actions for those characters who do not talk enough in the books (often first-person perspectives) could be used.

Action classification setup
In the system based on actions we use only a subset of the features described in 4.2. From the lexical features we focus on the 500 most frequent verbs and dependency word pairs. Semantic features are used the same way as in 4.2, profiting from LIWC, WordNet, Verbnet and the sentiment lexicons. Word embedding vectors for book characters are in this case computed by taking only the verbs into account rather than all content words. From the stylistic features we use the part-ofspeech bigrams and trigrams, verb modality and verb tense. Table 5 shows the performance of the classification models based on the protagonists' actions, using different feature groups. The overall performance is higher than for the direct speech model. .849 Table 5: Weighted precision (P), recall (R), F-score (F) and accuracy (A) for actions -in each line for a system using only the given group of features. WordNet stands for WordNet semantic labels, VerbNet setup uses the WordNet-VerbNet links. Highlighted F-scores differ from the majority baseline significantly (p <0.05), using an approx. randomization test.

Classification results on actions
Due to the lack of previous NLP experiments on this task, we compare our features to the actions measured in the International Personality Item Pool (Goldberg et al., 2006), frequently used personality assesment questionnaire ( Table 6).
The most predictive features of this model capture the activity and excitement seeking facets of extraversion. Stylistic features reflect the complexity difference of the verb phrases (John jumped vs. John thought about it), extraverts being characterized by plain verbs. Semantic features exhibit higher precision than stylistic ones. Sense-linked semantic classes of VerbNet demonstrate the preference of extraverts for being active and expressing themselves -they jump, fight, shout, run in and run out, eat and drink, see and hear and get easily bored. Extraverts in books also Extravert International Personality Item Pool: likes to party, feels comfortable around people, starts conversations, talks to many people, enjoys being a center of attention, makes friends easily, takes charge, captivates people, feels at ease with a company, is skilled in handling social situations Our experiment: bring (VN), consume (VN), contiguous location(VN), holding (VN), social (WN), motion (WN), emotion (WN) Leisure (LIWC), Home (LIWC), Family (LIWC), fight, march, care, take, jump, shriek, clear throat, bore, get to, come in, agree, hold, hear, inform, sell, come forward Introvert International Personality Item Pool: Doesn't talk much, stays in the background, has little to say, does not draw attention, has difficulties to approach others, is quiet around strangers, feels uncomfortable around others,does not show feelings, is a private person, waits to be lead Our experiment: snooze (VN), conceal (VN), wish (VN), stative (WN), creation (WN), walk, sleep, lay, know, maintain, expect, hope, find out, might, help, explain Table 6: Characteristic actions for extraverts and introverts as assessed in the IPIP personality questionaire, compared to our most informative features often bring or hold something. Introverts, on the other hand, seem to favor slow movements -while they are thinking, reflecting, creating, looking for explanations and find out solutions, they tend to lie down, sit or walk, eventually even sleep or snooze. The uncertainty typical for introverts is also notable in their actions, as they often hope or wish for something they might like to do. Additionally, semantic classes Social and Family, reported as correlated to extraversion by Pennebaker and King (1999) and not confirmed in our first model, became predictive in protaonists' actions.

Discussion
Also in this task, the VerbNet classes brought significant improvement in performance. The classification model based on actions outperforms not only the direct speech model, but also the state-of-the-art systems predicting authors' extraversion from the stream-of-consciousness essays (Mairesse et al., 2007;Celli et al., 2013;Neuman and Cohen, 2014). While surely not directly comparable, this result hints to the fact that the personality is easier to detect from behavior than from person's verbal expression. This would correspond to the findings of Mairesse et al. (2007), Biel and Gatica-Perez (2013) and Aran and Gatica-Perez (2013) on multimodal data sets.

Predicatives of fictional characters
Our third extraversion prediction system is subordinate to how fictional characters are described and to the manners in which they behave. We are not aware of a previous NLP work predicting extraversion using descriptive adjectives of the persons in question. We thus juxtapose the most predictive features of our system to the adjectival extraversion markers developed by Goldberg (1992).

Extraction of descriptive properties
In this setup we extract predicatives of the named entities PERSON in the books -relations amod (angry John) and cop (John was smart). As these explicit statements are very sparse in modern novels, we additionally include adverbial modifiers (advmod) related to person's actions (John said angrily). We extract data for 205 characters, with on average 43 words per character.

Classification setup
This system uses similar set of lexical, semantic and vectorial features similarly as in 5.2, this time with the focus on adjectives, nouns and adverbs instead of verbs. Stylistic and VerbNet features are hence not included, word vectors are as in 4.2. .849 Table 7: Weighted precision, recall, F-score and accuracy. Highlighted F-scores differ from the majority baseline significantly (p <0.05).

Discussion on errors
All our systems had issues with characters rated by less than five readers and with protagonists with low agreement. Other challenges arise from authorial style, age of the novel and speech individuality of characters (e.g. Yoda). Varied length of information for different characters poses issues in measuring normally distributed features (e.g. ratio of jumping verbs), being in shorter texts less reliable. Ongoing and future work on this task addresses the limitations of these initial experiments, especially the data set size and the gold standard quality. Extending the data will also enable us to examine different book genres as variables for the personality distribution and feature impact. It will be worth examining the relations between characters, since we observed certain patterns in our data, such as the main introvert character supported by his best friend extravert. Additionally, we want to verify if the system in Section 6 is overly optimistic due to the data size.

Conclusion and future work
Automated personality profiling of fictional characters, based on rigorous models from personality psychology, has a potential to impact numerous domains. We framed it as a text classification problem and presented a novel collaboratively built dataset of fictional personality. We incor-porate features of both lexical resource-based and vectorial semantics, including WordNet and Verb-Net sense-level information and vectorial word representations. In models based on the speech and actions of the protagonists, we demonstrated that the sense-linked lexical-semantic features significantly outperform the baselines. The most predictive features correspond to the reported findings in personality psychology and NLP experiments on human personality. Our systems based on actions and appearance of characters demonstrate higher performance than systems based on direct speech, which is in accordance with recent research on personality in social networks (Kosinski et al., 2014;Biel and Gatica-Perez, 2013), revealing the importance of the metadata. We have shown that exploiting the links between lexical resources to leverage more accurate semantic information can be beneficial for this type of tasks, oriented to actions performed by the entity. However, the human annotator agreement in our task stays high above the performance achieved. Considering that most of the sucessful novels were produced as movies, we cannot exclude that our annotators based their decision on the multimodal representation of the protagonists. In the future we aim on collecting a more detail and rigorous gold standard through gamification and expanding our work on all five personality traits from the Five-Factor Model and their facets, and ultimately extend our system to a semi-supervised model dealing with notably larger amount of data. We also plan to examine closer the differences between perceived human and fictional personality, and the relationship between the personality of the reader and the characters.