Mining Knowledge in Storytelling Systems for Narrative Generation

Storytelling systems are computational systems designed to tell stories. Every story generation system deﬁnes its speciﬁc knowledge representation for supporting the storytelling process. Thus, there is a shared need amongst all the systems: the knowledge must be expressed unambiguously to avoid inconsistencies. However, when trying to make a comparative assessment between the storytelling systems, there is not a common way for expressing this knowledge. That is when a form of expression that covers the different aspects of the knowledge representations becomes necessary. A suitable solution is the use of a Controlled Natural Language (CNL) which is a good half-way point between natural and formal languages. A CNL can be used as a common medium of expression for this heterogeneous set of systems. This paper proposes the use of Controlled Natural Language for expressing every storytelling system knowledge as a collection of natural language sentences. In this respect, an initial grammar for a CNL is proposed, focusing on certain aspects of this knowledge.


Introduction
Natural language is the most basic form of knowledge representation for the humans, because it allows communication and knowledge transmission. Natural languages provide an unbeatable expressivity for concept modelling and structuring. However, for the same reasons they are substantially complex for automatic processing.
A Controlled Natural Language (CNL) is an engineered subset of natural languages whose grammar and vocabulary have been restricted in a systematic way in order to reduce both ambiguity and complexity of full natural languages (Schwitter, 2010). CNL can be considered as a tradeoff between the expressivity of the natural languages, and the need for the orthogonality of a formal representation that can be handled by a computer.
Story generation systems are a form of expression for computational creativity. According to (Gervás, 2012), a story generator algorithm (SGA) refers to a computational procedure resulting in an artefact that can be considered a story. The term story generation system can be considered as a synonym of storytelling systems, that is, a computational system designed to tell stories.
Story generation system are faced with a significant challenge of acquiring knowledge resources in the particular representation formats that they use. They face a inherent difficulty when using formal languages in the detachment between the formulation of the needs in the real world and its representation in a formal construction.
In this context, the use of a CNL would provide the means for a quicker development of required resources in a format easier to write for human experts. So, the use of a CNL for codifying resources for storytelling systems might provide some advantage. If authors of storytelling systems were to develop the initial version of their resources in a commonly agreed CNL, and then develop the appropriate automated transformations to generate knowledge in their own preferred format, the same resources writ- ten in CNL might be of use to researchers developing different storytelling systems. Some previous studies on this matter can be found in (Schwitter, 2010), (Kuhn, 2009), (Power et al., 2009), (Davis et al., 2009), (Fuchs et al., 2008, and (Funk et al., 2007).
Particularly, the use of a CNL for knowledge representation has been documented previously (Kuhn, 2009), and (Barzdins, 2014). In both cases, these precedent works are quite convenient with respect to information extraction of and reasoning with the content of texts.
This paper proposes a model of a Controlled Natural Language understood as a means for mining knowledge from existing storytelling systems.
This process is part of a wider project which aims at the development of a collaborative environment involving several story generation systems.
The purpose of this environment is to establish a co-creation architectural model which allows the involved systems to take advantage of a shared knowledge base and use it for enhancing the quality of the generated texts. The architecture of this system is schematically depicted in the Figure 1. In the current stage of the model, the NLG step is used for translating the system-specific formalisms into the common CNL statements. In the final model, there will be an additional NLG step when generating the refined story.
The thrust of this approach is the use of a CNL as a shared representation for the various knowledge models of the different story generation sys- tems. Ideally, every custom representation should be translated into the common CNL for being subsequently employed. In this way, the story generation process will result in a build-up of contributions from different systems. For example, one of the depicted systems, labeled as Affinities (Méndez et al., 2016), is specialized in characters interactions and affinities, but it lacks in a deeper enhancement of the narrative discourse, that can be compensated by STellA (León and Gervás, 2011).
The drafted architecture aims at several objectives. Firstly, it intends to establish a collaborative model that allows the free exchange of knowledge between the different storytelling systems in order to develop an iterative improvement process of literary creation. Beside this objective, it promotes the development of a knowledge representation model for creating a common, system-agnostic knowledge base that can be feed in the future with the outcomes of new storytelling systems, without the need to adapt their knowledge representation models.
The scope of this paper relates only to the Natural Language Generation step, as depicted in the Figure 2.
2 Related Work

Natural Language Generation
The process of Natural Language Generation has been clearly defined (Reiter et al., 2000), as well as its six distinctive tasks: • Content Determination: in which the generation system makes a decision concerning the information that will be taken into account for generating the text.
• Discourse Planning: this task involves decisions about how the text should be globally articulated.
• Lexicalisation: in which the generation system makes a choice of the particular words and phrases it considers suitable to convey the semantics of the selected information, in the given natural language and its context.
• Aggregation: this task involves decisions concerning the composition of the generated sentences to form a natural discourse.
• Referring Expression Generation: this task involves the determination of the properties of a given linguistic element, which to be used when the element is mentioned again.
• Surface Realisation: the last task that reviews the text for checking that it presents syntactically, morphologically and orthographically correct sentences in the corresponding natural language.
Many different architectures have been proposed for NLG systems, reflecting the range of different applications and its purposes. Basically, there are two main models of the NLG process: the Abstract Generation System (Bateman, 1997) and the Abstract Referential Model, an outcome of the Reference Architecture for Generation Systems (RAGS) (Cahill et al., 2000) which is aiming at standards for NLG architecture.

Knowledge Representation in Storytelling Systems
From an historical perspective, formal languages have been the most common way of knowledge representation. The reason for using formal languages is simplicity; they have a well-defined syntax, an unambiguous semantics and they are very convenient for automated reasoning. Particularly, in the field of automatic story generation, there is an abundance of examples of this kind. TALE-SPIN (Meehan, 1977a) was one of the earlier story generators. It was a problem solver, top-down and goal-directed story generation engine. TALE-SPIN generated stories about the inhabitants of a forest taking a collection of characters with their corresponding objectives as inputs. TALE-SPIN found a solution for those characters goals, and wrote up a story narrating the steps performed for achieving those goals.
TALE-SPIN knowledge representation relied on Conceptual Dependency Theory (Schank and Abelson, 1975). It used a set of primitives for representing the problem domain. All its knowledge was expressed as a formal language.
Minstrel (Turner, 1993) was a story generation system that told stories about King Arthur and his Knights of the Round Table. Each story was focused on a moral, which also provided the seed for developing the story.
The knowledge representation in Minstrel used an extension of a Lisp library called Rhapsody. Rhapsody was a tools package for AI program development that provided the user with ways to declare and manipulate simple frame-style representations, and a number of tools for building programs that use them.
Mexica (Perez y Perez, 1999) was developed as a computer model whose purpose was studying the creative process. It generated short stories about the early inhabitants of Mexico. Mexica was a pioneer in that it took into account emotional links and tensions between the characters as a means for driving and evaluating ongoing stories.
Mexica used several knowledge structures for supporting its storytelling model: An actions library, a collection of stories for inspiring the new ones, and a group of characters and locations. The generation process also took several steps, in which data were progressively transformed.
Mexica knowledge management involves several concerns in order to provide a high-quality outcome, in terms of literary production. Its knowledge base included several types of structures for representing things like characters relationships, actions, emotional links, and a literary base composed of previously generated stories.
Brutus (Bringsjord and Ferrucci, 2000) was a system that generated short stories using betrayal as leitmotiv. It had a rich logical model for representing betrayal. This feature allowed it to generate complex stories. A very innovative aspect of Brutus was that it considered the existing body of knowledge about literature and grammar for generating stories.
Brutus structured its knowledge in several layers, including a grammar specific part. So, the process of converting the plot into the final output was carried out by the application of a hierarchy of grammars: story grammars, paragraph grammars and sentence grammars. This hierarchical procedure led to define every story as a sequence of paragraphs which in turn were sequences of sentences.
MAKEBELIEVE (Liu and Singh, 2002) was a short fictional story generation system that used common sense knowledge to generate stories. The user provided a story about a character as initial seed, and then MAKEBELIEVE attempted to continue that story by inferring possible sequences of events that might happen to that character. The system used common sense knowledge about causality and how the world works, mined from the Open Mind Common Sense knowledge base .
STellA (Story Telling Algorithm) ) is a story generation system that controls and chooses states in a non-deterministically generated space of partial stories until it finds a satisfactory simulation of events that is rendered as a story.
STellA uses a custom representation for the knowledge it needs. It manages several different structures, including a matrix representation of the world in which characters live, and a set of rules for evaluating the range of results associated to the actions.

Use of CNLs in Storytelling Systems
There is not a long record of application of CNLs in the context of storytelling.
Inform (Reed, 2010) was a toolset for creating interactive fiction. As from version 7, Inform provided a domain-specific language for defining the primary aspects of an interactive fiction like the world setting, the character features, and the story flow. The provided domain-specific language used a CNL, similar to Attempto Controlled English (Fuchs et al., 1998).
The StoryBricks (Campbell, 2011) framework was an interactive story design system. It provided a graphical editing language based on Scratch (Resnick et al., 2009) that allowed users to edit both the characters features and the logic that drove their behaviour in the game. By means of special components named story bricks, users could define the world in which characters live, define their emotions, and supply them with items. Story bricks were blocks containing words to create sentences in natural language when placed together. They served to define rules that apply under certain conditions during the development of the story in the game.
In the extended ATTAC-L version (Broeckhoven et al., 2015), authors introduced a model which combined the use of a graphical Domain Specific Modeling Language (DSML) for modelling serious games narrative, ATTAC-L (Broeckhoven and Troyer, 2013), with a CNL to open the use of the DSML to a broader range of users, for which they selected Attempto Controlled English (Fuchs et al., 1998). It allows describing things in logical terms, predicates, formulas, and quantification statements. All its sentences are built by means of two word classes: function words (determiners, quantifiers, negation words, etc.) and content words (nouns, verbs, adverbs and prepositions). The main advantage is that Attempto Controlled English defines a strict and finite set of unambiguous constructions and interpretation rules.

Conceptual Basis
Towards the definition of a shared representation, we will review previously the main aspects of the knowledge involved in storytelling systems.
Narrative has different aspects in terms of representation , each of which has a different natural structure. Every story generation system focuses in a subset of these aspects and holds them by means of a certain set of data structures that represents the system knowledge. For example, Brutus (Bringsjord and Ferrucci, 2000) and Minstrel (Turner, 1993) emphasised the thematic aspect of the narrative, that is the central topic a text treats. Brutus main theme was betrayal, while every Minstrel story started on a moral that was used as the initial seed.
Still on this subject, another relevant conclusion mentioned by ) is that the same information may be represented through different data structures without affecting its essence, or a data structure can be extended for representing additional types of information. For example, Brutus (Bringsjord and Ferrucci, 2000) used a specific representation for representing the betrayal. Brutus was developed using a logic-programming system called FLEX, which is based on the programming language Prolog. Its knowledge about betrayal was modelled by a set of statements in FLEX, called frames. Every frame formalized the essential characteristics of betrayal: the betrayer, the betrayed, the locations, the actions involved, etc. Mexica (Perez y Perez, 1999) used a wider representation of the relationships between the characters, not specifically focused on betrayal. Relations in Mexica are of two types: emotional links and tensions. Emotional links represent affective reactions between characters. They are defined in terms of three attributes: type (love or friendship), valence (positive or negative) and intensity. Tensions represent if there is a conflict between two characters. It is defined by a type (of conflict) and a state (on or off).
In both examples the same narrative aspect is represented differently in every system, but it can be conceptually identified as a shared concern.

Dimensions of the narrative
For the purpose of this paper, we are considering a previous work of ( , who analysed the most relevant classifications of the story generation systems according to the knowledge they managed, and proposed their own list of suitable dimensions obtained from the different aspects of a narrative: • The discourse sequence aspect: a sequential discourse of conceptually conveyed items.
• The simulation aspect: a representation of the activity of agents in terms of actions, interac-tions, mental states, and movement between locations.
• The causal aspect: a structured representation of causal relations between elements in the story.
• The intentional aspect: a representation of the motivations of agents.
• The thematic aspect: a representation of the theme of parts of the story.
• The emotional aspect: a representation of the emotions involved in or produced by the story.
• The authorial aspect: a representation of the intentions of the author.
• The narrative structure aspect: representations of the story in terms of narratological concepts of story structure.

Considerations for grammar definition
In addition to these semantic aspects, the proposed CNL grammar definition should meet the common requirements expressed by (Kuhn, 2010): • Concreteness: CNL grammars should be fully formalized and interpretable by computers.
• Declarativeness: CNL grammars should not depend on a concrete algorithm or implementation.
• Lookahead Features: CNL grammars should allow for the retrieval of possible next tokens for a partial text.
• Anaphoric References: CNL grammars should allow for the denition of nonlocal structures like anaphoric references.
• Implementability: CNL grammars should be easy to implement in different programming languages.
One of the major challenges that faces the target representation is to provide a unambiguous formalism while keeping Natural Language expressiveness.

A proposed representation for the narrative dimension
The dimension considered firstly in the CNL grammar is the narrative aspect. It focuses on identifying the underlying structure of the narrative, understood as the framework that supports the inner consistency of the story. From a procedural point of view, the narrative aspect defines the actions performed in order to enhance this skeleton, providing a progressively enriched narrative as a result. This dimension can be traced in the knowledge representation of several of the referred systems (Meehan, 1977b;Dehn, 1981;Turner, 1993;Perez y Perez, 1999; As noted by , a different fundamental aspect of narrative is the fact that it can be analyzed in terms of recurring structures that articulate its main ingredients into abstractions that allow its description at a higher level than simple enumerations of events. Along this same line, Propp work (Propp, 1968) is an effort for systematizing the representation of this aspect.
Lang works provided a very interesting step forward to this matter (Lang, 1999) by developing a declarative model for simple narratives. This model described stories in terms of a sequence of events, trying to provide a combined response to the two traditional approaches: declarative and procedural. In the declarative approach the generated text fits a structure that has been defined before (Rumelhart, 1975). By contrast, in the procedural approach, the text was modelled according to a creation process that emulated human authors (Lebowitz, 1985;Turner, 1993).

Story structure
The proposed structure for representing stories is conceptually based on previous work (Lang, 1999), in the sense of a story is composed by a setting and an episode list, which both have temporal intervals associated with them.
Every episode can be expressed as a N-tuple composed of four elements: • An initiating event • An emotional response on the part of the protagonist • An action response on the part of the protagonist • An outcome or state description which holds at the conclusion of the episode

Vocabulary definition
The vocabulary provides the terms for sustaining the conceptual model of every specific dimension. Each dimension can be considered as a domain itself, understood as a unit composed by a cohesive set of interconnected concepts. These concepts are provided by a collection of domain terms and their relations. So, they are the building units for expressing the knowledge relevant for the considered dimensions. In order to formalize this structure, the vocabulary is defined as follows: • A term designates a significant knowledge entity that can be represented by a common noun or a noun phrase.
• A name designates unambiguously a significant entity that represents a single thing. It is typically a proper name, referring a character, a place, an object, etc.
• A verb designates a relationship, situation, or action involving one or more terms or names. The verbs are both the richest and the most complex elements of the vocabulary. A verb can be expressed in an active or a passive form. Verbs can also be qualified by modal verbs, so they can communicate probability, ability, permission, obligation and advice.
• An adverbial serves for expressing the circumstances involving the action defined by the verb. It is an optional part of the sentence.

Grammar definition
The expression Subject + Verb defines an attribution or a state related to the Subject, that is a placeholder for a Term or a Name. The combination Subject + Verb + Object defines a semantic relationship and has two placeholders filled by Terms/Names. The particular case of the verb to be must be considered as a typical expression for building descriptions.
The combination Subject + Verb + [Adverbial] + Object defines an action performed over an object. The action defined by the verb can be better put into context by means of adverbials. These can be used for expressing the circumstances in which action takes place.
Sentences can be combined in order to create compound sentences or subordinate clauses.
The sentence will be expressed in a declarative manner. For example, the following statement shows a complete case: The main character finds accidentally a clue that allows him to finish his research.
The CNL grammar defines a collection of syntax rules and constrains for representing the knowledge as statements. It presupposes the existence of a vocabulary because it addresses the terms and verbs defined in the vocabulary. The general structure of every statement is composed of four parts: the initiating event, an optional part that expresses a change in the subject emotions, another optional part that expresses the actions taken by the subject, and a ending sentence that expresses the outcome.
So far, we have presented the general pattern of the grammar. It is a set of rules which allow going from a simple to a reasonably complex structure. This last point can be reached by means of connectives. The noun phrases can also be combined and qualified using different quantifications and prepositional phrases, but always with the certainty that it will produce sentences that are grammatically correct.

A proposed representation for the simulation dimension
Another relevant aspect of narrative is the representation of characters, their behaviour, and the expression of their mental state, their relations with one another, their motivations, and their beliefs. The simulation aspect has been frequently highlighted as the leitmotiv for the representation of narratives in some approaches to story generation (Lebowitz, 1985;Bringsjord and Ferrucci, 2000). Such approaches usually focus on representing characters and rules that may govern their behaviour and interaction.

Modelling the affinity
The simulation aspect refers to the characterization of the persona in terms of the interaction between each other. That is, this aspect covers a wide scope that ranges from the definition of characters attributes and traits, to the delimitation of their affinities. Naturally, this also relates to the way in which the characters interact with each other and the actions they perform motivated by the result of such interactions. The affinity aspect have been studied by several authors and systems (Imbert and De Antonio, 2005;Si et al., 2006;Méndez et al., 2016). The present work is related to the system developed by (Méndez et al., 2016). Usually, the authors apply an affinity factor for modelling the way in which social interaction affects the behaviour of the characters with each other. In other cases, affinity is affected by other factors, such as social obligations and characters goals. An additional aspect of affinity to keep in mind is that it is not symmetrical. Given two characters, their mutual affinity is likely to be dissimilar.
There are several possible ways for expressing the affinity between two characters. A first option is the use of a collection of symbolic values that allow reasoning about them and the ongoing simulation, but that difficult the operation. On the other hand, the use of numeric values makes easier operating with them, but hinders understanding the evolution of the simulation.
With a view to representing the affinity in terms of Natural Language, the simpler choice is to use a collection of adjectives that represent a range of numeric values.
In the referred model (Méndez et al., 2016), authors have modelled additionally four levels of affinity according to four different kinds of affinity: foe (no affinity), indifferent (slight affinity), friend (medium affinity) and mate (high affinity). These values can be suitable for expressing it in a first approach.

Vocabulary definition
As stated previously, the vocabulary provides the terms for defining the model of the corresponding dimension. In the domain of the simulation, the vocabulary is defined as follows: • A term designates a significant knowledge entity that can be represented by a common noun or a noun phrase. It is exactly as defined in the context of narrative structure.
• A name designates unambiguously a significant entity that represents a single thing. It is exactly the same entity as in the case of the narrative structure.
• A verb states a feature, a trait, or an action involving one or more terms or names. The verb in the simulation aspect provides basically a definition or an action performed by a character

Grammar definition
The simulation aspect is defined in terms of characters' traits and interactions. These special features need a specific way of being represented.
In this regard, the CNL created for expressing all these dimensions will contain basically sentences for describing traits and attributing features. It will also be a language for describing actions and interaction. So, the grammar for formalizing it is composed of expressions of the following kind: Subject + Verb + Attribute or: Subject + Verb + Object.
In the first case, the expression reflects the definition of a trait or a feature. The attribute will be represented by means of a term. The verb will typically be the to be and to have.
In the second case, the verb expresses an action. The character, that is the subject, performs some action that affects something or someone. So, the object can be either a term or a name.
This last type of expression can also be used for defining the affinity between characters, so there will be sentences like: John is a friend of Mary.

Conclusions and future work
This paper proposed the application of a CNL for eliciting and exchanging knowledge between story generation systems as a means of collaborative generation of stories. It also discusses a model for generating this CNL automatically from different knowledge representation formalisms. As explained above, there have been precedents of the use of CNL in the interactive storytelling domain with satisfactory results.
The aim of the proposed representation is to help bridging the variety of knowledge representation in a simple and formal way. The proposed syntax has been defined by a formal grammar but the resulting expressions keep a human-friendly nature.
In this paper, the developed work is centred on two dimensions of the knowledge: the story structure dimension and the simulation dimension. This is just one part of the needed multi-aspectual representation. As mentioned above, there are some other dimensions that must be addressed in future versions of the CNL: the authorial aspect, the emotional aspect, the intentional aspect and the theme aspect.
The future work will be focused on completing the set of the grammar generation rules for expressing these remaining aspects of knowledge involved in storytelling. This work will provide a completely expressive representation that hold co-creation between storytelling systems.
Benchmarking this work can really be complex, and will probably involve a shared effort with other research groups. So, we are working on proposing a shared task in which to compare the quality of stories generated by different systems using the same initial knowledge base. Collaborators will be provided with a grammar definition of a CNL that represents the narrative aspects mentioned in the previous section, along with a set of initial situations written using this grammar from which to generate different stories using the same CNL. They will be required to use as much information as possible in order to generate rich stories that cover one or more of the previously described narrative aspects. These stories must be expressed in the same CNL used to describe the initial situation so that, hypothetically, the output of a system might feed another system in order to provide more details about some of the aspects that may have been left uncovered by previous generators, following a co-creation process where a system can strengthen the weaknesses of another. The outcome of this collaborative process is expected to provide the means to develop an enhanced story creation model.

Aknowledgements
This work was partially supported by the projects WHIM (611560) and ConCreTe (611733), funded by the European Commission under FP7, the ICT theme, and the Future Emerging Technologies (FET) program.