Finding the Middle Ground - A Model for Planning Satisficing Answers

To establish sophisticated dialogue systems, text planning needs to cope with congruent as well as incongruent inter-locutor interests as given in everyday di-alogues. Little attention has been given to this topic in text planning in contrast to di-alogues that are fully aligned with anticipated user interests. When considering dialogues with congruent and incongruent interlocutor interests, dialogue partners are facing the constant challenge of ﬁnding a balance between cooperation and competition. We introduce the concept of fairness that operationalize an equal and adequate, i.e. equitable satisfaction of all interlocutors’ interests. Focusing on Question-Answering (QA) settings, we describe an answer planning approach that support fair dialogues under congruent and incongruent interlocutor interests. Due to the fact that fairness is subjective per se, we present promising results from an empirical study (N=107) in which human subjects interacted with a QA system implementing the proposed approach.


Introduction
For building dialogue systems that cope with contradictions and individual interests of dialog partners, text planning is required to process incongruent and congruent interests of interlocutors. So far, research on dialogue systems focusses on supporting dialogues that are fully aligned with anticipated user interests, e.g., (Hovy, 1991;Grosz and Kraus, 1996;Moore and Paris, 1993;Lochbaum, 1998;Rich and Sidner, 1997), and, thus, maximizing cooperativeness (Bunt and Black, 2000, 191 p. 5). Few approaches exist that investi-gate text planning with pure conflict, e.g., (Jameson et al., 1994;Hadjinikolis et al., 2013;Black and Atkinson, 2011;Prakken, 2006). When considering dialogues with congruent as well as incongruent interlocutors interests, dialogue partners are facing the constant challenge of finding a balance between cooperation and competition (Parikh, 2010). We introduce the concept of fairness that operationalize an equal and adequate, i.e. equitable satisfaction of all interlocutors' interests (Oxford Dictionaries, 2016). Focusing on Question-Answering (QA) settings, we describe an answer planning approach that support fair dialogues under congruent and incongruent interests of interlocutors. Due to the fact that fairness is subjective per se, we present results from an empirical study in which human subjects interacted with a QA system in various dialogue settings. When determining appropriate answers in text planning, approaches range from (1) wrong answer avoidance concepts technically checking the correctness of answers, e.g., Dong et al. (2011), and (2) opponent models in persuasion dialogues for choosing most suitable arguments, e.g., Hadjinikolis et al. (2013), to (3) the prediction of emotions of interlocutors to generate answers, e.g., Hasegawa et al. (2013). Here, related work is relevant that focuses on the determination of appropriate answers by processing concepts like users' intentions (e.g., Levelt (1993)), desires (e.g., Rao & Georgeff (1995)), preferences (e.g., Li et al. (2013)), objectives (e.g., Schelling (1960)) and goals (e.g., Traum et al. (2008)) which we will hereafter subsume under the term motives. Motives refer to objectives or situations that interlocutors would like to accomplish, e.g., to find the best price when shopping. According to the belief-desire-intention model, motives can be described as desires in the sense of a motivational state (Georgeff et al., 1998). Motives do not in-volve the mandatory purpose of being recognizable by other participants; so they are equivalent with the concept of intentions in (Levelt, 1993). Regarding the processing of congruent and incongruent motives, existing approaches rather focus on motives of single interlocutors or on joint motives, e.g., Paquette (2012), Li et al. (2013). In the following, the aggregation of congruent and incongruent interlocutor motives in dialogues will be described as mixed motives. In this work, we propose a model that formalizes answer planning as psychological game (Bjorndahl et al., 2013) embedded in text planning approaches (Mann and Thompson, 1986;Moore and Paris, 1993) for creating dialogues perceived as fair by all interlocutors. Since traditional formalization of motives by means of utility functions is not sufficient to handle complex interactions as given in the considered dialogue setting (Bjorndahl et al., 2013), psychological games enrich classical game settings with user models, i.e. in our case explicit representations of mixed motives. One appeal of the model is the consideration of answer planning as psychological game that lifts the process of finding appropriate answers from the short-term linguistic level to the long-term motive level in contrast to other approaches (van Deemter, 2009;Stevens et al., 2015). Interlocutors do not have preferences for answers, but try to satisfy motives. So, we assume that this approach enables a more sophisticated simulation of human behavior in mixed motive interactions as well as the establishment of "cooperativeness in response formulation" (Bunt and Black, 2000, p. 5) for creating dialogues perceived as fair. By exemplifying the model within a QA system as natural language sales assistant for conducting sales dialogues, we were able to evaluate the proposed approach in an empirical user study (N=107) in terms of perceived fairness of created dialogues with promising results.

Planning Answers given Mixed Motives
Adopting a computational pragmatics perspective, we intend to compute relevant linguistic aspects of answers based on contextual aspects given by mixed motives (Bunt and Black, 2000, p. 3).
When searching for answers that support an equitable satisfaction of mixed motives during dialogue, Ω represents the solution space with poten-tial answers. An objective function f : Ω → R assigns values to all answers x ∈ Ω for representing their potential in satisfying motives of interlocutor i ∈ I. Of course, all interlocutors I prefer answers x that satisfy best their motives; . So, the goal would be to find an answer x ∈ Ω with highest satisfaction of motives f (x) of interlocutor i ∈ I in the sense of an optimal solution x * ; i.e. f (x * ) = max{f (x)|x ∈ Ω}. But, in order to achieve fair outcomes regarding an equitable, i.e. equal and adequate satisfaction of mixed motives, this definition is not sufficient. First, decision making takes places in the context of social dialogue interaction, i.e. answers have to be selected based on multiple objective functions since motives of all interlocutors i ∈ I shall be satisfied; f i : Ω → R. For capturing the aspect of equal motive satisfaction, the potential of answers has to be represented absolutely and relatively. In other words, the performance of an answer in satisfying motives of an interlocutor i ∈ I is combined with its performance in satisfying motives of counterparts −i ∈ I; max{f i,-i (x)|x ∈ Ω}. Second, the aforementioned conflict between cooperation and competition needs to be solved adequately. Since it is impossible to find an answer satisfying all motives of all interlocutors at any time in the dialogue, we search for a compromise in form of a solution i.e. an answer x + with a minimum quality s so that f (x + ) ≥ s. Adopting the concept of satisficing by Simon (1956), an approach that attempts to find the best alternative available in contrast to optimal decision making, the goal is to find an answer x ∈ Ω with highest sufficient satisfaction of motives f (x) ≥ s of all interlocutors I in the sense of a satisficing solution

Model for Planning Satisficing Answers
To capture these issues, we defined a model for planning satisficing answers in dialogues with mixed motives. In the considered setting, a user with motives poses questions to a QA system that takes the role of a proxy for indirect interlocutors, e.g., retailers in online shopping scenarios.
The QA system adopts their motives and develops strategies to satisfy them. Adopted motives as well as user motives that are anticipated by the system represent mixed motives in the dialogue. Task of the system is to process these mixed mo-tives with the objective to create a dialogue that is perceived as fair by all interlocutors after a finite number of question answer pairs. As full satisfaction of all motives of all interlocutors at any time in a mixed motive dialogue is not possible, the QA system has to find a compromise, i.e. it has to plan answers that satisfice mixed motives during dialogue. Let us start by describing an example dialogue between a customer and a retailer in a shopping scenario: Q: Is the range of this wifi router appropriate for a house with 3 floors? A: In case of 3 floors, I would recommend an additional wifi repeater that got very good feedback by other customers. You can buy both router and repeater as a bundle with 15% discount.
In this dialogue snippet, the customer intends to get comprehensive product information regarding the wifi router; the retailer also wants to satisfy informational needs of the customer to establish excellent services. Beyond these congruent motives, the retailer wants to increase revenue and to raise sales figures. A balance between mixed motives is found by giving information regarding the wifi router as well as preferences of other customers followed by a discounted bundle offer.
In order to implement this kind of behavior into dialogue systems, the model for planning satisficing answers separates linguistics from conceptual non-linguistic aspects (Traum and Larsson, 2003;Allen et al., 2001) and consists of three main modules: linguistic module, mapper and mixed motive module (cf. Fig. 1). The linguistic module takes care for handling user questions as input as well for generating answers as output. Essential components of the linguistic module are the linguistic intention model and flexible text planning technologies. For the latter, we apply text plans according to the Rhetorical Structure Theory (Mann and Thompson, 1986) in form of plan operators (Moore and Paris, 1993) for generating answers. Each plan operator consists of a single compulsive part, called nucleus, that is related with diverse optional text segments, mentioned as satellites. We assume that beside supporting the effect of the nucleus, satellites represent an opportunity to satisfice mixed motives during dialogue. Satellites are linked with entities of the linguistic intention model, means linguistic intentions that capture the intended effects, i.e. functions of satellites within answers (Grosz and Sidner, 1986). By means of second module -the mapper -linguistic intentions are mapped onto motives and vice versa (cf. Fig. 1). Therefore domain-specific knowledge about correlations between linguistic intentions and mixed motives is required that is induced by a domain configurator and has to be derived empirically. Last, the mixed motive module combines an explicit representation and situated processing of mixed motives (Cohen and Levesque, 1990) with a game-theoretical equilibrium approach (Nash, 1951) to establish a psychological game setting (Bjorndahl et al., 2013) (cf. Fig. 1). Our approach operates by assuming that interlocutors are rational. That means they act strategically and purposively in pursuit of their own motives that they try to maximally satisfy. Therefore, we assume that game theory is an adequate prospect to deliver the analytical tools for planning answers in the context of mixed motives. In game theory literature, equilibrium concepts are widely applied, e.g., Nash equilibrium (Nash, 1951). A Nash equilibrium is an outcome that holds because no involved actor has a rational incentive to deviate from it, i.e., the final result is "good enough" for all actors in the sense of a happy medium. Adapted to this work, this refers to a satisficing combination of motives at a particular time in the dialogue, that is good enough for planning an answer that supports equitable satisfaction of mixed motives.

Concepts
From a conceptual perspective, the model uses several core entities. First, we have players p ∈ P that represent interlocutors I. Players have domain-specific motives m for participating in the dialogue. For each player p ∈ P , we assume a M otiveSet that consists of individual motives, IndM , as well as of motives the player, i.e. the interlocutor anticipates from counterparts, AntM .
Mixed motives M M are represented by the nonredundant aggregation of 1. . . n M otiveSet of players p ∈ P in the dialogue.
All motives m ∈ M M are operationalized by means of real-valued weights for each player covered by a weight vector − −−−− → W eight m . Motives are formed earlier and persist during dialogue, but players deliberate about weights of motives continuously (Bratman, 1987). The achievement and thereby satisfaction of motives is supported by linguistic intentions li ∈ LI that are satisfied by satellites sat that are offered by plan operators and integrated into an answer. That means motives are achieved, if answers were given, that contributed to satisfaction of these motives.

Algorithm and example
For introducing the proposed approach, we will give an example course of satisficing answer planning starting with user question and ending with system answer. The description of the process will be supported by a model view marked with step numbers in Fig. 1 as well as by an algorithmic view in Alg. 1. In the example, we apply domainspecific knowledge that was derived empirically in the retailing domain. Although, in literature review, customer and retailer motives in sales dialogues were specified. Combinations of these motives were analyzed in simulated sales conversations between real retailers (N=3) and subjects acting as customers (N=12). Recorded as video files, conversations and identified motives were validated in a web-based user study (N=120) regarding their naturalness and relevance. Sales conversations were transcribed, aggregated to a text corpus and analyzed regarding question and answer structures. So, the domain-specific knowledge representation used in the example bases on results of this empirical analysis and covers all core model concepts introduced before: a mixed motive model with empirically derived default weights consisting of 19 customer and 4 retailer motives (cf. Tab. 1); 39 question and 33 answer schemata (McKeown, 1985), 31 plan operators (Moore and   (Hobbs, 1978;Hovy, 1993;Mann and Thompson, 1986), and exemplary product information. Imagine a sales conversation regarding consumer electronics between customer and retailer represented by player (p a ) and player (p b ). Sets of motives by players are equal regarding the motives included but differ in weights of individual and anticipated motives by players (cf. Tab. 1).
The customer poses a question concerning products with a specific feature: "How many tablets offer the wifi features 802. 11A, 802.11B, 802.11G, 802.11n?" Based on the identified question schema as well as the determined communicative function of the question, a dialogue system that instantiates the proposed model selects an appropriate plan operator (cf. Fig. 1, step 1 & 2).
Overall objective is to determine set S + out of set S, that consists of satellites that -besides supporting the effect of the nucleus -contribute to satisficing mixes motives of customer and retailer during dialogue (cf. Alg. 1). According to (Grosz and Sidner, 1986;Moore and Paris, 1993), satellites are linked with linguistic intentions; i.e. they fulfill certain functions regarding the overall dialogue. Set S is sent to the linguistic intention handler that specifies the Satisf actionSet (cf. Fig.  1, step 3 and Alg. 1, line 1-4). This set covers linguistic intentions that can be satisfied by satellites of set S (cf. Tab. 2): Figure 2: Plan operator NUMBER OF PRODUCTS 3.2.2 Mapping linguistic intentions onto mixed motives Next, linguistic intentions have to be mapped onto motives. The m:n correlation between linguistic intentions and motives (Moore and Paris, 1993) is domain-specific, has to be specified empirically and is induced by the domain configurator (cf. Fig. 1, step 5). Each motive is supported by a set of linguistic intentions that contribute to the achievement of this motive (cf. Fig. 3). On the other hand, each linguistic intention can support the achievement of several motives. By processing RelevanceSet = {mQ, mR, mIPD, mHCS, mACB, mICR, mI, mSCD, mPB, mEU, mFP, mED, mHLP, mSI, mSP, mHLC}

Satisficing mixed motives
Having identified the RelevanceSet, we now intend to identify a satisficing combination of the involved motives. Therefore, the mapper sends the RelevanceSet to the mixed motive model handler for specifying the Satisf icingSet that consists of motives that (1) are sufficiently interesting for all interlocutors (i.e. weighted positively), and (2) have preferably low conflict potential (i.e. small differences in player weights) (cf. Fig. 1, step 6). Satisficing mixed motives is considered as multiplayer non-zero-sum game that is played for infinitely many rounds, more precisely pairs of user questions and system answers. In each round of the game, it has to be decided which motives of the RelevanceSet are selected as trigger for planning an answer that supports the creation of dialogues perceived as fair by all interlocutors. The equilibrium identifier specifies strategy sets S p = {s 1 . . . s n } for all players P by generating the power set of the RelevanceSet (cf. Fig. 1, step 7 and Alg. 1, line 9). Each of the 137 resulting strategies s = {m 1 . . . m n } represents a possible combination of motives or the empty set and is measured by a normalized local payout for each player based on weights of involved motives.
LocalP ayoutp a ,s 18 = 0.1280; LocalP ayoutp b ,s 18 = 0.0090 Strategy sets of players are identical regarding types of covered strategies, but differ in local payouts that can be expected by players when playing this strategy as shown in eq. (7). As players prefer those strategies that provide high local payouts, the equilibrium identifier identifies strategies s * ∈ S p for each player that represent best answers regarding the behavior of counterparts − → s -p : LocalP ayout(s * , − → s-p) ≥ LocalP ayout(s, − → s-p), ∀s ∈ Sp Best answers of players in the sense of highest local payouts are aggregated to 17 strategy profiles, each a vector consisting of two strategies one for each player (cf. Alg. 1, line 10-15): − → s = {s x , s y }; s x ∈ S pa , s y ∈ S p b .
Next, strategy profiles are selected that meet the Nash equilibrium condition, i.e. those strategy profiles exclusively cover strategies that represent mutual best answers of players (cf. Alg. 1, line 16-20): ∀s ∈ − → s1 . . . − → sn In our example, we find two Nash equilibria. Those two strategy profiles represent best answers for the player p as well as the whole group of players P in the sense of a solution with minimum quality. No player has an incentive to deviate from those strategy profile because then its local payout would decrease. With − → s = {s 36 , s 36 }, we select the non-pareto-dominant option for finding the strategy profile with the lowest difference in local payouts following the idea of the model to create a balance between mixed motives. With each answer planning, players generate local payouts that are added during the course of dialogue to global payouts. Instead of gaining high global payouts, the objective of the model is to balance payouts of players during dialogue or to approximate them in case of drifting apart. We assume that similar global payouts of players can be regarded as evidence for satisficed mixed motives. Based on the selected strategy profile, involved motives are aggregated to the Satisf icingSet = {m R , m ICR } that represents a combination of mixed motives that is satisficing for all players in this time in the dialogue (cf. Alg. 1, line 21).

Mapping mixed motives onto linguistic intentions
The resulting Satisf icingSet is forwarded to the mapper for mapping back motives onto linguistic intentions (cf. Fig. 1, step 8 & 9). In case, the Satisf icingSet covers zero motives, no mapping takes place, the process ends and none of the satellites in set S, cf. eq (4), will be considered in the actual answer planning. Otherwise, the mapper determines the set of supporting linguistic intentions by processing the inverse is-supported-byrelation between motives and linguistic intentions (cf. Alg. 1, line 22-30) (cf. Fig. 3). Comparing this set with the Satisf actionSet (cf. eq. (5)), an intersection called SupportSet is created that represents the set of linguistic intentions that will be satisfied in current answer planning: 3.2.5 Determination of set S + and generation of answer The linguistic intention handler determines the final set of satellites S + by analyzing 1:1 relations between linguistic intentions of the SupportSet and satellites of the set S (cf. Fig. 1, step 10 and Alg. 1, line 27). The resulting set S + = {sat AAS , sat EUP } consists of two satellites: Alternatives Advantages Survey (sat AAS ) and Emotion User Preferences (sat EUP ). The text plan lib handler adjusts the final text plan regarding the selected satellites before sending it to the answer generator (cf. Fig. 1, step 11 & 12). Last, the text plan provided by the plan operator NUM-BER OF PRODUCTS is transformed into an answer. Thereby, answer schemata referenced by nucleus as well as satellites of set S + are instantiated (cf. Fig. 1, step 13

Summary
In summary, satisficing answer planning is considered as a game consisting of four components P, S, F, A : the set of players P = {p a , p b }, strategies of players S = {S pa , S p b }, objective functions of players F = {f pa , f p b }, and a state space A = {a 1 . . . a t } that represents the rounds of the game, i.e. answers planned in the dialogue. The game starts in an initial state a 1 . At a particular time t in the dialogue, the equilibrium identifier observes the state a t characterized by P, S, and F and identifies best answers for all players; s t ∈ S p ; ∀p ∈ P . Consequential, a strategy profile meeting the Nash equilibrium condition, − → s t = {s t pa , s t p b }, is specified and resulting payouts are observed: f (a t , − → s t ) → LocalP ayout → R. The calculation of local payouts by means of objective functions f ∈ F in state a t does not depend solely on the selected strategy profile, but on results of former states in A, i.e. all answers planned in the dialogue until a t . That means, infinite playing of the described non-zero-sum game a 1 , − → s 1 , . . . , a t , − → s t , . . . generates a stream of pay- Besides relevant motives of the RelevanceSet, answer planning in state a t+1 is directly influenced by local payouts f t (a t , − → s t ) in a t leading to a continuous deliberation of the mixed motive model during dialogue. To evaluate our approach, we conducted a user study with the implemented prototype in German that was set up as lab experiment. Goal of this study was to assess the perceived fairness and naturalness of the dialogue with the QA system as well as the extent of motive satisfaction of participants. For that purpose, four randomized groups were formed. Each group was characterized by a combination of motives by users (fair price of product (m FP ) or exclusive design of product (m ED )) and the QA system representing the retailer (increasing revenue (m IR ) or improving customer relationship (m ICR )) (cf. Tab. 3). These Table 3: Groups and mixed motive combinations of user study mixed motives were combined systematically by means of scenarios given to users and a manipulated mixed motive model of the QA system. Before interacting with the QA system that was embedded into a web-based questionnaire, participants had to opportunity to get to know the QA system and interacting with it for the first time (cf. Fig. 8). Participants were then asked to pose questions to the QA system and to evaluate generated answers against the background of their motive (e.g., m FP ) and the related scenario, e.g.: "You are searching for a new tablet that shall be functional regarding standby and storage capacity. A fair price is important; no need for the latest innovation. You do not want to spend a lot of money for the new tablet. You are price conscious." Participants were told to interact with the QA system as long as it needed to gain the information that was required by the scenario. Finally, sevenpoint Likert scales ranging from strongly disagree (1), neither (4) to strongly agree (7) were used to assess the perceived fairness of the dialogue, the naturalness of the dialogue and the motive satisfaction. Tab. 4 lists the questionnaire items for each of these constructs.

Results
In summary, 120 subjects participated in the experiment. A complete dataset from 107 participants (58,3% female) with an average age of 24.3 (SD=6.9) was considered for analysis. On average, interactions between participants (N=107) and the QA system covered 5.19 question answer pairs (cf. example dialogue in appendix A). 556 questions were posed by subjects; 35.07% of them were propositional questions (e.g., "Is product A up-to-date?"), 62.41% set questions (e.g., "Where is the difference between product A and product B?") and 2.52% choice questions (e.g., "Which product is better than product A?"), cf. Bunt et al. (2010). Due to the fact that Cronbachs alpha values for all three multi-item constructs lie clearly above the recommended threshold of .70 (Nunnally, 1967), Figure 5: Subject during interaction with QA system in user study which indicates a good to excellent reliability of the scales, we calculated aggregated mean scores for each construct. The descriptive statistics of the three core constructs are presented in Tab. 4. Additionally, results of one-sample t-tests are provided to evaluate whether the aggregated scores lie significantly above or below the neutral scale value of 4. Results indicate that the participants were undecided with respect to the "Perceived Naturalness of Dialogue" with the QA system. We assume that this is owed to the restricted QA setting since there were no significant differences among the four groups (F(3,104) = 2.06, p = .11) (cf. Tab. 3). However, the data support the conclusion that participants perceived the dialogue as fair and that they were able to sufficiently satisfy their motives. Assuming rather conflicting motives of subject and QA system as given for instance in group #4 in Tab. 3, it could be assumed that perceived fairness and motive satisfaction should be smaller than in rather congruent motive combinations as shown in group #1. Nonetheless, the mean value of the construct "Perceived Fairness of Dialogue" was 5.17 across all groups (significant above mean value 4) and there were no significant differences between the randomized groups (F(3,104) = 1.59, p = .20). Furthermore, "Motive Satisfaction" was rated with a mean value of 5.16 across all groups (significant above mean value 4) and again, there was not a significant effect of the group on motive satisfaction at the .05 level of significance (F(3,104) = 2.33, p = .08).
Overall, this indicates a positive evaluation of the QA system regarding its ability to generate satisficing answers despite of mixed motives of interlocutors.

Conclusion
We considered dialogues with congruent as well as incongruent interlocutor motives, where dialogue partners are facing the constant challenge of finding a balance between cooperation and competition. Despite of the overall presence of dialogues with such mixed motives in everyday life, little attention has been given to this topic in text planning in contrast to scrutinized dialogue systems that support dialogues fully aligned with anticipated user interests. Focusing on Question-Answering (QA) settings, we introduced a model that formalizes answer planning as psychological game embedded in text planning approaches for supporting fair dialogues under mixed motives. The model was exemplified within a QA sales assistant with domain-specific world knowledge for conducting sales dialogues. Due to the fact that fairness is subjective per se, we presented results from an empirical study (N=107) in which human subjects interacted with the QA system in various mixed motive settings. Results indicate a positive evaluation of the systems performance in planning answers that support fair dialogues despite of mixed motives of interlocutors.