Visualization of the Topic Space of Argument Search Results in args.me

In times of fake news and alternative facts, pro and con arguments on controversial topics are of increasing importance. Recently, we presented args.me as the first search engine for arguments on the web. In its initial version, args.me ranked arguments solely by their relevance to a topic queried for, making it hard to learn about the diverse topical aspects covered by the search results. To tackle this shortcoming, we integrated a visualization interface for result exploration in args.me that provides an instant overview of the main aspects in a barycentric coordinate system. This topic space is generated ad-hoc from controversial issues on Wikipedia and argument-specific LDA models. In two case studies, we demonstrate how individual arguments can be found easily through interactions with the visualization, such as highlighting and filtering.


Introduction
For many controversial topics in life and politics, people disagree on what is the right stance towards them, be it the need for feminism, the influence of religion, or the assassination of dictators. Stance is affected by the subjective assessment and weighting of pro and con arguments on the diverse aspects of a topic (Kock, 2007). Building stance in a selfdetermined manner is getting harder and harder in times of fake news and alternative facts, due to the unclear reliability of many sources and their bias in stance and covered aspects. This was our societal motivation for the development of the first dedicated argument search engine, args.me. 1 args.me allows querying for arbitrary controversial topics. As search results, it opposes pro and con arguments from the web, ranked by their computed relevance to the topic. args.me is non-commercial and aims to avoid bias towards either stance. In 1 Also known as just args, found at: https://args.me (Wachsmuth et al., 2017b), we introduced the software framework and the initial argument index underlying args.me as well as its basic user interface.
In its first version, args.me presented arguments in textual form with linked sources, similar to the web page snippets of conventional search engines, but with color-encoded stance. Examples are given below in Figures 1 and 2. This is adequate for comprehending those arguments deemed most relevant. Unlike for many general information needs (Croft et al., 2009), however, reading the top results is not enough for building an informed stance. Rather, diverse aspects of a controversial topic need to be explored. In our recent study with 97 international users, aspect coverage was seen as the second most important ranking criterion -after source reliability but before recency, user ratings, and others. 2 A simple relevance ranking of possibly thousands of arguments provides weak support in that regard. This paper shows a novel way of presenting argument search results, which we designed and integrated into args.me to support a rapid exploration of the aspects of a topic. In particular, we visualize this topic space in a barycentric coordinate system (Riehmann et al., 2018), representing the distribution of pro and con arguments over the main covered aspects (see Figure 3). Possible aspects were derived offline from the Wikipedia list of controversial issues 3 as well as from LDA topic models built based on the 291k arguments in our index, whereas the aspects actually visualized are derived ad-hoc from the search results. Through interactions with the visualization, a user can easily highlight and filter arguments on the aspects of interest. In two case studies, we demonstrate how the visualization speeds up argument search notably.
To support achieving consensus in a discussion, SEAS (Lowrance et al., 2000), VUE (Baroni et al., 2015), and Dialectic Map (Niu, 2016) provide a combination of automatic argument analysis and visual argument summaries. With similar intentions, Lexical Episode Plots (Gold et al., 2015), ConToVi All these works focus on single arguments or the set of arguments within a single debate or text. In contrast, we present a visualization that summarizes arguments from many different texts. Unlike in (Wachsmuth et al., 2017a), where we illustrated structural argumentation patterns in the texts of a corpus, here we target the content of arguments. As the above-mentioned system ConToVi, we visualize the topic space covered by a set of arguments. While ConToVi provides insights into the flow of aspects during the discussion of a controversial topic, our visualization aims to make arguments on specific aspects easily findable. Moreover, we allow arguments to cover a weighted distribution of multiple aspects rather than only a single aspect. 3 Argument Search with args.me As presented in (Wachsmuth et al., 2017b), the initial version of args.me follows approved concepts of conventional search engines (such as Google or Bing), but it adapts them to the specific goals of argument search. Via the interface of args.me, users can enter free text queries on controversial topics, such as "feminism" and "assassination of dictators". While conventional search engines return links to web pages along with short textual excerpts as results, args.me directly returns all arguments found to be relevant, linked to their source web pages.
Originally, args.me provided two views that displayed the found arguments in a textual result list with color-encoded stance, as shown in Figures 1 and 2: a pro vs. con view that opposes the most relevant pros and cons, and an overall ranking view that ranks all arguments by their relevance irrespective of stance. A fundamental question in this regard is what arguments are actually deemed most relevant? Argument search implies specific ranking criteria, such as recency, perceived quality, aspect coverage, and source reliability. Assessing these criteria is all but trivial and partly unsolved. In its current state (mid 2018), args.me therefore still relies on standard information retrieval measures (Robertson and Zaragoza, 2009), i.e., it ranks arguments higher the more they match the words in the query.
So far, however, for many queries the top-ranked arguments neither appear to be the strongest on the given topic, nor do they cover the whole diversity of the topic. Having only a textual result list can then make it hard to find the best arguments or specific arguments of interest. This is why we aimed for new ways to support an efficient search space exploration -one main goal of information visualization (Munzner, 2014). While our recent user study suggested that source reliability is most important (see Section 1), the 291,440 arguments currently indexed by args.me anyway come from five selected sources only (Wachsmuth et al., 2017b). Instead, we thus focus on the second most important ranking criterion: aspect coverage.

Visualization of the Topic Space
For building an informed stance on a controversial topic, obtaining insights into the variety of aspects touched by the topic is crucial. We aid this process by accompanying the textual result list in args.me with a topic space visualization, which puts the aspects covered by the listed arguments into the focus, highlights the main aspect of each argument, and groups arguments covering similar aspects.

Determining Topical Aspects
The first step to develop the visualization was to build a topic model that can represent the aspects of each argument in the result list. We compared two alternative approaches for this purpose: First, we computed the relative distribution of all the over 1000 terms from the Wikipedia list of controversial issues in each indexed argument. For instance, if "Women" occurs ten times, "Woman" six times, "Feminism" four times, and no other term, then we have (Women 0.5, Woman 0.3, Feminism 0.2) with implicit zeros for all others. Second, we performed LDA topic modeling (Blei et al., 2003) based on the words in all arguments from our index. With an interval size of 10, we tested all numbers of topics from 10 to 100 and chose the number that minimized perplexity: 40. Each aspect is then represented by all words of one LDA topic, and the relative aspect distribution is calculated by counting the occurrence of all associated words in each argument. We found the Wikipedia-based topic model to be more convincing, which is why it is set as the default in args.me. 4

Visualizing Topical Aspects
To visualize the aspect-based topic space, we opted for generalized barycentric coordinates (Meyer et al., 2002), as they naturally fit our purpose: We represent an argument as a linear combination of weights for all aspects, while barycentric coordinates represent a point as a linear combination of the vertices of a polygon (both adding up to 1.0). Thus, the topic model can be used as input for the visualization without recalculation. Figure 3 shows the visualization of the results for the query "feminism", consisting of two main elements: the topic space and the argument glyphs within this space.
The topic space is depicted as a regular polygon with one vertex for each represented aspect. Both given topic model alternatives comprise too many aspects to depict them all. To reduce visual clutter in favor of a lean visualization, we limit the maximum number of visualized aspects, such that readability is not diminished. In particular, we keep only those eight aspects that are the most frequent in the argument search results. All other aspects are merged into a ninth aspect "other". The labels for the aspects are short terms in case of the Wikipedia-based topic model or visualized as word clouds in case of the LDA topic model.
Each argument glyph represents one argument in the form of a colored circle (green for pro, red for con) with a small arrow pointing to the main covered aspect. The glyphs are positioned based on their aspect distribution: the stronger one aspect, the stronger a glyph is "pulled" in that direction, as sketched in Figure 4. Accordingly, similar arguments are placed spatially near to each other. To ensure the visibility of all glyphs and to avoid overplotting, arguments placed on top of each other are aggregated into a single glyph. The glyph size depends on a logarithmic mapping of the number of represented arguments. Since arguments with both stances may be grouped, the color of an aggregate glyph represents the majority stance of all arguments contained, from green (all pro), over gray (balanced pro/con), to red (all con).

Interacting with Topical Aspects
The integration of our visualization into args.me is shown below in Figures 7 and 8. This new topic space view replaces the old overall ranking view: it includes the textual argument ranking and adds the visualization to the right. At first, the visualization shows only the information outlined above, but it provides further details upon interaction.
Barycentric coordinates are ambiguous and may place arguments with different aspects at similar locations. For disambiguation, users can hover over a glyph to reveal all covered aspects, as exemplified in Figure 5. The represented arguments are also highlighted in the textual list, given that they appear on the current result page. Vice versa, hovering over a textual argument highlights the respective glyph with a wide green or red border. In addition, the visualization enables a filtering of the textual results: A user can select one or more arguments by clicking or brushing (see Figure 6), in order to narrow down the list to the aspects of interest. All other arguments are grayed out.

Case Studies
To verify the benefit of our visualization, we finally explore two typical use cases of argument search: topic space exploration and search refinement.

Topic Space Exploration
First, we consider a query for "feminism". 659 arguments are returned by args.me for this topic, as shown in Figure 7. While the top-ranked arguments seem highly relevant in general, our visualization reveals that also some rather specific aspects are covered by the search results, such as "Abortion" and "United Nations". Interacting with the visualization helps explore the entire topic space.
In particular, hovering over the argument glyphs clarifies what aspects they exactly cover, such as "Woman", "Women", and "Feminism" itself for the highlighted argument in Figure 7. After a first exploration via hovering, a result subset of interest can be filtered through brushing, say, the four topmost glyphs (see Figure 6 above). The selected arguments are then shown at the top of the textual result list (all below are grayed out). From the selected arguments, we learn that Emma Watson has made the need for feminism a point at the United Nations, whereas the claimed necessity of abortion is used as an analogy to justify the necessity of  feminism. Without the visualization, these insights would have been hard to gain; the two respective arguments were ranked at position #43 and #46.

Search Refinement
As a second example, we assume that a user looks for new arguments on the "assassination of dictators", for which args.me provides 119 results. If the user wants to refine a search to restrict it to a specific aspect of the topic only (e.g., to arguments covering Muammar Gaddafi), a simple click on the respective argument glyph in the topic space visualization suffices, as illustrated in Figure 8. The associated arguments are filtered and placed at the top of the result list (only one argument in the illustrated case). With the existing interaction methods of args.me, the argument text can be extended and its source web page shows up after clicking on it. In the old overall ranking view, the argument would have been ranked at position #34.

Conclusion
This paper has presented a new visual interface for our argument search engine, args.me. The visualization supports users in learning about the topical aspects covered by the arguments returned in response to a query. In two case studies, we have demonstrated how this topic space visualization enables an efficient exploration and refinement of argument search results. Future research on modeling the aspects of an argument can further enhance the usability of the visualization.