TJUdeM: A Combination Classifier for Aspect Category Detection and Sentiment Polarity Classification

This paper describes the system we submitted to In-domain ABSA subtask of SemEval 2015 shared task on aspect-based sentiment analysis that includes aspect category detection and sentiment polarity classiﬁcation. For the aspect category detection, we combined an SVM classiﬁer with implicit aspect indicators. For the sentiment polarity classiﬁcation, we combined an SVM classiﬁer with a lexicon-based polarity classiﬁer. Our system outperforms the baselines on both the laptop and restaurant domains and ranks above average on the laptop domain.


Introduction
Sentiment analysis aims at identifying people's opinions, sentiments, attitudes, and emotions towards entities and their attributes (Liu, 2012), which has a wide range of applications on user-generated content, e.g., reviews, blogs, and tweets.
Most previous work in sentiment analysis mainly attempted to identify the overall polarity of a given text or text span (Pang and Lee, 2008;Wilson et al., 2009;Zhang et al., 2009). The document-level or sentence-level sentiment classification is often insufficient for applications. Each document may talk about different entities, or express different opinions about different aspects of the entity even if the document concerns a single entity. Therefore, we need to discover the aspects of entities and determine the sentiment polarity on each entity aspect. This task is called aspect-based sentiment analysis or featurebased opinion mining (Hu and Liu, 2004).
The aspect-based sentiment analysis (ABSA) task (Task 12) (Pontiki et al., 2015) in SemEval 2015 is a continuation of SemEval 2014 Task 4 (Pontiki et al., 2014). The ABSA task consists of two subtasks: In-domain ABSA and Out-domain ABSA. We participated in the former subtask that aims to identify the aspect category (i.e., an entity and attribute pair) and the sentiment polarity given a review text about a laptop or a restaurant.
Each entity and attribute pair is an aspect category chosen from the predefined inventories of entity types and attribute labels per domain. For the aspect category detection, an SVM classifier with the bagof-words features can be used, and this approach is used as our baseline method. However, if a token implying an aspect, e.g., "overpriced", is not taken as a feature, the SVM classifier cannot correctly identify its corresponding category. Therefore, we enhance the results from the SVM classifier by using implicit aspect indicators (Cruz- Garcia et al., 2014). For the sentiment polarity classification, an SVM classifier with the bag-of-words features plus the category feature is trained and this is used as our baseline. However, again, if a sentiment word does not appear in the training data, the SVM classifier cannot predict its polarity. Therefore, we combined the SVM classifier and a lexicon-based polarity classifier (Taboada et al., 2011).
The remainder of this paper is organized as follows. In Section 2, we describe our approach to the aspect category detection. In Section 3, our approach to the sentiment polarity classification is presented. Experimental results are shown in Section 4. Section 5 provides the conclusion.

Aspect Category Detection
The aspect category detection task is to identify the specific entities and their attributes about the laptop or restaurant reviews. We use an SVM classifier enhanced by implicit aspect indicators. The process of the whole system is illustrated in Figure 1. We will describe the details in the following subsections.

SVM Classifier
The SVM classifier uses words as features to determine the aspect categories. We use the LIBSVM package (Chang and Lin, 2011) to implement an SVM classifier. The "-t" option is set to 0 for linear kernel, and the "-b" option is set to 1 for probability estimates. The top n frequent tokens in the training data are used as the bag-of-words features. We set n = 1000 as the number of bag-of-words features.
An aspect category (C) is an entity (E) and attribute (A) pair, i.e., C = E#A. For instance, I received prompt service with a smile.→{Service#General} It would cost too much to repair it.→{Support#Price} For a test sentence s, the LIBSVM package can predict the probability of assigning each category E#A to s. The category should be assigned to s only if its probability is higher than a predefined threshold t. We set t to 0.2 for the restaurant reviews and to 0.12 for the laptop reviews. It's easy to see that our SVM classifier is configured in accordance with the SVM baseline system provided by the task organizers (Pontiki et al., 2015).

Implicit Aspect Indicator
If the tokens implying aspects are beyond bag-ofwords features, the SVM classifier is unable to predict it. For example, It was totally overpricedfish and chips was about $15.
Both "overpriced" and "$15" in the above sentence are associated with the "price" aspect. These tokens are considered as the implicit aspect indicators.
The different methods can be used to identify the implicit aspect indicators (Cruz- Garcia et al., 2014). In our case, we do it manually by setting a set of indicators for several aspects (see Table 1). The list of words associated with the "price" aspect includes "cost", "overpriced", "expensive", etc. The list for the "quality" aspect includes "feels", "durable", "taste", etc.
If the word W indicates the implicit aspect A , the aspects determined by implicit aspect indicators are denoted as follows: (2)

Combination Classifier
We find the SVM classifier often obtains the category like "E|General" which means that a general opinion is expressed and it is not specific to a particular aspect. On the other hand, for the same case, the implicit aspect indicators may suggest other specific aspect categories (e.g., "price"). This case occurs when the words corresponding to the implicit aspect indicators are not included in the features used by SVM. It is in this case that it is the most useful to combine the two classifiers. Our combination is done as follows: if the "General" category is suggested by the SVM classifier, then we replace it by the categories identified through the implicit aspect indicators. Otherwise, the categories given by the SVM classifier remain unchanged. The method is described in the following algorithm.

Sentiment Polarity Classification
The sentiment polarity classification task is to assign a polarity from a set {positive, negative, neutral} to each identified aspect category of a sentence. We use a similar method as for the previous task. The processes of the system are illustrated in Figure 2 that includes three parts: an SVM classifier, a lexiconbased polarity classifier, and their combination classifier.

SVM Classifier
We also use the LIBSVM package (Chang and Lin, 2011) to implement an SVM classifier with linear kernel. Again, n (n = 1000) bag-of-words features are extracted from the training data. In addition, a feature that indicates the aspect category is used. Our SVM configurations are also the same with that of the SVM baseline system (Pontiki et al., 2015).
The SVM classifier can predict a polarity (positive, negative, or neutral) for each aspect category C within a test sentence s. We represent three polarity labels with three respective numbers. P olarity svm (s, C) ∈ {1, −1, 0} (3)

Lexicon-Based Polarity Classifier
If the sentiment words are beyond the bag-of-words features, the SVM classifier assigns the neutral polarity, and what's worse, it assigns the reverse polarity if the sentence contains negation words (Zhu et al., 2014), like "not" and "no". In fact, the lexiconbased methods can also be effective in sentiment classification (Taboada et al., 2011). We therefore adopt a simple lexicon-based method in our system. The sentiment lexicons, such as Bing Liu's Opinion Lexicon (Hu and Liu, 2004) and MPQA Subjectivity Lexicon (Wilson et al., 2009), are used to generate our sentiment word list. We denote all positive words and negative words by P OS and N EG respectively.
We use the Stanford Parser package (Klein and Manning, 2003) for POS tagging and parsing. The typed dependency "neg(X, Y )" shows that one sentence contains a negation Y modifying X, and "root(ROOT, X)" shows that X is a core word.
Assume that one sentiment word X is in a test sentence s and X ∈ P OS ∪ N EG, if X ∈ P OS, then P olarity(X) = 1, otherwise P olarity(X) = −1. The polarity for the aspect category is determined by, where Y ∈ s is a negation word, and Z ∈ s but Z / ∈ P OS ∪ N EG.
The following examples are corresponding to three circumstances in the above equation: Overpriced and not tasty {neg(tasty, not)} Our experience did not ever get any better {neg(get, not), root(ROOT, get)}

Combination Classifier
If none of the sentiment words in the lexicon appear in a sentence, the lexicon-based polarity classifier is helpless, but the SVM classifier could still determine a reasonable polarity (Pang et al., 2002).
We propose a classifier combining the SVM classifier and the lexicon-based polarity classifier. It works as follows: If there is disagreement between the polarity of SVM classifier and the lexicon, we will rely on the polarity based on the lexicon if the latter is not neutral (0). Otherwise, we take the polarity of the SVM classifier.

Data Sets
The training and test data is described in Table 2  There are 22 entity labels and 9 attribute labels on the laptop domain, and there are 6 entity labels and 5 attribute labels on the restaurant domain. Table 3 lists the results of our system for the aspect category detection.  Table 3: F-score comparison for aspect category detection.

Aspect category detection
Our system clearly outperforms the SVM baseline on both two domains. This indicates that the implicit aspect indicators can further improve the performance. Our system ranks above average on the laptop domain. But our system is far from the top system. This is possibly due to the simple features used by the SVM classifier. Globally, our method is comparable to the average performance of all the participating systems. Table 4 lists the results of our system for the sentiment polarity classification. The majority baseline is obtained by majority voting in all the participating results.  The performance of our system is obviously better than two baselines on both two domains, but fails to reach the average on the restaurant domain. The conclusion of this experiment is that the lexiconbased method is helpful to sentiment classification when it is combined with a baseline method. As for the task of aspect category detection, a possible reason lies in the simple bag-of-words features we used. With more sophisticated features, one can likely improve the performance of the baseline methods, and as a result, the combination method.

Sentiment polarity classification
Comparing the results on the two domains, we observe that our system produced lower performance than average for the restaurant reviews, but higher performance for the laptop reviews. A possible reason can be the lexicon we defined for the two domains. The Opinion Lexicon is originally designed for the customer reviews about 5 digit products, which is more related to the laptop domain.

Conclusions
In this task, we proposed a combination classifier for the aspect category detection which combines an SVM classifier with implicit aspect indicators, and a combination classifier for the sentiment polarity classification which combines an SVM classifier with a lexicon-based polarity classifier. Our system ranks above average on the laptop domain and outperforms the baselines, but is still lower than the av-erage for the restaurant domain. Our experiments show that implicit aspect indicators and polarity lexicon are both useful in these tasks. For the future work, more and better features will be examined to help to improve the classification performance.