Sentiment Domain Adaptation with Multiple Sources

Domain adaptation is an important research topic in sentiment analysis area. Existing domain adaptation methods usually transfer sentiment knowledge from only one source domain to target domain. In this paper, we propose a new domain adaptation approach which can exploit sentiment knowledge from multiple source domains. We ﬁrst extract both global and domain-speciﬁc sentiment knowledge from the data of multiple source domains using multi-task learning. Then we transfer them to target domain with the help of words’ sentimen-t polarity relations extracted from the unlabeled target domain data. The similarities between target domain and different source domains are also incorporated into the adaptation process. Experimental re-sults on benchmark dataset show the effectiveness of our approach in improving cross-domain sentiment classiﬁcation performance.


Introduction
Sentiment classification is a hot research topic in natural language processing field, and has many applications in both academic and industrial areas (Pang and Lee, 2008;Liu, 2012;Wu and Huang, 2016). Sentiment classification is widely known as a domain-dependent task (Blitzer et al., 2007;Glorot et al., 2011). The sentiment classifier trained in one domain may not perform well in another domain. This is because sentiment expressions used in different domains are usually different. For example, "boring" * Corresponding author. and "lengthy" are frequently used to express negative sentiments in Book domain. However, they rarely appear in Electronics domain (Bollegala et al., 2011). Thus a sentiment classifier trained in Electronics domain cannot accurately predict their sentiments in Book domain. In addition, the same word may convey different sentiments in different domains. For example, in Electronics domain "easy" is usually used in positive reviews, e.g., "this digital camera is easy to use." However, it is frequently used as a negative word in Movie domain. For instance, "the ending of this movie is easy to guess." Thus, the sentiment classifier trained in one domain usually cannot be applied to another domain directly (Pang and Lee, 2008).
In order to tackle this problem, sentiment domain adaptation has been widely studied (Liu, 2012). For example, Blitzer et al. (2007) proposed to compute the correspondence among features from different domains using their associations with pivot features based on structural correspondence learning (SCL).  proposed a spectral feature alignment (SFA) algorithm to align the domain-specific words from different domains in order to reduce the gap between source and target domains. However, all of these methods transfer sentiment information from only one source domain. When the source and target domains have significant difference in feature distributions, the adaptation performance will heavily decline. In some cases, the performance of sentiment domain adaptation is even worse than that without adaptation, which is usually known as negative transfer .
In this paper we propose a new domain adaptation approach for cross-domain sentiment classification. Our approach can exploit the sentiment information in multiple source domains to reduce the risk of negative transfer effectively. Our approach consists of two steps, i.e., training and adaptation. At the training stage, we extract two kinds of sentiment models, i.e., the global model and the domain-specific models, from the data of multiple source domains using multi-task learning. The global sentiment model can capture the common sentiment knowledge shared by various domains, and has better generalization performance than the sentiment model trained in a single source domain. The domain-specific sentiment model can capture the specific sentiment knowledge in each source domain. At the adaptation stage, we transfer both kinds of sentiment knowledge to target domain with the help of the words' sentiment graph of target domain. The sentiment graph contains words' domain-specific sentiment polarity relations extracted from the syntactic parsing results of the unlabeled data in target domain. Since sentiment transfer between similar domains is more effective than dissimilar domains, we incorporate the similarities between target domain and different source domains into the adaptation process. In order to estimate the similarity between two domains, we propose a novel domain similarity measure based on their sentiment graphs. Extensive experiments were conducted on the benchmark Amazon product review dataset. The experimental results show that our approach can improve the performance of cross-domain sentiment classification effectively.

Related work
Sentiment classification is widely known as a domain-dependent task, since different expressions are used to express sentiments in different domains (Blitzer et al., 2007). The sentiment classifier trained in one domain may not perform well in another domain. Since there are massive domains, it is impractical to annotate enough data for each new domain. Thus, domain adaptation, or so called cross-domain sentiment classification, which transfers the sentiment knowledge from domains with sufficient labeled data (i.e., source domain) to a new domain with no or scarce labeled data (i.e., target domain), has been widely studied. Existing domain adaptation methods mainly transfer sentiment information from only one source domain. For example, Blitzer et al. (2007) proposed a domain adaptation method based on structural correspondence learning (SCL). In their method, a set of pivot features are first selected according to their associations with source domain labels. Then the correspondence among features from source and target domains is computed using their associations with pivot features. In order to reduce the gap between source and target domains,  proposed a spectral feature alignment (SFA) algorithm to align the domainspecific sentiment words from different domains into clusters. He et al. (2011) proposed to extract polarity-bearing topics using joint sentiment-topic (JST) model to expand the feature representations of texts from both source and target domains. Li et al. (2009) proposed to transfer sentiment knowledge from source domain to target domain using nonnegative matrix factorization. A common shortcoming of above methods is that if the source and target domains have significantly different distributions of sentiment expressions, then the domain adaptation performance will heavily decline (Li et al., 2013).
Using multiple source domains in cross-domain sentiment classification has also been explored. Glorot et al. (2011) proposed a sentiment domain adaptation method based on a deep learning technique, i.e., Stacked Denoising Auto-encoders. The core idea of their method is to learn a high-level representation that can capture generic concepts using the unlabeled data from multiple domains. Yoshida et al. (2011) proposed a probabilistic generative model for cross-domain sentiment classification with multiple source and target domains. In their method, each word is assigned three attributes, i.e., the domain label, the domain dependence/independence label, and sentiment polarity. Bollegala et al. (2011) proposed to construct a sentiment sensitive thesaurus for cross-domain sentiment classification using data from multiple source domains. This thesaurus is used to expand the feature vectors for both training and classification. However, the similarities between target domain and different source domains are not considered in these methods. In addition, although unlabeled data is utilized in these methods, the useful word-level sentiment knowledge in the unlabeled target domain data is not exploited.
General-purpose multiple source domain adaptation methods have also been studied. For example, Mansour et al. (2009) proposed a distribution weighted hypothesis combination approach, and gave theoretical guarantees for it. However, this method is based on the assumption that target distribution is some mixture of source distri-butions, which may not hold in sentiment domain adaptation scenario. Duan et al. (2009) proposed a Domain Adaptation Machine (DAM) method to learn a Least-Squares SVM classifier for target domain by leveraging the classifiers independently trained in multiple source domains. Chattopadhyay et al. (2011) explored to assign psuedo labels to unlabeled samples in the target domain using the classifiers from multiple source domains. Then target domain classifier is trained on these psuedo labeled samples. Compared with these generalpurpose domain adaptation methods with multiple source domains, our approach is more suitable for sentiment domain adaptation because our approach exploits more sentiment-related characteristics and knowledge, such as the general sentiment knowledge shared by different domains and the word-level sentiment polarity relations, which is validated by experiments.

Sentiment Graph Extraction and Domain Similarity Measure
In this section we introduce two important components used in our sentiment domain adaptation approach, i.e., the words' sentiment graph and domain similarity.

Sentiment Graph Extraction
Compared with labeled data, unlabeled data is usually much easier and cheaper to collect on a large scale. Although unlabeled samples are not associated with sentiment labels, they can still provide a lot of useful sentiment information for domain adaptation. For example, if "great" and "quick" are frequently used to describe the same target in the same review of Kitchen domain, then they probably convey the same sentiment polarity in this domain. Since "great" is a general positive word in both Book and Kitchen domains, we can infer that "quick" is also a positive word in Kitchen domain when transferring from Book domain to Kitchen domain. Motivated by above observations, in this paper we propose to extract sentiment polarity relations among words from massive unlabeled data for sentiment domain adaptation. Two kinds of polarity relations are explored, i.e., sentiment coherent relation and sentiment opposite relation. The former means that two words convey the same sentiment polarity while the latter indicates opposite sentiment polarities. These polarity relations are ex- tracted from the syntactic parsing results according to manually selected rules. Two rules are used to extract sentiment coherent relations. The first one is that two words are connected by coordinating conjunctions such as "and" and "as well as". For example, a review in Kitchen domain may be "it is so high-quality and professional." Since "high-quality" and "professional" are connected by the coordinating conjunction "and", we infer that they probably convey the same sentiment polarity. The second rule is that two words are not directly connected but are used to describe the same target in the same sentence. For example, a review in Electronics domain may be "It is a beautiful, durable, easy-to-use camera." Since "beautiful", "durable", and "easy-to-use" are all used to describe the same camera in the same review, they tend to convey the same sentiment polarity. We also propose two rules for extracting sentiment opposite relations. The first rule is that two words are connected by adversative conjunctions such as "but" and "however". The second rule is that two words are connected by coordinating conjunctions but there is a negation symbol before one of them. For example, a review may be "The battery of this camera is small and not durable." We can infer that "small" and "durable" may convey opposite sentiments when they are used to describe camera battery. An illustrative example of extracting sentiment polarity relations from syntactic parsing results is shown in Fig. 1.
Based on the sentiment polarity relations among words extracted from the unlabeled data, we can build a words' sentiment graph for each domain. The nodes of the sentiment graph represent words and the edges stand for sentiment polarity relations. We denote R ∈ R D×D as the words' sentiment graph of a specific domain. R i,j represents the sentiment polarity relation score between words i and j. In this paper we define R i,j as where n C i,j and n O i,j represent the frequencies of words i and j sharing coherent and opposite sentiment polarity relations respectively in all the unlabeled samples. Thus, R i,j ∈ [−1, 1]. If R i,j > 0, then words i and j tend to convey the same sentiment polarity. Similarly, if R i,j < 0, then these two words are more likely to convey opposite sentiments. The absolute value of R i,j represents the confidence of this sentiment polarity relation.

Domain Similarity
Different pairs of domains have different sentiment relatedness (Remus, 2012;. Researchers have found that sentiment domain adaptation between similar domains, such as Kitchen and Electronics, is much more effective than that between dissimilar domains, such as Kitchen and Book (Blitzer et al., 2007;. Thus, it is beneficial if we take the similarity between source and target domains into consideration when transferring sentiment knowledge. In this paper we explore two methods to measure domain similarity. The first one is based on term distribution. The assumption behind this method is that similar domains usually share more common terms than dissimilar domains. For example, Smart Phone and Digital Camera domains share many common terms such as "screen", "battery", "light", and "durable", while the term distributions of Digital Camera and Book domains may have significant difference. Term distribution based domain similarity measures, such as A-distance, have been explored in previous works (Blitzer et al., 2007). Inspired by (Remus, 2012), here we apply Jensen-Shannon divergence to measure domain similarity based on term distributions, which is more easy to compute than A-distance. Denote t m ∈ R D×1 as the term distribution of domain m, where t m w is the probability of term w appearing in domain m. Then the similarity between domains m and n is formulated as: where t = 1 2 (t m + t n ) is the average distribution, and D KL (·, ·) is the Kullback-Leibler divergence: We can verify that D JS (t m , t n ) ∈ [0, 1]. Thus, the range of T ermSim(m, n) is also [0, 1]. The term distribution based domain similarity can measure whether similar words are used in two domains. However, sharing similar terms does not necessarily mean that sentiment expressions are used similarly in these domains. For example, CPU and Battery are both related to electronics. The word "fast" is positive when used to describe CPU. However, it is frequently used as a negative word in Battery domain. For example, "this battery runs out fast." Thus, it is more useful to measure domain similarity based on sentiment word distributions. However, although we can infer the sentiment word distributions of source domains according to labeled samples, it is difficult to compute the sentiment word distribution of target domain, since the labeled data does not exist or is very scarce in target domain.
In order to tackle this problem, in this paper we propose to estimate the similarity between two domains based on their sentiment graphs. Similar domains usually share more common sentiment words and sentiment word pairs than dissimilar domains. In addition, the polarity relation scores of a pair of words in the sentiment graphs of similar domains are also more similar. In other words, they tend to be both positive or negative in these two domains. Motivated by above observations, the domain similarity based on sentiment graph is formulated as follows: where R m w,v is the sentiment polarity relation score between words w and v in domain m, and N m w,v is its frequency in this domain. N m∩n We can verify that SentiSim(m, n) ∈ [0, 1]. If two domains have more common sentiment word pairs and the polarity relation scores of these word pairs are more similar, then these two domains share higher domain similarity according to Eq. (3).

Sentiment Domain Adaptation with Multiple Sources
In this section we introduce our sentiment domain adaptation approach in detail. First we introduce several notations that will be used in following discussions. Assume there are M source domains.
Denote {X m ∈ R Nm×D , y m ∈ R Nm×1 } as the labeled data in source domain m, where N m is the number of labeled samples and D is the size of feature vector. x m i ∈ R D×1 is the feature vector of the i th sample in domain m, and its sentiment label is y m i . In this paper we focus on sentiment polarity classification, and y m i ∈ {+1, −1}. Denote w ∈ R D×1 as the global sentiment model extracted from multiple source domains and w m ∈ R D×1 as the domain-specific sentiment model of source domain m. Denote w t ∈ R D×1 as the domain-specific sentiment model of target domain. Denote f (x, y, w) as the loss of classifying sample x into label y under linear classification model w. Our approach is flexible to the selection of loss function f , which can be square loss, logistic loss, and hinge loss. Denote R m ∈ R D×D as the sentiment graph knowledge of domain m, and S m,t ∈ [0, 1] as the similarity between source domain m and target domain.
Our sentiment domain adaptation with multiple sources approach (SDAMS) consists of two steps, i.e., training and adaptation. At the training stage, the global and domain-specific sentiment knowledge are extracted from the data of multiple source domains. And at the adaptation stage, these two kinds of sentiment knowledge are transferred to target domain by incorporating the sentiment graph knowledge of target domain and the similarities between target and source domains.

Training
Given the labeled data and the sentiment graph knowledge of multiple source domains, at the training stage, our goal is to train a robust global sentiment model to capture the general sentiment knowledge shared by various domains and a domain-specific sentiment model for each source domain. The model of the training process is motivated by multi-task learning (Evgeniou and Pontil, 2004;Liu et al., 2009) and is formulated as: where α, λ 1 , and λ 2 are nonnegative regularization coefficients. The sentiment classification model of each source domain is decomposed into two components, i.e., a global one and a domain-specific one. The global sentiment model is shared by all source domains and is trained in these domains simultaneously. It is used to capture the general sentiment knowledge, such as the general sentiment words "great", "worst", "perfect" and so on. The domain-specific sentiment model is trained on the labeled data within one source domain and is used to capture the specific sentiment knowledge of this domain. For example, the domain-specific sentiment word "easy" is a positive word in Electronics domain but is used as a negative word in Movie domain. In Eq. (4), the first term means minimizing the empirical classification loss on the labeled data of multiple source domains. In this way we incorporate the sentiment information in labeled samples into sentiment classifier learning. In the second term we incorporate the sentiment graph knowledge of each source domain. It is motivated by graph-guided fused Lasso (Chen et al., 2012). If two words have strong coherent (or opposite) sentiment polarity relations, then we constrain that their sentiment scores are more similar (or dissimilar) with each other in the final classification model. The L 1 -norm regularization terms are motivated by Lasso (Tibshirani, 1996). It can set many minor sentiment scores in the models to exact zeros. Since not all the words convey sentiments, these terms can help conduct sentiment word selection. We also incorporate the L 2 -norm regularization terms in order to improve model stability in high-dimensional problems, which is inspired by elastic net regularization (Zou and Hastie, 2003).

Adaptation
At the adaptation stage, we incorporate the global sentiment knowledge, the domain-specific sentiment knowledge of multiple source domains, the sentiment graph knowledge of target domain, and the domain similarities between target and source domains into a unified framework to learn an accurate sentiment classifier for target domain. The model of our adaptation framework is formulated as follows: where β, λ 1 , and λ 2 are nonnegative regularization coefficients. The final sentiment classifier of the target domain is a linear combination of w and w t , i.e., w+w t , where w is the global sentiment model extracted from multiple source domains at the training stage, and w t is the domain-specific sentiment model of target domain learned at the adaptation stage. In the first term of Eq. (5), we transfer the knowledge in domain-specific sentiment models from multiple source domains to w t . Since the sentiment knowledge transfer between similar domains is more effective, the transfer of domainspecific sentiment knowledge is weighted by the similarities between target domain and different source domains. If target domain is more similar with source domain m than source domain n (i.e., S m,t > S n,t ), then more domain-specific sentiment knowledge will be transferred to w t from w m than w n . Through the last term we incorporate the sentiment graph knowledge extracted from massive unlabeled data of target domain into the adaptation process. If two words share strong coherent (or opposite) sentiment polarity relations in the target domain, then we constrain that their sentiment scores in the sentiment classification model of target domain are more similar (or dissimilar). This term can help transfer the sentiment knowledge from source domains to target domain more effectively. For example, if we know that "great" is a positive word in the global sentiment model and there is a strong coherent polarity relation between "easy" and "great" in Electronics domain, then we can infer that "easy" is also a positive word in this domain.

Dataset and Experimental Settings
The dataset used in our experiments is the famous Amazon product review dataset collected by Blitzer et al. (2007). It is widely used as a benchmark dataset for cross-domain sentiment classification. Four domains, i.e., Book, DVD, Electronics and Kitchen, are included in this dataset. Each domain contains 1,000 positive and 1,000 negative reviews. Besides, a large number of unlabeled reviews are provided. The detailed statistics of this dataset are shown in Table 1.
In our experiments, each domain was selected in turn as target domain, and remaining domains as source domains. In each experiment, we randomly selected N labeled samples from the source domains to train sentiment models at the training stage. These samples were balanced among different source domains. In order to perform fair comparisons with baseline methods, following (Bollegala et al., 2011), we limited the total number of training samples, i.e., N , to 1,600. The target domain sentiment classifier was tested on all the labeled samples of target domain. Following (Blitzer et al., 2007), unigrams and bigrams were used as features. The sentiment polarity relations of bigrams were extracted by expanding the polarity relations between unigrams using modifying relations. For example, from the review "this phone is very beautiful and not expensive," we extract not only sentiment polarity relation between "beautiful" and "expensive", but also polarity relation between "beautiful" and "not expensive" (coherent sentiment), and that between "very beautiful" and "expensive" (opposite sentiment), since "very" and "not" are used to modify "beautiful" and "expensive" respectively. Classification accuracy was selected as the evaluation metric. We manually set β in Eq. (5) to 0.01. The values of α, λ 1 , and λ 2 in Eq. (4) were selected using cross validation. The optimization problems in Eq. (4) and Eq. (5) were solved using alternating direction method of multipliers (ADM-M) (Boyd et al., 2011). Each experiment was repeated 10 times independently and average results were reported.

Comparison of Domain Similarity Measures
In this section, we conducted experiments to compare the effectiveness of the two kinds of domain similarity measures introduced in Section 3.2 in sentiment domain adaptation task. The experimental results are summarized in Fig. 2. The classification loss function used in our approach is hinge loss. The results of other loss functions show similar patterns. From Fig. 2 we can see that the domain similarity measure based on sentiment graph performs consistently better than that based on term distribution in our approach. This result validates our assumption in Section 3.2 that the sentiment graph based domain similarity can better model the sentiment relatedness between different domains than that based on term distribution in sentiment domain adaptation task. In all the following experiments, the sentiment graph based domain similarities were used in our approach.

Performance Evaluation
In this section we conducted experiments to evaluate the performance of our approach by comparing it with a series of baseline methods. The methods to be compared are: 1) SCL, domain adaptation based on structural correspondence learning (Blitzer et al., 2007); 2) SFA, domain adaptation based on spectral feature alignment ; 3) SCL-com and SFA-com, adapting SCL and SFA to multiple source domain scenario by first training a cross-domain sentiment classifier in each source domain and then combining their classification results using majority voting; 4) SST, cross-domain sentiment classification by using multiple source domains to construct a sentiment sensitive thesaurus for feature expansion (Bollegala et al., 2011); 5) IDDIWP, multiple-domain sentiment analysis by identifying domain dependent/independent word polarity (Yoshida et al., 2011); 6) DWHC, DAM and CP-MDA, three general-purpose multiple source domain adaptation methods proposed in (Mansour et al., 2009), (Duan et al., 2009) and (Chattopadhyay et al., 2011) respectively; 7) SDAMS-LS, SDAMS-SVM, and SDAMS-Log, our proposed sentiment domain adaptation approaches with square loss, hinge loss, and logistic loss respectively; 8) All-Training, all the domains were involved in the training phase of our approach and there is no adaptation phase. This method is introduced to provide an upper bound for the performance of our approach. The experimental results of these methods are summarized in Table 2. From Table 2 we can see that our approach achieves the best performance among all the methods compared here. SCL and SFA are famous cross-domain sentiment classification methods. In these methods, the sentiment knowledge is transferred from one source domain to target domain. According to Table 2, our approach performs significantly better than them. This result indicates that the sentiment knowledge extracted from one source domain may contain heavy domain-specific bias and may be inappropriate for the target domain. Our approach can tackle this problem by extracting the global sentiment model from multiple source domains. This global model can capture the general sentiment knowledge shared by various domains and has better generalization performance. It can reduce the risk of negative transfer effectively. Our approach also outperforms SCLcom and SFA-com. In SCL-com and SFA-com, the sentiment information in different source domains is combined at the classification stage, while in our approach it is combined at the learning stage. The superior performance of our approach compared with SCL-com and SFA-com shows that our approach is a more appropriate way to exploit the sentiment knowledge in different source domains. SST and IDDIWP also utilize data from multiple source domains as our approach. But our approach can still outperform them. This is because in these methods, the similarities between target domain and different source domains are not considered. Since different domains usually have different sentiment relatedness, our approach can exploit the sentiment information in multiple source domains more accurately by incorporating the similarities between target domain and each source domain into the adaptation process. Our approach also outperforms the state-of-theart general-purpose multiple source domain adaptation methods, such as DWHC, DAM, and CP-MDA. This is because our approach can exploit more sentiment-related characteristics and knowledge for sentiment domain adaptation, such as the general sentiment knowledge shared by various domains, the sentiment graph based domain similarities, and the word-level sentiment polarity relations. Thus, our approach is more suitable for sentiment domain adaptation than these generalpurpose multiple source domain adaptation methods. Another observation from Table 2 is that the performance of our approach is quite close to the upper bound, i.e., All-Training, especially in Electronics and Kitchen domains. This result validates the effectiveness of our approach in sentiment domain adaptation.

Case Study
In this section we conducted several case studies to further explore how our sentiment domain adaptation approach works. As an illustrative example, we selected Electronics domain as the target domain and remaining domains as source domains. The top sentiment words in the global and domainspecific sentiment models learned from the data of multiple source domains are shown in Table 3. A subgraph of the sentiment graph extracted from the unlabeled data of target domain (Electronics) is shown in Fig. 3. The top words in the final domain-specific sentiment model of target domain returned by our approach are shown in Table 3.
From Table 3 we have following observations. First, the global sentiment model extracted from multiple source domains can capture the general sentiment knowledge quite well. It contains many general sentiment words, such as "excellent", "great", "waste" and so on. These general sentiment words convey strong sentiment orientations. In addition, their sentiment polarities are consistent in different domains. Thus, the global sentiment model extracted from multiple source domains has good generalization ability and is more suitable for domain adaptation than the sentiment model trained in a single source domain, which may contain heavy domain-specific sentiment bias. Second, the domain-specific sentiment models can capture rich specific sentiment expressions in each source domain. For example, "easy" is a positive word in Kitchen domain while "return" is a negative word in this domain. Third, different domains have different domain-specific sentiment expressions. For example, "read" is frequently used as a positive word in Book domain, while it is a negative word in DVD domain. Thus, it is important to separate the global and the domain-specific sentiment knowledge. In addition, although different sentiment expressions are used in different domains, similar domains may share many common domain-specific sentiment expressions. For example, "easy" and "works" are positive words in both Electronics and Kitchen domains, and "return" and "broken" are both negative words in them. Thus, transferring the domainspecific sentiment models from similar source domains to target domain is helpful. From Fig. 3 we can see that the sentiment polarity relations in the sentiment graph extracted from massive unlabeled data are reasonable. Words with positive relation scores tend to convey similar sentiments, and words with negative relation scores usually convey opposite sentiments. In addition, this sentiment graph contains rich domain-specific sentiment information in target domain, which is useful to transfer the sentiment knowledge from multiple source domains to target domain. For example, "excellent", "easy", "simple", and "quick" share the same sentiment polarity in Electronics domain according to Fig. 3. We can infer that "easy" is positive in this domain using the sentiment of "excellent" in the global sentiment model and the sentiment relation between "easy" and "excellent". Then we can further infer the sentiments of the domain-specific sentiment words Positive excellent, great, perfect, best, love, easy to, easy, little, the best, works, good, nice, wonderful Negative disappointed, poor, waste, too, bad, worst, back, broken, return, horrible, off, tried, poorly "simple" and "quick" in target domain using the polarity of "easy" and their sentiment relations with it, even if they may be covered by no source domain.

Conclusion
This paper presents a sentiment domain adaptation approach which transfers the sentiment knowledge from multiple source domains to target domain. Our approach consists of two steps. First, we extract both global and domain-specific sentiment knowledge from the data of multiple source domains. Second, we transfer these two kinds of sentiment knowledge to target domain with the help of the words' sentiment graph. We proposed to build words' sentiment graph for target domain by extracting their sentiment polarity relations from massive unlabeled data. Besides, we proposed a novel domain similarity measure based on sentiment graphs, and incorporated the domain similarities between target and different source domains into the domain adaptation process. The experimental results on a benchmark dataset show that our approach can effectively improve the performance of cross-domain sentiment classification.