Detecting"Smart"Spammers On Social Network: A Topic Model Approach

Spammer detection on social network is a challenging problem. The rigid anti-spam rules have resulted in emergence of"smart"spammers. They resemble legitimate users who are difficult to identify. In this paper, we present a novel spammer classification approach based on Latent Dirichlet Allocation(LDA), a topic model. Our approach extracts both the local and the global information of topic distribution patterns, which capture the essence of spamming. Tested on one benchmark dataset and one self-collected dataset, our proposed method outperforms other state-of-the-art methods in terms of averaged F1-score.


Introduction
Microblogging such as Twitter and Weibo is a popular social networking service, which allows users to post messages up to 140 characters. There are millions of active users on the platform who stay connected with friends. Unfortunately, spammers also use it as a tool to post malicious links, send unsolicited messages to legitimate users, etc. A certain amount of spammers could sway the public opinion and cause distrust of the social platform. Despite the use of rigid anti-spam rules, human-like spammers whose homepages having photos, detailed profiles etc. have emerged. Unlike previous "simple" spammers, whose tweets contain only malicious links, those "smart" spammers are more difficult to distinguish from legitimate users via content-based features alone (Ferrara et al., 2014). *

Corresponding Author
There is a considerable amount of previous work on spammer detection on social platforms. Researcher from Twitter Inc. (Chu et al., 2010) collect bot accounts and perform analysis on the user behavior and user profile features. Lee et al. (2011) use the so-called social honeypot by alluring social spammers' retweet to build a benchmark dataset, which has been extensively explored in our paper. Some researchers focus on the clustering of urls in tweets and network graph of social spammers (Yang et al., 2012;Wang et al., 2015;Wang, 2010;Yang et al., 2011), showing the power of social relationship features.As for content information modeling, (Hu et al., 2013) apply improved sparse learning methods. However, few studies have adopted topic-based features. Some researchers (Liu et al., 2014) discuss topic characteristics of spamming posts, indicating that spammers are highly likely to dwell on some certain topics such as promotion. But this may not be applicable to the current scenario of smart spammers.
In this paper, we propose an efficient feature extraction method. In this method, two new topicbased features are extracted and used to discriminate human-like spammers from legitimate users. We consider the historical tweets of each user as a document and use the Latent Dirichlet Allocation (LDA) model to compute the topic distribution for each user. Based on the calculated topic probability, two topic-based features, the Local Outlier Standard Score (LOSS) which captures the user's interests on different topics and the Global Outlier Standard Score (GOSS) which reveals the user's interests on specific topic in comparison with other users', are extracted. The two features contain both local and global information, and the combination of them can distinguish human-like spammers effectively.
To the best of our knowledge, it is the first time that features based on topic distributions are used in spammer classification. Experimental results on one public dataset and one self-collected dataset further validate that the two sets of extracted topicbased features get excellent performance on humanlike spammer classification problem compared with other state-of-the-art methods. In addition, we build a Weibo dataset, which contains both legitimate users and spammers.
To summarize, our major contributions are twofold: • We extract topic-based features (GOSS and LOSS) for spammer detection, which outperform state-of-the-art methods. • We build a dataset of Chinese microblogs for spammer detection.
In the following sections, we first propose the topic-based features extraction method in Section 2, and then introduce the two datasets in Section 3. Experimental results are discussed in Section 4, and we conclude the paper in Section 5. Future work is presented in Section 6.

Methodology
In this section, we first provide some observations we obtained after carefully exploring the social network, then the LDA model is introduced. Based on the LDA model, the ways to obtain the topic probability vector for each user and the two topic-based features are provided.

Observation
After exploring the homepages of a substantial number of spammers, we have two observations. 1) social spammers can be divided into two categories. One is content polluters, and their tweets are all about certain kinds of advertisement and campaign. The other is fake accounts, and their tweets resemble legitimate users' but it seems they are simply random copies of others to avoid being detected by antispam rules. 2) For legitimate users, content polluters and fake accounts, they show different patterns on topics which interest them. • Legitimate users mainly focus on limited topics which interest him. They seldom post contents unrelated to their concern.
• Content polluters concentrate on certain topics.
• Fake accounts focus on a wide range of topics due to random copying and retweeting of other users' tweets.
• Spammers and legitimate users show different interests on some topics e.g. commercial, weather, etc.
To better illustrate our observation, Figure. 1 shows the topic distribution of spammers and legitimate users in two employed datasets(the Honeypot dataset and Weibo dataset). We can see that on both topics (topic-3 and topic-11) there exists obvious difference between the red bars and green bars, representing spammers and legitimate users. On the Honeypot dataset, spammers have a narrower shape of distribution (the outliers on the red bar tail are not counted) than that of legitimate users. This is because there are more content polluters than fake accounts. In other word, spammers in this dataset tend to concentrate on limited topics. While on the Weibo dataset, fake accounts who are interested in different topics take large proportion of spammers. Their distribution is more flat (i.e. red bars) than that of the legitimate users. Therefore we can detect spammers by means of the difference of their topic distribution patterns.  Each document i is deemed as a bag of words W = {w i1 , w i2 , ..., w iM } and M is the number of words. Each word is attributable to one of the document's topics Z = {z i1 , z i2 , ..., z iK } and K is the number of topics. ψ k is a multinomial distribution over words for topic k. θ i is another multinomial distribution over topics for document i. The smoothed generative model is illustrated in Figure. 2. α and β are hyper parameter that affect scarcity of the document-topic and topic-word distributions. In this paper, α, β and K are empirically set to 0.3, 0.01 and 15. The entire content of each Twitter user is regarded as one document. We adopt Gibbs Sampling (Griffiths and Steyvers, 2004) to speed up the inference of LDA. Based on LDA, we can get the topic probabilities for all users in the employed dataset as: X = [X i ; X 2 ; · · · ; X n ] ∈ R n×K , where n is the number of users. Each element X i = [p (z 1 ) p (z 2 ) · · · p (z K )] ∈ R 1×K is a topic probability vector for the i th document. X i is the raw topic probability vector and our features are developed on top of it.

Topic-based Features
Using the LDA model, each person in the dataset is with a topic probability vector X i . Assume x ik ∈ X i denotes the likelihood that the i th tweet account favors k th topic in the dataset. Our topic based features can be calculated as below.
Global Outlier Standard Score measures the degree that a user's tweet content is related to a certain topic compared to the other users. Specifically, the "GOSS" score of user i on topic k can be calculated as Eq. (1): The value of GOSS (x ik ) indicates the interest-ing degree of this person to the k th topic. Specifically, if GOSS (x ik ) > GOSS (x jk ), it means that the i th person has more interest in topic k than the j th person. If the value GOSS (x ik ) is extremely high or low, the i th person showing extreme interest or no interest on topic k which will probably be a distinctive pattern in the fowllowing classfication. Therefore, the topics interested or disliked by the i th person can be manifested by f i GOSS = [GOSS(x i1 ) · · · GOSS(x iK )], from which the pattern of the interested topics with regarding to this person is found. Denote f i GOSS = [GOSS(x i1 ) · · · GOSS(x iK )] our first topic-based feature, and it hopefully can get good performance on spammer detection.
Local Outlier Standard Score measures the degree of interest someone shows to a certain topic by considering his own homepage content only. For instance, the "LOSS" score of account i on topic k can be calculated as Eq. ( 2): (2) µ(x i ) represents the averaged interesting degree for all topics with regarding to i th user and his tweet content. Similarly to GOSS, the topics interested or disliked by the i th person via considering his single post information can be manifested by f i LOSS = [LOSS(x i1 ) · · · LOSS(x iK )], and LOSS becomes our second topic-based features for the i th person.

Dataset
We use one public dataset Social Honeypot dataset and one self-collected dataset Weibo dataset to validate the effectiveness of our proposed features. Preprocessing: Before directly performing the experiments on the employed datasets, we first delete some accounts with few posts in the two employed since the number of tweets is highly indicative of spammers. For the English Honeypot dataset, we remove stopwords, punctuations, non-ASCII words and apply stemming. For the Chinese Weibo dataset, we perform segmentation with "Jieba" 2 , a Chinese text segmentation tool. After preprocessing steps, the Weibo dataset contains 2197 legitimate users and 802 spammers, and the honeypot dataset contains 2218 legitimate users and 2947 spammers. It is worth mentioning that the Honeypot dataset has been slashed because most of the Twitter accounts only have limited number of posts, which are not enough to show their interest inclination.

Actual
Polluter TP FN Legitimate FP TN

Evaluation Metrics
The evaluating indicators in our model are show in Table 2 . We calculate precision, recall and F1-score (i.e. F1 score) as in Eq. (3). Precision is the ratio of selected accounts that are spammers. Recall is the ratio of spammers that are detected so. F1-score is the harmonic mean of precision and recall.

Performance Comparisons with Baseline
Three baseline classification methods: Support Vector Machines (SVM), Adaboost, and Random Forests are adopted to evaluate our extracted features. We test each classification algorithm with scikit-learn (Pedregosa et al., 2011) and run a 10fold cross validation. On each dataset, the em-

Comparison with Other Features
To compare our extracted features with previously used features for spammer detection, we use three most discriminative feature sets according to Lee et al. (2011)( Though there is a slight decline in terms of recall. By SVM, we get comparative results on recall and F1-score but about 3.5% improvement on precision.

Conclusion
In this paper, we propose a novel feature extraction method to effectively detect "smart" spammers who post seemingly legitimate tweets and are thus difficult to identify by existing spammer classification methods. Using the LDA model, we obtain the topic probability for each Twitter user. By utilizing the topic probability result, we extract our two topicbased features: GOSS and LOSS which represent the account with global and local information. Experimental results on a public dataset and a self-built Chinese microblog dataset validate the effectiveness of the proposed features.

Future Work
In future work, the combination method of local and global information can be further improved to maximize their individual strengths. We will also apply decision theory to enhancing the performance of our proposed features. Moreover, we are also building larger datasets on both Twitter and Weibo to validate our method. Moreover, larger datasets on both Twitter and Weibo will be built to further validate our method.