Which Tumblr Post Should I Read Next?

Microblogging sites have emerged as major platforms for bloggers to create and consume posts as well as to follow other bloggers and get informed of their updates. Due to the large number of users, and the huge amount of posts they create, it becomes extremely difﬁcult to identify relevant and interesting blog posts. In this paper, we propose a novel convex collective matrix completion (CCMC) method that effectively utilizes user-item matrix and incorporates additional user activity and topic-based signals to recommend relevant content. The key advantage of CCMC over existing methods is that it can obtain a globally optimal solution and can easily scale to large-scale matrices us-ing Hazan’s algorithm. To the best of our knowledge, this is the ﬁrst work which applies and studies CCMC as a recommendation method in social media. We conduct a large scale study and show significant improvement over existing state-of-the-art approaches.


Introduction
The usage of social media sites has significantly increased over the years. Every minute people upload thousands of new videos on YouTube, write blogs on Tumblr 1 , take pictures on Flickr and Instagram, and send messages on Twitter and Facebook. This has lead to an information overload that makes it hard for people to search and discover relevant information.
Social media sites have attempted to mitigate this problem by allowing users to follow, or subscribe to updates from specific users. However, as 1 www.tumblr.com the number of followers grows over time, the information overload problem returns. One possible solution to this problem is the usage of recommendation systems, which can display to users items and followers that are related to their interests and past activities.
Over time recommender methods have significantly evolved. By observing the history of useritem interactions, the systems learn the preferences of the users and use this information to accurately filter through vast amount of items and allowing the user to quickly discover new, interesting and relevant items such as movies, clothes, books and posts. There is a substantial body of work on building recommendation systems for discovering new items, following people in social media platforms, predicting what people like (Purushotham et al., 2012;Chua et al., 2013;Kim et al., 2013). However, these models either do not consider the characteristics of user-item adoption behaviors or cannot scale to the magnitude of data.
It is important to note that the problem of recommending blog posts differs from the traditional collaborative filtering settings, such as the Netflix rating prediction problem in two main aspects. First, the interactions between the users and blogs are binary in the form of follows and there is no explicit rating information available about the user's preference. The follow information can be represented as an unidirectional unweighted graph and popular proximity measures based on the structural properties of the graph have been applied to the problem (Yin et al., 2011). Second, the blog recommendation inherently has richer side information additional to the conventional user-item matrix (i.e. follower graph).
In Tumblr, text data includes a lot of information, since posts have no limitation in length, compared to other microblogging sites such as Twitter. While such user generated content charac-terizes various blogs, user activity is a more direct and informative signal of user preference as users can explicitly express their interests by liking and reblogging a post. This implies that users who liked or reblogged the same posts are likely to follow similar blogs. The challenge is how to combine multiple sources of information (text and activity) at the same time. For the purpose, we propose a novel convex collective matrix completion (CCMC) social media recommender model, which can scale to million by million matrix using Hazan's algorithm (Gunasekar et al., 2015). Our contributions are as follows: • We propose a novel CCMC based Tumblr blog post recommendation model. • We represent users and blogs with an extensive set of side information sources such as the user/blog activity and text/tags. • We conduct extensive experimental evaluations on Tumblr data and show that our approach significantly outperforms existing methods.

Convex Collective Matrix Completion
In this section, we formulate the Tumblr blog post recommendation task as collective matrix factorization problem and we describe our large-scale convect collective matrix completion method with Hazan's algorithm (Gunasekar et al., 2015).

Model Description
Let X 1 ∈ {0, 1} nr 1 ×nc 1 denote the user-blog (follower) matrix, where n r 1 is the number of users and n c 1 is the number of blogs. In this matrix, if the user i likes blog j, the (i, j)th element is set to 1. In addition to the user-blog matrix, we have other auxiliary matrices denoted by X 2 ∈ R nr 2 ×nc 2 and X 3 ∈ R nr 3 ×nc 3 . For example, if we have an user activity matrix, we can use it as X 2 , where n r 2 = n r 1 and n c 2 is the number of activities. Moreover, if we have the content information of articles, we can use them as X 3 . In this case, n c 1 = n c 3 is the number of blogs, and n r 3 is the number of topics in LDA. Note that, X 1 tends to be a sparse matrix, while X 2 and X 3 tend to be denser matrices. The final goal is to factorize X 1 with the help of the auxiliary matrices X 2 and/or X 3 . First, we form a large matrix M by concatenating all matrices [X v ] V v=1 and then factorizing M together with the regularizations.
For example, for V = 3 , the matrix M is given as This framework is called convex collective matrix completion (CMC) (Singh and Gordon, 2008).
The key advantage of the CCMC approach is that the sparse user-blog matrix X 1 is factorized precisely with the help of the dense matrices X 2 and/or X 3 . Moreover, it has been recently shown that the sample complexity of the CCMC algorithm can be smaller than that of the simple matrix factorization approach (i.e., only factorize X 1 ) (Gunasekar et al., 2015). Finally, the CCMC method can easily incorporate multiple sources of information. Over time if Tumblr provides new signals or if we decide to incorporate new features, CCMC can easily adopt them. Therefore, we believe that CCMC is very suitable for solving the Tumblr recommendation task.

CCMC-Hazan Algorithm
One of the key challenges of CCMC for Tumblr data is the scalability, since Tumblr has more than million users and hundred millions of blog posts. The original CCMC approach adopts Singular Value Thresholding (SVT) to solve the problem, and it works for small scale problems. However, SVT needs to solve N × N dimensional eigenvalue decomposition on each iteration, and thus it is not feasible to deal directly with the Tumblr data. Recently, Gunasekar et al. proposed an Atomic norm minimization algorithm for CCMC (Gunasekar et al., 2015) using the approximate SDP solver of Hazan (Hazan, 2008;Jaggi and Sulovsky, 2010). The optimization problem is given as where X F is the Frobenius norm of matrix X , P Ωv , which extracts the elements in the set, Ω v is the set of non-zero indexes of X v , P v (Z) = Z v ∈ R nr v ×nc v , and η ≥ 0 is a regularization parameter. The Hazan's algorithm for CMF is summarized in Algorithm 1.
The advantage of CCMC-Hazan is that it needs to compute only a top eigenvector on each iteration. Practically, on each iteration t in Algorithm 1, we just need to compute an 1 t 2 -approximate largest eigenvalue of the sparse matrix with |Ω| non-zero elements, which needs O( |Ω| t ) computation using Lanczos algorithm. On the other hand, the original CCMC algorithms adopt Singular Value Thresholding (SVT) method, which converges much faster than CCMC-Hazan. However, the SVT approach has to compute all eigenvalues in each iteration. Thus, CCMC-Hazan is more suited for large-scale dataset than CCMC-SVT. The details of CMC with Hazan's algorithm, please refer to (Gunasekar et al., 2015).

Task Definition
We define our task as given a set of users and their Tumblr post adoption behavior over a period of time, the goal is to build a model that can discover and recommend relevant Tumblr posts to the users.

Evaluation Setup and Data
We set up our Tumblr post evaluation framework by considering the posting or reblogging of an item j by a user i as an adopted item, and otherwise as unadopted. We present each user with top k items sorted by their predicted adoption score and evaluate how many of the recommended items (posts) were actually adopted by the users.
For our post recommendation study, we used Tumblr data from July until September. We used the data from July to August for training, and tested on the data from September. This experimental set up simulates A/B testing.
From the derived data, we sampled 15, 000 active users and 35, 000 posts resulting in 5 million user-item adoptions for training and 8.6 million user-item adoptions for testing. 2 approxEV(X, computes the approximate top eigen vector of X upto error. In post recommendation our CCMC-Hazan method uses an user-item matrix X 1 ∈ {0, 1} 15000×35000 and an item-topic matrix X 2 ∈ R 35000×1000 . To learn the topics we use Latent Dirichlet Allocation (LDA) (Blei et al., 2003). We represent a document as a collection of post description, captions and hashtags. We use 1000 topics for our experiments. Figure 1 shows some examples of the learned topics from the Tumblr posts.

Evaluation Metrics
To evaluate the performance of our collaborative matrix factorization approach for Tumblr post recommendation, we calculate precision (P), recall (R) and normalized discounted cumulative gain (nDCG) for top-k recommended posts.
-P@k as the fraction of adopted items by each user in top-k items in the list. We aveage precision@k across all users.
-R@k as the fraction of adopted items that are successfully discovered in top-k ranked list out of all adopted items by each user. We average recall@k across all users.
-nDCG@k computes the weighted score of adopted items based on the position in the top-k list. We average nDCG@k of all users. We set k to 10 since recommending too many posts is unrealistic. While nDCG@k uses the position of correct answer in the top-k ranked list, it does not penalize for unadopted posts or missing adopted posts in the top-k ranked list. Therefore, to judge the performance of the algorithms, one has to consider all three metrics together. Intuitively a good performing model is the one that has high P@k, R@k and nDCG@k.

Comparison Against State-of-art Models
In addition to evaluating the performance of our algorithm on Tumblr post recommendation, we also conducted a comparative study against existing state-of-the-art models. Item-based 3 The item-based model recommends items that are similar to what the users have already adopted (Karypis, 2001). The model does not use textual information and only uses adopted items to compute the similarity between the items. The similarity metric is the Tanimoto Coefficient, which is used to handle binary ratings. User-based The user-based model recommends items that are adopted by other users with similar taste (Herlocker et al., 1999). The model does not use textual information and only uses adopted items to compute the similarity between the users. Similar to the item-based recommendation, we use the Tanimoto Coefficient. We choose top k items using k-Nearest Neighbor of similar users. MC 4 Alternating least squares (ALS) is matrix completion (MC) based collaborative filtering model, which was originally introduced to model user-movie rating prediction using meansquare loss function with weighted λ regularization (Zhou et al., 2008). The model does not use textual information or signals for adopted items. PMC 5 Probabilistic Matrix Completion (Salakhutdinov and Mnih, 2008) is a probabilistic linear model with Gaussian observation noise that handles very large data sets and is robust to sparse user-item matrix. Similar to MC, PMC models the user-item adoption as the product of two K-dimensional lower-rank user and item hidden variables. The model does not use textual information, but unlike the previous methods it uses information on unadopted items. CF Collaborative Filtering model with softmax function (Guadagni and Little, 1983;Manski, 1975;McFadden, 1974) captures the adoption and un-adoption behavior of users on items in social media. The model does not use textual information, but it uses signals on unadopted items. CF allows us to study the gain of performance in post recommendation when softmax function is used instead of the objective functions used in MC and PMC. CTR Collaborative Topic Regression (Wang and Blei, 2011) was originally introduced to recommend scientific articles. It combines collaborative filtering PMC and probabilistic topic model- ing LDA. It captures two K-dimensional lowerrank user and item hidden variables from user-item adoption matrix and the content of the items. This model uses textual information and signal for unadopted items. Table 1 shows the obtained results of the proposed CCMC-Hazan method against the remaining recommendation models. The simple user and item based recommendations have the lowest performance. This shows that for accurate post recommendation using direct post and user information is insufficient and one needs stronger context driven signals. This is shown in the performance of the CF and CTR methods, which model context information with LDA and perform better than the rest of the models. However, when we compare the performance of our collaborative matrix completion method, we can see that the rest of the models have significantly lower performance. The main reasons are due to the dense information of CCMC-Hazan method and the fact that our method optimizes a convex function whereas the MC, CF and CTF models can get stuck in local optima.

Conclusions
Recommending blog posts is one of the major tasks for user engagement and revenue generation in online microblogging sites such as Tumblr. In this paper, we propose a convex collective matrix completion based recommendation method that effectively utilizes the user-item matrix as well as rich side information from users and/or items. We evaluate the proposed method on real-world dataset collected from Tumblr. Extensive experiments demonstrate the effectiveness of the proposed method in comparison to existing state-ofthe-art approaches.