egories [10].Memory-based methods,such as [9,19],try U E RDxN and V E RDxM,where typically D N,M. to predict new ratings by (weighted)averaging the ratings and use R UTV to approximate the rating matrix R. of similar users or items.On the other hand.model-based The column vectors U.i and V.;represent the user-specific methods,such as probabilistic matrix factorization(PMF)[17 and item-specific latent feature vectors,respectively. try to learn a model from data using statistical learning tech- Let Z be the tagging matrix,and each of its elements Zik niques.To the best of our knowledge,there exists only one is the tf*idf value of user i and tag k [18,23]: CF method 25 which attempts to utilize tagging informa- tion to improve item recommendation.This method is a Zk=tf(i,R)×log2 memory-based one.The experimental results in 25 show df() (1) that little improvement could be achieved on item recom- mendation by integrating tagging information into the CF where tf(i,k)is the normalized frequency of tag k appeared procedure under the memory-based framework. in user i's tagging history and df(k)is the number of users In this paper,we propose a novel framework,called tag who have used tag k. informed collaborative filtering (TagiCoFi),to seamlessly in- tegrate tagging information into the model-based CF proce- 2.2 Probabilistic Matrix Factorization dure.More specifically,we use tagging information to regu- PMF [17]seeks to derive the aforementioned low-rank ma- larize the matrix factorization (MF)procedure of PMF [17] trices U and V by analyzing the rating matrix R in a prob- which has been demonstrated to be one of the state-of-the- abilistic framework.The likelihood of the observed ratings art CF methods.Some promising properties of TagiCoFi R is defined as follows: are highlighted here: w To the best of our knowledge,TagiCoFi is the first p(R U,V,o)= iWR,1uV,2)],2② 1 work that incorporates tagging information into a model- based CF system for item recommendation. where N(ru,o2)denotes the (univariate)Gaussian distri- TagiCoFi outperforms its counterpart,PMF,which bution with mean u and variance o discards the tagging information even when it is avail- Putting zero-mean spherical Gaussian priors on the user- able.This shows that the tagging information does specific and item-specific feature vectors: contain useful information for item recommendation and TagiCoFi can utilize it very effectively. p(Ul)=IN(U.:10,I) TagiCoFi can overcome,or at least alleviate,the over- fitting problem 17 suffered by most MF-based CF methods due to the sparsity of the rating matrix. p(Vlav)= ΠN(V|0,I, TagiCoFi can solve the cold-start problem 8,11,20 in that it can give recommendations to novel users who we can obtain the marimum a posteriori (MAP)estimates have no preference on any items. of U and V by minimizing the following objective function defined based on the sum of squared errors: The rest of this paper is organized as follows.In Section 2, N M we will introduce the notations and some preliminaries.Sec- tion 3 describes the details of our model.Experimental re- E= 2∑∑，(R,-UgVP =1j=1 sults are presented in Section 4 and,finally,we conclude the paper in Section 5. +(UTU)+t(VTV). 2 2 (3) 2.NOTATIONS AND PRELIMINARIES where Au =02/ai and Av =a2/av In this section,we first introduce some notations used in this paper.We then briefly review PMF [17 which is closely 3. TAG INFORMED COLLABORATIVE FIL- related to our work. TERING 2.1 Notations Because PMF [17]has achieved state-of-the-art perfor- We use boldface uppercase letters,such as A,to denote mance for CF tasks,we use it as the base model to make matrices,and boldface lowercase letters,such as b,to denote further enhancement by integrating tagging information in a vectors.The ith row and the jth column of a matrix A are principled way.The result is our tag informed collaborative denoted as Ai and A.,respectively.The (i,j)th element filtering method,which will be abbreviated as TagiCoFi in of A is denoted as Aij and the ith element of b as bi. the sequel.The key idea of TagiCoFi is to use tagging in- Suppose there are N users,M items and K tags.Let R be formation to regularize the MF procedure of PMF.More the rating matrix in which Ri;represents the rating of user i specifically,we seek to make two user-specific latent feature for item j.The matrix R is sparse because many elements vectors as similar as possible if the two users have similar are missing,and each such element Rij is assigned the value tagging history. of 0 to indicate that item j has not been rated by user i. In the rest of this section,we first introduce some met- Y is the indicator matrix where Yij is an indicator variable rics for characterizing the similarity between users based on which is equal to 1 if user i rated item j and 0 otherwise tagging information.We then propose our TagiCoFi model MF-based methods 17 seek to find two low-rank matrices based on the computed user similarities.egories [10]. Memory-based methods, such as [9, 19], try to predict new ratings by (weighted) averaging the ratings of similar users or items. On the other hand, model-based methods, such as probabilistic matrix factorization (PMF) [17], try to learn a model from data using statistical learning tech￾niques. To the best of our knowledge, there exists only one CF method [25] which attempts to utilize tagging informa￾tion to improve item recommendation. This method is a memory-based one. The experimental results in [25] show that little improvement could be achieved on item recom￾mendation by integrating tagging information into the CF procedure under the memory-based framework. In this paper, we propose a novel framework, called tag informed collaborative filtering (TagiCoFi), to seamlessly in￾tegrate tagging information into the model-based CF proce￾dure. More specifically, we use tagging information to regu￾larize the matrix factorization (MF) procedure of PMF [17] which has been demonstrated to be one of the state-of-the￾art CF methods. Some promising properties of TagiCoFi are highlighted here: • To the best of our knowledge, TagiCoFi is the first work that incorporates tagging information into a model￾based CF system for item recommendation. • TagiCoFi outperforms its counterpart, PMF, which discards the tagging information even when it is avail￾able. This shows that the tagging information does contain useful information for item recommendation and TagiCoFi can utilize it very effectively. • TagiCoFi can overcome, or at least alleviate, the over- fitting problem [17] suffered by most MF-based CF methods due to the sparsity of the rating matrix. • TagiCoFi can solve the cold-start problem [8, 11, 20] in that it can give recommendations to novel users who have no preference on any items. The rest of this paper is organized as follows. In Section 2, we will introduce the notations and some preliminaries. Sec￾tion 3 describes the details of our model. Experimental re￾sults are presented in Section 4 and, finally, we conclude the paper in Section 5. 2. NOTATIONS AND PRELIMINARIES In this section, we first introduce some notations used in this paper. We then briefly review PMF [17] which is closely related to our work. 2.1 Notations We use boldface uppercase letters, such as A, to denote matrices, and boldface lowercase letters, such as b, to denote vectors. The ith row and the jth column of a matrix A are denoted as Ai∗ and A∗j , respectively. The (i, j)th element of A is denoted as Aij and the ith element of b as bi. Suppose there are N users, M items and K tags. Let R be the rating matrix in which Rij represents the rating of user i for item j. The matrix R is sparse because many elements are missing, and each such element Rij is assigned the value of 0 to indicate that item j has not been rated by user i. Y is the indicator matrix where Yij is an indicator variable which is equal to 1 if user i rated item j and 0 otherwise. MF-based methods [17] seek to find two low-rank matrices U ∈ R D×N and V ∈ R D×M, where typically D  N, M, and use Rˆ = UT V to approximate the rating matrix R. The column vectors U∗i and V∗j represent the user-specific and item-specific latent feature vectors, respectively. Let Z be the tagging matrix, and each of its elements Zik is the tf*idf value of user i and tag k [18, 23]: Zik = tf(i, k) × log2 „ N df(k) « , (1) where tf(i, k) is the normalized frequency of tag k appeared in user i’s tagging history and df(k) is the number of users who have used tag k. 2.2 Probabilistic Matrix Factorization PMF [17] seeks to derive the aforementioned low-rank ma￾trices U and V by analyzing the rating matrix R in a prob￾abilistic framework. The likelihood of the observed ratings R is defined as follows: p(R | U, V, σ 2 ) = YN i=1 YM j=1 h N (Rij | U T ∗iV∗j , σ 2 ) iYij , (2) where N (x | µ, σ2 ) denotes the (univariate) Gaussian distri￾bution with mean µ and variance σ 2 . Putting zero-mean spherical Gaussian priors on the user￾specific and item-specific feature vectors: p(U | σ 2 U ) = YN i=1 N (U∗i | 0, σ 2 U I) p(V | σ 2 V ) = YM j=1 N (V∗j | 0, σ 2 V I), we can obtain the maximum a posteriori (MAP) estimates of U and V by minimizing the following objective function defined based on the sum of squared errors: E = 1 2 XN i=1 XM j=1 Yij (Rij − U T ∗iV∗j ) 2 + λU 2 tr(U T U) + λV 2 tr(V T V), (3) where λU = σ 2 /σ2 U and λV = σ 2 /σ2 V . 3. TAG INFORMED COLLABORATIVE FIL￾TERING Because PMF [17] has achieved state-of-the-art perfor￾mance for CF tasks, we use it as the base model to make further enhancement by integrating tagging information in a principled way. The result is our tag informed collaborative filtering method, which will be abbreviated as TagiCoFi in the sequel. The key idea of TagiCoFi is to use tagging in￾formation to regularize the MF procedure of PMF. More specifically, we seek to make two user-specific latent feature vectors as similar as possible if the two users have similar tagging history. In the rest of this section, we first introduce some met￾rics for characterizing the similarity between users based on tagging information. We then propose our TagiCoFi model based on the computed user similarities