Expert Systems with Applications 38(2011)5101-5109 Contents lists available at ScienceDirect Expert Systems with Applications ELSEVIER journalhomepagewww.elsevier.com/locate/eswa Utilizing various sparsity measures for enhancing accuracy of collaborative recommender systems based on local and global similarities Deepa Anand", Kamal K Bharadwaj School of Computer and Systems Sciences, Jawaharal Nehru University, New Delhi 110 067, India ARTICLE INFO A BSTRACT Collaborative filtering is a popular recommendation technique, which suggests items ing past user-item interactions involving affinities between pairs of users or items. In spite of th uccess they suffer from a range of problems, the most fundamental being that of data sparsity. w heir buge ting matrix is sparse, local similarity measures yield a poor neighborhood set thus affecting the recom- Sparsity measures mendation quality. In such cases global similarity measures can be used to enrich the neighborhood set y considering transitive relationships among users even in the absence of any common experiences. In his work we propose a recommender system framework utilizing both local and global similarities, tak- ng into account not only the overall sparsity in the rating data, but also sparsity at the user-item level. Several schemes are proposed, based on various sparsity measures pertaining to the active user, for the estimation of the parameter a, that allows the variation of the importance given to the global user sim- clarity with regards to local user similarity. Furthermore, we propose an automatic scheme f he various sparsity measures, through evolutionary approach, to obtain a unified measure of sparsity (UMS). In order to take maximum possible advantage of the various sparsity measures relating to an ctive user, a scheme based on the UMs is suggested for estimating a. Experimental results demonstrate that the proposed estimates of a, markedly, outperform the schemes for which a is kept constant across all predictions(fixed- schemes), on accuracy of predicted rating e 2010 Elsevier Ltd. All rights reserved. 1 Introduction Nichols, Oki, Terry, 1992)is the automation of"word of mouth A The explosive growth of the web has led to the problem of infor- (Shardanand Maes, 1995), where opinions gleaned from people, mation overload'-the overwhelming plethora of choices and op- who share similar tastes as the active user, is used in the decision tions available to a user, often varying in quality. The need for a making process. It is based on the assumption that users who solution to this abundance of information and the drive to bridge have agreed in the past tend to agree in the future. the greatest the gap between the vendor and customer in e-commerce has led strength of collaborative techniques is that they are completely to popularity of Web personalization Recommender systems are independent of any machine-readable representation of the the most notable application of Web Personalization. Recom- objects being recommended, and work well for complex objects mender systems are personalization tools which enable users to such as music and movies where variations in taste are responsible be presented information suiting his interests, which are novel, ser- for much of the variation in preferences( Burke, 2002). endipitous and relevant, without being explicitly asked for, thus Collaborative filtering algorithms can be classified as memory- supporting discovery" rather than search". Recommender based or model-based algorithms( Breese, Heckerman, Kadie, systems have become ubiquitous, with their presence everywhere 1998). Memory-based algorithms(Candillier, Meyer, Fessant from recommending books, CDs (Amazon. com, Linden, Smith, 2008: Russell Yoon, 2008)are heuristics based algorithms, which York2003).music[last.fm(www.last.fm),movies[mOvielensutilizetheentireratinghistorytoarriveatpredictionsThesein- (www.movielens.umn.Edu)torecommendinghighriskproductscludethecommonlyimplementedclassofuser-basedanditem such as mutual funds and vacations based CF methods. Model-based recommender systems(Al-Shamri Among the different types of recommender systems, et al 2007: Bell, Koren, Volinsky, 2007)build a user-model in an collaborative filtering is the most widely used and effective off-line learning phase and then apply this model on-line for rec- ommendation. The accuracy offered by memory-based RS, since Corresponding author. Tel. +91 11 9810253296: fax: +91 11 26717528. they examine the entire rating database for prediction, and their E-mailaddress:deepanand209@gmail.com(D.Anand). simplicity, lend to their popularity 0957-4174/s- see front matter o 2010 Elsevier Ltd. All rights reserved doi:10.1016/eswa2010.09141Utilizing various sparsity measures for enhancing accuracy of collaborative recommender systems based on local and global similarities Deepa Anand ⇑ , Kamal K. Bharadwaj School of Computer and Systems Sciences, Jawaharlal Nehru University, New Delhi 110 067, India article info Keywords: Collaborative filtering Recommender systems Similarity measures Sparsity measures abstract Collaborative filtering is a popular recommendation technique, which suggests items to users by exploit￾ing past user-item interactions involving affinities between pairs of users or items. In spite of their huge success they suffer from a range of problems, the most fundamental being that of data sparsity. When the rating matrix is sparse, local similarity measures yield a poor neighborhood set thus affecting the recom￾mendation quality. In such cases global similarity measures can be used to enrich the neighborhood set by considering transitive relationships among users even in the absence of any common experiences. In this work we propose a recommender system framework utilizing both local and global similarities, tak￾ing into account not only the overall sparsity in the rating data, but also sparsity at the user-item level. Several schemes are proposed, based on various sparsity measures pertaining to the active user, for the estimation of the parameter a, that allows the variation of the importance given to the global user sim￾ilarity with regards to local user similarity. Furthermore, we propose an automatic scheme for weighting the various sparsity measures, through evolutionary approach, to obtain a unified measure of sparsity (UMS). In order to take maximum possible advantage of the various sparsity measures relating to an active user, a scheme based on the UMS is suggested for estimating a. Experimental results demonstrate that the proposed estimates of a, markedly, outperform the schemes for which a is kept constant across all predictions (fixed-a schemes), on accuracy of predicted ratings. 2010 Elsevier Ltd. All rights reserved. 1. Introduction The explosive growth of the web has led to the problem of ‘infor￾mation overload’-the overwhelming plethora of choices and op￾tions available to a user, often varying in quality. The need for a solution to this abundance of information and the drive to bridge the gap between the vendor and customer in e-commerce, has led to popularity of Web personalization. Recommender systems are the most notable application of Web Personalization. Recom￾mender systems are personalization tools which enable users to be presented information suiting his interests, which are novel, ser￾endipitous and relevant, without being explicitly asked for, thus supporting ‘‘discovery” rather than ‘‘search”. Recommender systems have become ubiquitous, with their presence everywhere from recommending books, CDs (Amazon.com, Linden, Smith, & York, 2003), music [last.fm(www.last.fm), movies [MovieLens (www.MovieLens.umn.edu)] to recommending high risk products such as mutual funds and vacations. Among the different types of recommender systems, collaborative filtering is the most widely used and effective recommendation technique. Collaborative filtering (Goldberg, Nichols, Oki, & Terry, 1992) is the automation of ‘‘word of mouth” (Shardanand & Maes, 1995), where opinions gleaned from people, who share similar tastes as the active user, is used in the decision making process. It is based on the assumption that users who have agreed in the past tend to agree in the future. The greatest strength of collaborative techniques is that they are completely independent of any machine-readable representation of the objects being recommended, and work well for complex objects such as music and movies where variations in taste are responsible for much of the variation in preferences (Burke, 2002). Collaborative filtering algorithms can be classified as memory￾based or model-based algorithms (Breese, Heckerman, & Kadie, 1998). Memory-based algorithms (Candillier, Meyer, & Fessant, 2008; Russell & Yoon, 2008) are heuristics based algorithms, which utilize the entire rating history to arrive at predictions. These in￾clude the commonly implemented class of user-based and item￾based CF methods. Model-based recommender systems (Al-Shamri et al., 2007; Bell, Koren, & Volinsky, 2007) build a user-model in an off-line learning phase and then apply this model on-line for rec￾ommendation. The accuracy offered by memory-based RS, since they examine the entire rating database for prediction, and their simplicity, lend to their popularity. 0957-4174/$ - see front matter 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2010.09.141 ⇑ Corresponding author. Tel.: +91 11 9810253296; fax: +91 11 26717528. E-mail address: deepanand209@gmail.com (D. Anand). Expert Systems with Applications 38 (2011) 5101–5109 Contents lists available at ScienceDirect Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa