Expert Systems with Applications 38(2011)14609-14623 Contents lists available at ScienceDirect Expert Systems with Applications ELSEVIER journalhomepagewww.elsevier.com/locate/eswa A framework for collaborative filtering recommender systems Jesus Bobadilla Antonio Hernando, Fernando ortega, Jesus Bernal Universidad Politecnica de Madrid E FilmAffinity. com research team, Crta De valencia, Km. 7, 28031 Madrid, Spain ARTICLE FO A BSTRACT As the use of recommender systems becomes more consolidated on the Net, an increasing need arises to recommender systems evelop some kind of evaluation framework for collaborative filtering measures and methods which is capable of not only testing the prediction and recommendation results, but also of other purposes which imilarity measures until now were considered secondary, such as novelty in the recommendations and the users'trust in these. This paper provides: (a)measures to evaluate the novelty of the users 'recommendations and trust uality in their neighborhoods, (b)equations that formalize and unify the collaborative filtering process and its collaborative fltering evaluation, (c)a framework based on the above- mentioned elements that enables the evaluation of the quality results of any collaborative filtering applied to the desired recommender systems, using four graphs: quality of the predictions, the recommendations, the novelty and the trust. 2011 Elsevier Ltd. All rights reserved 1 Introduction of"Canary Islands"of an important number of individuals who also rated destinations in the Caribbean very highly. This suggestion Recommender systems(RS )are developed to attempt to reduce (recommendation) will often provide the user of the service part of the information overload problem produced on the Net As inspiring information from the collective knowledge of all o opposed to other traditional help systems, such as search engines users of the service. (Google, Yahoo, etc. ) RS generally base their operation on a collab- RS cover a wide variety of applications(Baraglia Silvestri, ative Filtering( CF)process, which provides personalized recom- 2004: Bobadilla, Serradilla, Hernando, 2009: Fesenmaier et al mendations to active users of websites where different elements 2002: Jinghua, Kangning, Shaohong, 2007: Serrano, Viedma, (products, films, holidays, etc. ) can be rated. Olivas, Cerezo, Romero, 2011), although those related to movie RS are inspired by human social behavior, where it is common recommendations are by far the best and most widely-used in to take into account the tastes, opinions and experiences of our the research field(Antonopoulus Salter, 2006: Konstan, Miller, acquaintances when making all kinds of decisions(choosing films riedl, 2004 to watch, selecting schools for our children, choosing products to A substantial part of the research in the area of CF focuses on how buy, etc. ) Obviously, our decisions are modulated according to to determine which users are similar to the given one: in order to our interpretation of the similarity that exists between us and tackle this task, there are fundamentally three approaches: mem- our group of acquaintances, in such a way that we rate the opin- ory-based methods, model-based methods and hybrid approaches ions and experiences of some more highly than othe Memory-based methods(Bobadilla, Ortega, Hernando, in By emulating each step of our own behavior insofar as is press: Bobadilla, Serradilla, Bernal, 2010: Kong, Sun, & Ye ble, the CF process of RS firstly selects the group of users from the Rs 2005: Sanchez, Serradilla, Martinez, Bobadilla, 2008: Symeonidis, website that is most similar to us, and then provides us with a group Nanopoulos, Manolopoulos, 2008) use similarity metrics and act of recommendations of elements that we have not rated yet directly on the ratio matrix that contains the ratings of all users (assuming in this way that they are new to us)and which have been who have expressed their preferences on the collaborative service ated the best by the group of users with similar tastes to us. This these metrics mathematically express a distance between two ay, a trip to the Canary islands could be recommended to an indi- users based on each of their ratios. Model-based methods vidual who has rated different destinations in the Caribbean very ( Adomavicius tuzhilin, 2005) use the ratio matrix to create a highly, based on the positive ratings about the holiday destination model from which the sets of similar users will be established Among the most widely used models we have: Bayesian classifiers (Cho, Hong, Park, 2007), neural networks(Ingoo, Kyong, Tac Corresponding author. Address: Universidad Politecnica de Madrid,, Crta De 2003)and fuzzy systems(Yager, 2003).Generally, commercial RS Valencia, Km. 7, 28031 Madrid, Spain Tel. +34 3365133: fax: +34 913367522. use memory-based methods (Giaglis Lekakos, 2006), whilst model-based methods are usually associated with research RS 0957-4174/s- see front o 2011 Elsevier Ltd. All rights doi:10.1016/eswa2011.05.021A framework for collaborative filtering recommender systems Jesus Bobadilla ⇑ , Antonio Hernando, Fernando Ortega, Jesus Bernal Universidad Politecnica de Madrid & FilmAffinity.com research team, Crta. De Valencia, Km. 7, 28031 Madrid, Spain article info Keywords: Recommender systems Framework Similarity measures Trust Novelty Quality Collaborative filtering abstract As the use of recommender systems becomes more consolidated on the Net, an increasing need arises to develop some kind of evaluation framework for collaborative filtering measures and methods which is capable of not only testing the prediction and recommendation results, but also of other purposes which until now were considered secondary, such as novelty in the recommendations and the users’ trust in these. This paper provides: (a) measures to evaluate the novelty of the users’ recommendations and trust in their neighborhoods, (b) equations that formalize and unify the collaborative filtering process and its evaluation, (c) a framework based on the above-mentioned elements that enables the evaluation of the quality results of any collaborative filtering applied to the desired recommender systems, using four graphs: quality of the predictions, the recommendations, the novelty and the trust. 2011 Elsevier Ltd. All rights reserved. 1. Introduction Recommender systems (RS) are developed to attempt to reduce part of the information overload problem produced on the Net. As opposed to other traditional help systems, such as search engines (Google, Yahoo, etc.), RS generally base their operation on a Collab￾orative Filtering (CF) process, which provides personalized recom￾mendations to active users of websites where different elements (products, films, holidays, etc.) can be rated. RS are inspired by human social behavior, where it is common to take into account the tastes, opinions and experiences of our acquaintances when making all kinds of decisions (choosing films to watch, selecting schools for our children, choosing products to buy, etc.). Obviously, our decisions are modulated according to our interpretation of the similarity that exists between us and our group of acquaintances, in such a way that we rate the opin￾ions and experiences of some more highly than others. By emulating each step of our own behavior insofar as is possi￾ble, the CF process of RS firstly selects the group of users from the RS website that is most similar to us, and then provides us with a group of recommendations of elements that we have not rated yet (assuming in this way that they are new to us) and which have been rated the best by the group of users with similar tastes to us. This way, a trip to the Canary Islands could be recommended to an indi￾vidual who has rated different destinations in the Caribbean very highly, based on the positive ratings about the holiday destination of ‘‘Canary Islands’’ of an important number of individuals who also rated destinations in the Caribbean very highly. This suggestion (recommendation) will often provide the user of the service with inspiring information from the collective knowledge of all other users of the service. RS cover a wide variety of applications (Baraglia & Silvestri, 2004; Bobadilla, Serradilla, & Hernando, 2009; Fesenmaier et al., 2002; Jinghua, Kangning, & Shaohong, 2007; Serrano, Viedma, Olivas, Cerezo, & Romero, 2011), although those related to movie recommendations are by far the best and most widely-used in the research field (Antonopoulus & Salter, 2006; Konstan, Miller, & Riedl, 2004). A substantial part of the research in the area of CF focuses on how to determine which users are similar to the given one; in order to tackle this task, there are fundamentally three approaches: mem￾ory-based methods, model-based methods and hybrid approaches. Memory-based methods (Bobadilla, Ortega, & Hernando, in press; Bobadilla, Serradilla, & Bernal, 2010; Kong, Sun, & Ye, 2005; Sanchez, Serradilla, Martinez, & Bobadilla, 2008; Symeonidis, Nanopoulos, & Manolopoulos, 2008) use similarity metrics and act directly on the ratio matrix that contains the ratings of all users who have expressed their preferences on the collaborative service; these metrics mathematically express a distance between two users based on each of their ratios. Model-based methods (Adomavicius & Tuzhilin, 2005) use the ratio matrix to create a model from which the sets of similar users will be established. Among the most widely used models we have: Bayesian classifiers (Cho, Hong, & Park, 2007), neural networks (Ingoo, Kyong, & Tae, 2003) and fuzzy systems (Yager, 2003). Generally, commercial RS use memory-based methods (Giaglis & Lekakos, 2006), whilst model-based methods are usually associated with research RS. 0957-4174/$ - see front matter 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2011.05.021 Abbreviations: RS, recommender systems; CF, collaborative filtering. ⇑ Corresponding author. Address: Universidad Politecnica de Madrid, Crta. De Valencia, Km. 7, 28031 Madrid, Spain. Tel.: +34 3365133; fax: +34 913367522. E-mail address: jesus.bobadilla@upm.es (J. Bobadilla). Expert Systems with Applications 38 (2011) 14609–14623 Contents lists available at ScienceDirect Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa