J. Bobadilla et al/ Knowledge-Based Systems 24(2011)1310-1316 1313 3.4. Genetic operators proposed metric may only work for a specific RS As usual, we will use the typical quality measures: Mean Absolute Error(MAE).cov- As usual, our genetic algorithm uses the common operators in erage, precision and recall. genetic algorithms: selection, crossover (recombination) and The results of our proposed genetic algorithm are compare utation. Since we have obtained quick satisfactory results using with the ones obtained using traditional metrics on RS collabora hese three classical operators, we have not used other possible tive filtering: Pearson correlation, cosine and Mean Squared Differ perators like migration, regrouping or colonization-extinction. ences. these baseline metrics are used to compare our results The features of our genetic operators are: obtained from the genetic algorithm with the ones obtained from these metrics tion. That is to say, the selection probability of an individual G The genetic algorithm described previously is run 20 times Selection. The chosen method is the fitness proportional selec- ch time the genetic algorithm runs, it provides a similarity func depends on its fitness level. tion simw associated to the vector w (see Equation(1). Associat Crossover. We use the one-point crossover technique. The to this similarity function, simw, we calculate the typical quality crossover probability is 0.8 measures: MAE, coverage precision and recall. For each quality Mutation We use a single point mutation technique in order to measure we show graphics depicting the best(labeled'BestGA') introduce diversity. the mutation probability is 0.02 and worse value (labeled'WorstGA)obtained through the 20 run nings. The area between them is displayed in grey color. Each gra- 3.5. Reproduction and termination phic is described with the results obtained with the metrics Pearson correlation(COR), cosine(COS)and Mean Squared Differ- When using the reproduction operator, 1000 individuals are ences(MSD). generated in each generation using random crossover between As we have stated above, we only use a part of the whole rS the best-fit parent individuals (training users and training items)in order to obtain an optimal The number of individuals keeps constant through every gener- similarity function. Once obtained this similarity function, we car ation As we have stated above, the population is 100 when using ry out our experiments using only the test users(the users which Movielens and Netflix as recommender systems and 200 whe are not used in the Ga algorithm) and the test items (the items using FilmAffinity as RS. We only keep the 5% of the best individ- which are not used in the GA algorithm) uals from each generation to obtain the next one(elitist selection). When using Movielens and FilmAffinity, we use 20% of users The genetic algorithm stops when there is an individual in the (randomly chosen)as test users and 20% of items(randomly cho- population with a fitness value lower than a constant We use sen)as test items. We use the rest 80% of users and items as train- 7=0.78 for the Movielens and Netflix RS(in these databases, we ing users and training items. When using Netflix, we also use 20% have that m=l and M=5)and we use ?=1.15 for the RS FilmAf- of items as test items but we only use 5% of the users as test users nity(in this database, we have that m=1 and M=10). We have due to the huge number of users in the database. used these values according to the optimal values found in [7- The constant K related to the number of neighbors for each user. varies between 50 and 800. These values enable us to view the trends on the graphics. The precision and recall recommendation quality results have been calculated using different values of N 4.1. Prediction and recommendation quality measures number of recommendations made) from 2 to 20. When using Movielens and Netflix the relevant threshold value =5. and =9 when using FilmAffinity (as stated above while the possible value In this section, we show different experiments made in order to in the ranking of item is(1,.,5)in Netflix and Movielens, it is prove that the method proposed works for any different RS. In this vay, we have used in the experiments the following three different (1,.,10)in FilmAffinity ). Table 2 shows the numerical parame ters used in the experiments Movielens: This database is a reference in research or RS over 4.2. Performance experiment he last years. In this database, we can compare the results obtained using our method with the results previously obtained In this section, we describe an experiment which proves that using other methods. predictions and recommendations can be made much faster when FilmAffinity: This database contains a large amount of data. This using the similarity function obtained with our GA-method (see e database provides an experiment to prove that our gA method (1)than when using the traditional metrics(Pearson Correlation, is useful for databases with a large amount of data. cosine, Mean Squared Differences). Indeed, since the similarity Netflix: This database offers us a very large database on which functions in our GA method (see Eq (1)) are much simpler than metrics,algorithms, programming and systems are put to thethe formulae involved in the traditionalmetrics, we can obtain test the k-neighbors for a user much faster (in this way, prediction and recommendations can be obtained much faster). Since our method provides high values in the quality measures he experiment designed compares the performance of differ pplied on the three databases, we reduce the danger that the ent similarity measures by calculating the similarity between users Table 2 Main parameters used in the experiments K(MAE, coverage Precision/recall Test users (%) Test items(‰) Genetic Movielens 1 M Film affi Netflix 000 0003.4. Genetic operators As usual, our genetic algorithm uses the common operators in genetic algorithms: selection, crossover (recombination) and mutation. Since we have obtained quick satisfactory results using these three classical operators, we have not used other possible operators like migration, regrouping or colonization-extinction. The features of our genetic operators are: Selection. The chosen method is the fitness proportional selec￾tion. That is to say, the selection probability of an individual depends on its fitness level. Crossover. We use the one-point crossover technique. The crossover probability is 0.8. Mutation. We use a single point mutation technique in order to introduce diversity. The mutation probability is 0.02. 3.5. Reproduction and termination When using the reproduction operator, 1000 individuals are generated in each generation using random crossover between the best-fit parent individuals. The number of individuals keeps constant through every gener￾ation. As we have stated above, the population is 100 when using Movielens and Netflix as recommender systems and 200 when using FilmAffinity as RS. We only keep the 5% of the best individ￾uals from each generation to obtain the next one (elitist selection). The genetic algorithm stops when there is an individual in the population with a fitness value lower than a constant c. We use c = 0.78 for the Movielens and Netflix RS (in these databases, we have that m = 1 and M = 5) and we use c = 1.15 for the RS FilmAf- finity (in this database, we have that m = 1 and M = 10). We have used these values according to the optimal values found in [7]. 4. Experiments 4.1. Prediction and recommendation quality measures In this section, we show different experiments made in order to prove that the method proposed works for any different RS. In this way, we have used in the experiments the following three different databases (Table 1) of real RS: Movielens: This database is a reference in research or RS over the last years. In this database, we can compare the results obtained using our method with the results previously obtained using other methods. FilmAffinity: This database contains a large amount of data. This database provides an experiment to prove that our GA method is useful for databases with a large amount of data. Netflix: This database offers us a very large database on which metrics, algorithms, programming and systems are put to the test. Since our method provides high values in the quality measures applied on the three databases, we reduce the danger that the proposed metric may only work for a specific RS. As usual, we will use the typical quality measures: Mean Absolute Error (MAE), cov￾erage, precision and recall. The results of our proposed genetic algorithm are compared with the ones obtained using traditional metrics on RS collabora￾tive filtering: Pearson correlation, cosine and Mean Squared Differ￾ences. These baseline metrics are used to compare our results obtained from the genetic algorithm with the ones obtained from these metrics. The genetic algorithm described previously is run 20 times. Each time the genetic algorithm runs, it provides a similarity func￾tion simw associated to the vector w (see Equation (1)). Associated to this similarity function, simw, we calculate the typical quality measures: MAE, coverage, precision and recall. For each quality measure we show graphics depicting the best (labeled ‘BestGA’) and worse value (labeled ‘WorstGA’) obtained through the 20 run￾nings. The area between them is displayed in grey color. Each gra￾phic is described with the results obtained with the metrics: Pearson correlation (COR), cosine (COS) and Mean Squared Differ￾ences (MSD). As we have stated above, we only use a part of the whole RS (training users and training items) in order to obtain an optimal similarity function. Once obtained this similarity function, we car￾ry out our experiments using only the test users (the users which are not used in the GA algorithm) and the test items (the items which are not used in the GA algorithm). When using Movielens and FilmAffinity, we use 20% of users (randomly chosen) as test users and 20% of items (randomly cho￾sen) as test items. We use the rest 80% of users and items as train￾ing users and training items. When using Netflix, we also use 20% of items as test items but we only use 5% of the users as test users due to the huge number of users in the database. The constant K related to the number of neighbors for each user, varies between 50 and 800. These values enable us to view the trends on the graphics. The precision and recall recommendation quality results have been calculated using different values of N (number of recommendations made) from 2 to 20. When using Movielens and Netflix the relevant threshold value = 5, and = 9 when using FilmAffinity (as stated above, while the possible value in the ranking of item is {1, ... , 5} in Netflix and Movielens, it is {1, ... , 10} in FilmAffinity). Table 2 shows the numerical parame￾ters used in the experiments. 4.2. Performance experiment In this section, we describe an experiment which proves that predictions and recommendations can be made much faster when using the similarity function obtained with our GA-method (see Eq. (1)) than when using the traditional metrics (Pearson Correlation, cosine, Mean Squared Differences). Indeed, since the similarity functions in our GA method (see Eq. (1)) are much simpler than the formulae involved in the traditionalmetrics, we can obtain the k-neighbors for a user much faster (in this way, predictions and recommendations can be obtained much faster). The experiment designed compares the performance of differ￾ent similarity measures by calculating the similarity between users Table 2 Main parameters used in the experiments. K (MAE, coverage) Precision/recall Test users (%) Test items (%) Genetic runs Range Step K N h Movielens 1 M {50, ... , 800} 50 100 {2, ... , 20} 5 20 20 20 Film affinity {50, ... , 800} 50 150 {2, ... , 20} 5 20 20 20 Netflix {50, ... , 800} 50 200 {2, ... , 20} 9 5 20 20 J. Bobadilla et al. / Knowledge-Based Systems 24 (2011) 1310–1316 1313