J Bobadilla et aL/Knowledge-Based Systems 23(2010)520-528 525 vote is"not voted", which we represent with the symbol . All the newmetric(x,y)=0.5×(1-0.218)=0.39 lists have the same number of elements: I If the values of the votes are normalized in the interval [0.1]. r∈{1.5]}U{} (1-MSD(x, y))A Jaccard(x,y) newmetric(x, yE[O.1] rx:(4.5,·,3,2,·,1,1),ry:(4.3,1,2,·,3.4, using standardized values [01: 4. Planning the experiments rx:(0.75.1,·,0.5.0.25·,0.0) The RS databases [2, 30, 32] that we use in our experiments pres ry:(0.75.0.5.0.0.25,·,0.5.0.75,·) ent the general characteristics summarized in Table 1 We define the cardinality of a list: #l as the number of elements The experiments have been grouped in such a way that the fol- the list l different to· · Accuracy dy;(r…d) Number of perfect predictions. Precision / recall. d,=(-)≠·A≠·可,=m== We consider a perfect prediction to be each situation in which (10) the prediction of the rating recommended to one user in one film matches the value rated by that user for that film. he previous experiments were carried out, depending on the Th =(0.0.25,·,0.0625.·,·,0.562 size of the database for each of the following k-neighborhoods (2)We obtain the MSD(xy)measure computing the arithmetic ues: MovieLens [2..1500] step 50, FilmAffinity [2.2000)step 100, average of the values in the list d 10000 step100, due to the fac that depending on size of each particular rS database, it is necessary to use a different d MSD(x, y)= number of k-neighborhoods in order to display tendencies in the (1) graphs that display their results. The precision recall recommenda- tion quality results have been obtained using a range 2. 20]of rec ommendations and relevance thresholds 0=5 using MovieLens dxy=(0+0.25+0.0625+0.5625)/4=0.218 and NetFlix and 0=9 using FilmAffinity When we use MovieLens and FilmAffinity we use 20% of test MSD(x,y)(11)tends towards zero as the ratings of users x and users taken at random from all the users of the database; with y become more similar and tends towards 1 as they became the remaining 80% we carry out the training. When we use NetFlix, different (we assume that the votes are normalized in given the huge number of users in the database we only use 5% of he interval [o.1D). ts users as test users in all cases we use 20% of test items ()We obtain the Jaccard (x,y) measure computing the propor Table 2 shows the numerical data exposed in this section. tion between the number of positions [1. in which there are elements different to. in both rx and ry regarding the 5. results number of positions [1] in which there are elements differ- ent to· In x or In ry: In this section we present the results obtained using the Jaccard(x,y)=nt=rx+#一#d abases specified in Table 1. Fig. 6 shows the results obtained (12) MovieLens, Fig. 7 shows those obtained with NetFlix and Fig 8 ds to FilmAffinity in our example: 4/(6+6-4)=0.5. Graph 6a shows the MAE error obtained in MovieLens by apply (4)We combine the above elements in the final equation g Pearson correlation(dashed) and the proposed metric(contin- newmetric(x,y)=Jaccard(x y)x(1-MSD(x y)).(13) uous). The new metric achieves significant fewer errors in practically all the experiments carried out (by varying the number of k-neighborhoods ) The percentage improvement average is around 0.2 stars in the most commonly used values of k(50, 100, Main parameters of the databases used in the experiments. 150.200) Graph 6B shows us the coverage. Small values of k pl Movielens FilmAffinity NetFlix small percentages in the capacity for prediction, as it is more mprobable that the few neighbors of a test user have voted for a Number o 1128 17770 1000209 19126278 100480507 film that this user has not voted for. As the number of neighbo ax values 1-10 increases, the probability that at least one of them has voted for the film also increases, as shown in the graph. Table 2 Main parameters used in the experiments K(MAE, coverage, perfect predictions) Pre Test users(%) Test items(%) Movielens 1M 2.20 FilmAffinity NetFlix 2.201vote is ‘‘not voted”, which we represent with the symbol . All the lists have the same number of elements: I. Example: rb a 2 ½ f g 1::5 [ fg; rx : ð4; 5; ; 3; 2; ; 1; 1Þ; ry : ð4; 3; 1; 2; ; 3; 4; Þ: using standardized values [0..1]: rx : ð Þ 0:75; 1; ; 0:5; 0:25; ; 0; 0 ; ry : ð Þ 0:75; 0:5; 0; 0:25; ; 0:5; 0:75; : We define the cardinality of a list: #l as the number of elements in the list l different to . (1) We obtain the list dx;y : d1 x;y; d2 x;y; d3 x;y; ... ; dI x;y  j di x;y ¼ ri x ri y  2 8ijri x – ^ri y – ; di x;y ¼ 8ijri x ¼_ ri y ¼ ; ð10Þ in our example: dx;y ¼ ð0; 0:25; ; 0:0625; ; ; 0:5625; Þ: (2) We obtain the MSD(x,y) measure computing the arithmetic average of the values in the list dx,y MSDðx; yÞ ¼ dx;y ¼ P i¼1::I;di x;y– di x;y #dx;y ; ð11Þ in our example: dx;y ¼ ð0 þ 0:25 þ 0:0625 þ 0:5625Þ=4 ¼ 0:218 MSD(x,y) (11) tends towards zero as the ratings of users x and y become more similar and tends towards 1 as they became more different (we assume that the votes are normalized in the interval [0..1]). (3) We obtain the Jaccard(x,y) measure computing the propor￾tion between the number of positions [1..I] in which there are elements different to in both rx and ry regarding the number of positions [1..I] in which there are elements differ￾ent to in rx or in ry: Jaccardðx; yÞ ¼ rx \ ry rx [ ry ¼ #dx;y #rx þ #ry #dx;y ; ð12Þ in our example: 4/(6 + 64) = 0.5. (4) We combine the above elements in the final equation: newmetricð Þ¼ x; y Jaccardð Þ x; y ð Þ 1 MSDð Þ x; y ; ð13Þ in the running example: newmetricðx; yÞ ¼ 0:5  ð1 0:218Þ ¼ 0; 391: If the values of the votes are normalized in the interval [0..1], then: ð Þ^ 1 MSDð Þ x; y Jaccardð Þ^ x; y newmetricð Þ2½ x; y 0::1: 4. Planning the experiments The RS databases [2,30,32] that we use in our experiments pres￾ent the general characteristics summarized in Table 1. The experiments have been grouped in such a way that the fol￾lowing can be determined: Accuracy. Coverage. Number of perfect predictions. Precision/recall. We consider a perfect prediction to be each situation in which the prediction of the rating recommended to one user in one film matches the value rated by that user for that film. The previous experiments were carried out, depending on the size of the database, for each of the following k-neighborhoods val￾ues: MovieLens [2..1500] step 50, FilmAffinity [2..2000] step 100, NetFlix [2..10000] step100, due to the fact that depending on the size of each particular RS database, it is necessary to use a different number of k-neighborhoods in order to display tendencies in the graphs that display their results. The precision/recall recommenda￾tion quality results have been obtained using a range [2..20] of rec￾ommendations and relevance thresholds h = 5 using MovieLens and NetFlix and h = 9 using FilmAffinity. When we use MovieLens and FilmAffinity we use 20% of test users taken at random from all the users of the database; with the remaining 80% we carry out the training. When we use NetFlix, given the huge number of users in the database, we only use 5% of its users as test users. In all cases we use 20% of test items. Table 2 shows the numerical data exposed in this section. 5. Results In this section we present the results obtained using the dat￾abases specified in Table 1. Fig. 6 shows the results obtained with MovieLens, Fig. 7 shows those obtained with NetFlix and Fig. 8 cor￾responds to FilmAffinity. Graph 6A shows the MAE error obtained in MovieLens by apply￾ing Pearson correlation (dashed) and the proposed metric (contin￾uous). The new metric achieves significant fewer errors in practically all the experiments carried out (by varying the number of k-neighborhoods). The percentage improvement average is around 0.2 stars in the most commonly used values of k (50, 100, 150, 200). Graph 6B shows us the coverage. Small values of k produce small percentages in the capacity for prediction, as it is more improbable that the few neighbors of a test user have voted for a film that this user has not voted for. As the number of neighbors increases, the probability that at least one of them has voted for the film also increases, as shown in the Graph. Table 1 Main parameters of the databases used in the experiments. MovieLens FilmAffinity NetFlix Number of users 4382 26447 480189 Number of movies 3952 21128 17770 Number of ratings 1000209 19126278 100480507 Min and max values 1–5 1–10 1–5 Table 2 Main parameters used in the experiments. K (MAE, coverage, perfect predictions) Precision/recall Test users (%) Test items (%) Range Step N h MovieLens 1M [2..1500] 50 [2..20] 5 20 20 FilmAffinity [2..2000] 100 [2..20] 5 20 20 NetFlix [2..10000] 100 [2..20] 9 5 20 J. Bobadilla et al. / Knowledge-Based Systems 23 (2010) 520–528 525