J Bobadilla et aL /Expert Systems with Applications 38(2011)14609-14623 14615 CPC-WS -PC-DFM Fig. 3. Breakdown of the Mae obtained on MovieLens 1M using the similarity measures: Constrained Pearson Correlation (CPC)combined with Weighted Sum(ws)and Pearson Correlation(PC) combined with Deviation From Mean(DFM). 2.7. 2 Running example If we want to impose a minimum recommendation value: OER, Table 9 shows the coverage measures using MSD and values we add pui>0 K=2 and K=3 2.8.2. Running example 2.7.3. Case of study By making use of Eqs. 38)and(39). as an example, we obtain By comparing Fig. 6 with Fig 4 we can see that there is a reverse the recommendations that can be made to user U3 with N=2 to trend between accuracy and coverage, to the extent that when N=5, using K=2. Table 10 shows these values. choosing a metric we must not take only one of these measures as a reference In Fig. 6 the similarity measure Mean Square Differ- 2.9. Quality of the recommendation: precision and recall ences(MsD) shows much better results than the other metrics however, as we have seen, it also has the worst accuracy. Along 29.1 Formalization lines, Pearson Correlation using z-scores provides us with First, we very low coverage values, in contrast to its good accuracy results Fig7 shows the breakdown of the coverage results using Pear- Xu CIAⅵ∈Xn,ru≠· son correlation with and without z-scores. as we can see. the use of We will use tu to represent the quality preci this standardization process is justified in order to improve the recommendations obtained by making test accuracy, due to its minimum impact on the coverage to the user u, taking a 0 relevancy threshold Simil resent the recall measure obtained by making the 2.8.1 Formalization Assuming that all users accept N test recommendation th. We define Xu as the set of recommendations to user u, and Z as set of n recommendations to user u Zuru≥卟 The following must be true: HCIAⅵi∈Xl,rui=·,pui≠·, #{i∈ Furui≥+#{i∈u≠·Au≥ zgXa,#u=N,Wx∈Zu,Wy∈Xu:Pax≥Pay 9 口DFM■ws■ Z-DFM ZWS口DFM2zwsZ INIAL PR Fig. 4. MAE results obtained on Moviel g the similarity measures: Constrained Pearson Correlation(CPC), Pearson Correlation(PC). Sp SPR), cosine(Cos) and Mean Squared Differences(MSD). making use of the aggregation approaches: Weighted Sum (ws)and Deviation From Mean (DFm)and using z-scores in the input data(Z-)or in similarity values during the prediction process(-22.7.2. Running example Table 9 shows the coverage measures using MSD and values K = 2 and K = 3. 2.7.3. Case of study By comparing Fig. 6 with Fig. 4 we can see that there is a reverse trend between accuracy and coverage, to the extent that when choosing a metric we must not take only one of these measures as a reference. In Fig. 6 the similarity measure Mean Square Differ￾ences (MSD) shows much better results than the other metrics, however, as we have seen, it also has the worst accuracy. Along the same lines, Pearson Correlation using z-scores provides us with very low coverage values, in contrast to its good accuracy results. Fig. 7 shows the breakdown of the coverage results using Pear￾son correlation with and without z-scores. As we can see, the use of this standardization process is justified in order to improve the accuracy, due to its minimum impact on the coverage. 2.8. Top N recommendations 2.8.1. Formalization We define Xu as the set of recommendations to user u, and Z u as the set of N recommendations to user u. The following must be true: Xu  I ^ 8i 2 Xu; ru;i ¼ ; pu;i – ; ð38Þ Zu # Xu; #Zu ¼ N; 8x 2 Zu; 8y 2 Xu : pu;x P pu;y ð39Þ If we want to impose a minimum recommendation value: h 2 R, we add pu,i P h 2.8.2. Running example By making use of Eqs. (38) and (39), as an example, we obtain the recommendations that can be made to user U3 with N = 2 to N = 5, using K = 2. Table 10 shows these values. 2.9. Quality of the recommendation: precision and recall 2.9.1. Formalization First, we redefine the Eq. (38) Xu  I ^ 8i 2 Xu; ru;i – ; pu;i – We will use tu to represent the quality precision measure for recommendations obtained by making N test recommendations to the user u, taking a h relevancy threshold. Similarly, xu will rep￾resent the recall measure obtained by making the same N recom￾mendations to user u. Assuming that all users accept N test recommendations: tu ¼ #fi 2 Zujru;i P hg N ð40Þ xu ¼ #fi 2 Zujru;i P hg #fi 2 Zujru;i P hg þ #fi 2 Zc ujru;i – ^ru;i P hg ð41Þ Fig. 3. Breakdown of the MAE obtained on MovieLens 1M using the similarity measures: Constrained Pearson Correlation (CPC) combined with Weighted Sum (WS) and Pearson Correlation (PC) combined with Deviation From Mean (DFM). Fig. 4. MAE results obtained on MovieLens 1M using the similarity measures: Constrained Pearson Correlation (CPC), Pearson Correlation (PC), Spearman rank correlation (SPR), cosine (COS) and Mean Squared Differences (MSD), making use of the aggregation approaches: Weighted Sum (WS) and Deviation From Mean (DFM) and using z-scores in the input data (Z) or in similarity values during the prediction process (Z). J. Bobadilla et al. / Expert Systems with Applications 38 (2011) 14609–14623 14615