4.2 Evaluation criteria dataset. In our experiment we set the size of the hi- erarchy to be 12. Our greedy partitioning algorithm In this paper we adopt the Absolute Deviation metric arrived at the following basic classes: drama, comedy, [MRK97, BHK98] to assess the quality of our CF algo- classic, action, art-foreign Drama, thriller, romance rithms. We divide the data into a training and test set medy, none, family, horror, action Thriller, other using the method described above and build a PRM using the training data. We then iterate over each user n the test set. allowing each user to become the active Algorithm Absolute Deviation user. for the active user we then iterate over his set 0.994 of votes, Pa, allowing each vote to become the active 1.10 vote; the remaining votes are used in the PRM model N 1.066 (if needed ) The predicted vote for the active user a 2.136 on movie j is denoted pa, j, whereas the actual vote is PRM 1.060 denoted u The average absolute deviation wnere ma is the number of vote predictions made, is: Table 2: Absolute deviation scoring results for each- Movie dataset. Lower scores are better Sa ma jENa By applying hPRMs to the eachMovie dataset, we are able to reduce the absolute deviation error from 1.26 The absolute deviation for the dataset as a whole is (with standard PRMs)to 1.06. Again, for comparison arrived at by averaging this score over all the users in we include results from BHK98]; however, in since the test set of users hPRMs are able to leverage other he user has made in making predictions, we use the All-But-One 4. 3 Standard PrMs results presented in [BHK98, where the prediction al is able to use all of the ac Igorithm Absolute Deviation (except for the current active vote) in making a pre- 1.257 diction. As one can see by comparing Table 1 to Table 2, including the additional voting information results in a substantial reduction in error rate for most of the ⅤSIM other four algorithms PRM 1.26 hPRMs not only provide a significant performance ad- vantage over standard PRMs. but are also able to out- Table 1: Absolute Deviation scoring results for Each- l but one of the other four al Movie dataset. Lower scores are better In our experiments we were able to achieve an abso- 5 Future Work lute deviation error of 1. 26. For comparison, we have included the results from [BHK98]; in this paper four In this paper we learned a fairly broad hierarchy based CF algorithms were tested: correlation(CR), Bayesian on various sub-genres in the EachMovie dataset. How- Vector Similarity(VSIM). We have elected to include ever, hPRMs allow for arbitrarily specific class hierar chies, where a leaf-node entry for H[X might be a the results from[BHK98 where algorithms were given handful (or even just one)movie. This could be ex- two votes out of the non-active votes to use in making the prediction, since the standard PRM model does ploited when there are certain indicator movies in a not have any direct dependency on other Votes Genre that may accurately predict a user's votes on other movies in that(or other) Genres, and that fur In this experiment standard PRMs are able to outper- thermore most users have seen. For example, whether form the vSIM algorithm, and is competitive with the a user likes science fiction movies or not, they have correlation-based algorithm. However, both Bayesian likely seen the Star Wars Trilogy; this fact could b Clustering and the Bayesian Network model have su- leverage by making the Star Wars movies a basic sub- perior results in this context class in the class hierarchy, and subsequently used t learn new dependencies of the type if a user likes the 4. 4 Hierarchical PRMs Star Wars movies, they will in general like science fi tion movies. Learning such a hierarchy is a challenging The first part of the experiment for hPRMs was task that would likely significantly improve the perfor- constructing a class hierarchy from the EachMovie mance of hPRMs4.2 Evaluation Criteria In this paper we adopt the Absolute Deviation metric [MRK97, BHK98] to assess the quality of our CF algo￾rithms. We divide the data into a training and test set using the method described above, and build a PRM using the training data. We then iterate over each user in the test set, allowing each user to become the active user. For the active user we then iterate over his set of votes, Pa, allowing each vote to become the active vote; the remaining votes are used in the PRM model (if needed). The predicted vote for the active user a on movie j is denoted pa,j , whereas the actual vote is denoted va,j . The average absolute deviation, where ma is the number of vote predictions made, is: Sa = 1 ma X j∈Pa |pa,j − va,j | (1) The absolute deviation for the dataset as a whole is arrived at by averaging this score over all the users in the test set of users. 4.3 Standard PRMs Algorithm Absolute Deviation CR 1.257 BC 1.127 BN 1.143 VSIM 2.113 PRM 1.26 Table 1: Absolute Deviation scoring results for Each￾Movie dataset. Lower scores are better. In our experiments we were able to achieve an abso￾lute deviation error of 1.26. For comparison, we have included the results from [BHK98]; in this paper four CF algorithms were tested: correlation (CR), Bayesian Clustering (BC), a Bayesian Network model (BN), and Vector Similarity (VSIM). We have elected to include the results from [BHK98] where algorithms were given two votes out of the non-active votes to use in making the prediction, since the standard PRM model does not have any direct dependency on other V otes. In this experiment standard PRMs are able to outper￾form the VSIM algorithm, and is competitive with the correlation-based algorithm. However, both Bayesian Clustering and the Bayesian Network model have su￾perior results in this context. 4.4 Hierarchical PRMs The first part of the experiment for hPRMs was constructing a class hierarchy from the EachMovie dataset. In our experiment we set the size of the hi￾erarchy to be 12. Our greedy partitioning algorithm arrived at the following basic classes: drama, comedy, classic, action, art-foreignDrama, thriller, romance￾comedy, none, family, horror, actionThriller, other. Algorithm Absolute Deviation CR 0.994 BC 1.103 BN 1.066 VSIM 2.136 hPRM 1.060 Table 2: Absolute Deviation scoring results for Each￾Movie dataset. Lower scores are better. By applying hPRMs to the EachMovie dataset, we are able to reduce the absolute deviation error from 1.26 (with standard PRMs) to 1.06. Again, for comparison we include results from [BHK98]; however, in since hPRMs are able to leverage other votes the user has made in making predictions, we use the All-But-One results presented in [BHK98], where the prediction al￾gorithm is able to use all of the active user’s votes (except for the current active vote) in making a pre￾diction. As one can see by comparing Table 1 to Table 2, including the additional voting information results in a substantial reduction in error rate for most of the other four algorithms. hPRMs not only provide a significant performance ad￾vantage over standard PRMs, but are also able to out￾perform all but one of the other four algorithms. 5 Future Work In this paper we learned a fairly broad hierarchy based on various sub-genres in the EachMovie dataset. How￾ever, hPRMs allow for arbitrarily specific class hierar￾chies, where a leaf-node entry for H[X] might be a handful (or even just one) movie. This could be ex￾ploited when there are certain indicator movies in a Genre that may accurately predict a user’s votes on other movies in that (or other) Genres, and that fur￾thermore most users have seen. For example, whether a user likes science fiction movies or not, they have likely seen the Star Wars Trilogy; this fact could be leverage by making the Star Wars movies a basic sub￾class in the class hierarchy, and subsequently used to learn new dependencies of the type if a user likes the Star Wars movies, they will in general like science fic￾tion movies. Learning such a hierarchy is a challenging task that would likely significantly improve the perfor￾mance of hPRMs