Method coverage overall better recommended 63.1%士6.9%18.5%±5.6%23.6%±5.8%23.8%±6.1%19.4%±5 47.5%±7 ontent93.1%±1.3%36.1%±6.4%41.8%±6.4%30.7%±6.6%25.8%±5.9%46.4%±72% Table 1: Average across users of various performance metrics: the proportion of urls for which the recommended tags had n comparison to the common tags, a larger total number of true tags: some additional true tags: some true tags lost; and a recommended. Confidence intervals are 38. commended smaller total number of tags. "Uncommon recommended"refers to the average proportion of uncommon true tags that were where a url is represented as a vector, u=(u1, U2,. UM), The tagging-based method did generate new good tags in and we define u-u=E, Ui U All that remains is to specify nearly a quarter of cases, but the content-based method gen- how a particular url is given a vector space representation of erated new tags in 40% of cases, and was overall better in a the form required. larger proportion of cases than the content-based method. Tagging-based Similarity. For this metric we take the proportions of winners and losers were statistically indis- requencies scraped from delicio us. out numbered the losers, indicating that the content-based Content-based Similarity. For this metric we take method gives a better overall recommendation than common (U1, U2, .. UM)in(2)to be a vector of word frequencies gs. This is surprising b ecause user selection of tags is bi- found in the contents of the url itself ased towards the common tags that they see at the time o tagging. One way of interpreting the result is to say that, not surprisingly, a user's tagging behaviour is even more strongly activity that the median number of urls tagged is roughly 150, list of recently used tagging data, which is unfortunately not and 75% have tagged less than 400; only 6% have more than publicly available 1000, although the largest number found was 5188. Giver Surprisingly, given the superior performance of the content. this, we scraped common tag sets for 6180 urls tagged by 36 based method in general, in terms of the proportion of ur users of del icio us, selected for having close to the median common true tags found its performance was indistinguish- number of urls-a larger study was prevented by IP blocking able from that of the tagging-based method. This apparent that limited our ability to scrape data. conundrum is resolved by observing that the average is taken In order for the content-based similarity metric to be cal- only over those urls for which each method was able to give and readable in the sense that it contains at least some text all cases where there are no common tags-for r od fails in culable, the content pointed to by a url must be reachable, a recommendation at all. The tagging-based metl words. 268 urls, or approximately 4.5% were unreachable, true tag is therefore uncommon, and on which it is and a further 163, or approximately 2.5% had no words in fore correspondingly difficult for the content-based common with any other user url, and so were deemed to be to get a good score. Thus the urls covered by content unreadable tagging-based method are biased with respect to difficulty in In terms of number of common tags per url, the maximum recommending a high proportion of uncommon true tags, and number of common tags that delicio us reports is 25, and this explains why the content method does not do as well oughly 25% of urls have enough tagging data to reach the expected maximum number of common tags. The most frequent num- ber of common tags is unfortunately not 25 but O: about 35% 4. Conclusion have tagging-based similarity zero to all other urls tagged by In this paper we have presented a novel method for recom the user, which means that a tagging-based recommendation mending semantic tags on the basis of similarity metrics de- rived either from tagging data, or from content analysis. Our method gives personalized recommendations that provide a 3.1 Performance promising addition to existing tag recommendations based or The principal measures of performance we used were cover- commonly used tags Of the two methods, the tagging-based method is far more recommendation -and the proportion of users'urls for which lightweight to implement, since it does not require a separate the top n recommended tags had larger intersection with the index for the content of the urls itself, but it is less effec set of true tags than the set of n common tags. The results re shown in Table 1, which reveals that the content-based ethod, and has much lower coverage (although only slightly method is clearly significantly superior to the tagging-based lower than the coverage of common tags themselves) method From the point of view of performance, the content-based References method was clearly superior to the tagging-based method. [1]RA Baeza-Yates and B ARibeiro-Neto. Modern Another possibility is that the page to which the url points Information Retrieval. ACM Press/ Addison-Wesley is there solely for the purpose of ction. and is other 1999 rise emptyMethod coverage overall some some overall uncommon better gained lost worse recommended Tagging 63.1% ± 6.9% 18.5% ± 5.6% 23.6% ± 5.8% 23.8% ± 6.1% 19.4% ± 5.1% 47.5% ± 7.2% Content 93.1% ± 1.3% 36.1% ± 6.4% 41.8% ± 6.4% 30.7% ± 6.6% 25.8% ± 5.9% 46.4% ± 7.2% Table 1: Average across users of various performance metrics: the proportion of urls for which the recommended tags had, in comparison to the common tags, a larger total number of true tags; some additional true tags; some true tags lost; and a smaller total number of tags. “Uncommon recommended” refers to the average proportion of uncommon true tags that were recommended. Confidence intervals are 95%. where a url is represented as a vector, u = (v1, v2, . . . , vM), and we define u·u 0 = PM i=1 vi v 0 i . All that remains is to specify how a particular url is given a vector space representation of the form required. • Tagging-based Similarity. For this metric we take (v1, v2, . . . , vM) in (2) to be the vector of common tag frequencies scraped from del.icio.us. • Content-based Similarity. For this metric we take (v1, v2, . . . , vM) in (2) to be a vector of word frequencies found in the contents of the url itself. 3. Results We found from a sample of 200 users based on recent tagging activity that the median number of urls tagged is roughly 150, and 75% have tagged less than 400; only 6% have more than 1000, although the largest number found was 5188. Given this, we scraped common tag sets for 6180 urls tagged by 36 users of del.icio.us, selected for having close to the median number of urls – a larger study was prevented by IP blocking that limited our ability to scrape data. In order for the content-based similarity metric to be cal￾culable, the content pointed to by a url must be reachable, and readable in the sense that it contains at least some text words. 268 urls, or approximately 4.5% were unreachable, and a further 163, or approximately 2.5% had no words in common with any other user url, and so were deemed to be unreadable1 . In terms of number of common tags per url, the maximum number of common tags that del.icio.us reports is 25, and roughly 25% of urls have enough tagging data to reach the maximum number of common tags. The most frequent num￾ber of common tags is unfortunately not 25 but 0: about 35% of the urls we sampled had no common tags at all; these urls have tagging-based similarity zero to all other urls tagged by the user, which means that a tagging-based recommendation is impossible. 3.1 Performance The principal measures of performance we used were cover￾age – the number of urls for which it was possible to make a recommendation – and the proportion of users’ urls for which the top N recommended tags had larger intersection with the set of true tags than the set of N common tags. The results are shown in Table 1, which reveals that the content-based method is clearly significantly superior to the tagging-based method. From the point of view of performance, the content-based method was clearly superior to the tagging-based method. 1 Another possibility is that the page to which the url points is there solely for the purpose of redirection, and is other￾wise empty. The tagging-based method did generate new good tags in nearly a quarter of cases, but the content-based method gen￾erated new tags in 40% of cases, and was overall better in a larger proportion of cases than the content-based method. More promising still, although for the tagging-based method the proportions of winners and losers were statistically indis￾tinguishable, for the content method the winners significantly out numbered the losers, indicating that the content-based method gives a better overall recommendation than common tags. This is surprising because user selection of tags is bi￾ased towards the common tags that they see at the time of tagging. One way of interpreting the result is to say that, not surprisingly, a user’s tagging behaviour is even more strongly biased towards their own previously used tags. In this con￾text it would be valuable to evaluate our methods against the list of recently used tagging data, which is unfortunately not publicly available. Surprisingly, given the superior performance of the content￾based method in general, in terms of the proportion of un￾common true tags found its performance was indistinguish￾able from that of the tagging-based method. This apparent conundrum is resolved by observing that the average is taken only over those urls for which each method was able to give a recommendation at all. The tagging-based method fails in all cases where there are no common tags – for which every true tag is therefore uncommon, and on which it is there￾fore correspondingly difficult for the content-based method to get a good score. Thus the urls covered by content but not tagging-based method are biased with respect to difficulty in recommending a high proportion of uncommon true tags, and this explains why the content method does not do as well as expected. 4. Conclusion In this paper we have presented a novel method for recom￾mending semantic tags on the basis of similarity metrics de￾rived either from tagging data, or from content analysis. Our method gives personalized recommendations that provide a promising addition to existing tag recommendations based on commonly used tags. Of the two methods, the tagging-based method is far more lightweight to implement, since it does not require a separate index for the content of the urls itself, but it is less effec￾tive at finding good recommendations than the content-based method, and has much lower coverage (although only slightly lower than the coverage of common tags themselves). References [1] R. A. Baeza-Yates and B. A. Ribeiro-Neto. Modern Information Retrieval. ACM Press / Addison-Wesley, 1999