Www 2008/ Refereed Track: Rich Media April 21-25, 2008. Beijing, China MRR SO1 S@5 P@5 WordN Baseline(using Class I 79377000 5160 Location Class I7762.6800 54 Artifact or Object 61% Class III. 7542. 6400 .5160 Person or Group Class Iv. 7272. 6 Action or Event 51% Promotion(using Time Class I Other 53% Class II Class III Table 6: Acceptance ratio of tags of different Word- Class Iv.7673.600098005640 Net categories. Class i-0.1%0.0%2.1%-0.8% Class I3.6%5.9%0.09 WordNet, but that these two types of tags are well balanced Class Ii4.6%6.3%-4.2% for the other categories. Class Iv5.5%0.0%11.4%41.0% Table 6 shows the acceptance ratio for different Word- Net categories. From the figure and the table we see that Table 5: Performance of our system over different locations, artifacts, and objects have a relatively high accep- lasses of topics. tance ratio. However, people, groups and unclassified tags (tags that do not appear in WordNet)have relatively low 日A91用的1解 acceptance ratio. We conclude that our system is particu- larly good at recommending additional location-, artifact 6.5 Summary We conclude this section by recapitulating the main re- sults of the evaluation results presented in this section. First, we have shown that the proposed strategies are effective, ie. the recommended tags contain useful additions to the user-defined tags. For almost 70% of photos we give a good recommendation at the first position in the ranking(S@1 and for 94% of the photos we provide a good recommen- dation among the top 5 ranks. If 5 tags are recommended for each photo, than on average more than half of our rec- WordNet categories of initially assigned ommendations are good. Second, we proved that our pro- motion function has a positive effect on the performance in tags,recommended tags, and accepted recommen- general, and in particular on the precision at rank 5. We found a significant increase in the number relevant tags in the top five recommended tags. Third, we have shown that motion has an overall positive effect, but mainly increases our best strategy(votet) has a stable performance over dif- the performance of our syst ten on p tos that have mor ferent classes of photos. Fourth, we reported that our sys- user-defined tags tem is particularly good at recommending locations, arti- facts and objects, both in terms of volume and acceptance 6.4 Semantic Analysis ratio We finish the evaluation of the tag recommendation sy ten by analysing what type of tags are being recommended 7. CONCLUSIONS and accepted, to follow up on the tag characterisation pre- Annotating photos through tagging is a popular way to in- sented in Section 3. We will perform this analysis using our dex and organise photos. In this paper we first presented a best performing strategy, based on vote aggregation and pro- characterisation of tag behaviour in Flickr, which forms the otion(votet). We turn our attention to the WordNet cat foundation for the tag recommendation system and evalua- gories of the tags that are visible to the user in the recom tion presented in the second part of the paper. mendation application: the user-defined tags, recommended tags, and accepted tags Tag behaviour in Flickr. We have taken a random snap- Figure 5 shows the WordNet categories of all the tag hot of Flickr consisting of 52 million photos to analyse how that took part in the tag recommendation process. The fig users tag their photos and what type of tags they are prouid- ure shows results for the combination of training and testin sets. The first column in each group shows the categories of We found that the tag frequency distribution follows a per- the tags initially assigned by the Flickr photo owners, the ct power law, and we indicated that the mid section of this ext column shows the categories of the top 5 recommended power law contained the most interesting candidates for tag tags, and the third column shows the categories of the ac recommendation. Looking at the photo-tag distribution,we cepted tags(i.e, the tags judged as good or very good ). It observed that the majority of the photos is being annotated can be seen that there exists a gap between user-defined and with only a few tags. Yet, based on a mapping of tags on accepted tags for those tags which can not be classified using the WordNet classification scheme. we discovered that theMRR S@1 S@5 P@5 Baseline (using sum) Class I .7937 .7000 .9400 .5160 Class II .7762 .6800 .9000 .5400 Class III .7542 .6400 .9600 .5160 Class IV .7272 .6000 .8800 .4000 Promotion (using vote+) Class I .7932 .7000 .9600 .5120 Class II .8040 .7200 .9000 .5640 Class III .7887 .6800 .9200 .5280 Class IV .7673 .6000 .9800 .5640 Improvement Class I -0.1% 0.0% 2.1% -0.8% Class II 3.6% 5.9% 0.0% 4.4% Class III 4.6% 6.3% -4.2% 2.3% Class IV 5.5% 0.0% 11.4% 41.0% Table 5: Performance of our system over different classes of topics. Figure 5: WordNet categories of initially assigned tags, recommended tags, and accepted recommen￾dations. motion has an overall positive effect, but mainly increases the performance of our system on photos that have more user-defined tags. 6.4 Semantic Analysis We finish the evaluation of the tag recommendation sys￾tem by analysing what type of tags are being recommended and accepted, to follow up on the tag characterisation pre￾sented in Section 3. We will perform this analysis using our best performing strategy, based on vote aggregation and pro￾motion (vote+). We turn our attention to the WordNet cat￾egories of the tags that are visible to the user in the recom￾mendation application: the user-defined tags, recommended tags, and accepted tags. Figure 5 shows the WordNet categories of all the tags that took part in the tag recommendation process. The fig￾ure shows results for the combination of training and testing sets. The first column in each group shows the categories of the tags initially assigned by the Flickr photo owners, the next column shows the categories of the top 5 recommended tags, and the third column shows the categories of the ac￾cepted tags (i.e., the tags judged as good or very good). It can be seen that there exists a gap between user-defined and accepted tags for those tags which can not be classified using WordNet Acceptance ratio Unclassified 39% Location 71% Artifact or Object 61% Person or Group 33% Action or Event 51% Time 46% Other 53% Table 6: Acceptance ratio of tags of different Word￾Net categories. WordNet, but that these two types of tags are well balanced for the other categories. Table 6 shows the acceptance ratio for different Word￾Net categories. From the figure and the table we see that locations, artifacts, and objects have a relatively high accep￾tance ratio. However, people, groups and unclassified tags (tags that do not appear in WordNet) have relatively low acceptance ratio. We conclude that our system is particu￾larly good at recommending additional location-, artifact-, and object-tags. 6.5 Summary We conclude this section by recapitulating the main re￾sults of the evaluation results presented in this section. First, we have shown that the proposed strategies are effective, i.e., the recommended tags contain useful additions to the user-defined tags. For almost 70% of photos we give a good recommendation at the first position in the ranking (S@1) and for 94% of the photos we provide a good recommen￾dation among the top 5 ranks. If 5 tags are recommended for each photo, than on average more than half of our rec￾ommendations are good. Second, we proved that our pro￾motion function has a positive effect on the performance in general, and in particular on the precision at rank 5. We found a significant increase in the number relevant tags in the top five recommended tags. Third, we have shown that our best strategy (vote+) has a stable performance over dif￾ferent classes of photos. Fourth, we reported that our sys￾tem is particularly good at recommending locations, arti￾facts and objects, both in terms of volume and acceptance ratio. 7. CONCLUSIONS Annotating photos through tagging is a popular way to in￾dex and organise photos. In this paper we first presented a characterisation of tag behaviour in Flickr, which forms the foundation for the tag recommendation system and evalua￾tion presented in the second part of the paper. Tag behaviour in Flickr. We have taken a random snap￾shot of Flickr consisting of 52 million photos to analyse how users tag their photos and what type of tags they are provid￾ing. We found that the tag frequency distribution follows a per￾fect power law, and we indicated that the mid section of this power law contained the most interesting candidates for tag recommendation. Looking at the photo-tag distribution, we observed that the majority of the photos is being annotated with only a few tags. Yet, based on a mapping of tags on the WordNet classification scheme, we discovered that the 334 WWW 2008 / Refereed Track: Rich Media April 21-25, 2008. Beijing, China