Www 2008/ Refereed Track: Rich Media April 21-25, 2008. Beijing, China Flickr community as a whole annotates their photos using 8. REFERENCES ags that span a broad spectrum of the semantic space, i.e. hey annotate where their photos are taken, who or what 1 L A. Adamic Zipf, power-laws, and pareto-a on the photo, and when the photo was taken. This moti- rankingtutorialhttp://www.hpl.hp.com/research/ rated us to investigate whether the collective knowledge of idl/papers/ranking/ranking. html, 2002 the community as a whole could be used to help user extend 2S. Ahern, S King, M. Naaman, R Nair, and J H.-I heir annotations of individual photos Yang. Zone Tag: Rich, community-supported context-aware media capture and annotation In Extending Flickr photo annotations. Based on our ob- Mobile Spatial Interaction workshop(MSI) at the ervations, we introduced a novel and generic method for SIGCHI conference on Human Factors in computing recommending tags, i.e., our approach deploys the collective systems(CHI 2007), 2007. knowledge that resides in Flickr without introducing tag. 3 S Ahern, M. Naaman, R Nair, and J. Yang World lass specific heuristics. Based on a representative sample of explorer: Visualizing aggregate data from lickr, we have extracted tag co-occurrence statistics, which unstructured text in geo-referenced colle in combination with the two tag aggregation strategies, and Proceedings of the Seventh ACM/IEEE-CS Joint the promotion function allowed us to build a highly effective Conference on Digital Libraries, (JCDL 07), 2007 system for tag recommendation 4 M. Ames and M. Naaman. Why we tag: Motivations We have evaluated the four tag recommendation strate- or annotation in mobile and online media. In gies in an empirical experiment using 200 photos which are Proceedings of the SIGCHI conference on Human so available on Flickr. The evaluation results showed that Factors in computing systems(CHI 2007), San Jose both tag aggregation strategies are effective, but that it is CA. USA 2007 essential to take the co-occurrence values of the candidate 5 K. Barnard, P. Duygulu, D. Forsyth, N. de Freitas, tags into account when aggregating the intermediate result D. Blei, and M. Jordan. Matching words and pictures. in a ranked list of recommended tags. Journal of machine Learning Research, 3: 1107-1135 We showed that the promotion function is an effective way 2003. to incorporate the ranking of tags and allows us to focus on the candidate tag set, where we expect to find good de- 6Citeulike.http://www.citeulike.org scriptive tags. Furthermore, the promotion function further 7 R. Datta, D Joshi, J. Li, and J. Z. Wang. Image proves the results, and has a highly positive effect of the retrieval: Ideas. influences and trends of the precision at rank 5. The best combination, the voteT strat ACM Computing Surveys, 40, 2008. to appear gy, gives a relevant tag on the first position in the ranking 8del.icio.ushttp://www.del.ic n 67% of the cases, and we find a relevant tag in 94% of M. Dubinko, R. Kumar, J Magnan the cases when looking at the top 5. On average, more than P. Raghavan, and A. Tomkins. Visualizing tags over 4% of the recommended tags in the top 5 is accepted as time In www 06: Proceedings of the 15th useful tag in context of the photo. The vote+ strategy international conference on World Wide web, pages 193-202. ACM Press,2006. of tag-classes. Finally, we have shown that our system is [10 C Fellbaum, editor. WordNet: An Electronic Lezical particularly good at recommending locations, artifacts and 11flIckrhttp Open tagging systems like Flickr have continuously evolv- [12] Flickr blog: We're going down ing vocabularies. Our method is based on the statistics of http://blog.flickrcom/en/2007/05/29/were-going Flickr annotation patterns and our co-occurrence model can down/ be incrementally updated when new annotations become [13S. Golder and B A Huberman. The structure of available. Hence our method can gracefully handle the e collaborative tagging systems lution of the vocabulary http://www.hpl.hp.com/research/idl/papers/tags/, 2006 Future Work. Our future work includes implementing an [14] K Lerman and L. Jones. Social browsing on Flickr. In online system where users can be aided in extending the an- Proceedings of International Conference on Weblogs notations of their own photos. Having such a system allows and Social Media, Boulder, Co, USA, 2007 us to evaluate the tag recommendation task more extensively 15 J. Li and J Z Wang Real-time computerized in an on-line usability experiment annotation of pictures. In Proceedings of the ACM Our method is complementary to previously explored ap. Multimedia Conference, pages 911-920, 2006. proaches using either content-based methods [5, 15] or the [16] CMarlow, M. Naaman, M.Davis, and D.Boyd patial, temporal and social context of the user [2, 24.A HTo6, tagging paper, taxonomy, Flickr, academic mbination of different complimentary methods is likely to article, toread. In HT06: Proceedings of the ive a more robust performance. Further research into this seventeenth ACM conference on Hypertext and is left as future work hypermedia, 2006 [17 P. Mika. Ontologies are us: A unified model of social Acknowledgments networks and semantics. In Proceedings of the 4th International Semantic Web Conference(IswC This research is partially supported by the European Union 2005), volume 3729 of LNCS. Springer-Verlag, 2005 under contract FP6-045032, "Search Environments for Me dia-semedia"(httP://www.semedia.orgFlickr community as a whole annotates their photos using tags that span a broad spectrum of the semantic space, i.e., they annotate where their photos are taken, who or what is on the photo, and when the photo was taken. This moti￾vated us to investigate whether the collective knowledge of the community as a whole could be used to help user extend their annotations of individual photos. Extending Flickr photo annotations. Based on our ob￾servations, we introduced a novel and generic method for recommending tags, i.e., our approach deploys the collective knowledge that resides in Flickr without introducing tag￾class specific heuristics. Based on a representative sample of Flickr, we have extracted tag co-occurrence statistics, which in combination with the two tag aggregation strategies, and the promotion function allowed us to build a highly effective system for tag recommendation. We have evaluated the four tag recommendation strate￾gies in an empirical experiment using 200 photos which are also available on Flickr. The evaluation results showed that both tag aggregation strategies are effective, but that it is essential to take the co-occurrence values of the candidate tags into account when aggregating the intermediate results in a ranked list of recommended tags. We showed that the promotion function is an effective way to incorporate the ranking of tags and allows us to focus on the candidate tag set, where we expect to find good de￾scriptive tags. Furthermore, the promotion function further improves the results, and has a highly positive effect of the precision at rank 5. The best combination, the vote+ strat￾egy, gives a relevant tag on the first position in the ranking in 67% of the cases, and we find a relevant tag in 94% of the cases when looking at the top 5. On average, more than 54% of the recommended tags in the top 5 is accepted as a useful tag in context of the photo. The vote+ strategy also shows to be a very stable approach for different types of tag-classes. Finally, we have shown that our system is particularly good at recommending locations, artifacts and objects. Open tagging systems like Flickr have continuously evolv￾ing vocabularies. Our method is based on the statistics of Flickr annotation patterns and our co-occurrence model can be incrementally updated when new annotations become available. Hence our method can gracefully handle the evo￾lution of the vocabulary. Future Work. Our future work includes implementing an online system where users can be aided in extending the an￾notations of their own photos. Having such a system allows us to evaluate the tag recommendation task more extensively in an on-line usability experiment. Our method is complementary to previously explored ap￾proaches using either content-based methods [5, 15] or the spatial, temporal and social context of the user [2, 24]. A combination of different complimentary methods is likely to give a more robust performance. Further research into this is left as future work. Acknowledgments This research is partially supported by the European Union under contract FP6-045032, “Search Environments for Me￾dia – SEMEDIA” (http://www.semedia.org). 8. REFERENCES [1] L. A. Adamic. Zipf, power-laws, and pareto - a ranking tutorial. http://www.hpl.hp.com/research/ idl/papers/ranking/ranking.html, 2002. [2] S. Ahern, S. King, M. Naaman, R. Nair, and J. H.-I. Yang. ZoneTag: Rich, community-supported context-aware media capture and annotation. In Mobile Spatial Interaction workshop (MSI) at the SIGCHI conference on Human Factors in computing systems (CHI 2007), 2007. [3] S. Ahern, M. Naaman, R. Nair, and J. Yang. World explorer: Visualizing aggregate data from unstructured text in geo-referenced collections. In Proceedings of the Seventh ACM/IEEE-CS Joint Conference on Digital Libraries, (JCDL 07), 2007. [4] M. Ames and M. Naaman. Why we tag: Motivations for annotation in mobile and online media. In Proceedings of the SIGCHI conference on Human Factors in computing systems (CHI 2007), San Jose, CA, USA, 2007. [5] K. Barnard, P. Duygulu, D. Forsyth, N. de Freitas, D. Blei, and M. Jordan. Matching words and pictures. Journal of Machine Learning Research, 3:1107–1135, 2003. [6] CiteULike. http://www.citeulike.org. [7] R. Datta, D. Joshi, J. Li, and J. Z. Wang. Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys, 40, 2008. to appear. [8] del.icio.us. http://www.del.icio.us. [9] M. Dubinko, R. Kumar, J. Magnani, J. Novak, P. Raghavan, and A. Tomkins. Visualizing tags over time. In WWW ’06: Proceedings of the 15th international conference on World Wide Web, pages 193–202. ACM Press, 2006. [10] C. Fellbaum, editor. WordNet: An Electronic Lexical Database. The MIT Press, 1998. [11] Flickr. http://www.flickr.com. [12] Flickr blog: We’re going down. http://blog.flickr.com/en/2007/05/29/were-going￾down/. [13] S. Golder and B. A. Huberman. The structure of collaborative tagging systems. http://www.hpl.hp.com/research/idl/papers/tags/, 2006. [14] K. Lerman and L. Jones. Social browsing on Flickr. In Proceedings of International Conference on Weblogs and Social Media, Boulder, Co, USA, 2007. [15] J. Li and J. Z. Wang. Real-time computerized annotation of pictures. In Proceedings of the ACM Multimedia Conference, pages 911–920, 2006. [16] C. Marlow, M. Naaman, M. Davis, and D. Boyd. HT06, tagging paper, taxonomy, Flickr, academic article, toread. In HT’06: Proceedings of the seventeenth ACM conference on Hypertext and hypermedia, 2006. [17] P. Mika. Ontologies are us: A unified model of social networks and semantics. In Proceedings of the 4th International Semantic Web Conference (ISWC 2005), volume 3729 of LNCS. Springer-Verlag, 2005. 335 WWW 2008 / Refereed Track: Rich Media April 21-25, 2008. Beijing, China