《电子商务 E-business》阅读文献：Reasonable tag-based collaborative filtering for social tagging systems

Reasonable Tag-Based Collaborative Filtering For Social Tagging Systems Reyn Y Nakamoto Shinsuke Nakajima Jun Miyazaki Nara Institute of science and Nihonbashi Hakozaki Building Motoyama, Kamigamo, Kita Technology Kyoto, Japan, 603-8555 Hakozaki 24-1, Nihonbashi nakajima@ cse. kyoto-su ac jp Nara, Japan, 630-0192 Tokyo, Japan 103-0015 miyazaki@ is naist. jp reyn@ kizasi jp Shunsuke uemura Hirokazu Kato Youichi Inagaki lara Sangyo University Nara institute of science and kizasi Company, Inc. Tateno-Kita 3-12-1, Sango Technolog Nihonbashi Hakozaki Building Ikoma Takayama 8916-5, Ikoma Nara, Japan, 636-8503 Nara, Japan, 630-0192 Hakozaki 24-1 Nihonbashi uemurashunsuke@ nara. su ac jp kato@ is naist. jp Tokyo, Japan 103-0015 inagaki@ kizasi jp ABSTRACT 1. INTRODUCTION In this paper, we present a tag-based collaborative filter As the Internet continues to mature and becomes more ing recommendation method for use with recently popular accessible to the common user, the amount of information online social tagging systems. Combining the information available increases exponentially. Accordingly, finding useful provided by tagging systems with the effective recommen- and relevant information is becoming progressively difficult dation abilities given by collaborative filtering we provide Moreover, a lot of the information available-blogs, various a website recommendation system which provides relevant types of reviews, and so forth-are highly subjective and thus credible recommendations that match the user's changing hard to evaluate purely through content-based algorithms. interests as well as the user's bookmarking profile. Based For such subjective sources, one person may enjoy some- upon user testing, our system provides a higher level of rel- thing while the next may dislike the same-everyone has dif- evant recommendations over other commonly used search fering opinions on what is good. In these cases, people-more and recommendation methods. We describe this system as so than the current ability of content-based algorithms-are well as the relevant user testing results and its implication greatly effective in evaluating and filtering this information towards use in online social tagging systems. For this reason. Tag-based Contextual Collaborative Fil- Categories and Subject descriptors found to be effective in providing website recommendations for social bookmarking systems 7. This method combines H 4m Information Systems Applications]: Miscella- the strengths of both Collaborative Filtering (CF)as well neous as tagging information from social bookmarking systems to provide effective, personalized recommendations to the user. General terms By using CF techniques, we can match users with similar Algorithms, Experimentation preferences. By employing tagging, we can match only the Keywords Both methods employ users to organize and evaluate infor- mation, making them a good fit for each other. With strong collaborative filtering, so user involvement. there is higher confidence in the credibility as well as the quality of the information recommendations. We now extend our algorithm one step further to provide implicit recommendations by considering the user's current interest. Our new method-called Reasonable Tag-Based Collaborative Filtering or RCF-takes into account the user bookmarking profile and the currently viewed page to pro- Permission to make digital or hard copies of all or part of this work for vide relevant website recommendations. We assume that or classroom use is granted without fee provided that copies are these tags are synonymous with the reasons why they liked not made or distributed for profit or commercial advantage and that copies something, and thus, we derived the name for our algorithm bear this notice and the full citation on the first page. To copy otherwise, to Reasonable Tag-Based Collaborative Filtering. In this paper, we explore the effectiveness of our algorithm wICOw08, October 30, 2008, Napa Valley, California, USA. d perform user testing as well. In comparison to the other Copyright2008ACM978-1-605582597/08/10…5 recommendation methods we tested. our method was found

Reasonable Tag-Based Collaborative Filtering For Social Tagging Systems Reyn Y. Nakamoto kizasi Company, Inc. Nihonbashi Hakozaki Building 3F Hakozaki 24-1, Nihonbashi Tokyo, Japan 103-0015 reyn@kizasi.jp Shinsuke Nakajima Kyoto Sangyo University Motoyama, Kamigamo, Kita Kyoto, Japan, 603-8555 nakajima@cse.kyoto-su.ac.jp Jun Miyazaki Nara Institute of Science and Technology Takayama 8916-5, Ikoma Nara, Japan, 630-0192 miyazaki@is.naist.jp Shunsuke Uemura Nara Sangyo University Tateno-Kita 3-12-1, Sango, Ikoma Nara, Japan, 636-8503 uemurashunsuke@nara.su.ac.jp Hirokazu Kato Nara Institute of Science and Technology Takayama 8916-5, Ikoma Nara, Japan, 630-0192 kato@is.naist.jp Youichi Inagaki kizasi Company, Inc. Nihonbashi Hakozaki Building 3F Hakozaki 24-1, Nihonbashi Tokyo, Japan 103-0015 inagaki@kizasi.jp ABSTRACT In this paper, we present a tag-based collaborative filter￾ing recommendation method for use with recently popular online social tagging systems. Combining the information provided by tagging systems with the effective recommen￾dation abilities given by collaborative filtering, we provide a website recommendation system which provides relevant, credible recommendations that match the user’s changing interests as well as the user’s bookmarking profile. Based upon user testing, our system provides a higher level of rel￾evant recommendations over other commonly used search and recommendation methods. We describe this system as well as the relevant user testing results and its implication towards use in online social tagging systems. Categories and Subject Descriptors H.4.m [Information Systems Applications]: Miscella￾neous General Terms Algorithms, Experimentation Keywords collaborative filtering, social tagging, recommendation sys￾tems Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. WICOW’08, October 30, 2008, Napa Valley, California, USA. Copyright 2008 ACM 978-1-60558-259-7/08/10 ...$5.00. 1. INTRODUCTION As the Internet continues to mature and becomes more accessible to the common user, the amount of information available increases exponentially. Accordingly, finding useful and relevant information is becoming progressively difficult. Moreover, a lot of the information available–blogs, various types of reviews, and so forth–are highly subjective and thus, hard to evaluate purely through content-based algorithms. For such subjective sources, one person may enjoy some￾thing while the next may dislike the same–everyone has dif￾fering opinions on what is good. In these cases, people–more so than the current ability of content-based algorithms–are greatly effective in evaluating and filtering this information. For this reason, Tag-based Contextual Collaborative Fil￾tering (TCCF) was proposed and described in [8]. It was found to be effective in providing website recommendations for social bookmarking systems [7]. This method combines the strengths of both Collaborative Filtering (CF) as well as tagging information from social bookmarking systems to provide effective, personalized recommendations to the user. By using CF techniques, we can match users with similar preferences. By employing tagging, we can match only the users that liked the same information for the same reasons. Both methods employ users to organize and evaluate infor￾mation, making them a good fit for each other. With strong user involvement, there is higher confidence in the credibility as well as the quality of the information recommendations. We now extend our algorithm one step further to provide implicit recommendations by considering the user’s current interest. Our new method–called Reasonable Tag-Based Collaborative Filtering or RCF–takes into account the user’s bookmarking profile and the currently viewed page to pro￾vide relevant website recommendations. We assume that these tags are synonymous with the reasons why they liked something, and thus, we derived the name for our algorithm, Reasonable Tag-Based Collaborative Filtering. In this paper, we explore the effectiveness of our algorithm and perform user testing as well. In comparison to the other recommendation methods we tested, our method was found 11

to be the most effective for implicit live-updating recommen- steps to get satisfactory cluster sizes and document classi- dations. We will describe these results and its corresponding fication. Lastly, they explored creating a personalized tag implications search engine based upon their findings 2. RELATED WORK 2.3 TCCF Website Recommendation System TCCF is an website recommendation algorithm which com- 2.1 Collaborative Filtering Systems bines traditional CF systems and social tagging systems Collaborative Filtering(CF) is a process that uses the The essential idea is that CF provides personalization, and community of users to sort out relevant or important infor- tags provide clues of the reasons why users liked something ation from the non-relevant and non-important informa Unlike traditional CF models which use numeric ratings, the tion. The process is based upon the idea that users should TCCF model uses tags for calculating user similarity and enjoy the same items that their similar users like. From a score prediction. wider perspective, it is the idea of collaborating with other In a social bookmarking service, the act of bookmark- people and sharing knowledge to filter out the best infor- ing a website is a strong indicator of whether something is mation from the rest. Lastly, cF provides a high level of liked. Additionally, we also have tagging information. USu- information credibility since several users are evaluating and rating such information users perspective, and in most cases this is the reason why CF has been proven to work well under certain domains- they liked something. Thus, the key difference between tra- mainly entertainment domains-such as usenet recommenda ditional CF and TCCF: In addition to considering whether tions 9), movie recommendations 5l, product recommenda- users like something by using the tagging information rely f numerical ratings on resources by users 9. Once enough In the TCCF Website Recommendation System, users ratings are in place, similarity scores between users are cal bookmark websites they like using tags, and subsequently is then calculated using the similar users' ratings on other tags. Once the the user has added enough bookmarks, the items. Those resources with a score prediction above a cer- tain threshold are recommended The main drawback of CF is that it only considers if and lar user has rated. The score prediction value is basically the how much a user likes something. However, it does I level that the system thinks the user will like something-the into account the reasons why a user likes something igher score. the more the user should like the website. both of these steps consider bookmarking as well as the attached 2.2 Social Tagging Systems tags when generating recommendation candidates. In this paper, we extend this method and describe it in section 3 ther terms such as metadata, categorization, labels, and so forth. Tagging is the process of attaching natural language 3. REASONABLE TAG-BASED words as metadata to describe some resource like a movie COLLABORATIVE FILTERING FOR photo, book, etc. Tagging vocabulary can be controlled uncontrolled depending on the type and purpose of the ap SOCIAL TAGGING SYSTEMS We now explain our recommendation algorithm and its In recent years, the advent of Social Tagging Systems has use in our recommendation system. The basis of our system brought tagging back into the limelight. Currently, there is a website bookmarking system similar to del icio us. A are several popular online social tagging systems which are sample usage pattern would be as shown in figure 1 the subject of continuing research: they range from website Here, an arrow indicates a user has bookmarked a page bookmarking such as delicio us 3, photo sharing 4, re- and the tags above the arrow indicate the tags used while search paper searching [2 and so forth. All of these sites bookmarking. User A is bookmarking website I with the use tagging for many purposes, but in addition to that, they tags japaneseand'dictionary'. Similarly, user B is book focus on the social networking aspects of tagging to enhance marking website 2 with the tags and"news. Based the experience for end users. However, currently, tags are upon this type of social bookmarking system, our system only used for tag searching-user profile matching and sub- generates effective recommendations based upon the user's sequent recommendations are yet to be implemented. As bookmarking profile as well as their currently viewed page mentioned before, tags provide the clues as to why a user Since our recommendation algorithm is heavily based upon liked something. They are the who, what, when, where, and comparing websites using their topics, we first explain how why of the user's reason for tagging something [6. Because we generate topic domain vectors of this, as well as the similar use of social networ king,so- cial tagging systems provide an ideal choice for com 3.1 Generating Topic Domain Vectors with CF systems iteration of our algorithm, similarity cal- of the research for tagging systems has culation between websites was based on the cosine of the cused on user and tagging patterns. [10 uses a statistica website tag vectors. This tag vector was a feature vector approach towards deriving the emergent semantics of social where the parameters consisted of all the tags attached by tagging systems, namely delicio us. They use the Expec. users. For example, in figure 1, website 1's vector would tation Maximization algorithm(EM) to automatically clus. have a value of one for japanese'and'dictionary'. Web- ter documents, users, and tags. They explored the optimal site 2 would have values of two, two, one, one, and one for umber of domain clusters as well as the number of iterative apple, 'news, "games, 'reviews,, and'tech

to be the most effective for implicit live-updating recommen￾dations. We will describe these results and its corresponding implications. 2. RELATED WORK 2.1 Collaborative Filtering Systems Collaborative Filtering (CF) is a process that uses the community of users to sort out relevant or important infor￾mation from the non-relevant and non-important informa￾tion. The process is based upon the idea that users should enjoy the same items that their similar users like. From a wider perspective, it is the idea of collaborating with other people and sharing knowledge to filter out the best infor￾mation from the rest. Lastly, CF provides a high level of information credibility since several users are evaluating and rating such information. CF has been proven to work well under certain domains– mainly entertainment domains–such as usenet recommenda￾tions [9], movie recommendations [5], product recommenda￾tions [1], and so forth. Many CF systems rely upon a matrix of numerical ratings on resources by users [9]. Once enough ratings are in place, similarity scores between users are cal￾culated. Based upon these similarity scores, score prediction is then calculated using the similar users’ ratings on other items. Those resources with a score prediction above a cer￾tain threshold are recommended. The main drawback of CF is that it only considers if and how much a user likes something. However, it does not take into account the reasons why a user likes something. 2.2 Social Tagging Systems Tagging has been around for sometime, albeit known by other terms such as metadata, categorization, labels, and so forth. Tagging is the process of attaching natural language words as metadata to describe some resource like a movie, photo, book, etc. Tagging vocabulary can be controlled or uncontrolled depending on the type and purpose of the ap￾plication. In recent years, the advent of Social Tagging Systems has brought tagging back into the limelight. Currently, there are several popular online social tagging systems which are the subject of continuing research: they range from website bookmarking such as del.icio.us [3], photo sharing [4], re￾search paper searching [2] and so forth. All of these sites use tagging for many purposes, but in addition to that, they focus on the social networking aspects of tagging to enhance the experience for end users. However, currently, tags are only used for tag searching–user profile matching and sub￾sequent recommendations are yet to be implemented. As mentioned before, tags provide the clues as to why a user liked something. They are the who, what, when, where, and why of the user’s reason for tagging something [6]. Because of this, as well as the similar use of social networking, so￾cial tagging systems provide an ideal choice for combination with CF systems. Much of the research for tagging systems has been fo￾cused on user and tagging patterns. [10] uses a statistical approach towards deriving the emergent semantics of social tagging systems, namely del.icio.us. They use the Expec￾tation Maximization algorithm (EM) to automatically clus￾ter documents, users, and tags. They explored the optimal number of domain clusters as well as the number of iterative steps to get satisfactory cluster sizes and document classi- fication. Lastly, they explored creating a personalized tag search engine based upon their findings. 2.3 TCCF Website Recommendation System TCCF is an website recommendation algorithm which com￾bines traditional CF systems and social tagging systems. The essential idea is that CF provides personalization, and tags provide clues of the reasons why users liked something. Unlike traditional CF models which use numeric ratings, the TCCF model uses tags for calculating user similarity and score prediction. In a social bookmarking service, the act of bookmark￾ing a website is a strong indicator of whether something is liked. Additionally, we also have tagging information. Usu￾ally, the user will use tags to describe the resource from the user’s perspective, and in most cases this is the reason why they liked something. Thus, the key difference between tra￾ditional CF and TCCF: In addition to considering whether or not a user likes a resource, it also takes into account why users like something by using the tagging information . In the TCCF Website Recommendation System, users bookmark websites they like using tags, and subsequently, they can easily retrieve their bookmarks by searching with tags. Once the the user has added enough bookmarks, the first step is finding similar users. This is then followed by calculating a score prediction for every website that the simi￾lar user has rated. The score prediction value is basically the level that the system thinks the user will like something–the higher score, the more the user should like the website. Both of these steps consider bookmarking as well as the attached tags when generating recommendation candidates. In this paper, we extend this method and describe it in section 3. 3. REASONABLE TAG-BASED COLLABORATIVE FILTERING FOR SOCIAL TAGGING SYSTEMS We now explain our recommendation algorithm and its use in our recommendation system. The basis of our system is a website bookmarking system similar to del.icio.us. A sample usage pattern would be as shown in figure 1. Here, an arrow indicates a user has bookmarked a page and the tags above the arrow indicate the tags used while bookmarking. User A is bookmarking website 1 with the tags ‘japanese’ and ‘dictionary’. Similarly, user B is book￾marking website 2 with the tags ‘apple’ and ‘news’. Based upon this type of social bookmarking system, our system generates effective recommendations based upon the user’s bookmarking profile as well as their currently viewed page. Since our recommendation algorithm is heavily based upon comparing websites using their topics, we first explain how we generate topic domain vectors. 3.1 Generating Topic Domain Vectors In our previous iteration of our algorithm, similarity cal￾culation between websites was based on the cosine of the website tag vectors. This tag vector was a feature vector where the parameters consisted of all the tags attached by users. For example, in figure 1, website 1’s vector would have a value of one for ‘japanese’ and ‘dictionary’. Web￾site 2 would have values of two, two, one, one, and one for ‘apple’, ‘news’, ‘games, ‘reviews’, and ‘tech’. 12

Users websites In this case. n is the number of distinct tags attached to ebsite k by all users. P(tilD,)is the conditional proba a japanese, dictionary ility of tag term i(shown as ti) given domain j(shown as Di). tfi k idfi is as calculated by standard tf-idf, where tfi k is the number of times any user bookmarked website k with tag i divided by the total number of times that the apple, news website is bookmarked. Similarly, the idfi is determined by the total number of websites in the entire set divided by the number of websites bookmarked with tag i. So, instead of the website vector consisting of tag term counts, a websites domain vector parameters are now its topic domain weight to that particu- lar topic domain. An example calculation is shown in figure p(t D softwar Figure 1: Website Bookmarking System Overview This approach has the inherent problems that are associ- ted with natural language processing. For ex dle. vecto containing two synonyms, such asfunny'or humourous, DVk=(0.86,0.02,…) should be similar; however, when comparing only the base tag, these synonyms are considered as different terms, and thus, their vector cosine value will be low To deal with this Figure 2: Calculation of k apple coms website we applied a similar approach to that of [10], by using the topic domain vector EM algorithm to cluster the websites into topic domain Once this was done, we compared the websites based upon their topic vector rather than their tag vector In this case, apple. com has three tags attached by all users: Our data consists of three months worth of mining of apple,, 'mac', and 'software. Using example domain term delicio us RSS feeds. In total, we have over 100.000 users probability values, we show the calculations for hypothetical domains 1 and 2 for a website k 2.5 million webpages, 3.6 million bookmarks, and 870, 000 istinct tags. After mining the data, we subsequently stemmed Similarly, a bookmark topic domain vector created from r a bookmarking website k, DVA-k, would be calculated the tags using the well-known Porter stemming algorithm Then, the number of unique stemmed tags was about 780,000 the same fashion as shown in equation 3 Over this data, we ran the EM algorithm for 50 iterations DVA→k=(dm,A-k,du2,A→k,dtr3,A-k,…d100,A-k)(3) over 20,000 of the top bookmarked documents and the top 20,000 used tags and clustered them into 100 topic domains. The only exception is that the domain tag weight only The number of topic domains was chosen after a subjective considers the tags that user A used on website k as shown comparison of topic separation they produced. Finding the optimal number of topic domains is described in [10] and is beyond the scope of our research. d1,A-k=∑P(tl|D)·id4f,A→k Based on these topic domain clusters, website feature vec- tors were created for each document as shown in equation Since a user can only apply atmost one of the same tag term 1. We will from now refer to these as topic domain s to a bookmark, only the idf is used here. and abbreviate it 3.2 RCF Recommendations DVk =(dwi. k, dw2, k, dws, k., da100k) We now describe our Reasonable Tag-based Collaborative Here, dwj, k is the topic domain weight of a website k for Filtering(RCF) algorithm. The process to generate recom- a topic domain j. In other words, this is how strongly the mendations follows these steps: document belongs in a given topic. This is calculated by summing the product of the conditional probability and tag term weight for all tags attached to website k. This is cal 2. Finding recommendation candidates culated as shown in equation 2 3. Providing live-updating recommendations based upon dn,k=∑P(tlD)tkin We now describe these steps

A B C D 1 2 3 4 japanese, dictionary apple, news mac, rumors tennis games, reviews apple, tech, news tennis, sports Users Websites Figure 1: Website Bookmarking System Overview This approach has the inherent problems that are associ￾ated with natural language processing. For example, vectors containing two synonyms, such as ‘funny’ or ‘humourous’, should be similar; however, when comparing only the base tag, these synonyms are considered as different terms, and thus, their vector cosine value will be low. To deal with this, we applied a similar approach to that of [10], by using the EM algorithm to cluster the websites into topic domains. Once this was done, we compared the websites based upon their topic vector rather than their tag vector. Our data consists of three months worth of mining of del.icio.us RSS feeds. In total, we have over 100,000 users, 2.5 million webpages, 3.6 million bookmarks, and 870,000 distinct tags. After mining the data, we subsequently stemmed the tags using the well-known Porter stemming algorithm. Then, the number of unique stemmed tags was about 780,000. Over this data, we ran the EM algorithm for 50 iterations over 20,000 of the top bookmarked documents and the top 20,000 used tags and clustered them into 100 topic domains. The number of topic domains was chosen after a subjective comparison of topic separation they produced. Finding the optimal number of topic domains is described in [10] and is beyond the scope of our research. Based on these topic domain clusters, website feature vec￾tors were created for each document as shown in equation 1. We will from now refer to these as ‘topic domain vectors’ and abbreviate it as DV . DVk = (dw1,k, dw2,k, dw3,k..., dw100,k) (1) Here, dwj,k is the topic domain weight of a website k for a topic domain j. In other words, this is how strongly the document belongs in a given topic. This is calculated by summing the product of the conditional probability and tag term weight for all tags attached to website k. This is cal￾culated as shown in equation 2. dwj,k = Xn i=0 P(ti|Dj ) · tfi,k · idfi (2) In this case, n is the number of distinct tags attached to website k by all users. P(ti|Dj ) is the conditional proba￾bility of tag term i (shown as ti) given domain j (shown as Dj ). tfi,k · idfi is as calculated by standard tf-idf, where tfi,k is the number of times any user bookmarked website k with tag i divided by the total number of times that the website is bookmarked. Similarly, the idfi is determined by the total number of websites in the entire set divided by the number of websites bookmarked with tag i. So, instead of the website vector consisting of tag term counts, a website’s domain vector parameters are now its topic domain weight value, i.e. how much the document belongs to that particu￾lar topic domain. An example calculation is shown in figure 2. apple mac software website k's attached tags D mac (0.69) software (0.68) osx (0.66) tool (0.59) apple (0.58) D program(0.78) develop(0.56) refer(0.52) code(0.34) software(0.29) +0.58 * 0.81 +0.69 * 0.51 +0.58 * 0.07 tf * idf apple (0.81) mac (0.51) software (0.07) +0.29 * 0.07 dw = 0.86 1,k dw = 0.02 2,k DVk = (0.86, 0.02, ...) i 2 1 p(t |D )j Figure 2: Calculation of k = apple.com’s website topic domain vector In this case, apple.com has three tags attached by all users: ‘apple’, ‘mac’, and ‘software’. Using example domain term probability values, we show the calculations for hypothetical domains 1 and 2 for a website k. Similarly, a bookmark topic domain vector created from user A bookmarking website k, DVA→k, would be calculated in the same fashion as shown in equation 3. DVA→k = (dw1,A→k, dw2,A→k, dw3,A→k, ...dw100,A→k) (3) The only exception is that the domain tag weight only considers the tags that user A used on website k as shown in equation 4. dwj,A→k = Xn i=0 P(ti|Dj ) · idfi,A→k (4) Since a user can only apply atmost one of the same tag term to a bookmark, only the idf is used here. 3.2 RCF Recommendations We now describe our Reasonable Tag-based Collaborative Filtering (RCF) algorithm. The process to generate recom￾mendations follows these steps: 1. Finding similar users. 2. Finding recommendation candidates. 3. Providing live-updating recommendations based upon the user’s current interest. We now describe these steps. 13

3.2.1 Finding Similar Users website liked. This is especially important when considering We start with finding similar users. Similar larger sites, such as Yahoo! or Slashdot, because they tend tional collaborative filtering, it is based upon to cover many topics, and thus, users may like the same site liked items-in this case, commonly bookmarked However, we also consider the used tags-which o be the reason why a user liked a resource. For example 3.2.2 Finding Recommendation Candidates from our previously shown system in figure 1, suppose we Next, we find recommendation candidates by calculating want to find similar users for user B. It would be calculated core prediction-the score which represents how much the s shown in figure 3. ke a resource. Again, similar to traditiona collaborative filtering we recommend websites that the sim- ilar users liked, except that now we also add the topic domain that the original user and the similar user have a common interest on. The goal being to recommend websites that match the topic that the two users commonly apple, news 2 For example, in the system shown in figure 1, we will find recommendation candidates for user B(user B has a sim- ilar user C-shown in figure 4). User B and C both liked website 2 for the reason that the website is about apple technology news. Thus, we find websites that match this topic. In this case, similar user C has two other bookmarks. Since only website 3s domain vector matches the commonly bookmarked website 2s domain vector-indicated by the dot. ted arrow-it has a high score prediction, and thus, becomes Figure 3: Example User Similarity Calculation a recommendation candidate(denoted by a star). On the other hand. C's other bookmark. website 4 does not match the commonly bookmarked website's topic domain-marked Since user B and C have bookmarked the same website by the crossed-out dotted line-and thus, its score prediction with similar tags-indicated by the dotted arrow-their simi- is lower larity score is higher. In this case, C becomes a similar user, indicated by the star. On the other hand, users B and A have bookmarked the same site, but they do not use similar target website, x tags-indicated by the crossed-out dotted arrow-and thus common bookmark. k their similarity score is lower. User similarity for a user A and a user B is calculated as apple, tech, I simre,(A,B)=a.-2Isim(DVA-k, DVB-k)) Figure 4: Example score prediction calculation Here, user similarity is the average of the cosine of A and B's domain vectors for each commonly bookmarked website, k In other words, if they bookmarked the same websites with The score prediction algorithm is shown in equation 7 similar tags, we assume that they liked the website for the Here, we are finding the score prediction for a target website same reasons, and thus, they are similar users. The first half r for a user A. We first take all bookmarks from all similar of the equation calculates the similarity between the topic users Sk with a user similarity above a certain user simi- domain vectors of each common bookmark for both users. larity threshold-in our case, 0.75. We then compare each The second half of the equation represents the number of of those bookmark domain vector DVsk-r with the book common bookmarks and increases as the number of common mark domain vector on the commonly bookmarked website bookmarks goes k, DVS,=k. We average these scores from all users and Also n is the number of bookmarks that user A and user then those webpages with scores above a certain threshold- main vector comparison, which for our experiments was set ditionally, we record the association between each commonly to 0.9. Additionally, sim(DVA-k, DVB-k)is the cosine of bookmarked webpage k and its generated recommendations user A's bookmark domain vector on website k and user B's This is used in the final score calculation described in the bookmark domain vector on website k as shown in equation following section Also, a is the weight given to the domain vector compar- ison, and in this case, it is 0.90. The left side calculates stm(DVA→k A→k‖!DVB (6) the similarity between the topic domain vectors of the com- mon bookmark and the target bookmark, and the right side Different from traditional CF, users must use similar tags of the equation represents the number of similar users that on the common website in addition to having the common have bookmarked the target website aT

3.2.1 Finding Similar Users We start with finding similar users. Similar to tradi￾tional collaborative filtering, it is based upon commonly liked items–in this case, commonly bookmarked websites. However, we also consider the used tags–which we assume to be the reason why a user liked a resource. For example, from our previously shown system in figure 1, suppose we want to find similar users for user B. It would be calculated as shown in figure 3. B C 2 apple, tech, news apple, news A games, reviews common bookmark, k Figure 3: Example User Similarity Calculation Since user B and C have bookmarked the same website with similar tags–indicated by the dotted arrow–their simi￾larity score is higher. In this case, C becomes a similar user, indicated by the star. On the other hand, users B and A have bookmarked the same site, but they do not use similar tags–indicated by the crossed-out dotted arrow–and thus, their similarity score is lower. User similarity for a user A and a user B is calculated as shown in equation 5. simrcf (A, B) = α · 1 n Xn 1 {sim(DVA→k, DVB→k)} + (1 − α) · log2(1 + n) (5) Here, user similarity is the average of the cosine of A and B’s domain vectors for each commonly bookmarked website, k. In other words, if they bookmarked the same websites with similar tags, we assume that they liked the website for the same reasons, and thus, they are similar users. The first half of the equation calculates the similarity between the topic domain vectors of each common bookmark for both users. The second half of the equation represents the number of common bookmarks and increases as the number of common bookmarks goes up. Also, n is the number of bookmarks that user A and user B have in common. α is the weight given to the topic do￾main vector comparison, which for our experiments was set to 0.9. Additionally, sim(DVA→k, DVB→k) is the cosine of user A’s bookmark domain vector on website k and user B’s bookmark domain vector on website k as shown in equation 6. sim(DVA→k, DVB→k) = DVA→k · DVA→k |DVA→k||DVB→k| (6) Different from traditional CF, users must use similar tags on the common website in addition to having the common website liked. This is especially important when considering larger sites, such as Yahoo! or Slashdot, because they tend to cover many topics, and thus, users may like the same site for different reasons. 3.2.2 Finding Recommendation Candidates Next, we find recommendation candidates by calculating score prediction–the score which represents how much the user should like a resource. Again, similar to traditional collaborative filtering, we recommend websites that the sim￾ilar users liked, except that now we also additionally match the topic domain that the original user and the similar user have a common interest on. The goal being to recommend websites that match the topic that the two users commonly liked. For example, in the system shown in figure 1, we will find recommendation candidates for user B (user B has a sim￾ilar user C–shown in figure 4). User B and C both liked website 2 for the reason that the website is about apple technology news. Thus, we find websites that match this topic. In this case, similar user C has two other bookmarks. Since only website 3’s domain vector matches the commonly bookmarked website 2’s domain vector–indicated by the dot￾ted arrow–it has a high score prediction, and thus, becomes a recommendation candidate (denoted by a star). On the other hand, C’s other bookmark, website 4, does not match the commonly bookmarked website’s topic domain–marked by the crossed-out dotted line–and thus, its score prediction is lower. 2 C 3 4 mac, rumors apple, tech, news tennis, sports common bookmark, k target website, x Figure 4: Example score prediction calculation The score prediction algorithm is shown in equation 7. Here, we are finding the score prediction for a target website x for a user A. We first take all bookmarks from all similar users Sk with a user similarity above a certain user simi￾larity threshold–in our case, 0.75. We then compare each of those bookmark domain vector DVSk→x with the book￾mark domain vector on the commonly bookmarked website k, DVSk→k. We average these scores from all users and then those webpages with scores above a certain threshold– 0.50 in our tests–become recommendation candidates. Ad￾ditionally, we record the association between each commonly bookmarked webpage k and its generated recommendations. This is used in the final score calculation described in the following section. Also, α is the weight given to the domain vector compar￾ison, and in this case, it is 0.90. The left side calculates the similarity between the topic domain vectors of the com￾mon bookmark and the target bookmark, and the right side of the equation represents the number of similar users that have bookmarked the target website x. 14

corepred(a, r)=a h-1[simrey(A, Sk).sim(DVSK-k, DVsk -1) +(1-a).log2(1+2Isimrer(A, Sk))(7) ∑k=1{ sTore/(A,Sk) all users attached tags common bookmark, k recommendation candidates.x pple, mac, software mac. rumors c tennis × Figure 5: Example Final Score Calculation 3.2.3 Providing Live-Updating Recommendations higher than a threshold of 0.50 are shown to the user in the Based Upon User's Current nterest descending order of the score The last step in the process of generating recommenda tions is determining which of the recommendation candi- 3.3 Live-Updating Website recommendation dates match the current interest. In our case. we assume Prototype System the user's current interest to be the topic of the website Using our RCF algorith, we created a prototy they are presently viewing. We then generate the topic do- main vector for the current ly viewed website by using the mendation system. It was created to provide live-updating ags from all users bookmarks from our mined data set website recommendations that automatically update based on the page that they are currently viewing. Thus, it was For example, in the system shown in figure 1, we provide designed to be viewable during the user's normal browsing recommendations for user B as shown in figure 5 Here, B is looking at a site which has the tags apple, activity. It is currently implemented as a Firefox sidebar mac, and software attached to it by bookmarks from all 6. While the user is browsing, the recommendations are ers. B also has two recommendation candidates, website updated whenever the viewed website changes, thereby pro- ystem), which were generated from section 3. 2.2. Website 3 viding recommendations relevant to the user's current inter est. Here, the generated recommendations are shown in the has been associated with the commonly bookmarked website sidebar on the left of the image. If they feel the recommen- 2. and website 6 has been associated with website 5. W then compare the user's currently viewed webpage's domain dations are useful, they can click on it and the browser will vector to all of B,s bookmarks. If we find one that is above be redirected to the recommended website a certain threshold. we take the recommendation candidates that are attached to it and calculate its final score as shown e全a in equation 8. All websites above a certain threshold are then recommended to the end user In this case. since bs bookmark domain vector matches the current websites domain vector, its attached website Pary as 52 180or is used for final score calculation if its score is above a certain threshold. it is recommended. B also has an arbi. trary recommendation candidate, website 6. However, since Bs bookmark on website 5 does not match the currently viewed website. its associated recommendation candidate is not recommended at this time. if the current website were to change to something about sports or tennis, website 6 would be recommended if its final score was above the se. ected threshold In equation 8, user A is viewing webpage cur, k is the commonly bookmarked website, and z is a recommenda- tion candidate. Lastly, B is the weight given to the part of the equation which determines how similar the recommen Figure 6: System Interface dation candidate should be to the current page. For our experiments, this was set to 1. All resultant final scores

scorepred(A, x) = α · Pn k=1{simrcf (A, Sk) · sim(DVSk→k, DVSk→x)} Pn k=1{simrcf (A, Sk)} + (1 − α) · log2(1 + Xn k=1 {simrcf (A, Sk)} (7) B 2 C apple, news common bookmark, k 3 mac, rumors cur E 5 tennis, sports 6 tennis all users attached tags apple, mac, software recommendation candidates, x Figure 5: Example Final Score Calculation 3.2.3 Providing Live-Updating Recommendations Based Upon User’s Current Interest The last step in the process of generating recommenda￾tions is determining which of the recommendation candi￾dates match the current interest. In our case, we assume the user’s current interest to be the topic of the website they are presently viewing. We then generate the topic do￾main vector for the currently viewed website by using the tags from all user’s bookmarks from our mined data set. For example, in the system shown in figure 1, we provide recommendations for user B as shown in figure 5. Here, B is looking at a site which has the tags ‘apple’, ‘mac’, and ‘software’ attached to it by bookmarks from all users. B also has two recommendation candidates, website 3 and 6 (6 is an arbitary webpage not shown in the example system), which were generated from section 3.2.2. Website 3 has been associated with the commonly bookmarked website 2, and website 6 has been associated with website 5. We then compare the user’s currently viewed webpage’s domain vector to all of B’s bookmarks. If we find one that is above a certain threshold, we take the recommendation candidates that are attached to it and calculate its final score as shown in equation 8. All websites above a certain threshold are then recommended to the end user. In this case, since B’s bookmark domain vector matches the current website’s domain vector, its attached website, 3, is used for final score calculation. If its score is above a certain threshold, it is recommended. B also has an arbi￾trary recommendation candidate, website 6. However, since B’s bookmark on website 5 does not match the currently viewed website, its associated recommendation candidate is not recommended at this time. If the current website were to change to something about sports or tennis, website 6 would be recommended if its final score was above the se￾lected threshold. In equation 8, user A is viewing webpage cur, k is the commonly bookmarked website, and x is a recommenda￾tion candidate. Lastly, β is the weight given to the part of the equation which determines how similar the recommen￾dation candidate should be to the current page. For our experiments, this was set to 1. All resultant final scores higher than a threshold of 0.50 are shown to the user in the descending order of the score. 3.3 Live-Updating Website Recommendation Prototype System Using our RCF algorithm, we created a prototype recom￾mendation system. It was created to provide live-updating website recommendations that automatically update based on the page that they are currently viewing. Thus, it was designed to be viewable during the user’s normal browsing activity. It is currently implemented as a Firefox sidebar plugin. The recommendation interface is shown in figure 6. While the user is browsing, the recommendations are updated whenever the viewed website changes, thereby pro￾viding recommendations relevant to the user’s current inter￾est. Here, the generated recommendations are shown in the sidebar on the left of the image. If they feel the recommen￾dations are useful, they can click on it and the browser will be redirected to the recommended website. Figure 6: System Interface 15

scorefinal(A, cur, r)=sim(DVcur, DVA-k.scorepred(DV, r).sim(DVeur, DVA-) 4. EXPERIMENT 3. Users would then look at the recommended websites As mentioned before in section 3.1. we mined delicio. us screenshot thumbnail, top tags, and title. They could RSS feeds for three months. In total. we retrieved over choose to click and view it or not 100,000 users, 2.5 million webpages, 3.6 million bookmarks, and 870,000 distinct tags. After mining this data, we per- 4. If they did click on a recommendation, the browser formed user testing with our algorithm against other popular would open the website. Additionally, rating buttons search methods to gauge the effectiveness of the algorithm would appear at the bottom of the sidebar. Here, they as well as observe how users react to generated recommen were asked to evaluate the website as either Good, dations. We tested our method as described in the previou Poor' in terms of the following two condi- section as well as three other different methods: tions: First, how interesting the pag whether the website was related to the that they topic-This method takes the top five topic domains from the currently viewed webpage and then finds all The experiment was done with eleven student volunteers the bookmarks from the database with high values for We now explain our results of our user testing hose topic domains. All websites above a cosine value of 0.75 are then ordered by that value and shown to 4.1 Results tag-bookmark-count(tbc)-This method takes the 4.1.1 How many bookmarks are needed before rec top three stemmed tags from the currently viewed web- ommendations can be made? page and then pulls all bookmarks from the database One of the difficulties in CF systems is getting the user ith these tags. The results are then ranked by the to evaluate and rate enough items. If there are not enough lumber of bookmarks in descending order. items rated, the system cannot confidently find similar user and therefore cannot find accurate recommendations as well cf-topic- All bookmarks of all users with common Thus, we explore how many bookmarks are necessary before bookmarks with the current user were stored as rec- the system can produce recommendations ommendation candidates. Then we take the currentl Figure 7(a) shows a user's total number of bookmarks viewed website's domain vector and those stored book versus the ratio of users with at least one common bookmark marks'domain vectors and calculate the cosine. They with another user. For example, about 40% of the users with are then ranked by cosine value. This has similarity to one to five total bookmarks have a bookmark in common our method, but it does not consider tagging when cal- with another user. Similarly, 80% of users with eleven to culating the score predictions of the recommendation fifteen bookmarks have at least one common bookmark with candidates another user Next, figure 7(b) shows a users total number of book Using the Firefox plugin, users were asked to create a marks versus the ratio of users with at least one recom- profile by bookmarking twenty or more websites using tags nendation candidate with a score prediction above 0.50(ex- They were free to use any webpage they liked with any tags plained in section 3. 2. 2). At the sixteen to twenty bookmark hey felt were appropriate. After their profiles were cre- ated, similar users as well as recommendation candidates range, a little more than 80% of the users have recommen- rere calculated as shown in section 3.2.1 and 3.2. 2. This dation candidates. After this point, the rate of increase in step occurred beforehand and not while the user was brows- the proportion of users with recommendations decreases. ing. The last step of selecting recommendation candidates Even with a few bookmarks, the system is able to generate to display (using the final score as shown in section 3.2.3) recommendations for the candidates. It is also largely de- was done in real-time while the user was browsing. pendent on which website you bookmark and whether that Each user followed this recommendation evaluation pro- website has been bookmarked by another user cedure for fifteen or more websites of their choosing 4.1.2 How often are recommendation candidates be- 1. The user opens a website in firefox ing shown? We have shown that our algorithm can generate recom- 2. Based upon the website, the system generates up to mendation candidates with just a few bookmarks. Next, we six recommendations and displays them in the side take a look at how often these candidates are being shown bar plugin. This interface was shown previously in 6. to the user in the final step of providing live-updating rec- The system randomly chooses one of the previously ommendations. Thus, in table 1, we have listed the number described algorithm. In the case of our algorithm, it of recommendations generated per times requested calculates the final score as shown in section 3. It The first column shows the number of recommendation does so by comparing the domain tag vector of the requests. The second shows the number of recommenda- currently viewed site and the domain tag vector asso- tions that were returned in total. the third column shows ciated with each recommendation candidate. It would the average number of recommendations per request. For display the top 6 results that were above the final score each request, there was a maximum of six recommendations threshold of 0.5. The other methods would do their re- returned. Last, there is the adjusted rate, which is the ap- pective procedure using the current page's tags proximate ratio of the number of times that each method

scoref inal(A, cur, x) = sim(DVcur, DVA→k) · scorepred(DV, x) · sim(DVcur, DVA→x) β (8) 4. EXPERIMENT As mentioned before in section 3.1, we mined del.icio.us RSS feeds for three months. In total, we retrieved over 100,000 users, 2.5 million webpages, 3.6 million bookmarks, and 870,000 distinct tags. After mining this data, we per￾formed user testing with our algorithm against other popular search methods to gauge the effectiveness of the algorithm as well as observe how users react to generated recommen￾dations. We tested our method as described in the previous section as well as three other different methods: • topic - This method takes the top five topic domains from the currently viewed webpage and then finds all the bookmarks from the database with high values for those topic domains. All websites above a cosine value of 0.75 are then ordered by that value and shown to the user • tag-bookmark-count (tbc) - This method takes the top three stemmed tags from the currently viewed web￾page and then pulls all bookmarks from the database with these tags. The results are then ranked by the number of bookmarks in descending order. • cf-topic - All bookmarks of all users with common bookmarks with the current user were stored as rec￾ommendation candidates. Then we take the currently viewed website’s domain vector and those stored book￾marks’ domain vectors and calculate the cosine. They are then ranked by cosine value. This has similarity to our method, but it does not consider tagging when cal￾culating the score predictions of the recommendation candidates Using the Firefox plugin, users were asked to create a profile by bookmarking twenty or more websites using tags. They were free to use any webpage they liked with any tags they felt were appropriate. After their profiles were cre￾ated, similar users as well as recommendation candidates were calculated as shown in section 3.2.1 and 3.2.2. This step occurred beforehand and not while the user was brows￾ing. The last step of selecting recommendation candidates to display (using the final score as shown in section 3.2.3) was done in real-time while the user was browsing. Each user followed this recommendation evaluation pro￾cedure for fifteen or more websites of their choosing: 1. The user opens a website in firefox. 2. Based upon the website, the system generates up to six recommendations and displays them in the side￾bar plugin. This interface was shown previously in 6. The system randomly chooses one of the previously described algorithm. In the case of our algorithm, it calculates the final score as shown in section 3.2.3. It does so by comparing the domain tag vector of the currently viewed site and the domain tag vector asso￾ciated with each recommendation candidate. It would display the top 6 results that were above the final score threshold of 0.5. The other methods would do their re￾spective procedure using the current page’s tags. 3. Users would then look at the recommended website’s screenshot thumbnail, top tags, and title. They could choose to click and view it or not. 4. If they did click on a recommendation, the browser would open the website. Additionally, rating buttons would appear at the bottom of the sidebar. Here, they were asked to evaluate the website as either ‘Good’, ‘Fair’, or ‘Poor’ in terms of the following two condi￾tions: First, how interesting the page was. Second, whether the website was related to the page that they were currently viewing. The experiment was done with eleven student volunteers. We now explain our results of our user testing. 4.1 Results 4.1.1 How many bookmarks are needed before rec￾ommendations can be made? One of the difficulties in CF systems is getting the user to evaluate and rate enough items. If there are not enough items rated, the system cannot confidently find similar users, and therefore cannot find accurate recommendations as well. Thus, we explore how many bookmarks are necessary before the system can produce recommendations. Figure 7(a) shows a user’s total number of bookmarks versus the ratio of users with at least one common bookmark with another user. For example, about 40% of the users with one to five total bookmarks have a bookmark in common with another user. Similarly, 80% of users with eleven to fifteen bookmarks have at least one common bookmark with another user. Next, figure 7(b) shows a user’s total number of book￾marks versus the ratio of users with at least one recom￾mendation candidate with a score prediction above 0.50 (ex￾plained in section 3.2.2). At the sixteen to twenty bookmark range, a little more than 80% of the users have recommen￾dation candidates. After this point, the rate of increase in the proportion of users with recommendations decreases. Even with a few bookmarks, the system is able to generate recommendations for the candidates. It is also largely de￾pendent on which website you bookmark and whether that website has been bookmarked by another user. 4.1.2 How often are recommendation candidates be￾ing shown? We have shown that our algorithm can generate recom￾mendation candidates with just a few bookmarks. Next, we take a look at how often these candidates are being shown to the user in the final step of providing live-updating rec￾ommendations. Thus, in table 1, we have listed the number of recommendations generated per times requested. The first column shows the number of recommendation requests. The second shows the number of recommenda￾tions that were returned in total. The third column shows the average number of recommendations per request. For each request, there was a maximum of six recommendations returned. Last, there is the adjusted rate, which is the ap￾proximate ratio of the number of times that each method 16

1.0 E08 0.52 80.6 0.4 0.440 9g0.2 0.536 1-56-10l-l5|6-2022526-30 user's total number of bookmarks 0.594 User's total number of bookmarks ys. ratio of users with at least one common bookmark 00.0.20.3040.50.6 1.0 Figure 8: Precision(Good/Total) 是08 0.6 rate every recommendation they visited, with either 'Good Fair, 'Poor. Here, each method's precision is shown. As can be seen here, our method RCF provides the highest level among all the recommendation methods. It is followed by cf-topic, topic, and then finally tbe. Additionally, we take a look at the average ratings for -56-0|-l516-202|2526-30 the top three recommendations per request. This is shown in figure 9. Here, we have the average rating for the top user's total number of bookmarks (b)User's total number of bookmarks vs. ratio of users at least one recommendation Figure 7: Users Total Number of Bookmarks vs recom. adj. recom. 0.621 3332 0.621 f-topic 169 628 0.619 Table 1: Amount of commendations generated c-te had at least one recommendation for a request. For topic, tbc, cf-topic, this came out to roughly 2 /3 of the time. For CF, this came out to a little more than 1/3 of the time. Figure 9: Average Score of the Top Results per Re- Our system evenly spread the requests over the methods. le to the specific nature of our algorithm, our method ded to generate less than the other methods. Addition- ally, our mined data set was relatively small. listed result, followed by the top two, and finally top three 4.1.3 How effective are the recommendations? results per recommendation request. The ratings were given the following values: Good= 1, 'Fair= 0.5, and "Poor Now, we examine how effective each method was for rec- =0. Again, within the recommender interface, up to six ommendation. Figure 8 shows the precision of each of the recommendations are shown in the sidebar, ordered from ecommendation methods the highest final score to the lowest. RCF has the highest Precision is defined as the number of ratings over precision among all the methods when considering only the the total number of ratings. Again, each user was asked t top results per request

0 0.2 0.4 0.6 0.8 1.0 1-5 6-10 11-15 16-20 21-25 26-30 user's total number of bookmarks ratio of users with at least one common bookmark (a) User’s total number of bookmarks vs. ratio of users with at least one common bookmark 0 0.2 0.4 0.6 0.8 1.0 1-5 6-10 11-15 16-20 21-25 26-30 ratio of users with at least one recommendation user's total number of bookmarks (b) User’s total number of bookmarks vs. ratio of users at least one recommendation Figure 7: User’s Total Number of Bookmarks vs ... num. num. recom. adj. recom. req. recom. per req. rate topic 173 645 3.73 0.621 tbc 172 641 3.73 0.621 cf-topic 169 628 3.72 0.619 rcf 170 358 2.11 0.351 Table 1: Amount of Recommendations Generated had at least one recommendation for a request. For topic, tbc, cf-topic, this came out to roughly 2/3 of the time. For RCF, this came out to a little more than 1/3 of the time. Our system evenly spread the requests over the methods. Due to the specific nature of our algorithm, our method tended to generate less than the other methods. Addition￾ally, our mined data set was relatively small. 4.1.3 How effective are the recommendations? Now, we examine how effective each method was for rec￾ommendation. Figure 8 shows the precision of each of the recommendation methods. Precision is defined as the number of ‘Good’ ratings over the total number of ratings. Again, each user was asked to topic tbc cf-topic RCF 0 0.1 0.2 0.3 0.4 0.5 0.6 Figure 8: Precision (Good / Total) rate every recommendation they visited, with either ’Good’, ’Fair’, ’Poor’. Here, each method’s precision is shown. As can be seen here, our method RCF provides the highest level among all the recommendation methods. It is followed by cf-topic, topic, and then finally tbc. Additionally, we take a look at the average ratings for the top three recommendations per request. This is shown in figure 9. Here, we have the average rating for the top 0.4 0.5 0.6 0.7 0.8 top 1 top 2 top 3 RCF cf-topic topic tagbc Figure 9: Average Score of the Top Results per Re￾quest listed result, followed by the top two, and finally top three results per recommendation request. The ratings were given the following values: ‘Good’ = 1, ‘Fair’ = 0.5, and ‘Poor’ = 0. Again, within the recommender interface, up to six recommendations are shown in the sidebar, ordered from the highest final score to the lowest. RCF has the highest precision among all the methods when considering only the top results per request. 17

4.2 Results discussion ing the results. However, we believe it is an effective method We first showed that our algorithm is capable of generat- that can be employed in all sorts of online tagging systems ing recommendations even when only a few websites have Social tagging systems and CF rely heavily on the people us- been bookmarked. This is an advantage over traditional CF ing these systems, so combining them as done in our method systems, which require more input for it to be effective seems like the next logical step. Doing so provides relevant In terms of amount of recommendations generated, it can recommendations in which the user has confidence in the be seen that our recommendation method did not generate credibility of the source. Social tagging systems would ben- s many results as the other methods. Since our method is efit greatly from an effective recommendation system In the future, we plan to further refine the recommenda- dation candidates. Additionally, this can be attributed to tion method. We are evaluating what the optimal threshold he fact that del icio us is largely used by technology-savvy people-therefore, much of the bookmarking data is for web eedback into our method. Currently it is based sites that are largely technology-oriented. Since some of the bookmarks. However, adding positive or negative feedback testers did not bookmark websites in a technology-related of the recommendations generated would allow the system to tion candidates with a final score above the threshold. In the add recommendation grouping, where the results are sorted future, when social bookmarking becomes used by a wider and grouped by topic. This way, the users can have a better audience, the variety of data should expand and allow for idea of what the website contains before they choose to view recommendations in more domains it Looking at the precision data, our RCF method provides the most relevant recommendations to the user in terms of Acknowledgments Account tagging information, RCF is able to more effectively Information and Communications Technology, MEXT(Grant- er out the appropriate results. Additionally, it can recom mend websites that are related to what the user is currently in- Aid for Scientific Research on Priority Areas #19024058) iewing in real-time and MEXT(Grant-in-Aid for Young Scientists(B)#20700089) Topic searching was the third most effective method in our testing. Its algorithm is entirely based on the tagging 6. REFERENCES formation from the current page. It does not use the users JAmazoncomhttp://www.amazon.com/ bookmarking profile in any way. 2Citeulike.http://www.citeulike.org/ tbc ended up last in its precision value. It takes website 3del.icio.ushttp://del.icio.us/. popularity into consideration as its ranking method orders by bookmark count. However. since tbc does not consider 4Flickr.http://www.flickrcom the user's bookmarking profile, it is unable to provide per- 5movielens.http://www.movielens.t sonalized recommendations 6 S Golder and B Huberman. The Lastly, cf-topic came up second place. It is a basic imple collaborative tagging systems mentation of collaborative filtering: it considers the user's 7 R Nakamoto, S. Nakajima, J. Miyazaki, and H K. profile to some degree. However, it does not utilize the S. Uemura. Evaluation of tag-based contextual tagging information when performing collaborative filtering collaborative filtering effectiveness in website Since social bookmarking systems do not have numeric rat recommendation. Technical Report 131, IEICE, 2007 ings at all, traditional CF is very ineffective. A user can [8R Nakamoto, S. Nakajima, J. Miyazaki, and only indicate liking a website with a bookmark or not-there S. Uemura. Tag-based contextual collaborative is no rating in between, nor are there negative ratings. filtering. In 18th IEICE Data Engineering Workshop Regarding the top results' values, RCF provided a higher recision when considering the top results. This is important 9 P. Resnick, N. Iacovou, M. Suchak, P. Bergstorm, and in recommendation systems as there is a limit on how many J. Riedl. GroupLens: An Open Architecture for recommendations you can provide to the end user. Being Collaborative Filtering of Netnews. In Proceedings of so, it is more important to provide high precision over high ACM 1994 Conference on Computer Supported recall Cooperative Work, pages 175-186. Chapel North Carolina. 1994. ACM [10X. Wu, L. Zhang, and Y. Yu. Exploring social 5. CONCLUSIONS AND FUTURE WORK annotations for the semantic web. In www 06: We have described our Reasonable Tag-Based Collabora- Proceedings of the 15th international conference on tive Filtering method for use in social tagging systems. Ad- World wide Web, pages 417-426, New York, NY ditionally, we have described its place in our live-updating USA. 2006. ACM website recommendation system. Through our user testing we have shown that live-updating implicit recommendation is possible. We found it to be more effective than other tested methods, and additionally provided higher ratings for he top results per recommendation request domain-making the task of providing recommendations dif- ficult. Thus, there are still many unexplored factors effect-

4.2 Results Discussion We first showed that our algorithm is capable of generat￾ing recommendations even when only a few websites have been bookmarked. This is an advantage over traditional CF systems, which require more input for it to be effective. In terms of amount of recommendations generated, it can be seen that our recommendation method did not generate as many results as the other methods. Since our method is more specific, it resulted in a lower number of recommen￾dation candidates. Additionally, this can be attributed to the fact that del.icio.us is largely used by technology-savvy people–therefore, much of the bookmarking data is for web￾sites that are largely technology-oriented. Since some of the testers did not bookmark websites in a technology-related topic domain, our system could not find any recommenda￾tion candidates with a final score above the threshold. In the future, when social bookmarking becomes used by a wider audience, the variety of data should expand and allow for recommendations in more domains. Looking at the precision data, our RCF method provides the most relevant recommendations to the user in terms of the precision of the recommendation results. By taking into account tagging information, RCF is able to more effectively filter out the appropriate results. Additionally, it can recom￾mend websites that are related to what the user is currently viewing in real-time. Topic searching was the third most effective method in our testing. Its algorithm is entirely based on the tagging information from the current page. It does not use the user’s bookmarking profile in any way. tbc ended up last in its precision value. It takes website popularity into consideration as its ranking method orders by bookmark count. However, since tbc does not consider the user’s bookmarking profile, it is unable to provide per￾sonalized recommendations. Lastly, cf-topic came up second place. It is a basic imple￾mentation of collaborative filtering: it considers the user’s profile to some degree. However, it does not utilize the tagging information when performing collaborative filtering. Since social bookmarking systems do not have numeric rat￾ings at all, traditional CF is very ineffective. A user can only indicate liking a website with a bookmark or not–there is no rating in between, nor are there negative ratings. Regarding the top results’ values, RCF provided a higher precision when considering the top results. This is important in recommendation systems as there is a limit on how many recommendations you can provide to the end user. Being so, it is more important to provide high precision over high recall. 5. CONCLUSIONS AND FUTURE WORK We have described our Reasonable Tag-Based Collabora￾tive Filtering method for use in social tagging systems. Ad￾ditionally, we have described its place in our live-updating website recommendation system. Through our user testing, we have shown that live-updating implicit recommendation is possible. We found it to be more effective than other tested methods, and additionally provided higher ratings for the top results per recommendation request. Overall, the system we have proposed covers a large, vast domain–making the task of providing recommendations dif- ficult. Thus, there are still many unexplored factors effect￾ing the results. However, we believe it is an effective method that can be employed in all sorts of online tagging systems. Social tagging systems and CF rely heavily on the people us￾ing these systems, so combining them as done in our method seems like the next logical step. Doing so provides relevant recommendations in which the user has confidence in the credibility of the source. Social tagging systems would ben￾efit greatly from an effective recommendation system. In the future, we plan to further refine the recommenda￾tion method. We are evaluating what the optimal threshold is for the final score. Next, we also intend to add relevance feedback into our method. Currently, it is based solely on bookmarks. However, adding positive or negative feedback of the recommendations generated would allow the system to provide better recommendations later on. Lastly, we plan to add recommendation grouping, where the results are sorted and grouped by topic. This way, the users can have a better idea of what the website contains before they choose to view it. Acknowledgments This work was supported in part by the National Institute of Information and Communications Technology, MEXT (Grant￾in- Aid for Scientific Research on Priority Areas #19024058), and MEXT (Grant-in-Aid for Young Scientists (B) #20700089). 6. REFERENCES [1] Amazon.com. http://www.amazon.com/. [2] Citeulike. http://www.citeulike.org/. [3] del.icio.us. http://del.icio.us/. [4] Flickr. http://www.flickr.com/. [5] movielens. http://www.movielens.umn.edu/. [6] S. Golder and B. Huberman. The structure of collaborative tagging systems. [7] R. Nakamoto, S. Nakajima, J. Miyazaki, and H. K. S. Uemura. Evaluation of tag-based contextual collaborative filtering effectiveness in website recommendation. Technical Report 131, IEICE, 2007. [8] R. Nakamoto, S. Nakajima, J. Miyazaki, and S. Uemura. Tag-based contextual collaborative filtering. In 18th IEICE Data Engineering Workshop, 2007. [9] P. Resnick, N. Iacovou, M. Suchak, P. Bergstorm, and J. Riedl. GroupLens: An Open Architecture for Collaborative Filtering of Netnews. In Proceedings of ACM 1994 Conference on Computer Supported Cooperative Work, pages 175–186, Chapel Hill, North Carolina, 1994. ACM. [10] X. Wu, L. Zhang, and Y. Yu. Exploring social annotations for the semantic web. In WWW ’06: Proceedings of the 15th international conference on World Wide Web, pages 417–426, New York, NY, USA, 2006. ACM. 18