《电子商务 E-business》阅读文献：Recommender Systems in E-Commerce

Recommender Systems in E-Commerce J. Ben Schafer, Joseph Konstan, John Riedl GroupLens Research Project Department of Computer Science and Engineering University of Minnesota Minneapolis, MN 55455 1-612-625-4002 (schafer, konstan, riedl @cs. umn. edu ABSTRACT their needs. One solution to this information overload problem is the use of recommender systems Recommender systems are changing from novelties used by a few E-commerce sites, to serious business tools that are re-shaping Recommender systems are used by E-commerce sites to suggest the world of E-commerce. Many of the largest commerce Web products to their customers. The products can be recommended sites are already using recommender systems to help their based on the top overall sellers on a site, based on the customers find products to purchase. A recommender system demographics of the customer, or based on an analysis of the past learns from a customer and recommends products that she will buying behavior of the customer as a prediction for future buying find most valuable from among the available products. In this behavior. Broadly, these techniques are part of personalization paper we present an explanation of how recommender systems n a site, because they help the site adapt itself to each customer Ip E-commerce sites increase sales, and analyze six sites that Recommender systems automate personalization on the Web, use recommender systems including several sites that use more enabling individual personalization for each customer than one recommender system. Based on the examples, we Personalization to this extent is one way to realize Pines ideas create a taxonomy of recommender systems, including the on the Web. Thus, Pine would probably agree with Jeff Bezos, terraces they present to customers, the technologies used to cEO of Amazon comM when he said "If I have 2 millio eate the recommendations, and the inputs they need from customers on the Web. i should have 2 million stores on the customers. We conclude with ideas for new applications of Web recommender systems to E-commerce Recommender systems enhance E-commerce sales in three ways Keywords Browsers into buyers: Visitors to a Web site often look over the Electronic commerce, recommender systems, interface, customer site without ever purchasing anything Recommender systems loyalty, cross-sell, up-sell, mass customization. can help customers find products they wish to purchase Cross-sell: Recommender systems improve cross-sell by 1. NTRODUCTION suggesting additional products for the customer to purchase. If In his book Mass Customization(Pine, 1993), Joe the recommendations are good, the average order size should need to shift from the old worl s increase. For instance, a site might recommend additi oduction where"standardized products, homogene products in the checkout process, based on those products alre and long product life and development cycles were the rule"to pping cart the new world where " variety and customization supplant Loyalty: In a world where a site s competitors are only a click or standardized products. Pine argues that building one product is two away, gaining customer loyalty is an essential business simply not adequate anymore. Companies need to be able to, at minimum, develop multiple products that meet the multiple strategy (Reichheld and Sesser, 1990)(Reichheld, 1993) Recommende needs of multiple customers. The movement toward E-commerce relationship between the site and the customer. Sites invest in has allowed companies to provide customers with more options However, in expanding to this new level of customization, learning about their users, use recommender systems to businesses increase the amount of information that customers perationalize that learning, and present custom interfaces that must process before they are able to select which items meet match customer needs. Customers repay these sites by returning to the ones that best match their needs The more a custome uses the recommendation system - teaching it what they want the more loyal they are to the site. "Even if a competitor were to build the exact same capabilities, a customer. would have to spend an inordinate amount of time and energy teaching the competitor what the company already knows. (Pine, et al. 1995) Finally, creating relationships between customers can also increase loyalty the site that recommends people with whom they will like to interact

Recommender Systems in E-Commerce J. Ben Schafer, Joseph Konstan, John Riedl GroupLens Research Project Department of Computer Science and Engineering University of Minnesota Minneapolis, MN 55455 1-612-625-4002 {schafer, konstan, riedl}@cs.umn.edu ABSTRACT Recommender systems are changing from novelties used by a few E-commerce sites, to serious business tools that are re-shaping the world of E-commerce. Many of the largest commerce Web sites are already using recommender systems to help their customers find products to purchase. A recommender system learns from a customer and recommends products that she will find most valuable from among the available products. In this paper we present an explanation of how recommender systems help E-commerce sites increase sales, and analyze six sites that use recommender systems including several sites that use more than one recommender system. Based on the examples, we create a taxonomy of recommender systems, including the interfaces they present to customers, the technologies used to create the recommendations, and the inputs they need from customers. We conclude with ideas for new applications of recommender systems to E-commerce. Keywords Electronic commerce, recommender systems, interface, customer loyalty, cross-sell, up-sell, mass customization. 1. INTRODUCTION In his book Mass Customization (Pine, 1993), Joe Pine argues that companies need to shift from the old world of mass production where “standardized products, homogeneous markets, and long product life and development cycles were the rule” to the new world where “variety and customization supplant standardized products.” Pine argues that building one product is simply not adequate anymore. Companies need to be able to, at a minimum, develop multiple products that meet the multiple needs of multiple customers. The movement toward E-commerce has allowed companies to provide customers with more options. However, in expanding to this new level of customization, businesses increase the amount of information that customers must process before they are able to select which items meet their needs. One solution to this information overload problem is the use of recommender systems. Recommender systems are used by E-commerce sites to suggest products to their customers. The products can be recommended based on the top overall sellers on a site, based on the demographics of the customer, or based on an analysis of the past buying behavior of the customer as a prediction for future buying behavior. Broadly, these techniques are part of personalization on a site, because they help the site adapt itself to each customer. Recommender systems automate personalization on the Web, enabling individual personalization for each customer. Personalization to this extent is one way to realize Pine’s ideas on the Web. Thus, Pine would probably agree with Jeff Bezos, CEO of Amazon.com™ , when he said “If I have 2 million customers on the Web, I should have 2 million stores on the Web.” Recommender systems enhance E-commerce sales in three ways: Browsers into buyers: Visitors to a Web site often look over the site without ever purchasing anything. Recommender systems can help customers find products they wish to purchase. Cross-sell: Recommender systems improve cross-sell by suggesting additional products for the customer to purchase. If the recommendations are good, the average order size should increase. For instance, a site might recommend additional products in the checkout process, based on those products already in the shopping cart. Loyalty: In a world where a site’s competitors are only a click or two away, gaining customer loyalty is an essential business strategy (Reichheld and Sesser, 1990) (Reichheld, 1993). Recommender systems improve loyalty by creating a value-added relationship between the site and the customer. Sites invest in learning about their users, use recommender systems to operationalize that learning, and present custom interfaces that match customer needs. Customers repay these sites by returning to the ones that best match their needs. The more a customer uses the recommendation system – teaching it what they want – the more loyal they are to the site. “Even if a competitor were to build the exact same capabilities, a customer … would have to spend an inordinate amount of time and energy teaching the competitor what the company already knows.” (Pine, et al. 1995) Finally, creating relationships between customers can also increase loyalty. Customers will return to the site that recommends people with whom they will like to interact

This paper makes five contributions to the understanding of Amazon. com Delivers: Amazon. com Delivers is a variation on recommender systems in E-commerce. First, we provide a set of the Eyes feature. Customers select checkboxes to choose from a recommender system examples that span the range of different list of specific categories/genres (Oprah books, biographies, applications of recommender systems in E-commerce. Second, cooking). Periodically the editors at Amazon. com send email we analyze the way in which each of the examples uses the announcements to notify subscribers of their latest recommender system to enhance revenue on the site. Third, we recommendations in the subscribed categor describe a mapping from applications of recommender systems to Book Matcher: The book Matcher feature allows customers to a taxonomy of ways of implementing the applications. Fourth, we examine the effort required from give direct feedback about books they have read. Customers rate recommendations. Fifth, we describe a set of suggestions fo books they have read on a 5-point scale from"hated it" to"loved new recommender system applications based of our it. After rating a sample of books, customers may request recommendations for books they might like. At that point a half taxonomy that have not been explored by the existing dozen non-rated texts are presented which correlate with the applications user's indicated tastes. Feedback to these recommendations is aper is useful to two groups: academics studying provided by a"rate these books" feature where customers can mender systems in E-commerce, and implementers indicate a rating for one or more of the recommended books ering applying recommender systems in their site. For academics, the examples and taxonomies provide a useful initial Customer Comments: The Customer Comments feature allows framework within which their research can be placed. The of other customers. Located on the information page for each framework will undoubtedly be expanded to include future hook is a list of 1-5 star ratings and written comments provided applications of recommender systems. For implementers, the paper provides a way of making choices among the available by customers who have read the book in question and submitted a review. Customers have the option of incorporating these applications and technologies. An implementer can choose a recommendations into their purchase decision oneymaking goal, select the interfaces that will help achieve that goal, and select an implementation technique that supports 2.2 CDNOW the goal within the interface Album Advisor: The Album Advisor feature of CDNOWTM 2. Recommender system examples (www.cdnow.com)worksintwodifferentmodesInthesingle album mode customers locate the information page for a given In the following section we present six e-commerce businesses album. The system recommends 10 other albums related to the that utilize one or more variations of recommender system album in question. In the multiple artist mode customers enter we give a brief description of the features of the system. In later up to three artists. In turn, the system recommends 10 album recommendations provided, the type of technology used, and the My CDNOW: My CDNOW enables customers to set up their types of information gathered. For organizational purposes these ic store, based on albums and artists they like. sites have been alphabetized. The examples listed were correct Customers indicate which albums they own, and which artists are as of May 31, 1999. Due to the rapid changes in the Internet their favorites. Purchases from CDNOw are entered they may no longer be valid automatically into the " own it" list. Although"own it"ratings are initially treated as an indication of positive likes, customers 2.1 Amazon. com can go back and distinguish between "own it and like it" and We focus on recommender systems in the book section of"own it but dislike it. When customers request Amazon. com recommendations the system will predict 6 albums the custome Customers who Bought: Like many E-commerce site es might like based on what is already owned. A feedback option is Amazon.comm(www.amazon.comisstructuredwithan available by customers providing a " own it, "move to wish list information page for each book, giving details of the text and or"not for me comment for any of the albums in this prediction purchase information. The Customers who Bought feature is list. The albums recommended change based on the feedback. found on the information page for each book in their catalog. It 2.3 eBay recommends books frequently purchased by customers who FeedbackProfileTheFeedbackProfilefeatureatebAy.comtm purchased the selected book. The second recommends authors ebay. com) allows both buyers and sellers to contribute to whose books are frequently purchased feedback profiles of other customers with whom they have done purchased works by the author of the selected book. business. The feedback consists of a satisfaction rating (satisfied/neutral/dissatisfied) as well as a specific comment Eyes: The Eyes feature allows customers to be notified via about the other customer. Feedback is used to provide a email of new items added to the Amazon. com catalog recommender system for purchasers, who are able to view the ustomers enter requests based upon author, title, subject, ISBN, profile of sellers. This profile consists of a table of the number or publication date information. Customers can use both simple of each rating in the past 7 days, past month, and past 6 months, and more complex Boolean-base ed criteria (AND/OR)for well as an overall summary (e.g, 867 positives from 776 notification queries. Requests can also be directly entered from unique customers). Upon further request, customers can browse any search results screen, creating a persistent request based the individual ratings and comments for the sellers the search

This paper makes five contributions to the understanding of recommender systems in E-commerce. First, we provide a set of recommender system examples that span the range of different applications of recommender systems in E-commerce. Second, we analyze the way in which each of the examples uses the recommender system to enhance revenue on the site. Third, we describe a mapping from applications of recommender systems to a taxonomy of ways of implementing the applications. Fourth, we examine the effort required from users to find recommendations. Fifth, we describe a set of suggestions for new recommender system applications based on parts of our taxonomy that have not been explored by the existing applications. The paper is useful to two groups: academics studying recommender systems in E-commerce, and implementers considering applying recommender systems in their site. For academics, the examples and taxonomies provide a useful initial framework within which their research can be placed. The framework will undoubtedly be expanded to include future applications of recommender systems. For implementers, the paper provides a way of making choices among the available applications and technologies. An implementer can choose a moneymaking goal, select the interfaces that will help achieve that goal, and select an implementation technique that supports the goal within the interface. 2. Recommender System Examples In the following section we present six e-commerce businesses that utilize one or more variations of recommender system technology in their web sites. For each site, and each variation, we give a brief description of the features of the system. In later sections we refer to these examples as we explain the types of recommendations provided, the type of technology used, and the types of information gathered. For organizational purposes these sites have been alphabetized. The examples listed were correct as of May 31, 1999. Due to the rapid changes in the Internet they may no longer be valid. 2.1 Amazon.com We focus on recommender systems in the book section of Amazon.com. Customers who Bought: Like many E-commerce sites, Amazon.com™ (www.amazon.com) is structured with an information page for each book, giving details of the text and purchase information. The Customers who Bought feature is found on the information page for each book in their catalog. It is in fact two separate recommendation lists. The first recommends books frequently purchased by customers who purchased the selected book. The second recommends authors whose books are frequently purchased by customers who purchased works by the author of the selected book. Eyes: The Eyes feature allows customers to be notified via email of new items added to the Amazon.com catalog. Customers enter requests based upon author, title, subject, ISBN, or publication date information. Customers can use both simple and more complex Boolean-based criteria (AND/OR) for notification queries. Requests can also be directly entered from any search results screen, creating a persistent request based on the search. Amazon.com Delivers: Amazon.com Delivers is a variation on the Eyes feature. Customers select checkboxes to choose from a list of specific categories/genres (Oprah books, biographies, cooking). Periodically the editors at Amazon.com send email announcements to notify subscribers of their latest recommendations in the subscribed categories. Book Matcher: The Book Matcher feature allows customers to give direct feedback about books they have read. Customers rate books they have read on a 5-point scale from “hated it” to “loved it.” After rating a sample of books, customers may request recommendations for books they might like. At that point a half dozen non-rated texts are presented which correlate with the user’s indicated tastes. Feedback to these recommendations is provided by a “rate these books” feature where customers can indicate a rating for one or more of the recommended books. Customer Comments: The Customer Comments feature allows customers to receive text recommendations based on the opinions of other customers. Located on the information page for each book is a list of 1-5 star ratings and written comments provided by customers who have read the book in question and submitted a review. Customers have the option of incorporating these recommendations into their purchase decision. 2.2 CDNOW Album Advisor: The Album Advisor feature of CDNOW™ (www.cdnow.com) works in two different modes. In the single album mode customers locate the information page for a given album. The system recommends 10 other albums related to the album in question. In the multiple artist mode customers enter up to three artists. In turn, the system recommends 10 albums related to the artists in question. My CDNOW: My CDNOW enables customers to set up their own music store, based on albums and artists they like. Customers indicate which albums they own, and which artists are their favorites. Purchases from CDNOW are entered automatically into the “own it” list. Although “own it” ratings are initially treated as an indication of positive likes, customers can go back and distinguish between “own it and like it” and “own it but dislike it.” When customers request recommendations the system will predict 6 albums the customer might like based on what is already owned. A feedback option is available by customers providing a “own it,” “move to wish list” or “not for me” comment for any of the albums in this prediction list. The albums recommended change based on the feedback. 2.3 eBay Feedback Profile: The Feedback Profile feature at eBay.com™ (www.ebay.com) allows both buyers and sellers to contribute to feedback profiles of other customers with whom they have done business. The feedback consists of a satisfaction rating (satisfied/neutral/dissatisfied) as well as a specific comment about the other customer. Feedback is used to provide a recommender system for purchasers, who are able to view the profile of sellers. This profile consists of a table of the number of each rating in the past 7 days, past month, and past 6 months, as well as an overall summary (e.g., 867 positives from 776 unique customers). Upon further request, customers can browse the individual ratings and comments for the sellers

2. 4 Levis describes the recommendation technology used by the site, and Style Finder: Style Finder allows customers of the Levi the inputs required by that technology. The fourth column describes how users find recommendations using the application on articles of Levis clothing. Customers indicate whether they Each of the columns of Table I is the subject of one of the are male or female, then view three categories- Music, Looks Fun-and rate a minimum of 4"terms"or"sub-categories the table, and their role in supporting recommender systems for within each. They do this by providing a rating on a 7-point cale ranging trom leave it to "love it. They may also choose 3. Recommendation Interfaces and Ways to are entered customers may select "get recommendations."Here, Make Money they are provided with thumbnails of 6 items of recommended An old proverb states that there is "more than one way to skin a clothing. Customers may provide feedback by use of the"tell us cat. One would assume that the method selected depends on the what you think feature"which allows them to enter an opinion desired outcome. Similarly, there is more than one way to rating for the recommended article of clothing. Feedback may display recommendations to a customer. The method selected change one or all of the six items recommended may well depend on how the e-commerce site wants the customer to use the recommendation. In the following we will examine 2.5 Moviefinder. com seven recommendation interfaces and how each assists the site Match Maker: Moviefinder coms Match in making money. While some of these methods have their roots (www.moviefinder.com)allowscustomerstolocatemovies in traditional commerce, each of them draws upon the strengths a similar"mood, theme, genre or cast" to a given movie. From of the electronic medium to provide more powerful the information page of the movie in question, customers click on recommendations the Match Maker icon and are provided with the list of Browsing: In traditional commerce a customer might walk into a recommended movies, as well as links to other films by the video store and ask the clerk to recommend "a comedy from the original films director and key actors 50s. Ideally, the clerk would recommend several movies, and We Predict: We Predict recommends movies to customers the customer could go off to locate the recommended movies based on their previously indicated interests. Customers enter a browse the box covers, and see which ones appealed to them. rating on a 5-point scale- from a to F- for movies they have However, the quality of the recommendations provided was viewed. These ratings are used in two different ways. Most dependent on the particular clerk's knowledge of an enormous simply, as they continue, the information page for non-rated range of movies. Reel. com has several advantages when movies contains a personalized textual prediction(go see it implementing browsing into their Movie Map feature. First, the forget it). In a variation of this, customers can use Powerfind to recommendations of several clerks/editors can be combined so search for top picks based on syntactic criteria such as Genre, that higher quality recommendations can be provided no matter directors, or actors and choose to have these sorted by their what the query parameters. Furthermore, recommendations are personalized prediction or by the all customer average returmed with immediate links to the items being recommended no more searching the store for the obscure videos recommended 2. 6 Reel. com Recommended browsing helps the E-commerce lovie Matches: Similar to Amazon. com's Custe BoughtReel.comsMovieMatches(www.reel.com)providesnarrowdowntheirchoicesandfeelmoreconfident recommendations on the information page for each movie. These decision to buy by providing organized access to the recommendations consist of"close matches"and/or "creative recommendations matches."Each set consists of up to a dozen hyperlinks to the Similar Item: Another modification of traditional commerce information pages for each of these"matched"films. The hyperlinks are annotated with one sentence descriptions of how techniques is the similar item recommendation. Systems such as the new movie is similar to the original movie in question Reel coms Movie Matcher. Amazon coms Customer's whe ("Darker thriller raises similarly disturbing questions. " ) Bought and one variation of CDNOW's Album Advisor attempt to expose customers to items they may have forgotten about, or of Movie Map: The Movie Map feature of Reel. com recommends which they may have simply been unaware to customers based on syntactic features. Customers implementation in E-commerce sites allows for more specific and queries based on Genre, movie types, viewing format personalized recommendations. The items displayed can be prices, and request results be constrained to"sleepers"or entirely selected based on the item(s)in which a customer has best of this genre. The recommendations s are editor's shown interest. In doing so, sites increase customer's exposure recommendations for movies that fit the specified criteria. to their product line, and ideally are able to sell more items per 2.7 Summary Email: Recommendations can also be delivered directly to recommendation hnology, and how users find customers through email, in a extension of traditional direct mail recommendations for all of the example applications. The first Amazon coms Eyes feature allows them to notit column just names each application, under the E-commerce site customers the minute an item becomes commercially available that houses it The second column describes the interface that Eyes enables Amazon. com to attract customers into their store used for delivering the recommendations. The third column before other stores with the same product can reach those

2.4 Levis Style Finder: Style Finder allows customers of the Levi Straus™ (www.levis.com) website to receive recommendations on articles of Levi’s clothing. Customers indicate whether they are male or female, then view three categories -- Music, Looks, Fun -- and rate a minimum of 4 “terms” or “sub-categories” within each. They do this by providing a rating on a 7-point scale ranging from “leave it” to “love it.” They may also choose the rating of “no opinion.” Once the minimum number of ratings are entered customers may select “get recommendations.” Here, they are provided with thumbnails of 6 items of recommended clothing. Customers may provide feedback by use of the “tell us what you think feature” which allows them to enter an opinion rating for the recommended article of clothing. Feedback may change one or all of the six items recommended. 2.5 Moviefinder.com Match Maker: Moviefinder.com’s Match Maker (www.moviefinder.com) allows customers to locate movies with a similar “mood, theme, genre or cast” to a given movie. From the information page of the movie in question, customers click on the Match Maker icon and are provided with the list of recommended movies, as well as links to other films by the original film’s director and key actors. We Predict: We Predict recommends movies to customers based on their previously indicated interests. Customers enter a rating on a 5-point scale -- from A to F – for movies they have viewed. These ratings are used in two different ways. Most simply, as they continue, the information page for non-rated movies contains a personalized textual prediction (go see it – forget it). In a variation of this, customers can use Powerfind to search for top picks based on syntactic criteria such as Genre, directors, or actors and choose to have these sorted by their personalized prediction or by the all customer average. 2.6 Reel.com Movie Matches: Similar to Amazon.com’s Customers who Bought, Reel.com’s Movie Matches (www.reel.com) provides recommendations on the information page for each movie. These recommendations consist of “close matches” and/or “creative matches.” Each set consists of up to a dozen hyperlinks to the information pages for each of these “matched” films. The hyperlinks are annotated with one sentence descriptions of how the new movie is similar to the original movie in question (“Darker thriller raises similarly disturbing questions… ”). Movie Map: The Movie Map feature of Reel.com recommends movies to customers based on syntactic features. Customers enter queries based on Genre, movie types, viewing format and/or prices, and request results be constrained to “sleepers” or “best of this genre.” The recommendations are editor’s recommendations for movies that fit the specified criteria. 2.7 Summary In Table 1 we have summarized the applications, interfaces, recommendation technology, and how users find recommendations for all of the example applications. The first column just names each application, under the E-commerce site that houses it. The second column describes the interface that is used for delivering the recommendations. The third column describes the recommendation technology used by the site, and the inputs required by that technology. The fourth column describes how users find recommendations using the application. Each of the columns of Table 1 is the subject of one of the sections of this paper, describing the meaning of the entries in the table, and their role in supporting recommender systems for E-commerce. 3. Recommendation Interfaces and Ways to Make Money An old proverb states that there is “more than one way to skin a cat.” One would assume that the method selected depends on the desired outcome. Similarly, there is more than one way to display recommendations to a customer. The method selected may well depend on how the e-commerce site wants the customer to use the recommendation. In the following we will examine seven recommendation interfaces, and how each assists the site in making money. While some of these methods have their roots in traditional commerce, each of them draws upon the strengths of the electronic medium to provide more powerful recommendations. Browsing: In traditional commerce a customer might walk into a video store and ask the clerk to recommend “a comedy from the 50s.” Ideally, the clerk would recommend several movies, and the customer could go off to locate the recommended movies, browse the box covers, and see which ones appealed to them. However, the quality of the recommendations provided was dependent on the particular clerk’s knowledge of an enormous range of movies. Reel.com has several advantages when implementing browsing into their Movie Map feature. First, the recommendations of several clerks/editors can be combined so that higher quality recommendations can be provided no matter what the query parameters. Furthermore, recommendations are returned with immediate links to the items being recommended – no more searching the store for the obscure videos recommended. Recommended browsing helps the E-commerce site by converting browsers into buyers. It does so by helping the users narrow down their choices and feel more confident in their decision to buy by providing organized access to the recommendations. Similar Item: Another modification of traditional commerce techniques is the similar item recommendation. Systems such as Reel.com’s Movie Matcher, Amazon.com’s Customer’s who Bought and one variation of CDNOW’s Album Advisor attempt to expose customers to items they may have forgotten about, or of which they may have simply been unaware. Their implementation in E-commerce sites allows for more specific and personalized recommendations. The items displayed can be entirely selected based on the item(s) in which a customer has shown interest. In doing so, sites increase customer’s exposure to their product line, and ideally are able to sell more items per order. Email: Recommendations can also be delivered directly to customers through email, in a extension of traditional direct mail techniques. Amazon.com’s Eyes feature allows them to notify customers the minute an item becomes commercially available. Eyes enables Amazon.com to attract customers into their store before other stores with the same product can reach those

hers. Furthermore, both Eyes and Amazon. com Delivers site in making money by increasing both loyalty, and the number the site to keep a customer aware of the site and of items of re the customer may have missed. Customers appreciate the email recommendations because they help them watch out for new items they are interested in purchasing. These features assist the Business/Applications Recommendation Recommendation Finding Customers who Bought Similar Item Item to Item Correlation Organic Navigation Purchase data ttribute based KEywords/freeform azon. com delivers Attribute Based Selection options Book matcher Top N List People to People Correlation Request List Customer Comments Average Rating Aggregated Rati OrganIc Navigation Text Comments CDNOW Album advise Similar ltem Item to Item Correlation Organic Navigation Top N List Purchase data Keywords/freeform My CDNoW Top N List People to People Correlation Organic Navigation Request list ay Average Rating Aggregated Rating Organic Navigation Text Comments Likert Text Levis tyle Finder Top N list People to People Correlation Request list Moviefinder. com Match Maker Similar item Item to Item Correlation Navigate to an item Editor's choice We predict Top n list people to People Correlatic Keywords/freeform Ordered Search Results Selection options Average Rat Likert anic na REels Movie matches Similar Item Item to Item Correlation Organic Navigation Editor's choice Movie Map Attribute based Editor 's choice customers with recommendations based directly on the text street" is the average rating feature. Rather than comments of other customers. Amazon coms Customer customers to browse a list of text based opinions Comments and eBays Feedback Profile streamlines the customers can provide numerical ranking opinions. B gathering of"the word on the street" by allowing customers to aggregating these rankings into an average rating, Customer locate an item of interest and browse the comments of other Comments and Feedback Profile both provide users with a"one customers. This helps sites make money by providing impartial stop"check on the quality of an item. Similar to text comments, information on the goods/services being sold - the thought being average ratings should facilitate in converting browsers into if enough people claim that a book is good or a seller is credible buyers, and increasing customer loyalty to the an it is likely to be true. This not only helps convert browsers Top-N: Amazon. com's Book Matcher, Levi's Style Finder and into buyers, but should oyalty to a site. If customers My CdNOW, among others, take advantage of recommendations learn they can trust these third party recommendations, than they through a top-N list. Once each site has learned details about a are more likely to return the next time they are faced with a questionable decision. customer's likes and dislikes, each is able to provide the

customers.. Furthermore, both Eyes and Amazon.com Delivers allows the site to keep a customer aware of the site and of items the customer may have missed. Customers appreciate the email recommendations because they help them watch out for new items they are interested in purchasing. These features assist the site in making money by increasing both loyalty, and the number of return visits. Business/Applications Recommendation Interface Recommendation Technology Finding Recommendations Amazon.com Customers who Bought Similar Item Item to Item Correlation Purchase data Organic Navigation Eyes Email Attribute Based Keywords/freeform Amazon.com Delivers Email Attribute Based Selection options Book Matcher Top N List People to People Correlation Likert Request List Customer Comments Average Rating Text Comments Aggregated Rating Likert Text Organic Navigation CDNOW Album Advisor Similar Item Top N List Item to Item Correlation Purchase data Organic Navigation Keywords/freeform My CDNOW Top N List People to People Correlation Likert Organic Navigation Request List eBay Feedback Profile Average Rating Text Comments Aggregated Rating Likert Text Organic Navigation Levis Style Finder Top N List People to People Correlation Likert Request List Moviefinder.com Match Maker Similar Item Item to Item Correlation Editor’s choice Navigate to an item We Predict Top N List Ordered Search Results Average Rating People to People Correlation Aggregated Rating Likert Keywords/freeform Selection options Organic Navigation Reel.com Movie Matches Similar Item Item to Item Correlation Editor’s choice Organic Navigation Movie Map Browsing Attribute Based Editor’s choice Keywords/freeform Table 1: Recommender System Examples Text Comments: More and more frequently, sites are providing customers with recommendations based directly on the text comments of other customers. Amazon.com’s Customer Comments and eBay’s Feedback Profile streamlines the gathering of “the word on the street” by allowing customers to locate an item of interest and browse the comments of other customers. This helps sites make money by providing impartial information on the goods/services being sold – the thought being, if enough people claim that a book is good, or a seller is credible, than it is likely to be true. This not only helps convert browsers into buyers, but should increase loyalty to a site. If customers learn they can trust these third party recommendations, than they are more likely to return the next time they are faced with a questionable decision.. Average Rating: Even simpler access to “the word on the street” is the average rating feature. Rather than asking customers to browse a list of text based opinions, other customers can provide numerical ranking opinions. By aggregating these rankings into an average rating, Customer Comments and Feedback Profile both provide users with a “one stop” check on the quality of an item. Similar to text comments, average ratings should facilitate in converting browsers into buyers, and increasing customer loyalty to the site. Top-N: Amazon.com’s Book Matcher, Levi’s Style Finder and My CDNOW, among others, take advantage of recommendations through a top-N list. Once each site has learned details about a customer’s likes and dislikes, each is able to provide the

customer with recommendation is generated without explicit effort by items for that customer. It is as though one could gather all of the customer. The customer just interacts with the site as he or clothes that might interest a given client onto a single rack wishes, and suddenly a recommendation appears that without distracting them with items they will not be interested appropriate for the customer's interests. Manual means that the in. This helps the vendor in several ways. First, it is another customer takes explicit effort to seek out recommendations that conv ing browsers into buyers it provides will fit her interests. Note that recommendations that increased exposure to the vendors wares, but only to those items Manual from the perspective of the user may be generated by the that should truly interest the user. Second, it may help the site using a computer program. We consider these Manual, since customer in making a decision about items that they originally we are taking the customers perspective. Likewise held in doubt -the suggestion from the site may be another point recommendations that appear automatically for the customer, bu in favor of the item that are generated by hand by the site are considered Automatic Ordered Search Results: Finally, a less restrictive variation of Whether the site uses a computer or a human to implement its the top-N list are Ordered Search Results recommendations recommendation algorithms is unimportant to the customer. While top-N limits the predictions to some predefined number, The persistence axis ranges from completely Ephemeral ordered search results allow the customer to continue to look at recommendations to Persistent recommendations. Ephemeral We Predict feature allows customers to have query returns sorted recommendations are made based entirely on a single customer ession, and are not based on any information from previous by the predicted likelihood that the customer will enjoy the item. sessions of this customer. Persistent recommendations are based Once again, this helps convert browsers into buyers on the site recognizing the customer, and suggesting products to the customer based on the customer's likes and dislikes in persistent previous sessions. This section is structured at the high level around the four recommendation techniques: non-personalized, attribute based Style Finder item-to-item correlations, and people-to-people correlations. For each technique, we briefly introduce it, explain its place in the taxonomy, and give examples of it from our recommender system 4.1 Non-Personalized recommendations Manua Non-personalized recommender systems recommend products to customers based on what other customers have said about the products on average. The recommendations are independent of the customer, so each customer gets the same recommendations Non-personalized recommender systems are Automatic, because ittle customer effort t not recognize the customer from one session to the next since the recommendations are not based on the customer. Non- Ephemer ovie Finder. Amazon personalized recommender systems are common in physical Figure 1: Recommendation Taxonomy without change by every customer on a display that is viewed stores, since they can be set up 4. A Taxonomy for Mapping Applications to For instance, the average customer ratings displayed by Amazon.comandMoviefinder.comarenon-personalized Recommendation Techniques recommendations. These recommendations are completely In this section we describe the Recommendation Technology independent of the particular customer targeted by the column of Table I in detail. We first lay out the differer recommender system. eBay has a slightly different form of non recommender system applications in a taxonomy of personalized recommendation in its feedback profile. Customers recommendation types. We then describe the different user give feedback on each other, rather than on products! The inputs, which are the italicized entries in the table. The goal of average and individual feedback is then available for the taxonomy is to present a completely user-focused analysis of consideration by buyers to decide whether a particular seller is a the different recommender systems, so the taxonomy is based on good risk, and by sellers to decide whether a particular buyer is a the features most important to customers of the E-commerce good risk. All three of these systems are nearly completely on sites. The two key dimensions in the taxonomy are the degree of the Automatic and Ephemeral end of the axes. Another type of automation, and the degree of persistence in the non-personalized recommendation is the text comments recommendations( Figure 1) supported in Amazons Customer Comments and eBays The automation axis ranges from completely Automatic Feedback Profile. Both of these are still Ephemeral, but move recommendations to completely Manual recommendations From closer to the Manual end of the other axis. In a sense, the system the perspective of the customer, Automatic means that the is merely providing raw data to the user, who must then collate

customer with a personalized list of the top number of unrated items for that customer. It is as though one could gather all of the clothes that might interest a given client onto a single rack without distracting them with items they will not be interested in. This helps the vendor in several ways. First, it is another example of converting browsers into buyers – it provides increased exposure to the vendor’s wares, but only to those items that should truly interest the user. Second, it may help the customer in making a decision about items that they originally held in doubt – the suggestion from the site may be another point in favor of the item. Ordered Search Results: Finally, a less restrictive variation of the top-N list are Ordered Search Results recommendations. While top-N limits the predictions to some predefined number, ordered search results allow the customer to continue to look at items highly likely to be of interest to them. Moviefinder.com’s We Predict feature allows customers to have query returns sorted by the predicted likelihood that the customer will enjoy the item. Once again, this helps convert browsers into buyers. Amazom.com Delivers My CDNOW eBay Movie Map Album Advisor Match Maker Movie Match Customers who Bought Book Match We Predict Style Finder Average Movie Finder, Amazon Persistent Ephemeral Manual Auto Figure 1: Recommendation Taxonomy 4. A Taxonomy for Mapping Applications to Recommendation Techniques In this section we describe the Recommendation Technology column of Table 1 in detail. We first lay out the different recommender system applications in a taxonomy of recommendation types. We then describe the different user inputs, which are the italicized entries in the table. The goal of the taxonomy is to present a completely user-focused analysis of the different recommender systems, so the taxonomy is based on the features most important to customers of the E-commerce sites. The two key dimensions in the taxonomy are the degree of automation, and the degree of persistence in the recommendations (Figure 1). The automation axis ranges from completely Automatic recommendations to completely Manual recommendations. From the perspective of the customer, Automatic means that the recommendation is generated without explicit effort by the customer. The customer just interacts with the site as he or she wishes, and suddenly a recommendation appears that is appropriate for the customer’s interests. Manual means that the customer takes explicit effort to seek out recommendations that will fit her interests. Note that recommendations that are Manual from the perspective of the user may be generated by the site using a computer program. We consider these Manual, since we are taking the customer’s perspective. Likewise, recommendations that appear automatically for the customer, but that are generated by hand by the site are considered Automatic. Whether the site uses a computer or a human to implement its recommendation algorithms is unimportant to the customer. The persistence axis ranges from completely Ephemeral recommendations to Persistent recommendations. Ephemeral recommendations are made based entirely on a single customer session, and are not based on any information from previous sessions of this customer. Persistent recommendations are based on the site recognizing the customer, and suggesting products to the customer based on the customer’s likes and dislikes in previous sessions. This section is structured at the high level around the four recommendation techniques: non-personalized, attribute based, item-to-item correlations, and people-to-people correlations. For each technique, we briefly introduce it, explain its place in the taxonomy, and give examples of it from our recommender system examples. 4.1 Non-Personalized Recommendations Non-personalized recommender systems recommend products to customers based on what other customers have said about the products on average. The recommendations are independent of the customer, so each customer gets the same recommendations. Non-personalized recommender systems are Automatic, because they require little customer effort to generate the recommendation, and are Ephemeral, because the system does not recognize the customer from one session to the next since the recommendations are not based on the customer. Non￾personalized recommender systems are common in physical stores, since they can be set up on a display that is viewed without change by every customer. For instance, the average customer ratings displayed by Amazon.com and Moviefinder.com are non-personalized recommendations. These recommendations are completely independent of the particular customer targeted by the recommender system. eBay has a slightly different form of non￾personalized recommendation in its feedback profile. Customers give feedback on each other, rather than on products! The average and individual feedback is then available for consideration by buyers to decide whether a particular seller is a good risk, and by sellers to decide whether a particular buyer is a good risk. All three of these systems are nearly completely on the Automatic and Ephemeral end of the axes. Another type of non-personalized recommendation is the text comments supported in Amazon’s Customer Comments and eBay’s Feedback Profile. Both of these are still Ephemeral, but move closer to the Manual end of the other axis. In a sense, the system is merely providing raw data to the user, who must then collate the data and make sense out of it manually

4.2 Attribute- Based recommendations information items to individuals(Resnick et al. 1994, Hill et al Attribute based recommender systems recommend products to 1995, Shardanand Maes 1995, Konstan et al. 1997). Though customers based on syntactic properties of the products. For we use the word correlation in the name of this technique, instance. if the customer does a search for a historical romance hinting at nearest-neighbor techniques based on linear book, and the E-commerce site responds with a list of three correlation, the technique can be implemented with many other recommended books, that is an example of an attribute-based technologies as well Breese et al. 1998). Since we are focused ecommendation. Attribute-based recommendations are ofter on the effect of the technique on users, we differentiate according Manual, since the customer must directly request the to user experience rather than implementation details. People-to- recommendation by entering his desired syntactic product people correlation recommender systems are close to Automatic, properties. Attribute-based recommendations can be either since the recommendations themselves are generated Ephemeral or Personal, depending on whether the E-commerce automatically by the system. The system does have to learn over site remembers the attribute preferences for a customer. time from customers. In some systems this is done by having Reel coms Movie Map is an example of a attribute-based customers explicitly rate products, in which case the system is moved part of the way towards Manual. In other systems, the recommendation. The recommendations are entirely based on learning is implicit from the buying patterns or click-stream ehavior of the users, in which case the system is pure must explicitly go to Movie Map and navigate to a category to Automatic. These systems are most often Persistent, since obtain a recommendation, Movie Map is Manual. Since Movie Map does not remember a customer's interest from one visit to learning about patterns of agreement between users requires the next, it is Ephemeral. Amazon. com Delivers is also Manual substantial data which is most easily collected over time. In since customers must explicitly sign up and provide a set of principle, such a system could be Ephemeral if user sessions are long enough Interest categorie However. Amazon. com Delivers is Amazon coms Book Matcher, Moviefinder's We predict, and recommendations in selected categories until the customer turns Style Finder are all examples of Persistent but not qui off the request. Automatic people-to-people correlation recommender systems Users explicitly rate products and other products are 4.3 Item-to-Item Correlation recommended based on the ratings. Since the ratings are entered Item-to-item correlation recommender systems recommend only to get the recommendations, these systems are not products to customers based on a small set of products the considered fully Automatic customers have expressed interest in. For instance, if a customer My CDNOW is a fully Automatic example of this technique has placed a few products in her shopping basket, the since customer opinions are inferred by the actions a customer recommender system may recommend complementary products takes in setting up his personal music site within the CDNOW E- commerce site. Recommendations are provided organically systems can be Automatic, if they are based on observations of within the context of the personal music site he customers unchanged behavior. They can also require some Manual effort, if the customer must explicitly type in several 4.5 User Inputs items of interest in order to generate a recommendation. Item-to- Each of the previous four recommendation technologies requires item correlation recommender systems are usually Epher some form of input upon which to base the recommendations since they do not need to know any history about the customer to Typically this input is provided by the customer(s).However, it generate a recommendation based on the products she has is possible that the input may also be provided by the business as selected well. The systems in our examples utilize one or more of the Reel coms Movie Matches. Moviefinder's Match Maker. and following inputs Amazon coms Customers who Bought are similar from the Purchase data: Which products a customer has purchased perspective of customer experience. All three suggest other Systems such as Amazon coms Customers who bought and My products a customer might be interested in based single CDNOW make recommendations based entirely patterns of"co- other product that customer has expressed interest in. These purchase "between multiple customers. In principle, this may be systems are Automatic and Ephemeral, since they require neither augmented with how many of each product the customer has action from nor identification of the customer. CDNOW's purchased Album Advisor is different, since it is triggered by the user asking for recommendations by typing in a set of artists. This Likert: What a customer says he thinks of a product, typically on application is still Ephemeral, but is closer to Manual because it 1-5 or 1-7 scale. The scale may be numeric or textual, but requires some customer effort be totally ordered. Systems such as e Bays Feedback Profile and Levi's Style Finder utilize Likert inputs 4.4 People-to-People correlation Text: Written comments intended for other customers to read systems recon Usually not interpreted by the computer system. Currently roducts to a customer based on the correlation between that ncluded in systems such as Amazon coms Customer Comments. customer and other customers who have purchased products from the E-commerce site. This technology is often called Editors choice: Selections within a category made by humai collaborative filtering, because it originated as an information editors, usually employed by the E-commerce site, though fil that used group opinions to independent editors are possible in principle. Editors choice

4.2 Attribute-Based Recommendations Attribute based recommender systems recommend products to customers based on syntactic properties of the products. For instance, if the customer does a search for a historical romance book, and the E-commerce site responds with a list of three recommended books, that is an example of an attribute-based recommendation. Attribute-based recommendations are often Manual, since the customer must directly request the recommendation by entering his desired syntactic product properties. Attribute-based recommendations can be either Ephemeral or Personal, depending on whether the E-commerce site remembers the attribute preferences for a customer. Reel.com’s Movie Map is an example of a attribute-based recommendation. The recommendations are entirely based on the category of movie the customer selects. Since customers must explicitly go to Movie Map and navigate to a category to obtain a recommendation, Movie Map is Manual. Since Movie Map does not remember a customer’s interest from one visit to the next, it is Ephemeral. Amazon.com Delivers is also Manual, since customers must explicitly sign up and provide a set of interest categories. However, Amazon.com Delivers is Persistent, since Amazon.com continues to send out recommendations in selected categories until the customer turns off the request. 4.3 Item-to-Item Correlation Item-to-item correlation recommender systems recommend products to customers based on a small set of products the customers have expressed interest in. For instance, if a customer has placed a few products in her shopping basket, the recommender system may recommend complementary products to increase the order size. Item-to-item correlation recommender systems can be Automatic, if they are based on observations of the customer’s unchanged behavior. They can also require some Manual effort, if the customer must explicitly type in several items of interest in order to generate a recommendation. Item-to￾item correlation recommender systems are usually Ephemeral, since they do not need to know any history about the customer to generate a recommendation based on the products she has selected. Reel.com’s Movie Matches, Moviefinder’s Match Maker, and Amazon.com’s Customers who Bought are similar from the perspective of customer experience. All three suggest other products a customer might be interested in based on a single other product that customer has expressed interest in. These systems are Automatic and Ephemeral, since they require neither action from nor identification of the customer. CDNOW’s Album Advisor is different, since it is triggered by the user asking for recommendations by typing in a set of artists. This application is still Ephemeral, but is closer to Manual because it requires some customer effort. 4.4 People-to-People Correlation People-to-people correlation recommender systems recommend products to a customer based on the correlation between that customer and other customers who have purchased products from the E-commerce site. This technology is often called “collaborative filtering”, because it originated as an information filtering technique that used group opinions to recommend information items to individuals (Resnick et al. 1994, Hill et al. 1995, Shardanand & Maes 1995, Konstan et al. 1997). Though we use the word correlation in the name of this technique, hinting at nearest-neighbor techniques based on linear correlation, the technique can be implemented with many other technologies as well (Breese et al. 1998). Since we are focused on the effect of the technique on users, we differentiate according to user experience rather than implementation details. People-to￾people correlation recommender systems are close to Automatic, since the recommendations themselves are generated automatically by the system. The system does have to learn over time from customers. In some systems this is done by having customers explicitly rate products, in which case the system is moved part of the way towards Manual. In other systems, the learning is implicit from the buying patterns or click-stream behavior of the users, in which case the system is pure Automatic. These systems are most often Persistent, since learning about patterns of agreement between users requires substantial data which is most easily collected over time. In principle, such a system could be Ephemeral if user sessions are long enough. Amazon.com’s Book Matcher, Moviefinder’s We Predict, and Style Finder are all examples of Persistent but not quite Automatic people-to-people correlation recommender systems. Users explicitly rate products and other products are recommended based on the ratings. Since the ratings are entered only to get the recommendations, these systems are not considered fully Automatic. My CDNOW is a fully Automatic example of this technique, since customer opinions are inferred by the actions a customer takes in setting up his personal music site within the CDNOW E￾commerce site. Recommendations are provided organically within the context of the personal music site. 4.5 User Inputs Each of the previous four recommendation technologies requires some form of input upon which to base the recommendations. Typically this input is provided by the customer(s). However, it is possible that the input may also be provided by the business as well. The systems in our examples utilize one or more of the following inputs. Purchase data: Which products a customer has purchased. Systems such as Amazon.com’s Customers who Bought and My CDNOW make recommendations based entirely patterns of “co￾purchase” between multiple customers. In principle, this may be augmented with how many of each product the customer has purchased. Likert: What a customer says he thinks of a product, typically on a 1-5 or 1-7 scale. The scale may be numeric or textual, but must be totally ordered. Systems such as eBay’s Feedback Profile and Levi’s Style Finder utilize Likert inputs. Text: Written comments intended for other customers to read. Usually not interpreted by the computer system. Currently included in systems such as Amazon.com’s Customer Comments. Editor’s choice: Selections within a category made by human editors, usually employed by the E-commerce site, though independent editors are possible in principle. Editor’s choice is

portant in both Reel. co Movie Matches/Map ar systems in E-commerce sites. These range from simple Moviefinder com's match Ma variations on existing systems, to entirely new types of systems 5. Finding recommendations As discussed above, many sites currently use purchase data as an Just as sites can utilize different methods for calculating implicit, positive rating. CDNOW has realized in My CDNow displaying recommendations, so can they utilize different that owning something cannot always be interpreted as a methods for allowing customers to access the recommendations positive. Recall that CDNOW allows customers to later go back Through our recommender system examples we have identified and indicate "own it but dislike it". However. few sites are four different methods for ons ea attempting to extract implicit negative ratings from purchase interface and/or technology. These four methods are orderdi which may provide access to more than one recommendatio data. One way to do this would be through the analysis of data on returned products. While customers may return an item for a the amount of customer effort required to find the variety of reasons, in general any return could be considered as a negative rating on the item in question. Another model of mplicit negative rating can be derived from detail views. If the Organic Navigation: Requiring the least amount of work to site presents a few products in low detail and the customer actually access recommendations is the organic navigation chooses to view some products at higher detail, but ignores process. In applications such as Album Advisor, Movie Matches, others, a mild negative rating can be inferred for the unselected and Feedback Profile, customers do nothing extra in order to items. Many recommender system algorithms perform better receive recommendations. In each of these applications with both negative and positive ratings, so the negative data can recommendations appear as part of the item information page. be These recommendations can consist of additional items to consider, average ratings, or a list of other customer comments Another creative use of a recommender system would be to use it normal navigation of the site, customers are provided wIth.% However, the underlying similarity is that through the course in reverse to explain to a user what type of thing a product is For instance, a recommender system might be used to tell the user this product you're looking at is similar to these other products that you have liked in the past". Recommender system Request Recommendation List: Requiring not much more algorithms that correlate items can be used in this way. For best work from the customer is the request recommendation list results they should be modified to return items that the user has process. Customers using applications such as Book Matcher purchased in the past, rather than the usual set of items the user and Style Finder can access recommendations based on their has not purchased in the past previously recorded likes/dislikes. To do so, they simply have to equest these recommendations from the system Current recommender systems only use a small subset of th available information about the customer in making their Selection Options: In the selection options process customers recommendations. Some systems use demographic information, must truly interact with the system in order to receive some use purchase data information, some use explicit ratings, recommendations. Typically, customers choose from a set of some use ownership data, but no system effectively uses all thi redefined criterion/options up which to base their data simultaneously for real-time recommendations. How should recommendations. For example, users of Amazon. com Delivers these diverse types of data be combined? Should individual ave a choice from nearly 50 pre-defined categories in which to recommender systems running on each type of data produce receive periodic recommendations. Even more involved, users of independent recommendations? Or can better recommendations Moviefinder coms We Predict system can select from a finite be produced by using all of the available data simultaneousl list of title, format, length and genre options to define a search, Recommender system algorithms that use many different types of s well as customizing options such as ranking method and display features data create the possibility for"subtle personalization, in whicl the site provides a completely organic personalized experience to Keyword/Freeform: Arguably, the keyword freeform the customer. The customer interacts with the site just as she requires the most interaction from the customer. In applica would have before personalization. She does not need to take any ch as Eyes, customers provide a set of textual keywords upon explicit actions to inform the site of her interests or desires. The which to retrieve future recommendations. A version of Album site subtly changes the interface in nearly invisible ways to create Advisor takes the freeform input of multiple artists upon which a more personal experience for her, without her even noticing make recommendation matches. The We Predict and Movie that anything has changed!(Balabanovic Shoham, 1997) Map applications produce recommendations from the results of a(Basu, Hirsh, Cohen, 1998)(Sarwar et al, 1998) query conducted using the keywords provided. While each uses Recommender systems are currently used as virtual salespeople, the keywords in very different manners, each requires the user to rather than as marketing tools. The difference is that many know specifically what types of things they are interested in. recommender systems target each individual customer 6. E-Commerce Opportunities differently, making it difficult to produce the reports that have already explored multiple interfaces, technologies, ir Many varieties of recommender systems are already in use. We marketing professionals are used to. These reports usual partition the population into a manageable number of segments information needs for these types of systems. However, there One way to bring these two worlds together would be to use th remain many opportunities for the expansion of recommender people to people correlations used by some recommender system algorithms to create segments for the reports. Open questions

important in both Reel.com’s Movie Matches/Map and Moviefinder.com’s Match Maker. 5. Finding Recommendations Just as sites can utilize different methods for calculating or displaying recommendations, so can they utilize different methods for allowing customers to access the recommendations. Through our recommender system examples we have identified four different methods for finding recommendations each of which may provide access to more than one recommendation interface and/or technology. These four methods are ordered in the amount of customer effort required to find the recommendations. Organic Navigation: Requiring the least amount of work to actually access recommendations is the organic navigation process. In applications such as Album Advisor, Movie Matches, and Feedback Profile, customers do nothing extra in order to receive recommendations. In each of these applications, recommendations appear as part of the item information page. These recommendations can consist of additional items to consider, average ratings, or a list of other customer comments. However, the underlying similarity is that through the course of normal navigation of the site, customers are provided with a recommendations. Request Recommendation List: Requiring not much more work from the customer is the request recommendation list process. Customers using applications such as Book Matcher and Style Finder can access recommendations based on their previously recorded likes/dislikes. To do so, they simply have to request these recommendations from the system. Selection Options: In the selection options process customers must truly interact with the system in order to receive recommendations. Typically, customers choose from a set of predefined criterion/options upon which to base their recommendations. For example, users of Amazon.com Delivers have a choice from nearly 50 pre-defined categories in which to receive periodic recommendations. Even more involved, users of Moviefinder.com’s We Predict system can select from a finite list of title, format, length and genre options to define a search, as well as customizing options such as ranking method and display features. Keyword/Freeform: Arguably, the keyword/freeform option requires the most interaction from the customer. In applications such as Eyes, customers provide a set of textual keywords upon which to retrieve future recommendations. A version of Album Advisor takes the freeform input of multiple artists upon which to make recommendation matches. The We Predict and Movie Map applications produce recommendations from the results of a query conducted using the keywords provided. While each uses the keywords in very different manners, each requires the user to know specifically what types of things they are interested in. 6. E-Commerce Opportunities Many varieties of recommender systems are already in use. We have already explored multiple interfaces, technologies, and information needs for these types of systems. However, there remain many opportunities for the expansion of recommender systems in E-commerce sites. These range from simple variations on existing systems, to entirely new types of systems. As discussed above, many sites currently use purchase data as an implicit, positive rating. CDNOW has realized in My CDNOW that owning something cannot always be interpreted as a positive. Recall that CDNOW allows customers to later go back and indicate “own it but dislike it”. However, few sites are attempting to extract implicit negative ratings from purchase data. One way to do this would be through the analysis of data on returned products. While customers may return an item for a variety of reasons, in general any return could be considered as a negative rating on the item in question. Another model of implicit negative rating can be derived from detail views. If the site presents a few products in low detail and the customer chooses to view some products at higher detail, but ignores others, a mild negative rating can be inferred for the unselected items. Many recommender system algorithms perform better with both negative and positive ratings, so the negative data can be valuable. Another creative use of a recommender system would be to use it in reverse to explain to a user what type of thing a product is. For instance, a recommender system might be used to tell the user “this product you’re looking at is similar to these other products that you have liked in the past”. Recommender system algorithms that correlate items can be used in this way. For best results they should be modified to return items that the user has purchased in the past, rather than the usual set of items the user has not purchased in the past. Current recommender systems only use a small subset of the available information about the customer in making their recommendations. Some systems use demographic information, some use purchase data information, some use explicit ratings, some use ownership data, but no system effectively uses all this data simultaneously for real-time recommendations. How should these diverse types of data be combined? Should individual recommender systems running on each type of data produce independent recommendations? Or can better recommendations be produced by using all of the available data simultaneously? Recommender system algorithms that use many different types of data create the possibility for “subtle personalization”, in which the site provides a completely organic personalized experience to the customer. The customer interacts with the site just as she would have before personalization. She does not need to take any explicit actions to inform the site of her interests or desires. The site subtly changes the interface in nearly invisible ways to create a more personal experience for her, without her even noticing that anything has changed! (Balabanovic & Shoham, 1997) (Basu, Hirsh, & Cohen, 1998) (Sarwar et al, 1998) Recommender systems are currently used as virtual salespeople, rather than as marketing tools. The difference is that many recommender systems target each individual customer differently, making it difficult to produce the reports that marketing professionals are used to. These reports usually partition the population into a manageable number of segments. One way to bring these two worlds together would be to use the people to people correlations used by some recommender system algorithms to create segments for the reports. Open questions include “how can names be assigned to the automatically

generated segments?"and"are automatically generated segments."Provide point of delivery customization more useful for managing marketing campaigns than traditional recommender system directly customizes the point of delivery for the E-commerce site Recommender systems can be made more useful as marketing. "Provide quick response throughout the value chain":We ystems in other ways, too. Current recommender systems are predict that recommender systems will be used in the future ainly"buy-side systems. That is, they are designed to work o predict demand for products, enabling earlier on behalf of the customer in deciding what products they should communication back the supply chair urchase. However, modern marketing is designed not just to maximize utility to the customer, but to maximize value to the Recommender systems are a key way to automate mass business at the same time. The recommender system could customization for E-commerce sites. They will become produce an indication of the price sensitivity of the customer fo increasingly important in the future, as modern businesses are a given product, so the E-commerce site could offer each product business(Peppers Rogers 1997). E-commerce sites will be the site. For instance, one customer might be willing to purchase working hard to maximize the value of the customer to their site the product at a price that would earn the site ten cents of profit providing exactly the pricing and service they judge will create the most valuable relationship with the customer. Since customer while another customer might purchase the same product at a one retention will be very important to the sites, this relationship will dollar profit. There are challenging ethical issue in implementing often be to the benefit of the customer as well as the site- but systems like these that use information from studying the not always. Important ethical challenges will arise in balancing the value of recommendations to the site and to the customer customer. One economic study suggests that sites may need to pay customers for their information(Avery et al. 1999) There are many different techniques for implementi In a related concept, sell-side recommender systems could recommender systems, and the different techniques can be used allow businesses to decide which clients to make special offers nearly independently of how the recommender system is intended towards. In traditional commerce a company could offer a to increase revenues for the site. E-commerce sites can first coupon for a free pound of bananas with the purchase of a box of choose a way of increasing revenue, then choose the of ereal and a gallon of milk in order to increase sales of milk persistence and automation they desire, and finally The success of this depends on the custo recommender system technique that fits that profile mer vie and remembering to bring it to the store. a recommender system Technologists often assume that the holy grail of recommender could be designed which notices that a customer already has systems is fully Automatic, completely Ephemeral bananas and milk in his shopping cart and rarely purchases recommendations. Our study does not bear this assumption out cereal. This customer might be a good choice for the above offer at all. Many E-commerce sites use Persistent systems that One limitation to recommender systems is collecting enough data require Manual effort from the user. One reason for this to make effective recommendations for new users. One way to preference is that systems that are more Persistent create a ed the transition is for sites to share information about their relationship with the customers. If creating the relationship quires some degree of Manual effort from the customers, they users. Shared information benefits users, because they get more will prefer to return to the site in which they have invested the accurate recommendations in less time. but decreases the benefit effort, increasing the degree of"stickiness" of the relationship to individual sites because users are not as loyal to them. Since between the site and its customers. On the other hand, purely sites own the information they collect, they have little incentive Manual recommender systems are entirely portable: the customer share with competitors. However, it seems quite possible that consortia of non-competing sites may form with the goal of can freely go to another site with the same Manual features to sharing data to increase the value to companies within the obtain the desired recommendations. The optimal technology for consortia. Customers of these consortia will need assurances that the custom Ephemeral recommendations, since these recommendations leave their privacy will be carefully protected, even as their data are them free to visit any similar E-commerce site. However, most shared beyond the boundary of a single site recommender systems are deployed by the people running E 7. Conclusion commerce site Joe Pine's book mass Customization lists the five fundament Persistent, and is likely to be only partially Automatic, red methods for achieving the goal of mass customization. Each of some input from customers to increase"stickiness",but the first four of these goals can be realized through recommender rewarding the customers with valuable recommendations based on their input. Our prediction is that most recommender systems will be run by E-commerce sites, and will be Persistent an Customize services around standardized products and partially Automatic. A few recommender systems will be run by services": Recommender systems provide a customized groups whose goal is to support customers, and their service that enables E-commerce sites to sell their largely recommendations will tend to be ephemeral and fully automatic commodity products more efficiently to minimize customer effort, or Ephemeral and fully Manual, to ."Create customizable products and services" Recommender maximize customer control and portability (Schneiderman 1997) systems are a customizable product of the E-commerce site Recommender systems are creating value for both E-commerce sites and their customers. We hope our taxonomy of the ways in which recommender systems make money for sites, the way they

generated segments?” and “are automatically generated segments more useful for managing marketing campaigns than traditional segments?” Recommender systems can be made more useful as marketing systems in other ways, too. Current recommender systems are mainly “buy-side” systems. That is, they are designed to work on behalf of the customer in deciding what products they should purchase. However, modern marketing is designed not just to maximize utility to the customer, but to maximize value to the business at the same time. The recommender system could produce an indication of the price sensitivity of the customer for a given product, so the E-commerce site could offer each product at the price that maximizes the lifetime value of the customer to the site. For instance, one customer might be willing to purchase the product at a price that would earn the site ten cents of profit, while another customer might purchase the same product at a one dollar profit. There are challenging ethical issue in implementing systems like these that use information from studying the customer in determining how to get more money from the customer. One economic study suggests that sites may need to pay customers for their information (Avery et al. 1999). In a related concept, “sell-side” recommender systems could allow businesses to decide which clients to make special offers towards. In traditional commerce a company could offer a coupon for a free pound of bananas with the purchase of a box of cereal and a gallon of milk in order to increase sales of milk. The success of this depends on the customer viewing the coupon and remembering to bring it to the store. A recommender system could be designed which notices that a customer already has bananas and milk in his shopping cart and rarely purchases cereal. This customer might be a good choice for the above offer. One limitation to recommender systems is collecting enough data to make effective recommendations for new users. One way to speed the transition is for sites to share information about their users. Shared information benefits users, because they get more accurate recommendations in less time, but decreases the benefit to individual sites because users are not as loyal to them. Since sites own the information they collect, they have little incentive to share with competitors. However, it seems quite possible that consortia of non-competing sites may form with the goal of sharing data to increase the value to companies within the consortia. Customers of these consortia will need assurances that their privacy will be carefully protected, even as their data are shared beyond the boundary of a single site. 7. Conclusion Joe Pine’s book Mass Customization lists the five fundamental methods for achieving the goal of mass customization. Each of the first four of these goals can be realized through recommender systems: ¨ “Customize services around standardized products and services”: Recommender systems provide a customized service that enables E-commerce sites to sell their largely commodity products more efficiently. ¨ “Create customizable products and services”: Recommender systems are a customizable product of the E-commerce site. ¨ “Provide point of delivery customization”: The recommender system directly customizes the point of delivery for the E-commerce site. ¨ “Provide quick response throughout the value chain”: We predict that recommender systems will be used in the future to predict demand for products, enabling earlier communication back the supply chain. Recommender systems are a key way to automate mass customization for E-commerce sites. They will become increasingly important in the future, as modern businesses are increasingly focused on the long-term value of customers to the business (Peppers & Rogers 1997). E-commerce sites will be working hard to maximize the value of the customer to their site, providing exactly the pricing and service they judge will create the most valuable relationship with the customer. Since customer retention will be very important to the sites, this relationship will often be to the benefit of the customer as well as the site – but not always. Important ethical challenges will arise in balancing the value of recommendations to the site and to the customer. There are many different techniques for implementing recommender systems, and the different techniques can be used nearly independently of how the recommender system is intended to increase revenues for the site. E-commerce sites can first choose a way of increasing revenue, then choose the degree of persistence and automation they desire, and finally choose an recommender system technique that fits that profile. Technologists often assume that the holy grail of recommender systems is fully Automatic, completely Ephemeral recommendations. Our study does not bear this assumption out at all. Many E-commerce sites use Persistent systems that require Manual effort from the user. One reason for this preference is that systems that are more Persistent create a relationship with the customers. If creating the relationship requires some degree of Manual effort from the customers, they will prefer to return to the site in which they have invested the effort, increasing the degree of “stickiness” of the relationship between the site and its customers. On the other hand, purely Manual recommender systems are entirely portable: the customer can freely go to another site with the same Manual features to obtain the desired recommendations. The optimal technology for the customers may well be fully Automatic, completely Ephemeral recommendations, since these recommendations leave them free to visit any similar E-commerce site. However, most recommender systems are deployed by the people running E￾commerce sites. The optimal technology for them will be Persistent, and is likely to be only partially Automatic, requiring some input from customers to increase “stickiness”, but rewarding the customers with valuable recommendations based on their input. Our prediction is that most recommender systems will be run by E-commerce sites, and will be Persistent and partially Automatic. A few recommender systems will be run by groups whose goal is to support customers, and their recommendations will tend to be Ephemeral and fully Automatic to minimize customer effort, or Ephemeral and fully Manual, to maximize customer control and portability (Schneiderman 1997). Recommender systems are creating value for both E-commerce sites and their customers. We hope our taxonomy of the ways in which recommender systems make money for sites, the way they

can be implemented, and our analysis of future directions for Usenet news. Communications of the ACM, 40(3): pp 77 recommender systems in E-commerce helps to stimulate the creativity that is needed to produce the recommender systems of 87 the future Don Peppers and Martha Rogers 1997. The One to One Future: Building Relationships One Customer at a Time 8. ACKNOWLEDGMENTS Bantam Doubleday Dell Publishing Foundation under grants IIS 9613960, IS 9734442, and DGE B. Joseph Pine II 1993. Mass Customization. Harvard 9554517. Support was also provided by Net Perceptions Inc, a Business School Press. Boston, Massachusetts company that Konstan and Riedl co-founded, and that sells a B. Joseph Pine IL, Don Peppers, and Martha Rogers 1995 recommender system for E-Commerce. Do you want to keep your customers forever? Harvard 9. REFERENCES Business School Review, 1995(2): pp. 103-114 Christopher Avery, Paul Resnick, and Richard Zeckhauser Frederick F. Reichheld and W. Earl Sasser, Jr 1990. Zero 1999. The Market for Evaluations. American Economic Defections: Quality Comes to Services. Harvard Business Review89(3)pp564584. School Review, 1990(5): pp 105-111 Marko Balabanovic and Yoav Shoham 1997. Fab: Frederick F Reichheld 1993. Loyalty-Based Management Content-based collaborative recommendati lon Harvard Business School Review, 1993(2): pp 64-73 Communications of the ACM, 40(3 ) pp 66-72 Paul Resnick, Neophytos lacovo, Mitesh Suchak, Peter Chumki Basu, Haym Hirsh, and William Cohen 1998. Bergstrom, and John Riedl 1994. Grouplens: An open Recommendation as classification: using social and architecture for collaborative filtering Proceedings of the 1998 Workshop on Recommender Supported Cooperative Work, Pp 175-10e on Computer- content-based information in recommendation. In Proceedings of ACM CSCw 94 Confere Systems, pages 11-15 Badrul M. Sarwar, Joseph A. Konstan, Al Borchers, Jon John S Breese, David Heckerman, and Carl Kadie 1998. Herlocker, Brad Miller, and John Empirical analysis of predictive algorithms for filtering agents to improve prediction quality in the collaborative filtering. In Proceedings of the 14th grouplens research collaborative filtering system. In Conference on Uncertainty in Artificial Intelligence(UAl- Proceedings of 1998 Conference on Computer Support 98),pp43-52 Will Hill, Larry Stead, Mark Rosenstein, and George Ben Shneiderman 1997. Direct Manipulation for Furnas 1995 Recommending and evaluating choices in a Comprehensible, Predictable, and Controllable User virtual community of use. In Proceedings of ACM CHI95 Interfaces. Proceedings of 1U197, 1997 International Conference on Human Factors in Computing Systems, Conference on Intelligent User Interfaces, Orlando, FL, Joseph A. Konstan, Bradley N. Miller, David Maltz, Upendra Shardanand and Patti Maes 1995. Social Jonathan L. Herlocker, Lee R. Gordon, and John Riedl information filtering: Algorithms for automating"word of 1997. GroupLens: Applying collaborative filtering to mouth".In Proceedings of ACM CHI 95 Conference or Human Factors in Computing Systems, pages 210-217

can be implemented, and our analysis of future directions for recommender systems in E-commerce helps to stimulate the creativity that is needed to produce the recommender systems of the future. 8. ACKNOWLEDGMENTS We gratefully acknowledge the support of the National Science Foundation under grants IIS 9613960, IIS 9734442, and DGE 9554517. Support was also provided by Net Perceptions Inc, a company that Konstan and Riedl co-founded, and that sells a recommender system for E-Commerce. 9. REFERENCES Christopher Avery, Paul Resnick, and Richard Zeckhauser 1999. The Market for Evaluations. American Economic Review 89(3): pp 564-584. Marko Balabanovic and Yoav Shoham 1997. Fab: Content-based, collaborative recommendation. Communications of the ACM, 40(3): pp. 66-72. Chumki Basu, Haym Hirsh, and William Cohen 1998. Recommendation as classification: using social and content-based information in recommendation. In Proceedings of the 1998 Workshop on Recommender Systems, pages 11-15. John S. Breese, David Heckerman, and Carl Kadie 1998. Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence (UAI- 98), pp 43-52. Will Hill, Larry Stead, Mark Rosenstein, and George Furnas 1995. Recommending and evaluating choices in a virtual community of use. In Proceedings of ACM CHI'95 Conference on Human Factors in Computing Systems, pages 194-201. Joseph A. Konstan, Bradley N. Miller, David Maltz, Jonathan L. Herlocker, Lee R. Gordon, and John Riedl 1997. GroupLens: Applying collaborative filtering to Usenet news. Communications of the ACM, 40(3): pp 77- 87. Don Peppers and Martha Rogers 1997. The One to One Future : Building Relationships One Customer at a Time. Bantam Doubleday Dell Publishing. B. Joseph Pine II 1993. Mass Customization. Harvard Business School Press. Boston, Massachusetts B. Joseph Pine II, Don Peppers, and Martha Rogers 1995. Do you want to keep your customers forever? Harvard Business School Review, 1995(2): pp. 103-114. Frederick F. Reichheld and W. Earl Sasser, Jr 1990. Zero Defections: Quality Comes to Services. Harvard Business School Review, 1990(5): pp. 105-111. Frederick F. Reichheld 1993. Loyalty-Based Management. Harvard Business School Review, 1993(2): pp. 64-73. Paul Resnick, Neophytos Iacovou, Mitesh Suchak, Peter Bergstrom, and John Riedl 1994. Grouplens: An open architecture for collaborative filtering of netnews. In Proceedings of ACM CSCW'94 Conference on Computer￾Supported Cooperative Work, pp 175-186. Badrul M. Sarwar, Joseph A. Konstan, Al Borchers, Jon Herlocker, Brad Miller, and John Riedl 1998. Using filtering agents to improve prediction quality in the grouplens research collaborative filtering system. In Proceedings of 1998 Conference on Computer Supported Collaborative Work. Ben Shneiderman 1997. Direct Manipulation for Comprehensible, Predictable, and Controllable User Interfaces. Proceedings of IUI97, 1997 International Conference on Intelligent User Interfaces, Orlando, FL, January 6-9, 1997, 33-39. Upendra Shardanand and Patti Maes 1995. Social information filtering: Algorithms for automating "word of mouth". In Proceedings of ACM CHI'95 Conference on Human Factors in Computing Systems, pages 210-217