Implicit Feedback for Recommender Systems From: Aaal Technical Report Ws-98-08. Compilation copyright @ 1998, Aaal (wwW.aaai.org).Allrightsreserved Douglas W. Oard and Jinmook Kim Digital Library Research Grou College of Library and Information Services foard, jinmook]@glue umd. edu Abstract users interests. USENET newsreader software typicall records the identifiers of messages that users have seer Can implicit feedback substitute for explicit and Karlgren (1994)explored the design of a recom- nder systems? If so, we could avoid the mender system using such lists. Morita and Shinoda associated with gathering explicit ratings from hen, can we capture useful information unobtrusively, and ( 1994)and Konstan et al. (1997) found a positive correla ow might we use that information to make recom tion between reading time and explicit ratings in USENET this paper identify three types of in news applications, and we have generalized that source of feedback and suggest two strategies for using implic observations as "examination duration" to accommodate back to make recommendations other modalities such as audio and video. hill et al 1992) have developed this idea further, defining"edit wear"as Introduction an analogue to the useful effects of uneven wear that physical materials accumulate over time that provide other Recommender systems exploit ratings provided by an users with cues that help discover useful materials and tire user population to reshape an information space for the useful items of those items. In text browsing, for example benefit of one or more individuals(Oard, 1997b). In re- edit wear might be measured by using dwell times at spe- search systems, these ratings are often provided explicitly cific locations in the text to characterize scrolling behavior one or more ordinal or qualitative Examination may extend beyond more than a single inter action between user and system, and we seek to capture scales. The cognitive load effort to assign accurate ratings that source of observations by characterizing the repetition user populations and contributing to data sparsity within of the foregoing user behaviors. Finally, when information avoid this bottleneck by inferring something similar to the offer extremely strong evidence of the value ascribed bons existing populations. Implicit feedback techniques seek to access is priced on a per-item basis, purchase decis object. Similar information would be available at a some- ratings that a user would assign from observations that are what coarser scale when users purchase subscription ac- available to the system. Such an approach could greatly extend the range of applications for which recommender cess to certain types of content (e.g, subscription to a systems would be useful separately priced cable television channel) Sources of Implicit Feedback Category Observable Behavior Nichols(1997) surveyed the state of the art in implicit Duration feedback techniques with an eye toward their potential use for information filtering. Table I presents the sources identified by Nichols and some others that we believe will also be useful. In addition to explicit ratings we have Purchase(object or subscription identified three broad categories of potentially useful ob Save a reference or save an object servations: examination. retention and reference. Information systems often provide brief summaries of several promising documents using some sort of selection Retention with or without organization) interface display, and selection of individual objects for Print further examination can thus provide the first cue about a Delete Object->Object(forward, reply, post follow up) Nichols(1997) suggested two additional behaviors re- Refe lated to content-based retrieval: discovery of users that rtion->Object(hypertext link, citation present a common set of query terms and discovery of us Object->Portion(cut paste, quotation ers that retrieve similar documents. Both can be mapped into our framework by adopting the perspective that que Table 1. Observable behavior for implicit feedback ries are information objects in their own right
Implicit Feedback for Recommender Systems Douglas W. Oard and Jinmook Kim Digital Library Research Group College of Library and Information Services University of Maryland, College Park, MD 20742 {oard, jinmook} @glue.umd.edu Abstract Can implicit feedback substitute for explicit ratings in recommender systems? If so, we could avoid the difficulties associated with gathering explicit ratings from users. How, then, can we capture useful information unobtrusively, and how might we use that information to make recommendations? In this paper we identify three types of implicit feedback and suggest two strategies for using implicit feedback to make recommendations. Introduction Recommender systems exploit ratings provided by an entire user population to reshape an information space for the benefit of one or more individuals (Oard, 1997b). In research systems, these ratings are often provided explicitly by each user using one or more ordinal or qualitative scales. The cognitive load effort to assign accurate ratings acts as disincentive, making it difficult to assemble large user populations and contributing to data sparsity within existing populations. Implicit feedback techniques seek to avoid this bottleneck by inferring something similar to the ratings that a user would assign from observations that are available to the system. Such an approach could greatly extend the range of applications for which recommender systems would be useful. Sources of Implicit Feedback Nichols (1997) surveyed the state of the art in implicit feedback techniques with an eye toward their potential use for information filtering. Table 1 presents the sources identified by Nichols and some others that we believe will also be useful, t In addition to explicit ratings we have identified three broad categories of potentially useful observations: examination, retention and reference. Information systems often provide brief summaries of several promising documents using some sort of selection interface display, and selection of individual objects for further examination can thus provide the first cue about a l Nichols (1997) suggested two additional behaviors related to content-based retrieval: discovery of users that present a common set of query terms and discovery of users that retrieve similar documents. Both can be mapped into our framework by adopting the perspective that queries are information objects in their own right. user’s interests. USENET newsreader software typically records the identifiers of messages that users have seen, and Karlgren (1994) explored the design of a recommender system using such lists. Morita and Shinoda (1994) and Konstan et al. (1997) found a positive correlation between reading time and explicit ratings in USENET news applications, and we have generalized that source of observations as "examination duration" to accommodate other modalities such as audio and video. Hill et al. (1992) have developed this idea further, defining "edit wear" as an analogue to the useful effects of uneven wear that physical materials accumulate over time that provide other users with cues that help discover useful materials and useful items of those items. In text browsing, for example, edit wear might be measured by using dwell times at specific locations in the text to characterize scrolling behavior. Examination may extend beyond more than a single interaction between user and system, and we seek to capture that source of observations by characterizing the repetition of the foregoing user behaviors. Finally, when information access is priced on a per-item basis, purchase decisions offer extremely strong evidence of the value ascribed to an object. Similar information would be available at a somewhat coarser scale when users purchase subscription access to certain types of content (e.g., subscription to separately priced cable television channel). Category [[ Observable Behavior Selection Duration Examination Edit wear Repetition Purchase (object or subscription) Save a reference or save an object (with or without annotation) Retention (with or without organization) Print Delete Object->Object (forward, reply, post follow up) Reference Portion->Object (hypertext link, citation) Objeet->Portion (cut & paste, quotation) Table 1. Observable behavior for implicit feedback 81 From: AAAI Technical Report WS-98-08. Compilation copyright © 1998, AAAI (www.aaai.org). All rights reserved
Our"retention"category is intended to group those be- message to some form of group venue such as a mailing haviors that suggest some degree of intention to make fu list establishes the same sort of link. Goldberg et al re use of an object. Bookmarking a web page is a simple 1987) described a simple example of this in which users could construct an electronic mail filter to display mes- idea assave a reference "to accommodate a wider range sag of actions such as construction of symbolic links within a ypertext links from one web page to another and biblio- file system. Rucker Polanco(1997), for example, con- graphic citations in academic papers create links from a structed a recommender system using bookmark lists portion of an object (characterized, perhaps, by some Saving the object itself is the obvious alternative, some- neighborhood around the link itself) to another object, al thing Stevens(1993)used as implicit feedback for content- though the refinement to a portion of a document has not based filtering. In either case, the object may be saved been exploited often. Brin Page(1998)provide an ex with or without some form of annotation. For example, ample of how hypertext links might be used, although their web browsers typically default to using the page title in the focus is on a population statistics rather than individual bookmark list, but users may optionally provide a more preferences. Garfield(1979)describes the design of re meaningful entry if they desire. Although numerous con- trieval systems that are based on bibliographic citations founding factors would likely be present, it may be possi Alternatively, selective inclusion of another document ble to infer something about the value a user places on an using either cut-and-paste or a quotation, creates a link individual page by whether or not they go to the trouble of from an information object to a portion of another constructing an informative bookmark entry. Similarly, users may choose to save a reference or an object in an Using Implicit Feedback explicitly organized fashion or in the default manner. For example, storing electronic mail about this workshop in a The goal of a recommender system is to help users find new folder might provide greater support for an inference desirable information objects, That task combines infer that the user ascribes particular value to the message than ence and prediction, and Figures 1 and 2 show alternative originator. The salient issue in this case is not the act of duce ratings similar to those that a user would have ex organizing, but rather the way in which the organization plicitly assigned, and then the prediction stage uses those given to an individual object distinguishes it from the way estimated ratings to predict future ratings. Konstan et al in which similar forms of organization are assigned to adopted this perspective when evaluating how well ob other objects. This difference may not be easy to charac- terize, but it may be worth thinking about how to do it. We served reading time predicted explicit ratings for individ- ual articles. Figure 2 shows an alternative strategy in have chosen to group printing with retention because of the hich past observations are used to predict user behavior permanence of the printed page, but users may also print to facilitate examination because in response to new information, and then the inference per still has some decided advantages over electronic dis- stage seeks to estimate the value of the information based on the predicted behavior. We are not aware of any im plays in many applications. Printing overlaps with the plementations of this second approach, but Stevens(1993) next category(reference) as well, since users may print a implemented a simplified version of the strategy. He pre- document or image with the intention of forwarding them dicted the examination duration for a new usenet news to another individual or including portions in anothe document. Nevertheless, printing is often associated with a article based on the examination durations for similar arti- cles in the past and then constructed content-based queries desire for retention, so we find this grouping useful. A that would select articles with long predicted examination with examination, it may be possible to infer something durations. This essentially amounts to a degenerate infer- about the portions of a document that the user finds most valuable from the portions which he or she chooses to ence stage in which desirability is assumed to increase print. Finally, the retention category is distinguished by monotonically with examination duration The distinction between the two strategies is quite subtle the possibility of directly observing evidence of negative evaluations as well. when retention is a default condition in the case of content-based filtering. In as in some electronic mail systems, a decision by the user system, by contrast, the strategy shown in Figure 1 would characterize each article using the examination durations to delete an object might support to an inference that the reported by other users, while the strategy shown in Figure 2 would characterize each article using the predicted rat retained ngs for other users. Recommender systems based on the The"refer to" category may appear at first glance to of observable second strategy might be more flexible, since participating each has the effect of establishing some form of link be users might draw different inferences from the same ob- servations if they did not share a common set of objectives tween two objects. Forwarding a message, for example, establishes a link between the new message and the origi- On the other hand, recommender systems using the first strategy would likely have more context available locally nal. Similarly, replying individually or posting a follow up for interpreting observations than would be available at
Our "retention" category is intended to group those behaviors that suggest some degree of intention to make future use of an object. Bookmarking aweb page is a simple example of such a behavior, and we have generalized that idea as "save a reference" to accommodate a wider range of actions such as construction of symbolic links within a file system. Rucker & Polanco (1997), for example, constructed a recommender system using bookmark lists. Saving the object itself is the obvious alternative, something Stevens (1993) used as implicit feedback for contentbased filtering. In either case, the object may be saved with or without some form of annotation. For example, web browsers typically default to using the page title in the bookmark list, but users may optionally provide a more meaningful entry if they desire. Although numerous confounding factors would likely be present, it may be possible to infer something about the value a user places on an individual page by whether or not they go to the trouble of constructing an informative bookmark entry. Similarly, users may choose to save a reference or an object in an explicitly organized fashion or in the default manner. For example, storing electronic mail about this workshop in a new folder might provide greater support for an inference that the user ascribes particular value to the message than would the use of some default scheme such as placing it in the folder routinely used for mail from the message’s originator. The salient issue in this case is not the act of organizing, but rather the way in which the organization given to an individual object distinguishes it from the way in which similar forms of organization are assigned to other objects. This difference may not be easy to characterize, but it may be worth thinking about how to do it. We have chosen to group printing with retention because of the permanence of the printed page, but users may also print document or images to facilitate examination because paper still has some decided advantages over electronic displays in many applications. Printing overlaps with the next category (reference) as well, since users may print document or image with the intention of forwarding them to another individual or including portions in another document. Nevertheless, printing is often associated with a desire for retention, so we find this grouping useful. As with examination, it may be possible to infer something about the portions of a document that the user finds most valuable from the portions which he or she chooses to print. Finally, the retention category is distinguished by the possibility of directly observing evidence of negative evaluations as well. When retention is a default condition, as in some electronic mail systems, a decision by the user to delete an object might support to an inference that the deleted object is less valued than other objects that are retained. The "refer to" category may appear at first glance to contain a fairly eclectic group of observable activities, but each has the effect of establishing some form of link between two objects. Forwarding a message, for example, establishes a link between the new message and the original. Similarly, replying individually or posting a follow up message to some form of group venue such as a mailing list establishes the same sort of link. Goldberg et al. (1987) described a simple example of this in which users could construct an electronic mail filter to display messages that their colleagues had taken the time to reply to. Hypertext links from one web page to another and bibliographic citations in academic papers create links from a portion of an object (characterized, perhaps, by some neighborhood around the link itself) to another object, although the refinement to a portion of a document has not been exploited often. Brin & Page (1998) provide an example of how hypertext links might be used, although their focus is on a population statistics rather than individual preferences. Garfield (1979) describes the design of retrieval systems that are based on bibliographic citations. Alternatively, selective inclusion of another document, using either cut-and-paste or a quotation, creates a link from an information object to a portion of another. Using Implicit Feedback The goal of a recommender system is to help users fred desirable information objects. That task combines inference and prediction, and Figures 1 and 2 show alternative strategies for accomplishing this. Figure 1 depicts a modular strategy in which the inference stage seeks to produce ratings similar to those that a user would have explicitly assigned, and then the prediction stage uses those estimated ratings to predict future ratings. Konstan et al. adopted this perspective when evaluating how well observed reading time predicted explicit ratings for individual articles. Figure 2 shows an alternative strategy in which past observations are used to predict user behavior in response to new information, and then the inference stage seeks to estimate the value of the information based on the predicted behavior. We are not aware of any implementations of this second approach, but Stevens (1993) implemented a simplified version of the strategy. He predicted the examination duration for a new USENET news article based on the examination durations for similar articles in the past and then constructed content-based queries that would select articles with long predicted examination durations. This essentially amounts to a degenerate inference stage in which desirability is assumed to increase monotonically with examination duration. The distinction between the two strategies is quite subtle in the case of content-based filtering. In a recommender system, by contrast, the strategy shown in Figure 1 would characterize each article using the examination durations reported by other users, while the strategy shown in Figure 2 would characterize each article using the predicted ratings for other users. Recommender systems based on the second strategy might be more flexible, since participating users might draw different inferences from the same observations if they did not share a common set of objectives. On the other hand, recommender systems using the first strategy would likely have more context available locally for interpreting observations than would be available at 82
other points in the network. It might thus be worth consid- cantly expand impact and importance of recommender ering hybrid approaches in which some preliminary inter systems in a networked world pretation is performed locally when the observaton is made and then additional inferences are drawn at other points References the network Brin, S and Page, L. 1998. The Anatomy of a Large-Scale Hypertextual Web Search Engine. Dept. of Computer Sci- ence, Stanford Uniy Inference ↓mmg Goldberg, D, Nichols, D, Oki, B M, and Terry, D. 1992. Using Collaborative Filtering to Weave an Information Prediction Tapestry. Communication of the ACM, December, 35(12) Predicted ratings Garfield, E. 1979. Citation Indexing: Its Theory and Ap- plication in Science, Technology, and Hum York: Wiley-Interscience Figure 1. Rating estimation strategy Hill, w.C. hollan, D. wrobelwski, D and McCandless T. 1992. Read Wear and Edit Wear. In: Proceedings of Observations ACM Conference on Human Factors in Computing Sys- Prediction Karlgren, J. 1994. Newsgroup Clustering Based on User Predicted observations Behavior: A Recommendation Algebra. Tecl Research Reports from SICS, T94-01 Inference htp/wwsicsse/babstracshtml#T94/04. Konstan, J. A, Miller, B. N, Maltz, D, Herlocker, J. L Predicted rating Gordon, L R, and Riedl, J. 1997. GroupLens: Applying Collaborative Filtering to Usenet News Communications of the ACM, March, 40(3): 77-87 Figure 2. Predicted observations strategy. Morita, M. and Shinoda, Y. 1994. Information Filtering Conclusion Based on User Behavior Analysis and Best Match Text Retrieval. In Proceedings of the Seventeenth Annual Inter- We have presented three potential sources for implicit national ACM-SIGIR Conference on Research and Devel feedback and described two ways those sources could be opment in Information Retrieval, 272-281 used by recommender systems. Our "examination"cate- gory seeks to capture ephemeral interactions that begin and Nichols, D. M. 1997. Implicit Ratings and filtering. In groups user behaviors that suggest an intention for future Collaborative Filtering, 10-12. Budapaest, Hungary, se of the material. Our third category is reference, which ERCIM includes user behaviors that create explicit or explicit links between information objects. We believe these categories Oard, D. w. 1997. The State of the Art in Text Filtering group observable behavior in a way that is useful when User Modeling and User-Adapted Interaction, 7(3): 141 thinkingabouthowtomakepredictionsandtowardthat178.http:/lwwwglueumdedu/oard/researchhtml end we have suggested two strategies for using implicit feedback in recommender systems. Our present work is Rucker, J and Polanco, M.J. 1997. Personalized Naviga- focused on understanding how to relate observations to tion for the Web Communications of the ACM, March, predicted ratings, both individually and in various combi- 40(3):73-89 nations that could be more informative than single-source observations. We then hope to develop and implement a Stevens, C. 1993. Knowledge-Based Assistance for Ac prototype that will give us some insight into how implicit cessing Large, Poorly Structured Information Spaces Feedback can be used effectively in an application envi- Ph D dissertation, Dept of Computer Science, Univ of ronment. If successful, this approach could help transcend Colorado. Boulder the current reliance on explicit ratings and thus signifi
other points in the network. It might thus be worth considering hybrid approaches in which some preliminary interpretation is performed locally when the observaton is made and then additional inferences are drawn at other points in the network. Observations Inference I Estimated ratings Prediction I Predicted ratings Figure 1. Rating estimation strategy. Observations Prediction I Predicted observations Inference I Predicted ratings Figure 2. Predicted observations strategy. Conclusion We have presented three potential sources for implicit feedback and described two ways those sources could be used by recommender systems. Our "examination" category seeks to capture ephemeral interactions that begin and end during a single session, while the "retention" category groups user behaviors that suggest an intention for future use of the material. Our third category is reference, which includes user behaviors that create explicit or explicit links between information objects. We believe these categories group observable behavior in a way that is useful when thinking about how to make predictions, and toward that end we have suggested two strategies for using implicit feedback in recommender systems. Our present work is focused on understanding how to relate observations to predicted ratings, both individually and in various combinations that could be more informative than single-source observations. We then hope to develop and implement a prototype that will give us some insight into how implicit feedback can be used effectively in an application environment. If successful, this approach could help transcend the current reliance on explicit ratings and thus significantly expand impact and importance of recommender systems in a networked world. References Brin, S. and Page, L. 1998. The Anatomy of a Large-Scale Hypertextual Web Search Engine. Dept. of Computer Science, Stanford Univ. http_ ://google.stanford.edu/~backrub/google.html. Goldberg, D., Nichols, D., Oki, B. M, and Terry, D. 1992. Using Collaborative Filtering to Weave an Information Tapestry. Communication of the ACM, December, 35 (12 ): 61-70. Garfield, E. 1979. Citation Indexing: Its Theory and Application in Science, Technology, and Humanities. New York: Wiley-Interscience. Hill, W.C., Hollan, J. D., Wrobelwski, D. and McCandless, T. 1992. Read Wear and Edit Wear. In: Proceedings of ACM Conference on Human Factors in Computing Systems, CHI ’92: 3-9. Karlgren, J. 1994. Newsgroup Clustering Based on User Behavior: A Recommendation Algebra. Technical and Research Reports from SICS, T94-01. http_ ://www. ~ic s. se/libabstracts.html#T94/04. Konstan, J. A., Miller, B. N., Maltz, D., Herlocker, J. L., Gordon, L. R., and Riedl, J. 1997. GroupLens: Applying Collaborative Filtering to Usenet News. Communications of the ACM, March, 40(3): 77-87. Morita, M. and Shinoda, Y. 1994. Information Filtering Based on User Behavior Analysis and Best Match Text Retrieval. In Proceedings of the Seventeenth Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, 272-281. Nichols, D. M. 1997. Implicit Ratings and Riltering. In Proceedings of the 5 th DELOS Workshop on Filtering and Collaborative Filtering, 10-12. Budapaest, Hungary, ERCIM. Oard, D. W. 1997. The State of the Art in Text Filtering. User Modeling and User-Adapted Interaction, 7(3): 141- 178. http;//www.glue.umd.edu/-oard/research.html. Rucker, J. and Polanco, M. J. 1997. Personalized Navigation for the Web. Communications of the ACM, March, 40(3): 73-89. Stevens, C. 1993. Knowledge-Based Assistance for Accessing Large, Poorly Structured Information Spaces. Ph.D. dissertation, Dept. of Computer Science, Univ. of Colorado, Boulder. http;//www.holodeck.com/curt/m_vp, apers.html. 83