Case 1: 08-CV-03139-RRM-RER Document 51-6 Filed 03/17/10 Page 1 of 47 The Lane's Gifts v Google Report Alexander tuzhilin Table of contents 1. Dr. Tuzhilin's Background 1 2. Materials Reviewed 2 3. Google Personnel Interviewed 3 4. Development of the Internet 4 5. Growth of Search Engines and Googles History 5 6. Development of the Pay-per-Click Advertising Model 6 7. Googles Pay-per-Click Advertising Model 9 8. Invalid Clicks and Google's Definition 15 9. Google's Approach to Detecting Invalid Clicks 21 10. Conclusions 46 Executive Summary I have been asked to evaluate google's invalid click detection efforts and to conclude whether these efforts are reasonable or not. As a part of this evaluation, I have visited Googles campus three times, examined various internal documents, interviewed several Google's employees, have seen different demos of their invalid click inspection system and examined internal reports and charts showing various aspects of performance of Googles invalid click detection system. Based on all these studied materials and the information narrated to me by Google's employees, I conclude that Google's efforts to combat click fraud are reasonable. In the rest of this report, I elaborate on this point 1. Dr. Tuzhilin's Background I have recently been appointed as a Professor of Information Systems at the Stern School of Business at New York University(NYU), having previously served as an Associate Professor at the Stern School. I received my Ph. D. in Computer Science from the Courant Institute of Mathematical Sciences. NyU in 1989. M.S. in engineering Economics from the School of Engineering at Stanford University in 1981, and BA.in Mathematics from NYU in 1980 My current research interests include knowledge discovery in databases(data mining), personalization, Customer Relationship Management(CRM) and Internet marketing. My
1 The Lane’s Gifts v. Google Report Alexander Tuzhilin Table of Contents 1. Dr. Tuzhilin’s Background 1 2. Materials Reviewed 2 3. Google Personnel Interviewed 3 4. Development of the Internet 4 5. Growth of Search Engines and Google’s History 5 6. Development of the Pay-per-Click Advertising Model 6 7. Google’s Pay-per-Click Advertising Model 9 8. Invalid Clicks and Google’s Definition 15 9. Google’s Approach to Detecting Invalid Clicks 21 10. Conclusions 46 Executive Summary I have been asked to evaluate Google’s invalid click detection efforts and to conclude whether these efforts are reasonable or not. As a part of this evaluation, I have visited Google’s campus three times, examined various internal documents, interviewed several Google’s employees, have seen different demos of their invalid click inspection system, and examined internal reports and charts showing various aspects of performance of Google’s invalid click detection system. Based on all these studied materials and the information narrated to me by Google’s employees, I conclude that Google’s efforts to combat click fraud are reasonable. In the rest of this report, I elaborate on this point. 1. Dr. Tuzhilin’s Background I have recently been appointed as a Professor of Information Systems at the Stern School of Business at New York University (NYU), having previously served as an Associate Professor at the Stern School. I received my Ph.D. in Computer Science from the Courant Institute of Mathematical Sciences, NYU in 1989, M.S. in Engineering Economics from the School of Engineering at Stanford University in 1981, and B.A. in Mathematics from NYU in 1980. My current research interests include knowledge discovery in databases (data mining), personalization, Customer Relationship Management (CRM) and Internet marketing. My Case 1:08-cv-03139-RRM -RER Document 51-6 Filed 03/17/10 Page 1 of 47
Case 1: 08-CV-03139-RRM-RER Document 51-6 Filed 03/17/10 Page 2 of 47 prior research was done in the areas of temporal databases, query-driven simulations and the development of specification languages for modeling business processes. I have co- authored over 70 papers on these topics published in major Computer Science and Information Systems journals, conferences and other outlets i currently serve on the Editorial Boards of the IEEE Transactions on Knowledge and Data Engineering, the Data Mining and Knowledge Discovery Journal, the INFORMS Journal on Computing, and the Electronic Commerce Research Journal. I have also co-chaired the Program Committees of the IEEE International Conference on Data Mining(ICDM)in 2003 and the 2005 International Workshop on Customer Relationship Management that brought together researchers from the data mining and marketing communities to explore and promote an interdisciplinary focus on CRM. i have also served on numerous program and organizing committees of major conferences in the fields of Data Mining and Informatio Systems. I have also had visiting academic appointments at the Wharton School of University of Pennsylvania, Computer Science Department of Columbia University, and Ecole Nationale Superieure des telecommunications in Paris, france On the industrial side, I worked as a developer at Information Builders, Inc in New York for two years and consulted for various companies, including Lucent's Bell Laboratorie on a data mining project and Click Forensics on a click fraud detection project Additional information about my background can be found in my Cv in the appendix 2. Materials Reviewed During this project, I reviewed the following materials 1. Internal documents provided to me by Google, including the following documents Type of data collected and statistics/signals used for the detection of invalid click: Description of the filtering methods Description of the log generation and log transformation/aggregation system used for the analysis and detection of invalid clicks Description of the AdSense auto-termination system Description of the duplicate AdSense account detection system Description of the ad conversion system Description of the AdSense publisher investigation, flagging and termination Click Quality investigative processes, including the rules on when and how to terminate the publishers Description of the advertiser credit processes and systems Description of the inquiry handling processes and guidelines Description of the attack simulation system Description of the alerting system
2 prior research was done in the areas of temporal databases, query-driven simulations and the development of specification languages for modeling business processes. I have coauthored over 70 papers on these topics published in major Computer Science and Information Systems journals, conferences and other outlets. I currently serve on the Editorial Boards of the IEEE Transactions on Knowledge and Data Engineering, the Data Mining and Knowledge Discovery Journal, the INFORMS Journal on Computing, and the Electronic Commerce Research Journal. I have also co-chaired the Program Committees of the IEEE International Conference on Data Mining (ICDM) in 2003 and the 2005 International Workshop on Customer Relationship Management that brought together researchers from the data mining and marketing communities to explore and promote an interdisciplinary focus on CRM. I have also served on numerous program and organizing committees of major conferences in the fields of Data Mining and Information Systems. I have also had visiting academic appointments at the Wharton School of University of Pennsylvania, Computer Science Department of Columbia University, and Ecole Nationale Superieure des Telecommunications in Paris, France. On the industrial side, I worked as a developer at Information Builders, Inc. in New York for two years and consulted for various companies, including Lucent’s Bell Laboratories on a data mining project and Click Forensics on a click fraud detection project. Additional information about my background can be found in my CV in the Appendix. 2. Materials Reviewed During this project, I reviewed the following materials: 1. Internal documents provided to me by Google, including the following documents: • Type of data collected and statistics/signals used for the detection of invalid clicks • Description of the filtering methods • Description of the log generation and log transformation/aggregation system used for the analysis and detection of invalid clicks. • Description of the AdSense auto-termination system • Description of the duplicate AdSense account detection system • Description of the ad conversion system • Description of the AdSense publisher investigation, flagging and termination systems • Description of various Click Quality investigative processes, including the rules on when and how to terminate the publishers • Description of the advertiser credit processes and systems • Description of the inquiry handling processes and guidelines • Description of the attack simulation system • Description of the alerting system Case 1:08-cv-03139-RRM -RER Document 51-6 Filed 03/17/10 Page 2 of 47
Case 1: 08-CV-03139-RRM-RER Document 51-6 Filed 03/17/10 Page 3 of 47 History of the doubleclicking action Overview of the Click Quality teams high-priority projects Investigative reports generated by 3 different inspection systems that investigated three different cases of invalid clicking activities. One was an attack on an advertiser by an automated system, another one was an attack on a publisher by an automated system, and the third one was a general investigation of certain suspicious clicking activities. These reports were generated as a part of giving me demos on how Googles inspection systems worked and how manual offline investigations are typically conducted by Google personnel Different internal reports and charts showing various aspects of performance of Google's invalid click detection systems 2. Demos of various invalid click detection and inspection systems developed by the Click Quality team. Of course, these demos were provided only for the Click Quality systems that can be demoed(e.g, have appropriate User Interfaces) 3. Interviews with Google personnel, as described in the next section This report is based on this reviewed information and on the information narrated to me by google personnel during the interviews 3. Google Personnel Interviewed All the invalid click detection activities are performed by the Click Quality team at Google. The Click Quality team consists of the following two subgroups Engineering Responsible for the design and development of online filters and other nvalid click detection software. It consists primarily of engineers and currently has about a dozen staff members on the team Spam Operations Responsible primarily for the offline operations, inspections of invalid clicking activities including investigations of customer's inquiries. The group currently has about two dozens staff members on the team In addition, several other groups at Google, including Web spam, Ads quality Publications quality and others interact with the Click Quality team and provide their expertise on the issues that are related to invalid clicks(.g, Web spam and click fraud have some issues in common). Overall, the Click Quality team can draw upon the knowledge and expertise of a few dozens of other people on these teams, whenever equired The two groups, although located in different parts of the Google campus, interact closely with each other
3 • History of the doubleclicking action • Overview of the Click Quality team’s high-priority projects • Investigative reports generated by 3 different inspection systems that investigated three different cases of invalid clicking activities. One was an attack on an advertiser by an automated system, another one was an attack on a publisher by an automated system, and the third one was a general investigation of certain suspicious clicking activities. These reports were generated as a part of giving me demos on how Google’s inspection systems worked and how manual offline investigations are typically conducted by Google personnel. • Different internal reports and charts showing various aspects of performance of Google’s invalid click detection systems. 2. Demos of various invalid click detection and inspection systems developed by the Click Quality team. Of course, these demos were provided only for the Click Quality systems that can be demoed (e.g., have appropriate User Interfaces). 3. Interviews with Google personnel, as described in the next section. This report is based on this reviewed information and on the information narrated to me by Google personnel during the interviews. 3. Google Personnel Interviewed All the invalid click detection activities are performed by the Click Quality team at Google. The Click Quality team consists of the following two subgroups • Engineering Responsible for the design and development of online filters and other invalid click detection software. It consists primarily of engineers and currently has about a dozen staff members on the team. • Spam Operations Responsible primarily for the offline operations, inspections of invalid clicking activities including investigations of customer’s inquiries. The group currently has about two dozens staff members on the team. In addition, several other groups at Google, including Web spam, Ads quality, Publications quality and others interact with the Click Quality team and provide their expertise on the issues that are related to invalid clicks (e.g., Web spam and click fraud have some issues in common). Overall, the Click Quality team can draw upon the knowledge and expertise of a few dozens of other people on these teams, whenever required. The two groups, although located in different parts of the Google campus, interact closely with each other. Case 1:08-cv-03139-RRM -RER Document 51-6 Filed 03/17/10 Page 3 of 47
Case 1: 08-CV-03139-RRM-RER Document 51-6 Filed 03/17/10 Page 4 of 47 In addition, the Product Manager of the Trust and Safety Group works closely with the Click Quality team on more business oriented and public relat g invalid click detection During this project, I visited Google campus three times and interviewed over a dozen of the Click Quality team members from the Spam Operations and the Engineering groups, as well as the Product Manager of the Trust and Safety Group. I found the members of both groups to be well-qualified and highly competent to perform their jobs. Most of them have relevant prior backgrounds and strong credentials Before focusing on the Pay-per-Click advertising model and Google's efforts to combat valid clicks, I first provide some background materials on the Internet and the growth of the search engines to put these main topics into perspective 4. Development of the Internet The Internet is a worldwide system of interconnected computer networks that transmit data using packet switching methods of the Internet Protocol (IP). Computing devises attached to the Internet can exchange data of various types, from emails to text documents to video and audio files, over the pathways connecting computer networks These documents are partitioned into pieces, called packets, by the Internet Protocol and travel over the pathways in a flexible manner determined by routers and other devices controlling the Internet traffic. These packets are assembled back in the proper order at the destination site using the well-developed principles of the Internet Protocol Internet was developed long time ago. The predecessor of the Internet(called the ARPANET) was developed in late 1960s and early 1970s. The first wide area Internet network was operational by January 1983 when the National Science Foundation constructed a network connecting various universities. The Internet was opened to commercial interests in 1985 Prior to the 1990s, Internet was predominately used by the people with strong technical skills because most of the Internet applications at that time required such skills, and only relatively few people had these skills in those days. This situation changed dramatically and the Internet became much more accessible to the general public after the invention of the world wide web (www) by Tim Berners-Lee in 1989 WWw is a globally connected network of Web servers and browsers that allows transferring different types of Web pages and other documents containing text, images, audio, video and other multimedia resources over the Internet using a special type of protocol developed specifically for the Web(the so-called Http protocol). Each esource on the www(such as a Web page)has a unique global identifier(Uniform Resource Identifier(or Locator)-URI (URL), So that each such resource can be found
4 In addition, the Product Manager of the Trust and Safety Group works closely with the Click Quality team on more business oriented and public relations issues pertaining to invalid click detection. During this project, I visited Google campus three times and interviewed over a dozen of the Click Quality team members from the Spam Operations and the Engineering groups, as well as the Product Manager of the Trust and Safety Group. I found the members of both groups to be well-qualified and highly competent to perform their jobs. Most of them have relevant prior backgrounds and strong credentials. Before focusing on the Pay-per-Click advertising model and Google’s efforts to combat invalid clicks, I first provide some background materials on the Internet and the growth of the search engines to put these main topics into perspective. 4. Development of the Internet The Internet is a worldwide system of interconnected computer networks that transmit data using packet switching methods of the Internet Protocol (IP). Computing devises attached to the Internet can exchange data of various types, from emails to text documents to video and audio files, over the pathways connecting computer networks. These documents are partitioned into pieces, called packets, by the Internet Protocol and travel over the pathways in a flexible manner determined by routers and other devices controlling the Internet traffic. These packets are assembled back in the proper order at the destination site using the well-developed principles of the Internet Protocol. Internet was developed long time ago. The predecessor of the Internet (called the ARPANET) was developed in late 1960’s and early 1970’s. The first wide area Internet network was operational by January 1983 when the National Science Foundation constructed a network connecting various universities. The Internet was opened to commercial interests in 1985. Prior to the 1990’s, Internet was predominately used by the people with strong technical skills because most of the Internet applications at that time required such skills, and only relatively few people had these skills in those days. This situation changed dramatically and the Internet became much more accessible to the general public after the invention of the World Wide Web (WWW) by Tim Berners-Lee in 1989. WWW is a globally connected network of Web servers and browsers that allows transferring different types of Web pages and other documents containing text, images, audio, video and other multimedia resources over the Internet using a special type of protocol developed specifically for the Web (the so-called HTTP protocol). Each resource on the WWW (such as a Web page) has a unique global identifier (Uniform Resource Identifier (or Locator) – URI (URL)), so that each such resource can be found Case 1:08-cv-03139-RRM -RER Document 51-6 Filed 03/17/10 Page 4 of 47
Case 1: 08-Cv-03139-RRM-RER Document 51-6 Filed 03/17/10 Page 5 of 47 and accessed. Web pages are created using special markup languages, such as HTML or XML that contain commands telling the browser how to display information contained in these pages. The markup languages also contain commands for linking the page to other pages, thus creating a hypertext environment that lets the Web user navigate from one Web page to another using these links(clicking on them) and thus letting the users to surf the Web The development of the World wide web. Web documents and Web browsers for displaying these documents in a user-friendly fashion, made Internet much more user- friendly. This opened Internet to the less technologically savvy general public that simply wanted to display, access and exchange various types of information without resorting to complicated technical means that were needed before to achieve these goals. B developing the Web and thus making the tasks of displaying, accessing and exchanging information over the Internet much simpler, spawned the development of various types of websites that collect, organize and provide systematic access to Web documents. The number of these websites experienced explosive growth in the 1990s and continued to grow rapidly worldwide up until now Massive volumes of Web documents were created over a short period of time since the invention of the Www. To deal with this information overload it was necessary to search and find relevant documents among millions(and later billions) of Web pages spread all over the world among numerous websites. This gave rise to the creation and growth of search engines designed to search and find relevant information in the massive volumes of Web documents 5. Growth of Search Engines and Google's History A search engine finds information requested by the user that is located somewhere on the World wide Web or other places, including proprietary networks and sites, and on a personal computer. The user formulates a search query, and the search engine looks for documents and other content satisfying the search criteria of the query. Typically, these search queries contain a list of keywords or phrases and retrieve documents that match these queries. Although the search can be done in various environments, including corporate intranets, the majority of the search has been done on the Web for different kinds of documents and information available on the Web. Since searching these documents directly on the Web is prohibitively time consuming, all the search engines indexes to provide efficient retrieval of the searched information. These indexes ar maintained regularly in order to keep them current The history of search engines goes back to Archie and Gopher, two tools designed in 1990-1991 for searching files located at the publicly accessible FTP sites over the Internet(and not over the Www which did not exist at that time ). The early commercial search engines for the Web documents were Lycos, Infoseek, Alta Vista and Excite which were launched around 1994-1995
5 and accessed. Web pages are created using special markup languages, such as HTML or XML that contain commands telling the browser how to display information contained in these pages. The markup languages also contain commands for linking the page to other pages, thus creating a hypertext environment that lets the Web user navigate from one Web page to another using these links (clicking on them) and thus letting the users to “surf” the Web. The development of the World Wide Web, Web documents and Web browsers for displaying these documents in a user-friendly fashion, made Internet much more userfriendly. This opened Internet to the less technologically savvy general public that simply wanted to display, access and exchange various types of information without resorting to complicated technical means that were needed before to achieve these goals. By developing the Web and thus making the tasks of displaying, accessing and exchanging information over the Internet much simpler, spawned the development of various types of websites that collect, organize and provide systematic access to Web documents. The number of these websites experienced explosive growth in the 1990’s and continued to grow rapidly worldwide up until now. Massive volumes of Web documents were created over a short period of time since the invention of the WWW. To deal with this information overload, it was necessary to search and find relevant documents among millions (and later billions) of Web pages spread all over the world among numerous websites. This gave rise to the creation and growth of search engines designed to search and find relevant information in the massive volumes of Web documents. 5. Growth of Search Engines and Google’s History A search engine finds information requested by the user that is located somewhere on the World Wide Web or other places, including proprietary networks and sites, and on a personal computer. The user formulates a search query, and the search engine looks for documents and other content satisfying the search criteria of the query. Typically, these search queries contain a list of keywords or phrases and retrieve documents that match these queries. Although the search can be done in various environments, including corporate intranets, the majority of the search has been done on the Web for different kinds of documents and information available on the Web. Since searching these documents directly on the Web is prohibitively time consuming, all the search engines use indexes to provide efficient retrieval of the searched information. These indexes are maintained regularly in order to keep them current. The history of search engines goes back to Archie and Gopher, two tools designed in 1990 – 1991 for searching files located at the publicly accessible FTP sites over the Internet (and not over the WWW which did not exist at that time). The early commercial search engines for the Web documents were Lycos, Infoseek, AltaVista and Excite, which were launched around 1994 – 1995. Case 1:08-cv-03139-RRM -RER Document 51-6 Filed 03/17/10 Page 5 of 47
Case 1: 08-CV-03139-RRM-RER Document 51-6 Filed 03/17/10 Page 6 of 47 Google co-founders started working on developing Google search engine in 1997 and Google Inc. was founded in September 1998. The beta label came off the Google website in September 1999. The co-founders have developed innovative patented search technologies based on the Page Rank concept that turned out to be highly effective in generating good search results. Google popularity grew rapidly, and the company w handling more than 100 million search queries a day by the end of 2000. Around that time, Google started launching various additional offerings, such as Google Toolbar, and this trend continued since then. Currently, Google supports a couple of dozens of such offerings publicly available on the Google's website Currently, the main competing search engines for Google include(a) Yahoo! acquired Inktomi search engine in 2002 and also Overture which owned Alta Vista. (b) Microsoft which launched its own independent msn Search engine in early 2005 Google is currently the market leader in the search engine field, accounting for over 50% of all the Web search queries Google realized the power of the keyword-based targeted advertising back in 2000 when it launched its initial version of AdWords, which was quite different from its current version and even from the version launched in February 2002. The Pay-per-Click overhauled version of Ad Words was launched in February 2002. It was followed by the AdSense program in March 2003 The AdWords and AdSense programs will be described later in Section 7 in the context of Google's overall Pay-per-Click advertising model. However, before doing this, I will first present a general overview of the Pay-per-Click advertising model in Section 6 6. Development of the Pay-per-click Advertising Model The idea of delivering targeted ads to an internet user has been around for a long time For example, such companies as Double Click have been involved in this effort since the 90s. The key question in this problem is: what is the basis for targeting these ads? The ads can be targeted based on 1. personal characteristics of a web page visitor known to the party delivering an ad 2. keywords of a search query launched by the user 3. content of a web page visited by the user The first source of targeting, based on personal characteristics of a web page visitor, has been adopted by various companies in the personalization and Customer Relationshil Management area. The two other sources of targeting are adopted by the search engines including google The second issue dealing with the delivery of targeted ads is the payment model. When the ads are delivered to the user, for what exactly should advertisers pay and when? The alternative choices for charging an advertiser are when the ad is being shown to the user 6
6 Google co-founders started working on developing Google search engine in 1997 and Google Inc. was founded in September 1998. The beta label came off the Google website in September 1999. The co-founders have developed innovative patented search technologies based on the PageRank concept that turned out to be highly effective in generating good search results. Google popularity grew rapidly, and the company was handling more than 100 million search queries a day by the end of 2000. Around that time, Google started launching various additional offerings, such as Google Toolbar, and this trend continued since then. Currently, Google supports a couple of dozens of such offerings publicly available on the Google’s website. Currently, the main competing search engines for Google include (a) Yahoo! that acquired Inktomi search engine in 2002 and also Overture which owned AltaVista, and (b) Microsoft which launched its own independent MSN Search engine in early 2005. Google is currently the market leader in the search engine field, accounting for over 50% of all the Web search queries. Google realized the power of the keyword-based targeted advertising back in 2000 when it launched its initial version of AdWords, which was quite different from its current version and even from the version launched in February 2002. The Pay-per-Click overhauled version of AdWords was launched in February 2002. It was followed by the AdSense program in March 2003. The AdWords and AdSense programs will be described later in Section 7 in the context of Google’s overall Pay-per-Click advertising model. However, before doing this, I will first present a general overview of the Pay-per-Click advertising model in Section 6. 6. Development of the Pay-per-Click Advertising Model The idea of delivering targeted ads to an internet user has been around for a long time. For example, such companies as DoubleClick have been involved in this effort since the 90’s. The key question in this problem is: what is the basis for targeting these ads? The ads can be targeted based on: 1. personal characteristics of a web page visitor known to the party delivering an ad 2. keywords of a search query launched by the user 3. content of a web page visited by the user. The first source of targeting, based on personal characteristics of a web page visitor, has been adopted by various companies in the personalization and Customer Relationship Management area. The two other sources of targeting are adopted by the search engines, including Google. The second issue dealing with the delivery of targeted ads is the payment model. When the ads are delivered to the user, for what exactly should advertisers pay and when? The alternative choices for charging an advertiser are: • when the ad is being shown to the user Case 1:08-cv-03139-RRM -RER Document 51-6 Filed 03/17/10 Page 6 of 47
Case 1: 08-CV-03139-RRM-RER Document 51-6 Filed 03/17/10 Page 7 of 47 when the ad is being clicked by the user when the ad has" influenced" the user in the sense that its presentation lead to a conversion event, such as the actual purchase of the product advertised in the ad or other related conversion events, such as placing the related product into the user's shopping basket From the advertiser's point of view, the weakest form of delivery is when an ad is only shown to the user because the user may not even look at it and may simply ignore the ad Clicking on an ad indicates some interest in the product or service being advertised Finally, the most powerful user reaction to an ad is the conversion event when the user actually acts in response to the ad, with the most powerful type of action being actual purchase of the advertised product or service. For these reasons, advertisers value these three activities differently and, generally, are willing to pay more money per conversion event than per clicking event and than per ad viewing event(however, there are also some exceptions to this observation, which I will not cover in this report because they have only tangential relevance) The two key measures of how effective an advertisement is are Click-Through Rate (CTR): it specifies on how many ads X, out of the total number of ads Y shown to the visitors, the visitors actually clicked; in other words. CTR= X/Y CTR measures how often visitors click on the ad Conversion Rate: it specifies the percentage of visitors who took the conversion action. Conversion rate gives a sense of how often visitors actually act on a given ad. which is a better measure of ads effectiveness than the ctr measure Conversion actions are actually very relevant to click fraud because proper conversion actions following clicking activities, such as a purchase of an advertised product, are really good indicators that the clicks are valid. However, less direct conversion actions such as putting a product into a shopping cart, may still not be indicative of a valid click since it can be a part of a comversion fraud(an unethical user may do it on purpose without a true intent to purchase the product, but just simply to confuse an invalid click detection system) The three situations described above give rise to the following three different internet dvertising payment methods CPM-Cost per Mille-an advertiser pays per one thousand impressions of the ad (" stands for thousand"in Latin); an alternative term used in the industry for this payment model is CPI(Cost per Impression) CPC-Cost per Click(a k a. Pay per Click or PPC, we will use these terms interchangeably)-an advertiser pays only when a visitor clicks on the ad, as is clearly stated in the name of this payment model CPA-Cost per Action-an advertiser only pays when a certain conversion action takes place, such as a product being purchased, an advertised item was placed into a shopping cart, or a certain form being filled. This is the best option for an advertiser to pay for the ads from the advertisers point of view since it gives the
7 • when the ad is being clicked by the user • when the ad has “influenced” the user in the sense that its presentation lead to a conversion event, such as the actual purchase of the product advertised in the ad or other related conversion events, such as placing the related product into the user’s shopping basket. From the advertiser’s point of view, the weakest form of delivery is when an ad is only shown to the user because the user may not even look at it and may simply ignore the ad. Clicking on an ad indicates some interest in the product or service being advertised. Finally, the most powerful user reaction to an ad is the conversion event when the user actually acts in response to the ad, with the most powerful type of action being actual purchase of the advertised product or service. For these reasons, advertisers value these three activities differently and, generally, are willing to pay more money per conversion event than per clicking event and than per ad viewing event (however, there are also some exceptions to this observation, which I will not cover in this report because they have only tangential relevance). The two key measures of how effective an advertisement is are • Click-Through Rate (CTR): it specifies on how many ads X, out of the total number of ads Y shown to the visitors, the visitors actually clicked; in other words, CTR = X/Y. CTR measures how often visitors click on the ad. • Conversion Rate: it specifies the percentage of visitors who took the conversion action. Conversion rate gives a sense of how often visitors actually act on a given ad, which is a better measure of ad’s effectiveness than the CTR measure. Conversion actions are actually very relevant to click fraud because proper conversion actions following clicking activities, such as a purchase of an advertised product, are really good indicators that the clicks are valid. However, less direct conversion actions, such as putting a product into a shopping cart, may still not be indicative of a valid click since it can be a part of a conversion fraud (an unethical user may do it on purpose without a true intent to purchase the product, but just simply to confuse an invalid click detection system). The three situations described above give rise to the following three different internet advertising payment methods: • CPM – Cost per Mille – an advertiser pays per one thousand impressions of the ad (“Mille” stands for “thousand” in Latin); an alternative term used in the industry for this payment model is CPI (Cost per Impression). • CPC – Cost per Click (a. k. a. Pay per Click or PPC; we will use these terms interchangeably) – an advertiser pays only when a visitor clicks on the ad, as is clearly stated in the name of this payment model. • CPA – Cost per Action – an advertiser only pays when a certain conversion action takes place, such as a product being purchased, an advertised item was placed into a shopping cart, or a certain form being filled. This is the best option for an advertiser to pay for the ads from the advertisers’ point of view since it gives the Case 1:08-cv-03139-RRM -RER Document 51-6 Filed 03/17/10 Page 7 of 47
Case 1: 08-CV-03139-RRM-RER Document 51-6 Filed 03/17/10 Page 8 of 47 best indication among the three alternatives that the ad actually " worked"(as I said before, however, there are certain exceptions to this general observation were mainly CPM-based. For example, the CPm model between 2000 and February 2002 However, the CPC model is more attractive for many(but not all) advertisers than the CPM model, and it replaced the CPM as a predominant internet advertising payment model. For example, this is certainly the case for Google since most of its advertisers currently use the CPC model The origins of the CPC model go back to mid-90s when different payment models were debated in the internet marketing community. The first major commercial keyword- basedCpcmodelwasintroducedbyoVerture(previouslyknownasgOto.com,nowpart of Yahoo! )that has developed certain patented technologies for implementing this model that go back to 1999. Google introduced its keyword-and CPC-based AdWords program in February 2002. Besides Google and Yahool, Microsoft has also recently deployed the CPC payment model through its adCenter program. Also, several other online advertising programs use the CPC/PPC payment model If one combines a particular ad payment method with a particular targeting method, this combination determines a specific targeted ad delivery model. For Google and Yahoo the two main models are the keyword-based PPC and the content-based PPC model Although currently popular, the CPC/PPC model has two fundamental problems Although correlated, good click-through rates(CTRs)are still not indicative of good conversion rates, since it is still not clear if a visitor would buy an advertised product once he or she clicked on the ad. In this respect, the CPA-based models provide better solutions for the advertisers(but not necessarily for the search engines), since they are more indicative that their ads are"working It does not offer any"built-in"fundamental protection mechanisms against the click fraud since it is very hard to specify which clicks are valid vS. invalid in general, as will be explained in Section 8 (it can be done relatively easily in some special cases, but not in general). For this reason, major search engines launched extensive invalid click detection programs and still face problems combating click fraud In response to these two problems and for various other business reasons, Google is currently testing a CPa payment model, according to some reports in the media. Some analysts believe that the conversion-based CPa model is more robust for the advertisers and also less prone to click fraud. Therefore, they believe that the future of the online advertising payments lies with the CPA model. Although this is only a belief that is not supported by strong evidence yet, Google is getting ready for the next stage of the online dvertising"marathon 8
8 best indication among the three alternatives that the ad actually “worked” (as I said before, however, there are certain exceptions to this general observation). Early forms of internet advertising models were mainly CPM-based. For example, Google initially based the AdWords program only on the CPM model between 2000 and February 2002. However, the CPC model is more attractive for many (but not all) advertisers than the CPM model, and it replaced the CPM as a predominant internet advertising payment model. For example, this is certainly the case for Google since most of its advertisers currently use the CPC model. The origins of the CPC model go back to mid-90’s when different payment models were debated in the internet marketing community. The first major commercial keywordbased CPC model was introduced by Overture (previously known as GoTo.com, now part of Yahoo!) that has developed certain patented technologies for implementing this model that go back to 1999. Google introduced its keyword- and CPC-based AdWords program in February 2002. Besides Google and Yahoo!, Microsoft has also recently deployed the CPC payment model through its adCenter program. Also, several other online advertising programs use the CPC/PPC payment model. If one combines a particular ad payment method with a particular targeting method, this combination determines a specific targeted ad delivery model. For Google and Yahoo! the two main models are the keyword-based PPC and the content-based PPC models. Although currently popular, the CPC/PPC model has two fundamental problems: • Although correlated, good click-through rates (CTRs) are still not indicative of good conversion rates, since it is still not clear if a visitor would buy an advertised product once he or she clicked on the ad. In this respect, the CPA-based models provide better solutions for the advertisers (but not necessarily for the search engines), since they are more indicative that their ads are “working.” • It does not offer any “built-in” fundamental protection mechanisms against the click fraud since it is very hard to specify which clicks are valid vs. invalid in general, as will be explained in Section 8 (it can be done relatively easily in some special cases, but not in general). For this reason, major search engines launched extensive invalid click detection programs and still face problems combating click fraud. In response to these two problems and for various other business reasons, Google is currently testing a CPA payment model, according to some reports in the media. Some analysts believe that the conversion-based CPA model is more robust for the advertisers and also less prone to click fraud. Therefore, they believe that the future of the online advertising payments lies with the CPA model. Although this is only a belief that is not supported by strong evidence yet, Google is getting ready for the next stage of the online advertising “marathon.” Case 1:08-cv-03139-RRM -RER Document 51-6 Filed 03/17/10 Page 8 of 47
Case 1: 08-CV-03139-RRM-RER Document 51-6 Filed 03/17/10 Page 9 of 47 7. Google's Pay-per-Click Advertising Model As stated in Section 6. Google introduced the CPC/PPC model in addition to the previously deployed CPM model for the AdWords program in February 2002. The PPC model is widely adopted by Google now and its two main programs, Ad Words and AdSense, are based on it. These two programs are described below, including how the PPC advertising model is used in them 7.1. The AdWords Progra AdWords is a program allowing advertisers to purchase CPC-based advertising that targets the ads based on the keywords specified in users'search queries. An advertiser chooses the keywords for which the ad will be shown on Google's web page Google. com) or some other"network partner" pages, such as AOL and EarthLink(to be discussed below in Section 7.4), and specifies the maximum amount the advertiser is willing to pay for each click on this ad associated with this keyword. For example, an accounting firm signs with Google AdWords program and is willing to pay up to $10/click for showing its ad (a link to its home page combined with a short text message) on Google. com when the user types the query"tax return"on Google When a user issues a search query on Google. com or a network partner site, ads for relevant words are shown along with search results on the site on the right side of the Web page as"sponsored links and also above the main search results The ordering of the paid listings on the side of the page is determined according to the Ad Rank for the candidate ads that is defined Ad Rank= CPC X Quality Score where Quality Score is a measure identifying the " quality"of the keyword/ad pair. It depends on several factors, one of the main ones being the clickthrough rate(CtR)on the ad. In other words, the more the advertiser is willing to pay(CPC) and the higher the clickthrough rate on the ad(CtR), the higher the position of the ad in the listing is. There exists the whole science and art of how to improve the Ad Rank of advertisers'ads collectively known as Ad Optimization, so that the ad would be placed higher in the list by Google. Various tips on how to improve the results are presented on Googles website athttps://adwords.google.com/support/bin/static.py?page=tips.html&hl=enUs.thEtop of-the-page placement rank is also determined by the above Ad Rank formula; however the value of the Quality Score for the top-of-the-page placement is computed somewhat differently than for the side ads The actual amount of money paid when the user clicks on an ad is determined by the lowest cost needed to maintain the clicked ad's position on the results page and is usually less than the maximal CPC specified by the advertiser. Although the algorithm is known, the advertiser does not know a priori how much the click on the ad will actually cost
9 7. Google’s Pay-per-Click Advertising Model As stated in Section 6, Google introduced the CPC/PPC model in addition to the previously deployed CPM model for the AdWords program in February 2002. The PPC model is widely adopted by Google now and its two main programs, AdWords and AdSense, are based on it. These two programs are described below, including how the PPC advertising model is used in them. 7.1. The AdWords Program AdWords is a program allowing advertisers to purchase CPC-based advertising that targets the ads based on the keywords specified in users’ search queries. An advertiser chooses the keywords for which the ad will be shown on Google’s web page (Google.com) or some other “network partner” pages, such as AOL and EarthLink (to be discussed below in Section 7.4), and specifies the maximum amount the advertiser is willing to pay for each click on this ad associated with this keyword. For example, an accounting firm signs with Google AdWords program and is willing to pay up to $10/click for showing its ad (a link to its home page combined with a short text message) on Google.com when the user types the query “tax return” on Google. When a user issues a search query on Google.com or a network partner site, ads for relevant words are shown along with search results on the site on the right side of the Web page as “sponsored links” and also above the main search results. The ordering of the paid listings on the side of the page is determined according to the Ad Rank for the candidate ads that is defined as Ad Rank = CPC x QualityScore, where QualityScore is a measure identifying the “quality” of the keyword/ad pair. It depends on several factors, one of the main ones being the clickthrough rate (CTR) on the ad. In other words, the more the advertiser is willing to pay (CPC) and the higher the clickthrough rate on the ad (CTR), the higher the position of the ad in the listing is. There exists the whole science and art of how to improve the Ad Rank of advertisers’ ads, collectively known as Ad Optimization, so that the ad would be placed higher in the list by Google. Various tips on how to improve the results are presented on Google’s website at https://adwords.google.com/support/bin/static.py?page=tips.html&hl=en_US. The topof-the-page placement rank is also determined by the above Ad Rank formula; however, the value of the QualityScore for the top-of-the-page placement is computed somewhat differently than for the side ads. The actual amount of money paid when the user clicks on an ad is determined by the lowest cost needed to maintain the clicked ad’s position on the results page and is usually less than the maximal CPC specified by the advertiser. Although the algorithm is known, the advertiser does not know a priori how much the click on the ad will actually cost Case 1:08-cv-03139-RRM -RER Document 51-6 Filed 03/17/10 Page 9 of 47
Case 1: 08-CV-03139-RRM-RER Document 51-6 Filed 03/17/10 Page 10 of 47 because this depends on the actions of other bidders which are unknown to the advertiser beforehand. However. it is lower than the maximal cPC that the advertiser is willing to An advertiser has a certain budget associated with a keyword, which is allocated for a specified time period, e.g. for a day. For example, the accounting firm wants to spend no more than $100/day for all the clicks on the ad for the keyword"tax return. Each click on the ad decreases the budget by the amount paid for the ad, until it finally reaches zero during that time period(note that more money is added to the budget during the next time period, e.g., the next day ) If the balance reaches zero, the ad stops showing until the end of the time period (actually, the situation is somewhat more complex because Google has developed a mechanism to extend the ad exposure over the whole time period, but do it over short time intervals with long blackout periods; however, in the first approximation, we can assume that the ad stops showing when the balance reaches zero). For example, if the budget for the key word"tax return"reached zero by the mid-day, then no ads for the accounting firm are shown for the"tax return" query for the rest of the day(modulo the previous remark). However, the ad is resumed the next day, assuming that the accounting firm has signed up with Google for the next day This is one of the motivations for the click fraud with the purpose to hurt other advertisers. If an advertiser or its partner can deplete the budget of a competitor by repeatedly clicking on the ad, the competitors ad is not being shown for the rest of the time period, and the advertiser's ad has less competition and should appear higher in the paid ads list. Moreover, the advertiser may also end up paying less for his/her ad since there is less competition among the advertisers. Therefore, unethical advertisers or their partners not only hurt their competitors financially by repeatedly clicking on their ads they also knock them out of the auction competition for the rest of the day by depleting their advertising budgets and thus improving their positions in the sponsored link lists and also paying less for their own ads When search queries are launched on the network partners' websites, such as AOL or EarthLink, the PPC model works the same way as on Google. com with two caveats:(a) the ads are displayed somewhat differently on these websites than on Google. com and (b) Google shares parts of its advertising revenues with these partners AdWords based on the CPC/PPC advertising model described above was launched in February 2002. It changed Google's business model and was responsible for generating major revenue streams for the company 7.2. The AdSense Program Google AdSense is a program for the website owners(known as publishers)to display Google's ads on their websites and earn money from Google as a result. To participate in this program, website publishers need to register with Google and be accepted into the program by Google. These ads shown on the publishers' websites are administered by 10
10 because this depends on the actions of other bidders which are unknown to the advertiser beforehand. However, it is lower than the maximal CPC that the advertiser is willing to pay. An advertiser has a certain budget associated with a keyword, which is allocated for a specified time period, e.g. for a day. For example, the accounting firm wants to spend no more than $100/day for all the clicks on the ad for the keyword “tax return.” Each click on the ad decreases the budget by the amount paid for the ad, until it finally reaches zero during that time period (note that more money is added to the budget during the next time period, e.g., the next day). If the balance reaches zero, the ad stops showing until the end of the time period (actually, the situation is somewhat more complex because Google has developed a mechanism to extend the ad exposure over the whole time period, but do it over short time intervals with long blackout periods; however, in the first approximation, we can assume that the ad stops showing when the balance reaches zero). For example, if the budget for the keyword “tax return” reached zero by the mid-day, then no ads for the accounting firm are shown for the “tax return” query for the rest of the day (modulo the previous remark). However, the ad is resumed the next day, assuming that the accounting firm has signed up with Google for the next day. This is one of the motivations for the click fraud with the purpose to hurt other advertisers. If an advertiser or its partner can deplete the budget of a competitor by repeatedly clicking on the ad, the competitor’s ad is not being shown for the rest of the time period, and the advertiser’s ad has less competition and should appear higher in the paid ads list. Moreover, the advertiser may also end up paying less for his/her ad since there is less competition among the advertisers. Therefore, unethical advertisers or their partners not only hurt their competitors financially by repeatedly clicking on their ads, they also knock them out of the auction competition for the rest of the day by depleting their advertising budgets and thus improving their positions in the sponsorded link lists and also paying less for their own ads. When search queries are launched on the network partners’ websites, such as AOL or EarthLink, the PPC model works the same way as on Google.com with two caveats: (a) the ads are displayed somewhat differently on these websites than on Google.com and (b) Google shares parts of its advertising revenues with these partners. AdWords based on the CPC/PPC advertising model described above was launched in February 2002. It changed Google’s business model and was responsible for generating major revenue streams for the company. 7.2. The AdSense Program Google AdSense is a program for the website owners (known as publishers) to display Google’s ads on their websites and earn money from Google as a result. To participate in this program, website publishers need to register with Google and be accepted into the program by Google. These ads shown on the publishers’ websites are administered by Case 1:08-cv-03139-RRM -RER Document 51-6 Filed 03/17/10 Page 10 of 47
