Available online at www.sciencedirect.com DIRECT Journal of Mathematical ACADEMIC Psychology PRESS Journal of Mathematical Psychology 47(2003)90-100 http://www.elsevier.com/locate/jmp Tutorial Tutorial on maximum likelihood estimation In Jae Myung* Department of Psychology,Ohio State University,1885 Neil Arenue Mall,Columbus,OH 43210-1222,USA Received 30 November 2001:revised 16 October 2002 Abstract In this paper.I provide a tutorial exposition on maximum likelihood estimation(MLE).The intended audience of this tutorial are researchers who practice mathematical modeling of cognition but are unfamiliar with the estimation method.Unlike least-squares estimation which is primarily a descriptive tool,MLE is a preferred method of parameter estimation in statistics and is an indispensable tool for many statistical modeling techniques,in particular in non-linear modeling with non-normal data.The purpose of this paper is to provide a good conceptual explanation of the method with illustrative examples so the reader can have a grasp of some of the basic principles. C 2003 Elsevier Science (USA).All rights reserved. 1.Introduction (i.e.r2),and root mean squared deviation.LSE,which unlike MLE requires no or minimal distributional In psychological science,we seek to uncover general assumptions,is useful for obtaining a descriptive laws and principles that govern the behavior under measure for the purpose of summarizing observed data, investigation.As these laws and principles are not but it has no basis for testing hypotheses or constructing directly observable,they are formulated in terms of confidence intervals. hypotheses.In mathematical modeling,such hypo- On the other hand.MLE is not as widely recognized theses about the structure and inner working of the among modelers in psychology,but it is a standard behavioral process of interest are stated in terms of approach to parameter estimation and inference in parametric families of probability distributions called statistics.MLE has many optimal properties in estima- models.The goal of modeling is to deduce the form of tion:sufficiency (complete information about the para- the underlying process by testing the viability of such meter of interest contained in its MLE estimator); models. consistency (true parameter value that generated the Once a model is specified with its parameters,and data recovered asymptotically,i.e.for data of suffi- data have been collected,one is in a position to evaluate ciently large samples):efficiency (lowest-possible var- its goodness of fit,that is,how well it fits the observed iance of parameter estimates achieved asymptotically); data.Goodness of fit is assessed by finding parameter and parameterization invariance (same MLE solution values of a model that best fits the data-a procedure obtained independent of the parametrization used).In called parameter estimation. contrast,no such things can be said about LSE.As such, There are two general methods of parameter estima- most statisticians would not view LSE as a general tion.They are least-squares estimation (LSE)and method for parameter estimation,but rather as an maximum likelihood estimation (MLE).The former approach that is primarily used with linear regression has been a popular choice of model fitting in psychology models.Further,many of the inference methods in (e.g.,Rubin,Hinton,Wenzel,1999;Lamberts,2000 statistics are developed based on MLE.For example, but see Usher McClelland,2001)and is tied to many MLE is a prerequisite for the chi-square test,the G- familiar statistical concepts such as linear regression, square test,Bayesian methods,inference with missing sum of squares error,proportion variance accounted for data,modeling of random effects,and many model selection criteria such as the Akaike information *Fax:+614-292-5601 criterion (Akaike,1973)and the Bayesian information E-mail address:myung.1@osu.edu criteria (Schwarz,1978). 0022-2496/03/S-see front matter C 2003 Elsevier Science (USA).All rights reserved. doi:10.1016/S0022-2496(02)00028-7Journal of Mathematical Psychology 47 (2003) 90–100 Tutorial Tutorial on maximum likelihood estimation In Jae Myung* Department of Psychology, Ohio State University, 1885 Neil Avenue Mall, Columbus, OH 43210-1222, USA Received 30 November 2001; revised 16 October 2002 Abstract In this paper, I provide a tutorial exposition on maximum likelihood estimation (MLE). The intended audience of this tutorial are researchers who practice mathematical modeling of cognition but are unfamiliar with the estimation method. Unlike least-squares estimation which is primarily a descriptive tool, MLE is a preferred method of parameter estimation in statistics and is an indispensable tool for many statistical modeling techniques, in particular in non-linear modeling with non-normal data. The purpose of this paper is to provide a good conceptual explanation of the method with illustrative examples so the reader can have a grasp of some of the basic principles. r 2003 Elsevier Science (USA). All rights reserved. 1. Introduction In psychological science, we seekto uncover general laws and principles that govern the behavior under investigation. As these laws and principles are not directly observable, they are formulated in terms of hypotheses. In mathematical modeling, such hypo￾theses about the structure and inner working of the behavioral process of interest are stated in terms of parametric families of probability distributions called models. The goal of modeling is to deduce the form of the underlying process by testing the viability of such models. Once a model is specified with its parameters, and data have been collected, one is in a position to evaluate its goodness of fit, that is, how well it fits the observed data. Goodness of fit is assessed by finding parameter values of a model that best fits the data—a procedure called parameter estimation. There are two general methods of parameter estima￾tion. They are least-squares estimation (LSE) and maximum likelihood estimation (MLE). The former has been a popular choice of model fitting in psychology (e.g., Rubin, Hinton, & Wenzel, 1999; Lamberts, 2000 but see Usher & McClelland, 2001) and is tied to many familiar statistical concepts such as linear regression, sum of squares error, proportion variance accounted for (i.e. r2), and root mean squared deviation. LSE, which unlike MLE requires no or minimal distributional assumptions, is useful for obtaining a descriptive measure for the purpose of summarizing observed data, but it has no basis for testing hypotheses or constructing confidence intervals. On the other hand, MLE is not as widely recognized among modelers in psychology, but it is a standard approach to parameter estimation and inference in statistics. MLE has many optimal properties in estima￾tion: sufficiency (complete information about the para￾meter of interest contained in its MLE estimator); consistency (true parameter value that generated the data recovered asymptotically, i.e. for data of suffi- ciently large samples); efficiency (lowest-possible var￾iance of parameter estimates achieved asymptotically); and parameterization invariance (same MLE solution obtained independent of the parametrization used). In contrast, no such things can be said about LSE. As such, most statisticians would not view LSE as a general method for parameter estimation, but rather as an approach that is primarily used with linear regression models. Further, many of the inference methods in statistics are developed based on MLE. For example, MLE is a prerequisite for the chi-square test, the G￾square test, Bayesian methods, inference with missing data, modeling of random effects, and many model selection criteria such as the Akaike information criterion (Akaike, 1973) and the Bayesian information criteria (Schwarz, 1978). *Fax: +614-292-5601. E-mail address: myung.1@osu.edu. 0022-2496/03/$ - see front matter r 2003 Elsevier Science (USA). All rights reserved. doi:10.1016/S0022-2496(02)00028-7