An Inconsistent Maximum Likelihood Estimate THOMAS S.FERGUSON* An example is given of a family of distributions on [-1,then there exists a sequence of roots,0,of the likelihood 1]with a continuous one-dimensional parameterization equation. that joins the triangular distribution(when 0 =0)to the uniform (when 0=1),for which the maximum likelihood 31ogLn0)=0. a01 estimates exist and converge strongly to 0 1 as the sample size tends to infinity,whatever be the true value that converges in probability to 0o as no.Moreover, of the parameter.A modification that satisfies Cramer's any such sequence 0 is asymptotically normal and conditions is also given. asymptotically efficient.It is known that Cramer's theo- KEY WORDS:Maximum likelihood estimates;Incon- rem extends to the multiparameter case. To emphasize the point that this is a local result and sistency;Asymptotic efficiency;Mixtures. may have nothing to do with maximum likelihood esti- 1.INTRODUCTION mation,we consider the following well-known example, a special case of some quite practical problems mentioned There are many examples in the literature of estimation recently by Quandt and Ramsey (1978).Let the density problems for which the maximum likelihood principle f(x|0)be a mixture of two normals,N(0,1)and N(, does not yield a consistent sequence of estimates,notably o2),with mixing parameter, Neyman and Scott(1948),Basu (1955),Kraft and LeCam (1956),and Bahadur(1958).In this article a very simple f(x|μ，o)=克p(x)+是p(x-u)o)/o, example of inconsistency of the maximum likelihood where o is the density of the standard normal distribution, method is presented that shows clearly one danger to be and the parameter space is ={(u,o):o>0.It is clear wary of in an otherwise regular-looking situation.A re- that for any given sample,X1,...,Xn,from this density cent article by Berkson(1980)followed by a lively dis- the likelihood function can be made as large as desired cussion shows that there is still interest in these problems.by taking =X,say,and o sufficiently small.Never- The discussion in this article is centered on a sequence theless,Cramer's conditions are satisfied and so there of independent,identically distributed,and,for the sake exists a consistent asymptotically efficient sequence of of convenience,real random variables,X1,X2,..., roots of the likelihood equation even though maximum distributed according to a distribution,F(x0),for some likelihood estimates do not exist. 0 in a fixed parameter space It is assumed that there A more disturbing example is given by Kraft and is a o-finite measure with respect to which densities,f(x LeCam(1956),in which Cramer's conditions are satis- 0),exist for all 0 e 0.The maximum likelihood estimate fied,the maximum likelihood estimate exists,is unique, of 0 based on X1,...,Xn is a value,0n(x1,...,x)of and satisfies the likelihood equation,but is not consistent. 0∈⊙，if any,that maximizes the likelihood function In such examples,it is possible to find the asymptotically efficient sequence of roots of the likelihood equation by Ln(0)=Πfx|0) first finding a consistent extimate and then finding the i-1 closest root or improving by the method of scoring as in The maximum likelihood method of estimation goes back Rao(1965).See Lehmann(1980)for a discussion of these to Gauss,Edgeworth,and Fisher.For historical points, problems. see LeCam (1953)and Edwards (1972).For a general Other more practical examples of inconsistency in the survey of the area and a large bibliography,see Norton maximum likelihood method involve an infinite number (1972). of parameters.Neyman and Scott (1948)show that the The starting point of our discussion is the theorem of maximum likelihood estimate of the common variance of Cramer (1946,p.500),which states that under certain a sequence of normal populations with unknown means regularity conditions on the densities involved,if 0 is real based on a fixed sample size k taken from each population valued and if the true value 0o is an interior point of converges to a value lower than the true value as the number of populations tends to infinity.This example led directly to the paper of Kiefer and Wolfowitz (1956)on Thomas S.Ferguson is Professor,Department of Mathematics, the consistency and efficiency of the maximum likelihood University of California,Los Angeles,CA 90024.Research was sup- ported in part by the National Science Foundation under Grant MCS77- 2121.The author wishes to acknowledge the help of an exceptionally Journal of the American Statistical Association good referee whose very detailed comments benefited this article December 1982,Volume 77,Number 380 substantially. Theory and Methods Section 831An Inconsistent Maximum Likelihood Estimate THOMAS S. FERGUSON* An example is given of a family of distributions on [ - 1, 1] with a continuous one-dimensional parameterization that joins the triangular distribution (when 0 = 0) to the uniform (when 0 = 1), for which the maximum likelihood estimates exist and converge strongly to 0 = 1 as the sample size tends to infinity, whatever be the true value of the parameter. A modification that satisfies Cramer's conditions is also given. KEY WORDS: Maximum likelihood estimates; Incon￾sistency; Asymptotic efficiency; Mixtures. 1. INTRODUCTION There are many examples in the literature of estimation problems for which the maximum likelihood principle does not yield a consistent sequence of estimates, notably Neyman and Scott (1948), Basu (1955), Kraft and LeCam (1956), and Bahadur (1958). In this article a very simple example of inconsistency of the maximum likelihood method is presented that shows clearly one danger to be wary of in an otherwise regular-looking situation. A re￾cent article by Berkson (1980) followed by a lively dis￾cussion shows that there is still interest in these problems. The discussion in this article is centered on a sequence of independent, identically distributed, and, for the sake of convenience, real random variables, Xl, X2, . . distributed according to a distribution, F(x I 0), for some 0 in a fixed parameter space 0. It is assumed that there is a ur-finite measure with respect to which densities, f(x I 0), exist for all 0 E 0. The maximum likelihood estimate of 0 based on X1,. .., X is a value, On(x, . .., xn) of 0 E 0, if any, that maximizes the likelihood function n Ln (0) = H f(xi I 0) i = I The maximum likelihood method of estimation goes back to Gauss, Edgeworth, and Fisher. For historical points, see LeCam (1953) and Edwards ('972). For a general survey of the area and a large bibliography, see Norton (1972). The starting point of our discussion is the theorem of Cramer (1946, p. 500), which states that under certain regularity conditions on the densities involved, if 0 is real valued and if the true value 00 is an interior point of 0, * Thomas S. Ferguson is Professor, Department of Mathematics, University of California, Los Angeles, CA 90024. Research was sup￾ported in part by the National Science Foundation under Grant MCS77- 2121. The author wishes to acknowledge the help of an exceptionally good referee whose very detailed comments benefited this article substantially. then there exists a sequence of roots, 0), of the likelihood equation, -log Lnf(O) = 0, ao that converges in probability to Oo as n m. Moreover, any such sequence 0,, is asymptotically normal and asymptotically efficient. It is known that Cramer's theo￾rem extends to the multiparameter case. To emphasize the point that this is a local result and may have nothing to do with maximum likelihood esti￾mation, we consider the following well-known example, a special case of some quite practical problems mentioned recently by Quandt and Ramsey (1978). Let the density f(x I 0) be a mixture of two normals, N(O, 1) and N(i, (c2), with mixing parameter 2, f(x I P, a) = 2 p(x) + 2 ((- )Io)I, where 'p is the density of the standard normal distribution, and the parameter space is 0 = {(p, r): u > 0}. It is clear that for any given sample, XI, . . , X, from this density the likelihood function can be made as large as desired by taking 11 = XI, say, and r sufficiently small. Never￾theless, Cramer's conditions are satisfied and so there exists a consistent asymptotically efficient sequence of roots of the likelihood equation even though maximum likelihood estimates do not exist. A more disturbing example is given by Kraft and LeCam (1956), in which Cramer's conditions are satis￾fied, the maximum likelihood estimate exists, is unique, and satisfies the likelihood equation, but is not consistent. In such examples, it is possible to find the asymptotically efficient sequence of roots of the likelihood equation by first finding a consistent extimate and then finding the closest root or improving by the method of scoring as in Rao (1965). See Lehmann (1980) for a discussion of these problems. Other more practical examples of inconsistency in the maximum likelihood method involve an infinite number of parameters. Neyman and Scott (1948) show that the maximum likelihood estimate of the common variance of a sequence of normal populations with unknown means based on a fixed sample size k taken from each population converges to a value lower than the true value as the number of populations tends to infinity. This example led directly to the paper of Kiefer and Wolfowitz (1956) on the consistency and efficiency of the maximum likelihood ? Journal of the American Statistical Association December 1982, Volume 77, Number 380 Theory and Methods Section 831