832 Joumal of the American Statistical Association,December 1982 estimates with infinitely many nuisance parameters.An- 2.THE EXAMPLE other example,mentioned in Barlow et al.(1972),in- volves estimating a distribution known to be star-shaped The following densities on[-1,1]provide a continuous (i.e,F(λx)≤λF(x)for all0≤λ≤1 and all x such that parameterization between the triangular distribution (when F(x)<1).If the true distribution is uniform on (0,1),the 0 =0)and the uniform (when 0 =1)with parameter maximum likelihood estimate converges to F(x)=x2 on space [0,1]: (0,1). The central theorem on the global consistency of max- fx|0)=(1-0) imum likelihood estimates is due to Wald (1949).This theorem gives conditions under which the maximum like- ×1A闲+2-1.， lihood estimates and approximate maximum likelihood estimates(values of 0 that yield a value of the likelihood where A represents the interval [0 -5(0),0 +5(0)], function that comes within a fixed fraction c,0<c<1, 8(0)is a continuous decreasing function of 0 with 8(0) of the maximum)are strongly consistent.Other formu-1 and 0<8(0)s1 -0 for 0<0<1,and Is(x) lations of Wald's Theorem and its variants may be found represents the indicator function of the set S.For 0= in LeCam(1953),Kiefer and Wolfowitz (1956),Bahadur 1,f(x0)is taken to be(x).It is assumed that (1967),and Perlman(1972).A particularly informative independent identically.distributed observations X1,X2, exposition of the problem may be found in Chapter 9 of ..are available from one of these distributions.Then Bahadur(1971). conditions 1 through 4 of the introduction are satisfied. The example contained in Section 2 has the following These conditions imply the existence of a maximum like- properties: lihood estimate for any sample size because a continuous function defined on a compact set achieves its maximum 1.The parameter space is a compact interval on the on that set. real line. 2.The observations are independent identically dis- Theorem.Let 0 denote a maximum likelihood estimate tributed according to a distribution F(x|0)for some 0 of 0 based on a sample of size n.If 8(0)-0 sufficiently ∈0. fast as 0-1 (how fast is noted in the proof),then 0 3.Densities f(x0)with respect to some o-finite 1 with probability I as n,whatever be the true value of0∈[0，ll. measure (Lebesgue measure in the example)exist and are continuous in 0 for all x. Proof.Continuity of f(x 0)in 0 and compactness of 4.(Identifiability)If 00',then F(x 0)is not iden- implies that the maximum likelihood estimate,0,some tical to F(x 0'). value of 0 that maximizes the log-likelihood function It is seen that whatever the true value,0o,of the pa- ln(0)=∑log f() =1 rameter,the maximum likelihood estimate,which exists because of 1,2,and 3,converges almost surely to a fixed exists.Since for 0<1 value (I in the example)independent of 0o. Example 2 of Bahadur(1958)(Example 9.2 of Bahadur f610)s1-9+0_1 8(0) 1971)also has the properties stated previously,and the 26+五， example of Section 2 may be regarded as a continuous we have that for each fixed positive number a<1, version of Bahadur's example.However,the distribu- tions in Bahadur's example seem rather artificial and the maxIn(0)≤6a 十是<∞ 0≤0≤a1 parameter space is countable with a single limit point The example presented here is more natural;the sample since 8(0)is decreasing.We complete the proof by show- space is [-1,+1],the parameter space is [0,1],and the ing that whatever be the true value of 0, distributions are familiar,each being a mixture of the 1 uniform distribution and a triangular one. m)→。with probablityne In Section 3,it is seen how to modify the example using beta distributions so that Cramer's conditions are satis- provided 8(0)->0 sufficiently fast as 0-1,since then fied.This gives an example in which asymptotically ef- 0,will eventually be greater than a for any preassigned ficient estimates exist and may be found by improving a<I.Let Mn=max{Xi,.·,Xn}.Then M→1with any convenient 0(Vn)-consistent estimate by scoring, probability one whatever be the true value of 0,and since and yet the maximum likelihood estimate exists and even- 0<M<1 with probability one, tually satisfies the likelihood equation but converges to a fixed point with probability 1 no matter what the true max1ln(0)≥ln(Mn) 0≤0s1n n value of the parameter happens to be.Such an example was announced by LeCam in the discussion of Berkson's (1980)papr. log 2 n n832 Joumal of the American Statistical Association, December 1982 estimates with infinitely many nuisance parameters. An￾other example, mentioned in Barlow et al. (1972), in￾volves estimating a distribution known to be star-shaped (i.e., F(Ax) s XF(x) for all 0 < A s 1 and all x such that F(x) < 1). If the true distribution is uniform on (0, 1), the maximum likelihood estimate converges to F(x) = X2 on (0, 1). The central theorem on the global consistency of max￾imum likelihood estimates is due to Wald (1949). This theorem gives conditions under which the maximum like￾lihood estimates and approximate maximum likelihood estimates (values of 0 that yield a value of the likelihood function that comes within a fixed fraction c, 0 < c < 1, of the maximum) are strongly consistent. Other formu￾lations of Wald's Theorem and its variants may be found in LeCam (1953), Kiefer and Wolfowitz (1956), Bahadur (1967), and Perlman (1972). A particularly informative exposition of the problem may be found in Chapter 9 of Bahadur (1971). The example contained in Section 2 has the following properties: 1. The parameter space 0 is a compact interval on the real line. 2. The observations are independent identically dis￾tributed according to a distribution F(x I 0) for some 0 E0. 3. Densities f(x I 0) with respect to some ur-finite measure (Lebesgue measure in the example) exist and are continuous in 0 for all x. 4. (Identifiability) If 0 # 0', then F(x I 0) is not iden￾tical to F(x I 0'). It is seen that whatever the true value, 00, of the pa￾rameter, the maximum likelihood estimate, which exists because of 1, 2, and 3, converges almost surely to a fixed value (1 in the example) independent of 00. Example 2 of Bahadur (1958) (Example 9.2 of Bahadur 1971) also has the properties stated previously, and the example of Section 2 may be regarded as a continuous version of Bahadur's example. However, the distribu￾tions in Bahadur's example seem rather artificial and the parameter space is countable with a single limit point. The example presented here is more natural; the sample space is [- 1, + 1], the parameter space is [0, 1], and the distributions are familiar, each being a mixture of the uniform distribution and a triangular one. In Section 3, it is seen how to modify the example using beta distributions so that Cramer's conditions are satis￾fied. This gives an example in which asymptotically ef￾ficient estimates exist and may be found by improving any convenient O(\/)-consistent estimate by scoring, and yet the maximum likelihood estimate exists and even￾tually satisfies the likelihood equation but converges to a fixed point with probability 1 no matter what the true value of the parameter happens to be. Such an example was announced by LeCam in the discussion of Berkson's (1980) paper. 2. THE EXAMPLE The following densities on [ - 1, 1] provide a continuous parameterization between the triangular distribution (when 0 = 0) and the uniform (when 0 = 1) with parameter space 0 =[0, 1]: f(x I 0)=(1 - 0)5(0) I I - 0) X IA(X) + 2 where A represents the interval [0 - 8(0), 0 + 8(0)], 8(0) is a continuous decreasing function of 0 with 8(0) = 1 and 0 < 8(0) c 1 - 0 for 0 < 0 < 1, and Is(x) represents the indicator function of the set S. For 0 = 1, f(x I 0) is taken to be ' I[l l](x). It is assumed that independent identically distributed observations X1, X2, ... are available from one of these distributions. Then conditions 1 through 4 of the introduction are satisfied. These conditions imply the existence of a maximum like￾lihood estimate for any sample size because a continuous function defined on a compact set achieves its maximum on that set. Theorem. Let 0, denote a maximum likelihood estimate of 0 based on a sample of size n. If 8(0) -O 0 sufficiently fast as 0 --*1 (how fast is noted in the proof), then 0,n --*1 with probability 1 as n c m, whatever be the true value of 0 E [0, 1]. Proof. Continuity of f(x I 0) in 0 and compactness of 0 implies that the maximum likelihood estimate, 0,n, some value of 0 that maximizes the log-likelihood function n ln (0) = log f(xi 0) i= 1 exists. Since for 0 < 1 1 -0 0 1 f(x I ) '5(0) 2 5(0) 2 we have that for each fixed positive number o t 1, 1 1 max o -lIn(f0) <- + 'I 0 o--cot n b((o) since 8(0) is decreasing. We complete the proof by show￾ing that whatever be the true value of 0, maxI l() ---> ?O with probability one o-o-i n provided 8(0) -> 0 sufficiently fast as 0 1-> , since then On will eventually be greater than a for any preassigned a < 1. Let Mn = max{X ,... , Xn}. Then Mn,-> 1 with probability one whatever be the true value of 0, and since O < Mn < 1 with probability one, max -I ln(0) - I ln(Mn) O-OI fl fn _n-i1 Mn, 1 1 -Mn, -log 2y+-nlog 5(M,,)