as a function of for fixed ybs and fixed The expectation is actually the conditional expectation of the complete-data log-likelihood,conditional on yobs The tth M-step then finds (1)to maximize (:0()i.e.,finds1)such that Q(0+1:0)≥Q(0:0), for all ee.To verify that this iteration produces a sequence of iterates that converges to a maximum of lobs(0;yobs),first note that by taking conditional expectation of both sides of Cobs(0;yobs)=e(e;y)-log f2(ymislyobsi0), over the distribution ofy given y at the current estimate()can be expressed in the form lobs(:yb)0:y)f(ylyb)dy-log fa(ymislybf(ylybsidy E[(;y)lyt]-Ego[log f2(ymislybs)lyobs] = Q(0:0)-H(0:09) where Q(0;0())is as defined earlier and H(:0=Egllog f2(ymislyobsi )lybs] The following Lemma will be useful for proving a main result that the sequence of iterates ()resulting from EM algrithm will converge at least to a local maximum of bs(yobs). Lemma:For anyθ∈Θ， H(0:0)≤H(0):0). Theorem:The EM algorithm increases lobs(0;yobs)at each iteration,that is, obs(;yobs)lobs(;yobs) with equality if and only if Q(0+1:0)=Q(00:0o). This Theorem implies that increasing Q(0;0()at each step leads to maximizing or at least constantly increasing lobs(0;yobs). Although the general theory of EM applies to any model,it is particularly useful when the complete data y are from an exponential family since,as seen in examples,in such cases the E-step reduces to finding the conditional expectation of the complete-data sufficient sta- tistics,and the M-step is often simple.Nevertheless,even when the complete data y are from an exponential family,there exist a variety of important applications where complete-data maximum likelihood estimation itself is complicated;for example,see Little Rubin(1987) on selection models and log-linear models,which generally require iterative M-steps. 12as a function of θ for fixed yobs and fixed θ (t) . The expectation is actually the conditional expectation of the complete-data log-likelihood, conditional on yobs. The t th M-step then finds θ (t+1) to maximize Q(θ; θ (t) ) i.e., finds θ (t+1) such that Q(θ (t+1); θ (t) ) ≥ Q(θ; θ (t) ), for all θ ∈ Θ . To verify that this iteration produces a sequence of iterates that converges to a maximum of `obs(θ; yobs), first note that by taking conditional expectation of both sides of `obs(θ; yobs) = `(θ; y) − log f2(ymis|yobs; θ), over the distribution of y given yobs at the current estimate θ (t) , `obs(θ; yobs) can be expressed in the form `obs(θ; yobs) = Z `(θ; y)f(y|yobs; θ (t) )dy − Z log f2(ymis|yobs; θ)f(y|yobs; θ (t) )dy = Eθ (t) [`(θ; y)|yobs] − Eθ (t) [log f2(ymis|yobs; θ)|yobs] = Q(θ; θ (t) ) − H(θ; θ (t) ) where Q(θ; θ (t) ) is as defined earlier and H(θ; θ (t) ) = Eθ (t) [log f2(ymis|yobs; θ)|yobs]. The following Lemma will be useful for proving a main result that the sequence of iterates θ (t) resulting from EM algrithm will converge at least to a local maximum of `obs(θ; yobs). Lemma: For any θ ∈ Θ, H(θ; θ (t) ) ≤ H(θ (t) ; θ (t) ). Theorem: The EM algorithm increases `obs(θ; yobs) at each iteration, that is, `obs(θ (t+1); yobs) ≥ `obs(θ (t) ; yobs) with equality if and only if Q(θ (t+1); θ (t) ) = Q(θ (t) ; θ (t) ). This Theorem implies that increasing Q(θ; θ (t) ) at each step leads to maximizing or at least constantly increasing `obs(θ; yobs). Although the general theory of EM applies to any model, it is particularly useful when the complete data y are from an exponential family since, as seen in examples, in such cases the E-step reduces to finding the conditional expectation of the complete-data sufficient sta￾tistics, and the M-step is often simple. Nevertheless, even when the complete data y are from an exponential family, there exist a variety of important applications where complete-data maximum likelihood estimation itself is complicated; for example, see Little & Rubin (1987) on selection models and log-linear models, which generally require iterative M-steps. 12