CHAPTER 4 FINITE-SAMPLE PROPERTIES OF THE LSE 4.3 Estimating the variance of the least squares estimator E(E), a natural estimat or of 02is But t his estimator is biased as discussed now Since Th FleeT Etr(MEEIX (ME(EELX)) tr(M)=tr In-X(X'X) tr(In)-tr((XX)X'X (In)-tr(Ik Therefore Elem=(n-k) and an unbiased estimator of a2 ot The estimator s" is also unbiased unconditionally, because E[2]=E{E[sx}=E(o2)=a sing s?, we obtain an estimator of Var[IX The standard error of the estimat or bk isCHAPTER 4 FINITE—SAMPLE PROPERTIES OF THE LSE 3 4.3 Estimating the variance of the least squares estimator Since σ 2 = E (ε 2 i ), a natural estimator of σ 2 is σˆ 2 = 1 n n i=1 e 2 i = 1 n e ′ e. But this estimator is biased as discussed now. Since e = My = M (Xβ + ε) = Mε, e ′ e = ε ′Mε. Thus E [e ′ e|X] = E [ε ′Mε|X] = E [tr (ε ′Mε)|X] = E [tr (Mεε′ )|X] = tr (ME (εε′ |X)) = tr  Mσ2 I  = σ 2 tr (M). But tr (M) = tr
In − X (X ′X) −1 X ′ = tr (In) − tr  (X ′X) −1 X ′X  = tr (In) − tr (IK) = n − K. Therefore, E [e ′ e|M] = (n − K) σ 2 . and an unbiased estimator of σ 2 is s 2 = e ′ e n − K , not σˆ 2 . The estimator s 2 is also unbiased unconditionally, because E s 2  = Ex E s 2 |X  = Ex  σ 2  = σ 2 . Using s 2 , we obtain an estimator of V ar [b|X] V ar  [b|X] = s 2 (X ′X) −1 . The standard error of the estimator bk is s 2 (X′X) −1  kk