CHAPTER 4 FINITE-SAMPLE PROPERTIES OF THE LSE Chapter 4 Finite-Sample properties of the LSE Finnite-sample the n is assumed to be fixed normal dist n assumed Large-sample theory n is sent to oo, general distn assumed 4.1 Unbiasedness Write (XX Xy=(XX)X(XB+e) 6+(XXX The E(bIX)=B+E(X'X)XEl B+(X'X)XE(EX) Therefore E Er E[X= Er[BI center of the true parameter distribution b vect or 4. 2 The variance of the lse and the gaussMarkov theorem The Ols est imator of B is b=(X XX (X'X)X' is an k x n vector. Thus each element of b can be written as a linear combination of 3, .. 3. We call b a linear estimat or for this reason The covariance matrix of b is V (bX) E[(b-B)(b-B)IX E(X'X)XEEX(XX)IX (X'XXE(EEXX(XX) (xX)-x(21)X(Xx) Consider an arbitrary linear est imator of B, bo= Cy where C is a k x n matrix. For bo to be unbiased, we should have E(CXB+CEX)CHAPTER 4 FINITE—SAMPLE PROPERTIES OF THE LSE 1 Chapter 4 Finite—Sample Properties of the LSE Finnite—sample theory : n is assumed to be fixed, normal distn assumed Large—sample theory : n is sent to ∞, general distn assumed 4.1 Unbiasedness Write b = (X ′X) −1 X ′ y = (X ′X) −1 X ′ (Xβ + ε) = β + (X ′X) −1 X ′ ε. Then E (b|X) = β + E
(X ′X) −1 X ′ ε|X = β + (X ′X) −1 XE (ε|X) = β. Therefore E (b) = Ex {E [b|X]} = Ex [β] = β.

center of the true parameter distribution b vector 4.2 The variance of the LSE and the Gauss—Markov theorem The OLS estimator of β is b = (X ′X) −1 X ′ y. (X′X) −1 X′ is an k × n vector. Thus each element of b can be written as a linear combination of y1, · · · , yn. We call b a linear estimator for this reason. The covariance matrix of b is V ar (b|X) = E (b − β) (b − β) ′ |X  = E
(X ′X) −1 X ′ εε′X (X ′X) −1 |X = (X ′X) −1 X ′E (εε′ |X) X (X ′X) −1 = (X ′X) −1 X ′  σ 2 I  X (X ′X) −1 = σ 2 (X ′X) −1 Consider an arbitrary linear estimator of β, b0 = Cy where C is a k × n matrix. For b0 to be unbiased, we should have E (Cy|X) = E (CXβ + Cε|X) = β.