16.322 Stochastic Estimation and Control, Fall 2004 Prof vander velde Y=a+bX YY=aX+br2 y2=a2+abx +b2X aX+bx2-X(a+bX (a2+2abX+62X-a2-2abX-b2r) bIX-X 6(x-x Degree of linear Dependence At every observation, or trial or the experiment, we observe a pair x, y. We ask how well can we approximate y as a linear function of X? =a+bX Choose a and b to minimize the mean squared error, &, in the approximation. Y=a+bX-y 22=a2tb2x2+y2+2abX-2bXY-2ay a2+62x2+2+2abx-2bXY-2ay Page 2 of 9x 2 16.322 Stochastic Estimation and Control, Fall 2004 Prof. Vander Velde Y = + a bX XY = a X + b X 2 Y 2 2 2 = a + 2abX + b X 2 aX + b X 2 − X (a + bX ) ρ = 2 2 2 2 2 2 2 σ (a + 2abX + b X − a − 2abX − b X ) bX 2 − X 2 ) 2 = ( = bσ x = ±1 sgn( = b) 2 22 ( σ bX 2 − X bσ x 2 x ) Degree of Linear Dependence At every observation, or trial or the experiment, we observe a pair x,y. We ask: how well can we approximate Y as a linear function of X? Y = + a bX approx . 2 Choose a and b to minimize the mean squared error, ε , in the approximation. ε = Y approx . −Y = + a bX −Y 2 2 2 2 ε = a + b X + Y + 2abX − 2bXY − 2aY 2 2 Y 2 = a + b X 2 + + 2abX − 2b XY − 2aY Page 2 of 9