The Simple regression model y=Bo+Bx+u Economics 20- Prof anderson
Economics 20 - Prof. Anderson 1 The Simple Regression Model y = b0 + b1 x + u
Some Terminology o In the simple linear regression model where y=Po+ Bx+ u, we typically refer to y as the Dependent variable, or a Left-Hand Side variable. or a Explained Variable, or Regressand Economics 20- Prof anderson
Economics 20 - Prof. Anderson 2 Some Terminology In the simple linear regression model, where y = b0 + b1 x + u, we typically refer to y as the ◼ Dependent Variable, or ◼ Left-Hand Side Variable, or ◼ Explained Variable, or ◼ Regressand
Some terminology, cont o In the simple linear regression of y on X we typically refer to x as the Independent variable, or Right-Hand Side variable or a Explanatory variable, or Regressor, or a Covariate or a Control variables Economics 20- Prof anderson
Economics 20 - Prof. Anderson 3 Some Terminology, cont. In the simple linear regression of y on x, we typically refer to x as the ◼ Independent Variable, or ◼ Right-Hand Side Variable, or ◼ Explanatory Variable, or ◼ Regressor, or ◼ Covariate, or ◼ Control Variables
A Simple assumption o The average value of u, the error term, in the population is 0. That is ◆E(l)=0 This is not a restrictive assumption, since we can al ways use Bo to normalize E(u) to O Economics 20- Prof anderson 4
Economics 20 - Prof. Anderson 4 A Simple Assumption The average value of u, the error term, in the population is 0. That is, E(u) = 0 This is not a restrictive assumption, since we can always use b0 to normalize E(u) to 0
Zero conditional mean We need to make a crucial assumption about how u and x are related o We want it to be the case that knowing something about x does not give us any information about u, so that they are completely unrelated. That is, that DE(ux=e(u=o, which implies ◆E(vx)=B0+Bx Economics 20- Prof anderson 5
Economics 20 - Prof. Anderson 5 Zero Conditional Mean We need to make a crucial assumption about how u and x are related We want it to be the case that knowing something about x does not give us any information about u, so that they are completely unrelated. That is, that E(u|x) = E(u) = 0, which implies E(y|x) = b0 + b1 x
EGlx)as a linear function of x, where for any x the distribution of y is centered about E(x) fy) E(x)=Bo+ Bx x Economics 20- Prof anderson 6
Economics 20 - Prof. Anderson 6 . . x1 x2 E(y|x) as a linear function of x, where for any x the distribution of y is centered about E(y|x) E(y|x) = b0 + b1x y f(y)
Ordinary least squares o Basic idea of regression is to estimate the population parameters from a sample o Let ((,yi: i=1,.,n) denote a random sample of size n from the population o For each observation in this sample, it will be the case that y1=B0+Bx1+1 Economics 20- Prof anderson 7
Economics 20 - Prof. Anderson 7 Ordinary Least Squares Basic idea of regression is to estimate the population parameters from a sample Let {(xi ,yi ): i=1, …,n} denote a random sample of size n from the population For each observation in this sample, it will be the case that yi = b0 + b1 xi + ui
Population regression line, sample data points and the associated error terms ECx)=Bo+ Bx 3 2 Economics 20- Prof anderson 8
Economics 20 - Prof. Anderson 8 . . . . y4 y1 y2 y3 x1 x2 x3 x4 } } { { u1 u2 u3 u4 x y Population regression line, sample data points and the associated error terms E(y|x) = b0 + b1x
Deriving ols estimates To derive the ols estimates we need to realize that our main assumption ofe(ulx) E(u=0 also impli that ◆Cov(x,l)=E(xn)=0 e Why? Remember from basic probability that Cov(X,Y=E(XY)-E(XE(Y) Economics 20- Prof anderson 9
Economics 20 - Prof. Anderson 9 Deriving OLS Estimates To derive the OLS estimates we need to realize that our main assumption of E(u|x) = E(u) = 0 also implies that Cov(x,u) = E(xu) = 0 Why? Remember from basic probability that Cov(X,Y) = E(XY) – E(X)E(Y)
Deriving ols continued o We can write our 2 restrictions just in terms of x, y, Boand B,, since u=y-Bo-Bix ◆F(y-BD-Bx)=0 ◆E[x(y-B-B1x)=0 These are called moment restrictions Economics 20- Prof anderson 10
Economics 20 - Prof. Anderson 10 Deriving OLS continued We can write our 2 restrictions just in terms of x, y, b0 and b1 , since u = y – b0 – b1 x E(y – b0 – b1 x) = 0 E[x(y – b0 – b1 x)] = 0 These are called moment restrictions
