Regression Method
1 Regression Method
Chapter Topics Multiple regression ● Autocorrelation Slide 2
Slide 2 Chapter Topics • Multiple regression • Autocorrelation
Regression Methods To forecast an outcome (response variable, dependent variable) of a study based on a certain number of factors(explanatory variables, regressors The outcome has to be quantitative but the factors can either by quantitative or categorical Simple regression deals with situations with one explanatory variable, whereas multiple regression tackles case with more than one regressors Slide 3
Slide 3 Regression Methods • To forecast an outcome (response variable, dependent variable) of a study based on a certain number of factors (explanatory variables, regressors). • The outcome has to be quantitative but the factors can either by quantitative or categorical. • Simple Regression deals with situations with one explanatory variable, whereas multiple regression tackles case with more than one regressors
Simple linear regression Collect data Population Random Sampl le Y=bo+bx+e Unknown Relationship s Y =Bo+BX,+E, $ ②$ ②$ $ ②$ Slide 4
Slide 4 Simple Linear Regression – Collect data Population J $ J $ J $ J $ J $ Unknown Relationship Yi Xi i = + + 0 1 Random Sample J $ J $ J $ J $ Y = b +b X + e 0 1
Multiple regression Two or more explanatory variables Multiple linear regression model Y=Bo+BX1+B2X2++B,X,+e where s is the error term and E-N(O, 0) Multiple linear regression equation E()=Bo+B,X1+B2x2+.+BpXp Estimated Multiple linear regression equation Y=b+bx1+b2X2+…+bnX Slide 5
Slide 5 Multiple Regression • Two or more explanatory variables • Multiple linear regression model where is the error term and ~ N(0, 2 ) • Multiple Linear Regression Equation • Estimated Multiple Linear Regression Equation = + + + + + Y X X p X p ... 0 1 1 2 2 E Y = + X + X + + p X p ( ) ... 0 1 1 2 2 Y = b +b X +b X + +bp X p ... ˆ 0 1 1 2 2
Multiple regression Least squares criterion min e2=min(-. The formulae for the regression coefficients bo by b2 b, involve the use of matrix algebra. We will rely on computer software packages to perform the calculations b i represents an estimate of the change in Y corresponding to a one-unit change in X; when all other independent variables are held constant Slide 6
Slide 6 Multiple Regression • Least Squares Criterion • The formulae for the regression coefficients b0 , b1 , b2 , . . . bp involve the use of matrix algebra. We will rely on computer software packages to perform the calculations. • bi represents an estimate of the change in Y corresponding to a one-unit change in Xi when all other independent variables are held constant. = = = − n i i i n i ei Y Y 1 2 1 2 ) ˆ min min (
Multiple regression RZ-SSRSST=1-SSE/SST ° Adjusted R2(R2) R2=1 SSE/n-p n-1 =1-(1-R2) SST/(n-1) n-p-1 where n is the number of observations and p is the number of independent variables The Adjusted r2 compensates for the number of independent variables in the model. It may rise or fall It will fall if the increase in r2 due to the inclusion of additional variables is not enough to offset the reduction in the degrees of freedom Slide 7
Slide 7 Multiple Regression • R2=SSR/SST=1-SSE/SST • Adjusted R2 ( ) where n is the number of observations and p is the number of independent variables • The Adjusted R2 compensates for the number of independent variables in the model. It may rise or fall. • It will fall if the increase in R2 due to the inclusion of additional variables is not enough to offset the reduction in the degrees of freedom. 2 Ra 2 2 /( 1) 1 1 1 (1 ) /( 1) 1 a SSE n p n R R SST n n p − − − = − = − − − − −
Test for Significance Test for Individual Significance: t test ypothesis H0:B1=0 HG:B1≠0 Test statistic Decision rule: reject the null hypothesis at a level of significance if 2),Or p-values a Slide 8
Slide 8 Test for Significance • Test for Individual Significance: t test – Hypothesis – Test statistic – Decision rule: reject the null hypothesis at α level of significance if • , or • p-value < α : 0 : 0 0 = a i i H H i b i s b t = ) 2 ( 1; − − n p t t
Test for Significance Testing for Overall Significance: F test Test whether the multiple regression model as a whole is useful to explain y, i. e, at least one X variable in the regression model is useful to explain Y ypothesis Ho: all slope coefficients are equal to zero e.A1=B2=…=Bp=0) H a: not all Slope coefficients are equal to zero Slide 9
Slide 9 Test for Significance • Testing for Overall Significance: F test – Test whether the multiple regression model as a whole is useful to explain Y, i.e., at least one X– variable in the regression model is useful to explain Y. – Hypothesis H0 : all slope coefficients are equal to zero (i.e. β1 = β2 =…= βp =0) Ha : not all slope coefficients are equal to zero
Test for Significance Testing for Overall Significance: F test Test statistic MSR F SSR/p ∑(-万)/p MSE SSE/(n-p-1) E(-Y,)/(n-p-1 Decision rule: reject null hypothesis if F>F is based on an f distribution with p degrees of freedom in the numerator and n-p-1 degrees of freedom in the denominator or alue< a Slide
Slide 10 Test for Significance • Testing for Overall Significance: F test – Test statistic – Decision rule: reject null hypothesis if • F > Fα is based on an F distribution with p degrees of freedom in the numerator and n – p –1 degrees of freedom in the denominator, or • p-value < α ) ( 1) ˆ ( ) ˆ ( ( 1) 2 2 − − − − = − − = = Y Y n p Y Y p SSE n p SSR p MSE MSR F i i i