1 Model Selection in Linear Regression Basic Ideas "Model Selection"in linear regression attempts to suggest the best model for a given purpose. Recall that the two main purposes of linear regression models are: Estimate the effect of one or more covariates while adjusting for the possible confounding effects of other variables. Prediction of the outcome for the next set of similar subjects Also,we must again recall the famous quite from George Box: “All models are wrong,but some are useful..” The above quote is important here for two reasons: If we are looking to estimate effects,it reminds us that we do not need to find a single "final model",and draw all conclusions from that model alone.Since no single model is correct,we are free to estimate a large number of different models,and draw overall conclusions about the effect of a variable,confounding, and so on,from that model. So,in some sense,model selection is not relevant to estimating effects,where we will almost always want to look at more that one model,sometimes perhaps as many as 20 or 30 models will be helpful in drawing overall conclusions. We have already seen examples of this when we investigated confounding.Nev- ertheless,it turns out that a Bayesian program for model selection will,because of the way its output is formatted,be very useful for investigating effects.Thus, in learning a program for model selection,we will simultaneously see a way to easily draw overall conclusions for parameter estimation in the face of confound- ing.1 Model Selection in Linear Regression Basic Ideas “Model Selection” in linear regression attempts to suggest the best model for a given purpose. Recall that the two main purposes of linear regression models are: • Estimate the effect of one or more covariates while adjusting for the possible confounding effects of other variables. • Prediction of the outcome for the next set of similar subjects Also, we must again recall the famous quite from George Box: “All models are wrong, but some are useful.” The above quote is important here for two reasons: • If we are looking to estimate effects, it reminds us that we do not need to find a single “final model”, and draw all conclusions from that model alone. Since no single model is correct, we are free to estimate a large number of different models, and draw overall conclusions about the effect of a variable, confounding, and so on, from that model. So, in some sense, model selection is not relevant to estimating effects, where we will almost always want to look at more that one model, sometimes perhaps as many as 20 or 30 models will be helpful in drawing overall conclusions. We have already seen examples of this when we investigated confounding. Nev￾ertheless, it turns out that a Bayesian program for model selection will, because of the way its output is formatted, be very useful for investigating effects. Thus, in learning a program for model selection, we will simultaneously see a way to easily draw overall conclusions for parameter estimation in the face of confound￾ing