Implementing Statistical Criteria to Select Return Forecasting Models Table 4 displays the results.Exploiting the information in the cross sec- tion of stock returns,we used SUR regression to improve the estimate of the t-ratios.For comparison,other popular out-of-sample statistics [the adjusted R2 and root mean square prediction error of the out-of-sample regressions, that is,of Equation (3)]are provided as well.In order to put the root mean square prediction error of the out-of-sample regressions [see Equation(3)] into perspective,we also report the root mean square excess stock return over the estimation and testing samples (in brackets). Table 4 generates the following initial reactions about out-of-sample predictability. For models based on the adjusted R2,an enormous disparity emerges between in-sample and out-of-sample adjusted R2.For the U.S.stock market,for instance,the in-sample adjusted R2 is 8.3%.It drops to 0.5%out of sample.? The in-sample adjusted R2s of models selected on the basis of BIC, FIC,PIC,PLS,and PLS-MDC are a somewhat more reliable indicator of the out-of-sample goodness-of-fit.For FIC and PIC,for instance,the out-of-sample adjusted R2 is at least as high as the in-sample adjusted R2 in 6 of 14 cases.8 A comparison of the out-of-sample root mean square prediction error and the root mean square excess return reveals an across-the-board dismal performance of the models selected by the formal criteria.As a matter of fact,the decision by BIC not to include predictors at all in many cases is corroborated by this finding. Our model selection criterion,PLS-MDC,overfits as much as FIC or PIC. All this indicates poor out-of-sample predictability.A formal test of external validity of the retained models,however,can be obtained from the t-ratios in Table 4.Except for Japan,we find that none of the t-ratios of the slope coefficient in the SUR regression of out-of-sample outcomes onto predictions are significant!Consequently,we fail to find that models chosen on the basis of formal selection criteria have any external validity. Table 5 summarizes the tests by reporting F-tests of the joint significance of the slope coefficients in an SUR regression of out-of-sample excess stock The poor out-of-sample record in terms of R2is actually a bit exaggerated,because the out-of-sample use of this measure supposes that the forecaster knew heforehand in what direction the selected model would be biased. When BIC decides not to include any predictors at all,the in-sample and out-of-sample adjusted Rs are set equal to zero. When running simple OLS (as opposed to SUR),the t-statistics are significant (at the 5%level)only for four stock markets:France (BIC,FIC.PIC,and PLS).Japan (BIC).The Netherlands (BIC.PLS,and PLS-MDC),and the United Kingdom (AIC and PIC). 417