CAN STOCK MARKET FORECASTERS FORECAST? a paper read before a joint meeting of the Econometric Society and the American Statistical Association, Cincinnati, Ohio, December 31, 1932 INTRODUCTION THIS paper presents results of analyses of the forecasting efforts of 45 professional agencies which have attempted, either to select specific common stocks which should prove superior in investment merit to the general run of equities, or to predict the future movements of the stock market itself. The paper falls into two main parts. The first deals with the attempts of two groups, 20 fire insurance companies and 16 finan al services, to foretell which specific securities would prove most profitable. The second part deals with the efforts of 25 financial publi- cations to foretell the future course of the stock market. various sta- tistical tests of these results are given These investigations were instituted five years ago as a means of testing the success of applied economies in the investment field. It seemed a plausible assumption that if we could demonstrate the exist- ence in individuals or organizations of the ability to foretell the elusive fluctuations, either of particular stocks, or of stocks in general, this might lead to the identification of economic theories or statistical prac- tices whose soundness had been established by successful prediction The forecasters include well-known organizations in the diffe fields represented, many of which are large and well financed, employ ing economists and statisticians of unquestioned ability. The names of these organizations are omitted, since their publication would be likely to invite wholesale controversy over the interpretation of their records Some of the forecasters seem to have taken a page from the book of the Delphic Oracle expressing their prophecies in terms susceptible of more than one construction. It would frequently be possible, therefore, for an editor, after the event, to present a plausible challenge of our interpretation. Most of the forecasts appear through the medium of weekly publications and each of these has been read and recorded on the day it became available to us, which in practically every case was before the event. In this way certain possible elements of bias have been eliminated. It was impossible that hindsight could infuence our judgment, either in the selection of publications for analysis or in the interpretations placed on their forecasts. In the case of the fire insur ance companies, however, the analyses were made annually, based the transactions reported in Kimber' s Record of Insurance Company Security Purchases. The companies were selected as fairly representa 309
310 ECONOMETRICA tive of their class. The analysis of the 26-year forecasting record of William Peter Hamilton, former editor of the Wall Street Journal, also falls in a different category, in that it was undertaken because of the reputation for successful forecasting which he had established over a long period of years. FORECASTING THE COURSE OF INDIVIDUAL STOCK PRICES We turn first to the records of two groups the financial services and the fire insurance companies, which have attempted to select individual stocks that would prove more profitable for in vestment than the aver- age issue. The first part of this section deals with the records, over the 43 years ending july, 1932, of 16 leading financial services which have made a practice of regularly submitting to their subscribers selected lists of common stocks for investment. Our analysis includes about 7,500 separate recommendations, requiring approximately 75, 000 en- tries. The first step was to record each week the name and price of each stock recommended for purchase or sale by each service. Next came the tabulation of the advice to sell or cover the commitment previously advised Reiterated advice was not considered, action being assumed to have been taken as of the date when the recommendation was first published. The percentage gain or loss on each such transac tion was recorded and, in a parallel column, the gain or loss of the stock market for the identical period. a balance was struck every six months which summarized the total results secured by each service as com pared with the action of the stock market. Proper corrections were of course, made to offset the effect of changes in capital structure resulting from the issue of rights, stock dividends, etc. Since a tendency existed among some services to emphasize their conspicuously successful stock recommendations and ignore more unfortunate commitments adopted a practice of automatically dropping a stock from the list six months after it had been last recommended, when specific advice to sell was not given a redistribution of funds in equal amounts among all stocks recom- mended has been assumed for each service at the beginning of every si months'period analy zed. It could be maintained of course, that this equalizing process should take place as often as once a week but this would increase the labor of computation to overwhelming proportions Provisional experiments demonstrated that it would yield conclusions practically identical with those secured by the shorter method. Com pounding the successive six months'records gives the percentage by which each service s recommendations have exceeded, or fallen behind the stock market as shown in table i Only six of the 16 services achieved any success. To arrive at an
ALFRED COWLES BRD 311 average performance, the record of each service was reduced to an ef. fective annual rate which was then weighted in accordance with the length of the period represented. The average annual effective rate of all the services, thus arrived at, is.43 per cent TABLE I SULTS OF COMMITMENTB IN STOCKS RECOMMENDED BY 16 FINANCIAL SERVICES(RELATED TO MARKET AVERAGES) Service Weel Per cent 234 123456 234 17.2 34 234 234 462 860 104 0.5 234 52 o123456 020 2284 04 33.0 PROBABILITY TESTS In an attempt to determine whether the service having the best rec- ord achieved its result through skill or chance, we resorted to the the- ories of compound and inverse probability. Our conclusion is thus rendered consistent by obtaining approximately the same answer in two different ways. With the aid of various checks, involving 1250 computations of the action of individual stocks selected at random we derived a formula A, D.(0)=5.42+1.5t(A D,=average deviation, t, in units of 4 weeks 21), representing the deviation, for all periods from one month up to one year, of the a verage individual stock from the average of all stocks ce Number 1, for the 9 six months' periods from January 1 July 1, 1932, was successful 7 times and unsuccessful 2 times with the aid of the table referred to, the averages of "chances in 1000 to do worse''for the 7 periods in which it was successful and the 2 pe- riods in which it was unsuccessful were found to be 842 and 66 re spectively. By the theory of direct probabilities, the probability of a single service being right at least 7 times in 9 is equal to the sum of the first3 terms of the binomial(}+号)° p=1/2+9/29+36/20=46/512=090 The probability that a single service could in 9 predictions be 7 times
312 ECONOMETRICA on the positive side and in these 7 forecasts equal the achievement of ervice Number P=090×(1-842)=014 However, the record of the best service is marred by its failure in the two negative cases. The average of the two chances to do worse in these ases is 066. We then have Q=(7/9)×842+2/9×066=.670 as the probability of a single random service having a record worse than that of Service Number 1. We therefore conclude that the proba bility that a random service can, first, be on the right side of the market 7 times out of 9, and second, equal in performance the record of Service Number l, is P=090×(1-670)=.030 This means that in 16 services we should expect to find 16X.030=48 services which will equal the record of Service Number 1. That is to say, the chance is even that we should get at least one service as good Because of the assumptions implied in this computation, we shall ar gue this another way We shall assume that the probability that a serv- ice for its total forecast shall be on the positive side of the market is 1/2. Then the estimate of its success must be made by a different evaluation of Q. For this purpose we shall adopt a formula suggested by Bayes'rule in inverse probability in which the weights. 910 and. 090 instead of 7/9 and 2/9 are used. We get 910(842) 910(842)+(090)(934) Hence, if a service was on the right side of the market, the proba Thus the compound probability would be Ce Number 1 would be 1-Q bility of its achieving the success of Servic P=1/2(1-901)=050 Among the 16 services the probability of the most successful one equal- ling the record of Service Number l would be P=16 X.050=80, that is to say, we should expect to get among 16 random services about one service which would equal Number 1. Since this answer is quite con- sistent with our previous answer, our analysis suggests the conclusion that the record of Service Number l could not be definitely attributed to skill
ALFRED COWLES 3RD 313 TWENTY FIRE INSURANCE COMPANIES The second analysis deals with the common stock investments, from 1928 to 1931 inclusive, of 20 of our leading fire insurance companies Its significance lies in the fact that these companies are representative of a class of common stock investor which has had long years of experi ence and large amounts of capital at its disposal. Fire insurance dates from the great London Fire of 1666, and active investment in stocks developed during the nineteenth century. The fire insurance companies are much older hands at the business of investment than either the financial services, which are a twentieth century product, or American investment trusts, which are largely a development of the last few years. The investment policies of these companies are based on the ccumulated knowledge of successive boards of directors whose judg ment might be presumed, over the years, to have been well above that of the average investor. The 20 companies which were selected for analysis hold assets totalling several hundred million dollars, and seem a fair sample of their kind Fire insurance companies carry between 20 and 30 per cent of their total investments in common stocks Their average turnover amounts to only some 5 per cent a year For this reason it was thought best to confine our analysis to the record of the actual purchases and sales made during the period under examination, rather than to compute the record of the entire common stock portfolio. To simplify the labor, allitems of stock purchased were given equal weights, regardless of the amounts involved. While the conclusion does not exactly refect the actual investment results secured by these companies, it should, how ever, provide a satisfactory test of the success of these organizations in selecting stocks which performed better than the average The method employed in the analysis is essentially the same as that ed in the case of the investment services. a second purchase of an item was omitted from the record unless a sale of this item intervened A record of the sale of an item, of course determined the date as of which it was dropped from the list. Also, any item of which there had been no purchase recorded for 12 months was automatically considered The compounded records of the 20 companies for the 4-year are shown in Table It Six of the companies show evidence of success, and the average of the 20 is -4.72 per cent. The average record of the companies in the stocks which they selected for investment fell below the average of the stock market at the effective annual rate of 1. 20 per cent. a comparable result could have been achieved through a purely random selection of
314 ECONOMETRICA stocks. The analysis of the fire insurance companies'records thus con firms the results secured in appraising the records of the financial serv- TABLE II RESULTS OF COMMITMENTS IN STOCKS MADE BY TWENTY FIRE INSURANCE COMPANIES(RELATED TO STOCK MARKET AVERAGES) All companies 1928-31, inc. Company Per cent +27.35 +25.11 18.34 234567890123 +10.38 10.12 3.20 2.06 3.63 5.06 6.67 10.44 10.55 -11.76 14. 2.92 15. 13.82 6789 14.96 18.03 21.89 23.44 33.72 FORECASTING THE STOCK MARKET ACCORDING TO THE DOW THEORY Having dealt with the efficiency of two great groups of profession als, fire insurance companies and fnancial services, in selecting com mon stocks for investment we turn now to a consideration of skill in predicting the course of the stock market as a whole. This section also is in two principal sub-divisions. First we consider the record of william Peter hamilton This analysis was undertaken because several decades of editorials in the country 's leading financial newspaper have built up a great popu- lar following for the Dow Theory, of which Hamilton was the principal sponsor. The Dow Theory was the creation of Charles H. Dow, founder of the dow ones financial news service, founder and editor of the wall Street Journal. After Dows death in 1902 Hamilton succeeded him as ditor of the Wall Street Journe inuing in this position until death in December, 1929 During 26 years of his incumbency Hamilton wrote 255 editorials which presented forecasts for the stock market based on the Dow The- ory. These were sufficiently definite to permit scoring as bullish, bear ish, or doubtful. This we did by a majority vote of five readers. When
ALFRED COWLES BRD 315 doubtful we assumed that he abstained from trading When bullish it as assumed that he bought equal dollar amounts of the stocks in cluded in the Dow Jones railroad and industrial averages, and sold them only when he became bearish or doubtful. when bearish we as- sumed that he sold short equal dollar amounts of these stocks and covered only when he became doubtful or bullish. The percentage gain or loss on each such transaction has been calculated, and the results impounded through the 26 years. Since the Dow Jones averages have only recently been corrected to offset the effect of stock rights, stock dividends, and stock splits, such adjustments have been made for all the previous years on the basis of tables published by Dwight, C. Rose in his book Investment Management. Corrections have also been made to allow for the effect of brokerage charges, cash dividends, and the interest presumably earned by Hamilton's funds when they were not in the stock market. The fully adjusted figures were then reduced to an effective annual rate of gain which is presented as a measure of the esult accomplished From December 1903 to December 1929, Hamilton, through the ap. plication of his forecasts to the stocks composing the dow Jones in dustrial averages, would have earned a return, including dividend and interest income, of 12 per cent per annum. In the same period the stocks composing the industrial averages showed a return of 15.5 per cent per annum. Hamilton therefore failed by an appreciable margin to gain as much through his forecasting as he would have made by a continuous outright investment in the stocks composing the industrial averages. He exceeded by a wide margin, however, a supposedly nor- mal investment return of about 5 per cent. Applying his forecasts to the stocks composing the Dow Jones railroad averages, the result is an annual gain of 5.7 per cent while the railroad averages themselves show a return of 7.7 per cent. Hamilton was long of stocks 55 per cent, short 16 per cent, and out of the market 29 per cent, of the 26 years under review. Counting only changes of position, he made bullish forecasts 29 times. applying these to the industrial averages, 16 were profitable, 13 unprofitable. He an- nounced bearish forecasts 23 times, 10 were profitable, 13 unprofitable He advised 38 times that funds be withdrawn from the stock market 19 of these withdrawals being profitable, 19 unprofitable. In all, 45 of his changes of position were unsuccessful, 45 successful. The applica tion of the forecasts to the railroad averages confirms these conclusiong cept that in this case 41 changes of position were successful and 49 unsuccessful. For the period from 1909 to 1914 inclusive when the ir dustrial averages displayed what, in effect, was a horizontal trend, his hypothetical fund shrank 7.8 per cent per annum below what, it would
316 ECONOMETRICA have been if loaned at 5 per cent interest The result of applying his forecasts to the railroad averages deserves attention in view of the fact that this group displayed an almost horizontal secular trend for the 26 years under consideration. His average annual gain of 5.7 per cent in this group would have been approximately equalled, in the case of continuous outright investment, by the dividend income STOCK MARKET FORECASTS OF TWENTY-FOUR FINANCIAL PUBLICATION For the analysis of other results secured in forecasting the course of he stock market, we selected during the period from January 1, 1928, to June 1, 1932, 24 publications(among which were 18 professional financial services, 4 financial weeklies, one bank letter, and one invest ment house letter). More than 3, 300 forecasts were tabulated. The method used has been for each reader to ask himself the question, "In the light of what this particular bulletin says, would one be led to buy stocks with all the funds at his disposal, or place a portion only of his funds in stocks, or withdraw entirely from the market? "The reader graded the advice in each instance by means of one of nine possible entries, namely 100 per cent of funds in the market, 873, 75, 621, 50 373, 25, 122, or 0 per cent. The great majority of forecasters confine themselves to general discussions of the investment situation, leaving to the reader the decision as to what proportion of his funds he shall place in the market. The tabulation, therefore, cannot be mathemati- cally conclusive. Our method, in general, has been to have the vote of three readers of competent intelligence determine the interpretation of each forecast. Marginal commitments have not been incorporated in our tabulations because in no case have they been advised by any of the forecasters. Similarly, short commitments are not in general as- sumed because of the entire 24 forecasters, only one recommended them. His record has been computed on a special bas The tabulated forecasts have been tested in the light of the actual fluctuations of the stock market as reflected by the Standard Statistics Company index of 90 representative stocks. If a forecast is 100 per cent bullish and the market rises 10 per cent in the subsequent week, the forecaster is scored as +10 per cent. If the forecaster, after weighing the favorable and unfavorable factors, leaves the decision hanging in the balance, the score is +5 per cent or one-half of the market advance This is on the assumption that the investor being in doubt as to the future course of the market and being, by definition, committed to common stocks as a possible investment medium, would be led to dopt a hedged position with half of his funds in stocks and half in eserve. If the forecast is 100 per cent bearish, the score is zero, regard
ALFRED COWLES 3RD 3I7 less of the subsequent action of the market, on the assumption that, under such conditions, the investor would withdraw all of his funds rom stocks. On the other hand, if the forecast is 100 per cent bullish and the market drops 10 per cent in the ensuing week, the score is 10 per cent. If the forecast is doubtful when the market drops 10 per cent, the score is -5 per cent. The compounding of all these weekl scores for the period covered gives a cumulative record for each fore- caster. This permits comparisons which reveal relative success and average performance. While it may be thought that accurate week-to- week forecasting is a hopeless ideal, it should be emphasized that our analysis of weekly results also measures accurately the efficiency of long swing forecast A figure representing the average of all possible forecasting result for the period was arrived at by compounding one-half of every weekly percentage change in the level of the stock market. The final scores given below for the 24 forecasters were computed by dividing the actual performance of each by the average of all possible results re- ferred to above, and subtracting 100 TABLE III RESULTS OF STOCK MARKET FORECASTS Forecaster Weel Per cent +72.4 +31.5 123456789012 +28,3 +24.2 +.0 38357033 0.G 1.3 2.1 3.6 6.0 13. 6.7 14. 6,9 15 12.5 16. 13.5 17 17,2 18. 21.5 19 29.4 33.0 0128 35.3 41.5 45.3 49.1 The records show that only one-third of the list met with any suc- cess. In order to derive a significant average of the performance of the
318 ECONOMETRICA entire group the results listed above have been reduced to effective annual rates, and each has been given a weight to conform with the length of the record analysed. After these adjustments, we are enabled to conclude that the average forecasting agency fell approximately 4 per cent per annum below a record representing the average of all performances achievable by pure chance. This would seem to indicate that, in general, these stock market forecasters failed to accomplish their objective. The most that can be said in extenuation is that the long-continued decline in securities has been, naturally, a handicap to a group which, taking warning from the experience of Cassandra,usu- ally seems constrained to look on the bright side During the 41 year period under analysis the number of weeks in which the stock market declined almost exactly equalled the number of weeks in which ad vances were recorded, and the total amount of the declines consider ably exceeded the total amount of the advances. Yet we recorded during this period 2035 bullish, 804 bearish, and 479 doubtful fore- casts. Further we note that in 1928, the only year the market showed a net gain, the excess of bullish over bearish forecasts was smaller than in any succeeding year. Taking a glaring example, in the rising market of 1928 the ratio of bullish to bearish forecasts was only four to three. In 1931, when the market declined 54 per cent, there were sixteen bullish forecasts to every three bearish STATISTICAL INTERPRETATIONS OF RESULTS In an attempt to illuminate the problem of whether the records of all these forecasters lay within the limits of pure chance, we compiled 24 records, identical with those of the 24 forecasters as to the total period covered, but having purely fortuitous advices applied to random intervals within these perids. For example, to compile a purely chance record to compare with the actual record of a forecaster whose opera- tions covered 230 weeks from January 1, 1928, to June 1, 1932, we first determined the average number of changes of advice for such a period which was 33. Cards numbered from 1 to 229 were shuffled, drawn, re- shuffled, drawn, in all 33 times. Thus 33 random dates were selected as of which forecasts were to be changed. The investment policies which were to apply to the intervals between those dates were derived in similar fortuitous fashion, by drawing 33 times from nine cards on each of which a different one of the nine possible investment policies It only remained to relate these random advices to a stock market index, cumulate the results, relate them as we had done with the records of the actual forecasters to the average of all chances for the period, and subtract 100. Thus we had a list of 24 purely chance fore-
