ambiguity when Performance is Measured by the securities Market Line TORIo Richard roll The Journal of Finance, Vol 33, No 4.(Sep, 1978), pp. 1051-1069 Stable url: http://inks.jstor.org/sici?sici=0022-1082%028197809%2933%3a4%3c105193aawpimb%3e2.0.c0%03b2-4 The Journal of finance is currently published by American Finance Association Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://www.jstor.org/about/terms.htmlJstOr'sTermsandConditionsofUseprovidesinpartthatunlessyou have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at http://wwwjstor.org/journals/afina.html Each copy of any part of a jSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission jStOR is an independent not-for-profit organization dedicated to creating and preserving a digital archive of scholarly journals. For more information regarding JSTOR, please contact support@jstor. org http://www」]stor.or Thu Apr607:37:312006
JOURNAL OF FINANCE. VOL. XXXIII, NO. 4.SEPTEMBER 1978 The fournal of FINANCE VOL. XXXIII SEPTEMBER 1978 AMBIGUITY WHEN PERFORMANCE IS MEASURED BY THE SECURITIES MARKET LINE RICHARD ROLLS INTRODUCTION IMAGINE AN IDEALIZED ANALoG to the activities of professional money managers, a contest whose rules are as follows (a)Each contestant selects a portfolio from a specified set of individual assets (c) After each period of return observation, the portfolios are re-balanced to the nitial selections (d)After an interval consisting of several periods, winners""and"losers"are declared for that interval (e) Contestants choose a new portfolio, or keep the old one, and the process(b) through()is repeated (f After several intervals, consistent winners are declared to be superior port folio managers and consistent losers are declared inferior. In the absence of any consistency, everyone is declared non-superior The sponsors of the contest face only a single problem of intellectual interest. They must develop criteria to partition contestants at step(d)into winners and losers Of course,the criteria must be acceptable to participants and to disinterested obser vers. There should be a correspondence between"consistency in winning "and an tuitive notion of ability in portfolio selection We might think of many desirable qualities to be possessed by such criteria. For example, they should be robust to stochastic changes in the return sequence; true ability should be detectable over many intervals regardless of the sequence. If the criteria are employed by different sponsors, the same judgements about ability should be obtained. It should not be possible to reverse judgements by making hanges in the computation of the criteria, if such changes are deemed insignificant by all observers. In other words, the criteria must provide decisions about ability that are unambiguous to rational judges. criterion that is widely employed in the financial community for assessing portfolio performance is the"securities market line, the (linear) relation betwe Graduate-School of Management U C LA. Comments and suggestions by Alan Kraus, David Mayers, Stephen Ross and Eduardo Schwartz are gratefully acknowledged
1052 The Journal of finance mean returns on assets or portfolios and the betas of these assets or portfolios calculated against a market index. Judging from the academic literature, this criterion is even more widely-accepted by scholars as a tool for assessing the ex ante or ex post qualities of securities, portfolios and investment projects. There eems little doubt that it is currently the most widely-accepted criterion for inferences about the quality of risky assets It is quite simple to employ, particularly in a situation such as the contest mentioned above. There are only two steps to accomplish. First, an index must be osen by the judges; second, the betas must be computed against this index for each asset (and portfolio). The second step can be accomplished in many ways, including purely subjective, but the most common method is to employ historical data over the same interval as the contest itsel Given the computed betas and the returns on assets or portfolios, the criterion an be illustrated as in Figure 1. A line, R=Yo+y B is fit to the observations and assets are declared"winners "if they are above the line(such as a and b)or losers elow(C or D) My purpose here is to expose the ambiguity in this criterion. It is not robust, is likely to yield different judgements when employed by different judges, and can completely reverse its judgements after seemingly innocuous changes in its compu- curities Market Line FIGURE'I. The Securities Market Line Employed as a Criterion for Assessing Asset Quality l. The"beta "is the covariance of the asset and the market index returns divided by the variance of the market index'return The method of fit is not crucial for this ion. It is often done by regression, sometimes with sophisticated econometric corrections of the betas. It could also be done by choosing Yo and y, based on theories of market equilibrium. For example, the Sharpe [1964], Lintner [1965] theory requires that Ro the riskless rate of interest, and that YI=E(Rm)-Res the difference between the expecte eturn on the market index and the riskless rate of interest. Thus, estimates of E(Rm ) and of RF can be sed to fix the line. Alternatively, the Black[1972] theory would require y,=E(Rm-R)and Yo=E(R2) where z is a portfolio with minimal variance uncorrelated with m
Ambiguity When Performance is Measured by the Securities Market Line 1053 ation. Reasons for these deficiencies will be explained in detail. By implication, the concept of the beta as an unambiguous measure of risk will be disputed A Numerical Ey To illustrate the ambiguity of the securities market line criterion, let us consider specific numerical example-the idealized contest conducted with 15 hypothetical contestants and a hypothetical four asset universe. Table 1 begins the example with portfolios selected by the fifteen contestants Nothing is unusual about the selections. The first four contestants plunged into the individual assets and contestants 14 and 15 sold short some securities, but there seems little reason to exclude such possibilities. The idealization of the contest is evident only because trading costs and restrictions on short sales have not even been mentioned PORTFOLIOS SELECTED BY FIFTEEN CONTESTANTS Asset Percentage Invested in Individual Asset 0. l00 20 10 7.69 13 3l9 13.4 14 42 29.0 33.3 496 24 53.l Note: The last four portfolios' weights may not sum to 100% because of nding. The exact weights were used in all calculations After the portfolios were selected, a sample period was observed with the results for individual assets reported in Table 2. Again, there is nothing abnormal nor pathological about these numbers. The mean returns were different on different ssets and the covariance matrix was non-singular, certainly the most usual and desirable feature of real asset return samples. Indeed, all of the qualitative results to be reported hereafter could be obtained from any other numbers with these same general characteristics. For the purposes of illustration, there is not even any need to be concerned with the statistical properties of the sample. The ambiguities are
1054 The Journal of finance not related in any way to the sampling error in the estimates. The same problems nen the true population means, va and used in computing the criterion TABLE 2 SAMPLE STATISTICS FOR INDIVIDUAL ASSETS WHICH CONSTITUTED THE UNIVERSE OF SECURITIES FOR THE PORTFOLIO SELECTION CONTEST Asset asset Mean return Sample varian ance 6. 5. The observed means and variances of the 15 portfolios are easy to compute by applying the compositions of Table I to the observed individual asset returns, variances and covariances of Table 2 Because of its wide acceptance, the securities market line criterion will be used y the three hypothetical sponsors of the contest in order to distinguish winners from losers. Let us suppose, however, that a dispute arises about the best index to use in calculating the"betas. "Sponsor/judge no. 1 admits total ignorance about this question. He concludes that an index composed of equal weights in the individual assets would be the most sensible portrayal of this ignorance and the fairest to all contestants. Sponsor/judge no. 2, however, has studied asset pricing theory and argues that the appropriate index should have weights proportional to the aggregate market values of individual assets. He finds that the aggregate values of assets 2 and 3 are roughly four times larger than those of assets I and 4, so he suggests the index(10%, 40%, 40%, 10%) Sponsor/judge no. 3, also a theorist, thinks that a good index should be mean- variance efficient in the sense of Markowitz(1959), so he makes some calculations and obtains an index with the same mean return as the indices of the other judges but with a different composi- tion. His index turns out to be (18.2%, 37.0%, 21.5%, 23.3%), the proportions being rounded to three significant digits The compositions of the indices, their observed mean returns and variances, and the securities market lines computed against them are reported in Table 3. Notice that the three indices have equal means, similar variances and closely adjacent securities market lines. ( The securities market lines would have been identical they had been fit using an asset pricing theory rather than by regression. See Note 3. I. e, a portfolio which has the smallest sample variance of return for a given level of sample mean
Ambiguity When Performance is Measured by the Securities Market Line 1055 INDICES AND SECURITIES MARKET LINES USED BY THE THREE JUDGES IN THE PORTFOLIO SELECTION CONTEST Return Slope Intercept Percentage of Index in Security) Index Securities Market linea l14 (44 31823721523.375 11.0 a Standard errors are in parentheses. These lines were fitted cross-sectionally by ordinary least quare applied to mean returns and betas of the fifteen portfolios in the contest b Judge three actually specified more precise weights. The weights reported here have been rounded. (For example, the proportion of his index in asset 4 was actually 23. 3214% )The exact weights were used in all calculations Coefficients of Correlation Between Indices 3 982 920 What about their assessments of"winners"and"losers"? Judge no. I ranks th 15 contestants from best (largest positive deviation from his securities market line) to worst as follows Rankings of Contestants by Judge no. 1 (above the line) (below the line) 613 Losers 9871241131←( Worst a graph of the contestants' positions relative to the judge's criterion is given by Figure 2(A) The second judge has a different set of assessments, as shown in the following list and depicted in Figure 2(B) Losers
1056 The Journal of Finance 391014 FIGURE 2. Securities Market Lines and Positions of Selected Portfolios as Perceived by the Three Judges of the Contest nes are ordinary least squares estimated securities market lines fitted through the fifteen portfolios. The dotted lines pass through portfolios 12-15, which were exactly efficient ex post Although some contestants were similarly rated by both judges, (e.g, contestant 15 was ranked second by both and contestant 3 was ranked 14th by judge no, I and 15th by judge no. 2), other contestants were rated quite differently; (e.g,the number one winner according to judge no. I was a loser and ranked 13th out of 15 by judge no. 2. )The rank correlation between the decisions of these two judges is only 0036 and the lack of agreement is clearly evident in the figure As for judge no. 3, after calculating his securities market line and plotting the selected portfolios, he observes Figure 2(). Every single contestant is exactly on the line. Of course, this judge is unable to construct a ranking and can draw no inference about the relative abilities of contestants c This example was not constructed to generate bizarre results. The same results an be obtained from every sample of asset returns. They were not caused by any of the example's parameters, by the numbers of assets and contestants, nor by the pattern of returns The results in the example and the ambiguity in the securities market line criterion can be attributed to the following fact: corresponding to every index, there is a beta for every individual asset(and thus for every portfolio); but these betas can be different for different indices and will be different for most. To consider the beta as an attribute of the individual asset alone is a significant mistake. For every asset, an index can be found to produce a beta of any desired magnitude, however large or small. Thus, for every asset(or portfolio) judicious hoice of the index can produce any desired measured"performance, (positive or negative), against the securities market line. IL. MATHEMATICS OF THE SECURITIES MARKET LINE CRITERION In this section, the mathematical causes of the preceding numerical results will be made more precise. Since there are general principles involved that would bring
Ambiguity When Performance is Measured by the Securities Market Line 1057 similar results to every ex post sample and ex ante application of the criterion, it seems worthwhile to have a unified list of the principles in one place Let us presuppose the existence of a mean return vector, R, and a non-singular covariance matrix, V, of N individual assets. There is no need to require that these be of any particular dimension nor that they be ex ante(nor ex post sample statistics). The following statements are true for any R and V, however obtained These statements constitute the mathematics that explains the behavior of the S1: Let X, be the N x l column vector of investment proportions(or"weights") that defines a portfolio p. Let X define the index I used by a judge. Then the"beta "for portfolio p with respect to the index is given by B, /X VX/ 0 where of=XvX Proof: obvious S2: If the portfolio q is mean/variance efficient(in the Markowitz sense)with respect to R and V, then its weights are given by (2) where NxI unit vector. a is the efficient set information matrix and is the return of portfolio q P 1972]orRo1977] S3: If the selected index is mean/variance efficient, then the betas of all assets are related to their mean returns by the same linear function. Proof: Set g=I in(2)and use the results in(1). Then simplify to B2=()4-(n1)/ A,rr,and o are cross-sectional constants. Q.E.D S4: If, for some selected index, the betas of all assets are related to their mean returns by the same linear function, then that index is mean /variance Proof: Roll [1977, p. 165.] S5: The betas of all mean/ variance efficient portfolios are related to their mean returns by the same linear function, even if the index used to compute the betas is not itself mean/variance efficient. Proof: Identical to S3 S6 For every ranking of performance obtained with a mean /variance non efficient index, there exists another non-efficient index which reverses the ranking. R'V-IR IV-IR 5. It can be shown easily that r,Yo+ Y1 B, I where Yo is the return on a portfo thogonal to l and
1058 The Journal of finance Proof: Let z be the (KXN) matrix of selected portfolios (each row containing the weights for N individual assets in the portfolio selected by one of the K contestants). Let a be a subscript which denotes the first nking,(based on index 4). The scatter of selected portfolios in the return/ beta space can be denoted ZR=aA+Yo k+Y14(ZVXA/oA) where yod and yu are, respectively, the(estimated intercept and slope of the securities market line and aa is the(K x 1) vector of deviations. The jth element of aa is the"performance measure"for contestant j('k is the K-element unit vector). A reversal of rankings can be achieved by finding a second index, denoted B, such that aB=-da, where d is a positive scalar constant. Choose d=of/a3. Then(4)can be written twice, once for A and once for B, and the equations can be added to obtain a4+a=0=zR(+)-(4+7) V(YIAXA+Y1BXB) have Zin=Ik and thus in=(Zz)Z'lk Multiplying both sides of (5)by Z' and simplifying, we find R(Of+0B)-(oAYo +bYob)IN=V(YA X4+YiBXB) In the equation system(6), there are N+3 unknowns: the N elements of XB plus YoB, YiB, and aB(Recall that X,, of, YoA, and ya were already known) There are only quations in(6) but we must also add the portfolio additivity condition, XBIN=l, the definition of the e varlance XRVX and the usual requirement Y18=XBR-YoB, which guarantees that the securities market line passes through the mean return of the index (XBR) at a beta of unity. Thus, there obtains an identified system of three non-linear equations in three unknowns and at least one solution is guaranteed. When there are fewer contestants than assets, the system (5) is under entified and there is an infinity of solutions. Q.E.D. the system of equations is non-linear. rankings can be obtained by ignoring the difference in the securities market which he Xa obtained in this manner corresponding to the index selected by the first judge (which was equally-weighted), is the portfolio X=(-0422, 526, 232,, 285). It provides the anking of the 15 selected portfolios from"best "to"worst", of 3, 1, 11, 4, 9, 8, 12, 7/13, 6, 10, 14, 5, 15, 2. ompare judge no. I's rankings above. The rank correlation between them is -.961
Ambiguity When Performance is Measured by the Securities Market Line 1059 Statements SI through S6 explain all the facts of the numerical example. For example, Judge no. 3,s inability to discriminate among the contestants was caused by his choice of a mean/ variance efficient index. As S3 indicates, any such index will cause the scatter of returns and betas to fall exactly along a line. The portfolios selected by contestants 12 through 15 fall exactly along another line(shown dashod in all three panels of Figure 2). These selections turned out to be mean/variance efficient portfolios which, as $5 indicates, always fall exactly along a line regardless of the index. Note that this line is not necessarily the securities market line. The latter is usually a line of best fit to all selected portfolios and this can deviate from the line through efficient portfolios when the index itself is not efficient Finally, the deviations of portfolios from the securities market lines used by Judge no. 1 and Judge no. 2 are a consequence of s4. These judges did not us efficient portfolios as their indices so the indiuidual asset returns could not be exactly linear in the individual betas. Of course, contestants could have made choices such that their portfolio's betas and returns were linear; but this was not the case for the example nor is it likely in general. S6 shows that these judges might even have selected indices which reversed the rankings of contestants It is quite easy to show how the beta of an arbitrary portfolio varies with the index. Suppose we simplify matters by assuming that all the indices being con idered have the same return but different compositions(as in the example). Of course, only one will be mean/variance efficient. If I* is the index that is mean/variance efficient and I is another index with the same return, the difference in the beta of p from using I* rather than I is △B=x[(0)Xx]1呀 The sum of elements in the bracketed vector of(9)is equal to ol/07.-1, which is positive(because af<of). Thus, the numerator of 9)is proportional to the covariance between p and a hybrid portfolio whose investment proportions vector is[(ol /0)Xr The closer the pattern of p's weight pattern of weights in this hybrid portfolio, the larger is beta. Roughly speaking, if the weights of p match the difference in weights between an efficient index and the index actually employed, the observed beta will be"small"relative to the magni tude of beta had the efficient index been employed. Since the return of p is invariant to the index employed, a relatively smaller beta corresponds to better measured performance and vice versa As an example, consider contestant no. 2, who was ranked first by Judge no. 1 and 13th by Judge no. 2. His portfolio and the vectors [(o1/07)Xr-X,1/(02/02. 1)for the two judges are given in table 4. Clearly, contestant no. 2 selected a portfolio that was"different""from the first judge's index. The beta perceived by 12 was selected to be the global minimum variance portfolio and its return was 6.58 portfolios 13, 14, and 15 all had returns of at least 7 percent and since they were efficient, they were all located on the positively-sloped segment of the ex-post efficient frontier 8. Different in the sense that the largest weight of contestant no. 2's portfolio corresponded algebraically) smallest value of Xr -Xr(of/af) for hybrid portfolio for Judge no. I matched the largest weight for contestant no. 2
