You are here: Knowledge Center Reading Room

Performance Evaluation

E-mail Print PDF
Is it possible to evaluate the relative "goodness" of an investment manager?  If so, how should one go about it?  What should be avoided?  The concern here is to avoid evaluation strategies which might tend to cause one to embrace unskilled managers andor to exclude skilled managers.  Also, see the section on Risk Measures and Performance Measurement.

"Good clients will, if they decide to use active managers, insist that their managers adhere to the discipline of following through on agreed-upon investment policy. In other words, the investor client will be equally justified and reasonable to terminate a manager for out-of-control results above the market as for out-of-control results below the market. Staying with a manager who is not conforming his or her portfolio performance [to agreed-upon investment policy] or to prior promises is speculation — and ultimately will be 'punished.'

But staying with the competent investment manager who is conforming to his or her own promises — particularly when out of phase with the current market environment — shows real 'client prudence' in investing and ultimately will be well rewarded." — Charles Ellis, Winning the Loser's Game

Elroy Dimson and Andrew Jackson, "High-Frequency Performance Monitoring," Journal of Portfolio Management, Fall 2001, pp. 33-43 (519kb).  Also here.  This excellent paper warns that basing a judgment of an investment manager's relative "skill" on very short-term results increases the probability of incorrectly judging an unskilled manager to have skill and, just as badly, increases the probability of incorrectly judging a skilled manager to NOT have skill.

Eugene F. Fama and Kenneth R. French, "Luck versus Skill in the Cross-Section of Mutual Fund Returns," Journal of Finance, Forthcoming (363kb).  This paper uses an approach similar to that used in the Murphy paper below, only using more real-world data.  This paper takes real data of mutual fund performance (i.e., which shows a range of estimated alphas) and deliberately adjusts each fund's performance data set so that its alpha is zero (i.e., its estimated alpha is subtracted from each data point).  Thus, each synthetic fund now has a known alpha exactly equal to zero.  This data base of synthetic zero alpha funds was then used to generate 10,000 monthly returns using a bootstrap sampling approach (with replacement).  These monthly returns yielded an example of what might be expected in real life if it were true that the true alphas of each fund were, in fact, exactly equal to zero.  The paper found that the distribution of realized alphas from the simulation (i.e., where it was known that the true alpha was zero -- that any resulting apparent alpha was merely due to good luck) was similar to that found for actual funds.  In fact, if anything, the distribution of alpha estimates for real funds showed that the actual alpha of real funds was probably negative, but no better than approximately zero.  Thus, good alpha estimates for real funds were consistent with what would be expected through luck alone (i.e., suggesting that good results of actively managed funds are due to good luck and not skill).

Eugene F. Fama and Kenneth R. French, "Luck versus Skill in Mutual Fund Performance," FamaFrench Forum, November 30, 2009.  This is a slightly less technical description of the above FamaFrench paper.  The paper uses a simulation to show that the actual alphas of actively managed funds are no better than, and probably worse than, those expected purely through good luck alone (i.e., worse than they would be through simulations with alphas pre-programmed to be exactly zero).  This suggests that apparently good performance of actively managed funds is generally consistent with what would be expected through luck alone (i.e., good performance should generally be attributed to good luck and not to skill).  However, this is not evident from merely evaluating an active manager's performance, even if it is adjusted for risk (e.g., a calculated "alpha").

S.P. Kothari and Jerold B. Warner, "Evaluating Mutual Fund Performance," Journal of Finance, October 2001, pp. 1985-2010.  An earlier version is available here.  An interesting study.  They generated synthetic stock portfolios, simulated their operating as mutual funds and then analyzed their performance as though they were mutual funds.  "... standard mutual fund performance measures are unreliable and can result in false inferences.  It is hard to detect abnormal performance when it exists, particularly for a fund whose style characteristics differ from those of the value-weighted market portfolio."  In particular, the study found that it is easy to detect abnormal performance and market-timing ability when none exists.

J. Michael Murphy, "Why No One Can Tell Who's Winning," Financial Analysts Journal, MayJune 1980, pp. 49-57.  This outstanding paper demonstrates why it is so hard to identify truly skilled investment managers (assuming that they exist at all).  The author created 100 synthetic investment managers, each with a predetermined probability distribution of returns.  10 were pre-programmed to be more likely to outperform the market, 10 were pre-programmed to underperform the market, and 80 were pre-programmed to average the market returns.  He simulated ten years of returns.  The results were stunning: while the outperformers as a group dramatically outperformed both the random performers and the underperformers, the top two funds, four of the top five, and six of the top nine best performing funds over the simulated ten year period were random performers (i.e., those who were preprogrammed to have an expected return equaling the market return).  In other words, two managers with absolutely no skill got lucky to an extent which allowed them to outshine one hundred percent of the managers who definitely had skill — OVER A TEN YEAR PERIOD.  This really drives home how dangerous it is to assume skill based on performance measurements over any "short" time period.  "Given enough time, the outperformers should produce results significantly superior to those of the random performers.  But the time required undoubtedly exceeds the lifetimes of the managers being measured."

Back to Reading Room