Survivorship bias in backtesting: why historical samples overstate average returns

A backtest of any equity strategy on the current S&P 500 universe runs the strategy on the 500 companies that exist today—none of which has gone bankrupt, been acquired, or otherwise dropped out of the index. The backtest implicitly assumes the strategy could have selected only winners, ignoring the dozens of names that failed along the way. Survivorship bias is the formal name for this distortion, and it is one of the most pervasive and least-noticed problems in quantitative-strategy evaluation.

What survivorship bias is

Survivorship bias is the systematic distortion that occurs when a backtest computes historical returns from a sample that includes only currently-surviving entities, while excluding those that have failed or otherwise dropped out. The resulting estimate of average returns is biased upward, sometimes substantially, because the failures whose returns would have dragged the average down are missing from the calculation.

The bias appears in many investment contexts. A backtest of a stock strategy on a current equity-index universe ignores the stocks that have been removed from the index; a backtest of a hedge fund strategy ignores the funds that closed; a backtest of a mutual fund category ignores the funds that merged or liquidated. In each case, the historical record looks better than the realistic experience an investor would have had at the time, because the losers are no longer in the dataset.

Brown, Goetzmann, and Ross (1992) provided one of the earliest empirical estimates of the magnitude. They documented that mutual fund returns from databases of currently-existing funds typically overstated true historical performance by 100–300 basis points per year, depending on the survival mechanism and the sample period.

How it works

The mechanism is selection. A current sample of funds, stocks, or strategies represents only those that have made it to the present—by definition. Anything that failed before the current date is excluded; anything that delivered very poor returns and was therefore terminated, delisted, or removed is missing. The historical returns computed from this sample are conditional on having survived, and the conditional distribution is right-shifted relative to the unconditional distribution.

Equity-index examples make the magnitude visible. The S&P 500 today does not contain the constituents of the S&P 500 in 1990—many companies have been removed (acquired, gone private, failed, dropped to small-cap) and many new ones added. Backtesting a strategy on the current 500 names from 1990 onward would assume the investor in 1990 could have held the names that 35 years of selection has now identified as winners. The strategy looks much better than it would have looked using the actual 1990 universe applied forward in time.

Hedge fund database examples are even more extreme. Most hedge fund databases include only currently-reporting funds; funds that closed or stopped reporting are removed. Studies by Malkiel & Saha (2005), Aiken, Clifford & Ellis (2013), and others document that hedge fund index returns derived from current databases overstate the achievable return by 200–400 basis points per year, with the exact figure depending on the strategy category and the sample.

What the evidence shows

The most-cited corrections in the academic literature adjust historical returns by reconstructing the original universe at each point in time and running the strategy on the as-it-was sample. The CRSP US stock database, the Bloomberg point-in-time fund databases, and similar resources provide the historical universe membership needed for proper backtesting. Strategies tested on point-in-time universes consistently produce lower estimated returns than the same strategies tested on current-universe data.

The implication for retail investors evaluating quantitative strategies is direct. A strategy backtested on a current asset universe is almost certainly survivorship-biased. The realistic out-of-sample return is typically 100–300 basis points lower than the headline backtest figure. Strategies marketed on the basis of impressive backtests should be discounted for the bias before any investment decision is made.

The bias is particularly insidious in factor strategies that select on characteristics like quality, profitability, or low leverage. The current universe of high-quality stocks has, by selection, avoided the quality-collapse failures that would have appeared in a real-time selection. The backtested premium of the quality factor is therefore overstated by an amount that is hard to quantify precisely but is meaningful in any honest analysis.

Limitations and trade-offs

Eliminating survivorship bias entirely requires point-in-time data that includes every entity that existed at each historical date—including the ones that no longer exist. This data is expensive, often proprietary, and incomplete for older samples. Most retail-accessible backtesting tools use current-universe data and therefore inherit some degree of the bias.

Even with point-in-time data, judgement is required about how to handle delisted entities. A stock that goes to zero in bankruptcy produces a clear −100% return contribution to the strategy's historical performance; a stock that is acquired at a premium produces a positive contribution. Both events are common; backtests should handle both rather than treating delisting uniformly.

For retail investors, the practical implication is to discount backtested returns substantially before relying on them for investment decisions. The discount depends on the strategy's selection mechanism—pure passive strategies have minimal survivorship bias; high-turnover active strategies operating on a curated universe have potentially large bias. The headline backtest figure should be treated as an upper bound on the achievable return, not as a base case.

Survivorship bias in pfolio

Survivorship bias affects any backtest that uses a current asset universe to compute historical returns. pfolio's asset database holds currently-listed instruments only; delisted assets are not included in the universe. Investors evaluating systematic strategy results should be aware that the historical figures derived from a current universe are not adjusted for survivorship, and the realised long-run premium is typically smaller than the raw figure suggests.

Related articles

• Backtesting investment strategies: methodology, limitations, and how to avoid overfitting
• Overfitting in quantitative investing: why backtested strategies fail in practice
• Walk-forward optimisation: testing strategy robustness beyond the in-sample window
• Look-ahead bias in backtesting: when the test uses information that was not yet available