Base rate neglect in investing: ignoring statistical probabilities in favour of specific stories
Base rate neglect is the tendency to ignore the statistical background probability of an outcome when evaluating a specific case, overweighting the specific details of the situation and underweighting the general prior. In investing, this means that investors regularly evaluate the probability of an investment succeeding by focusing on the characteristics of this particular investment, without adequately accounting for how investments with similar characteristics have historically performed as a group. The result is systematic overconfidence in specific investment cases and systematic underestimation of the probability of adverse outcomes that are common at the category level.
What a base rate is
A base rate is the background frequency of an event across a reference class. The base rate for equity mutual funds outperforming their benchmark over ten years after fees is approximately 20%—roughly four in five fail to beat a passive benchmark over that period. This is the base rate that an investor should apply before evaluating whether a specific manager is likely to outperform. An investor who does not know or ignores this base rate—and instead evaluates the fund purely on the basis of the manager's track record, investment thesis, and credentials—is committing base rate neglect.
The base rate does not determine the answer for any individual investment; the specific case matters and can shift probabilities away from the base rate. A manager with a 20-year track record, a clearly articulated systematic process, and a disciplined risk management framework is more likely to be in the 20% that outperforms than a randomly selected manager. But the appropriate starting point is the base rate—adjusted upward or downward by the specific evidence—not the specific evidence alone.
Kahneman and Tversky's engineers and lawyers experiment
The canonical demonstration of base rate neglect is the engineers and lawyers experiment by Kahneman and Tversky (1973). Participants were given personality descriptions of individuals drawn from a group of either 70 engineers and 30 lawyers, or 30 engineers and 70 lawyers, and asked to assess whether each individual was an engineer or a lawyer. When given a personality description that fitted the stereotype of an engineer—"Jack is conservative, careful, and has no interest in politics"—participants assigned the same probability of Jack being an engineer regardless of whether the group was 70% or 30% engineers. They judged by representativeness, not by the base rate. The description swamped the prior.
Manifestations in investment decisions
Base rate neglect in investing takes many forms. Investors who evaluate an IPO based on the company's narrative—a compelling business model, strong early growth, experienced management—often fail to incorporate the base rate: that the average IPO underperforms the market in the three years following listing, and that the fraction of IPOs that deliver outstanding long-run returns is small relative to the fraction that disappoint. The story is vivid and specific; the base rate is statistical and abstract. The heuristic favours the vivid over the statistical.
The same pattern appears in the evaluation of economic forecasts. When a prominent economist predicts a recession based on a detailed causal analysis—inverted yield curve, deteriorating credit conditions, tightening monetary policy—investors often accept the forecast at face value without considering the base rate accuracy of economic recession predictions made twelve months in advance, which is poor across all forecasters including the most prominent ones.
In portfolio construction, base rate neglect underlies the overweighting of specific investment theses at the expense of diversification. An investor highly confident in a specific investment may concentrate a large fraction of the portfolio in it—without adequately weighting the base rate probability that specific bets, even well-reasoned ones, fail more often than the investor expects. This is directly related to overconfidence bias.
The inside view versus the outside view
Daniel Kahneman distinguishes between the inside view—evaluating a situation based on its specific features, timeline, and plan—and the outside view—asking how similar situations have typically turned out in the past. The inside view is what comes naturally; the outside view requires deliberate effort. Base rate neglect is the consistent overweighting of the inside view. The corrective is to explicitly identify the reference class for any investment decision, establish the base rate for that class, and use it as the starting point for the assessment rather than as an afterthought.
Implications for systematic investing
Systematic investing is, in part, a structural defence against base rate neglect. A quantitative strategy that allocates based on factor signals—momentum, value, quality—is implicitly incorporating base rate information: the signals are derived from historical analysis of large samples of investment situations, not from the specific narrative of one investment at one point in time. The strategy replaces inside-view specific assessment with outside-view statistical inference, applied consistently across a diversified portfolio.
Base rate neglect in pfolio
pfolio's systematic strategies operate on statistical signals applied uniformly across the asset universe. Each asset is evaluated on the same momentum and volatility measures, computed from its own price history, with no weighting given to specific narratives about why this time is different for any individual asset. The selection process is, in effect, a base-rate framework: assets earn allocation by passing the same statistical threshold, not by being supported by a compelling story. The signal methodology is documented at how we build portfolios.
Related articles
Disclaimer
Get started now
