Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
Post Reply
User avatar
SimpleGift
Posts: 3121
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Post by SimpleGift » Sat Apr 28, 2018 4:39 pm

This topic has been discussed previously on the Forum, but I always find it fascinating. S&P recently published the chart below, showing the distribution of S&P 500 stocks' cumulative returns for the 20-year period from 1997-2017. Unlike in Lake Wobegon (where all children are above average), in the real world of positively skewed returns, most stock returns are below average.
My understanding is this skewness in returns is common in stock markets worldwide and is a primary reason why active managers underperform most of the time. Active managers hold only a minority of names in the index. Every stock they select has a 50/50 chance of being above median — but since returns are so skewed, less than a 50/50 chance of being above average.

Thoughts on this?
Last edited by SimpleGift on Sat Apr 28, 2018 4:49 pm, edited 1 time in total.
Cordially, Todd

rkhusky
Posts: 5898
Joined: Thu Aug 18, 2011 8:09 pm

Re: Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Post by rkhusky » Sat Apr 28, 2018 4:44 pm

SimpleGift wrote:
Sat Apr 28, 2018 4:39 pm
Every stock they select has a 50/50 chance of being above median — but since returns are so skewed, less than a 50/50 chance of being above average.
That's assuming that the odds of a stock performing above average is random. It is likely that there is some merit to financial analysis, which closes the gap for active investors. Unfortunately, for the majority of them, it is not sufficient to overcome the demonstrated skewness.

User avatar
SimpleGift
Posts: 3121
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Re: Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Post by SimpleGift » Sat Apr 28, 2018 6:15 pm

If interested in more on this topic, these two recent papers offer in-depth perspectives:

Why Indexing Works, Heaton, Polson & Witte, May 2017.
Heaton et. al. wrote:We develop a simple stock selection model to explain why active equity managers tend to underperform a benchmark index. We motivate our model with the empirical observation that the best performing stocks in a broad market index often perform much better than the other stocks in the index. Randomly selecting a subset of securities from the index may dramatically increase the chance of underperforming the index.

The relative likelihood of underperformance by investors choosing active management likely is much more important than the loss those same investors take due to the higher fees of active management relative to passive index investing. Thus, active management may be even more challenging than previously believed, and the stakes for finding the best active managers may be larger than previously assumed.
Do Stocks Outperform Treasury Bills?, Hendrik Bessembinder, November 2017.
Bessembinder wrote:Four out of every seven common stocks that have appeared in the CRSP database since 1926 have lifetime buy-and-hold returns less than one-month Treasuries. When stated in terms of lifetime dollar wealth creation, the best-performing four percent of listed companies explain the net gain for the entire U.S. stock market since 1926, as other stocks collectively matched Treasury bills. These results highlight the important role of positive skewness in the distribution of individual stock returns, attributable both to skewness in monthly returns and to the effects of compounding. The results help to explain why poorly-diversified active strategies most often underperform market averages.
Cordially, Todd

asset_chaos
Posts: 1405
Joined: Tue Feb 27, 2007 6:13 pm
Location: Melbourne

Re: Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Post by asset_chaos » Sat Apr 28, 2018 9:29 pm

It looks like what's called a log-normal distribution, and at least some other economic processes seem to approximately follow such a distribution. For example, one I know of is real estate prices. The text book way to get a log-normal distribution is to multiply random numbers. Multiply a few numbers that are much larger than the typical number, and you get that large tail. This is in contrast to the more familiar normal distribution which occurs when the underlying process has a typical value with variations around it, like height. Normal distributions come from adding random numbers.

In the text book examples determining the average of a log-normal distribution is hard because the average moves depending on the amount you're able to sample the underlying distribution. This data says that the S&P 500 return is dominated by the relatively few companies in that long tail. This data reinforces for me two stock investing messages. (1) It is always a stock pickers market. Why don't they do better? If active managers can identify in advance even the above median companies, they should always be able to beat the market averages. That that's not observed says that that identification process cannot be reliably done. (2) As Bogle has said, don't look for the needle, buy the haystack. A total market index fund owns all the companies in the positive return tail.

Good stuff; thanks for sharing.
Regards, | | Guy

User avatar
SimpleGift
Posts: 3121
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Re: Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Post by SimpleGift » Sat Apr 28, 2018 11:23 pm

asset_chaos wrote:
Sat Apr 28, 2018 9:29 pm
This data reinforces for me two stock investing messages. (1) It is always a stock pickers market. Why don't they do better? If active managers can identify in advance even the above median companies, they should always be able to beat the market averages. That that's not observed says that that identification process cannot be reliably done. (2) As Bogle has said, don't look for the needle, buy the haystack. A total market index fund owns all the companies in the positive return tail.
Good points. One often hears an argument by active managers along the lines of "we only invest in our best ideas, rather than all those obvious loser stocks in the index." But it would seem, when returns have such a positive skew, that concentrated portfolios would be less likely to outperform.

In other words, if performance is driven by a small number of stocks, the more concentrated a portfolio is, the less likely it is to own the big winners — unless managers happen to be extremely lucky with their picks.
Cordially, Todd

User avatar
celia
Posts: 8600
Joined: Sun Mar 09, 2008 6:32 am
Location: SoCal

Re: Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Post by celia » Sat Apr 28, 2018 11:53 pm

Question: The first two columns are for negative returns. What does -100% mean? Did the company lose all of its value (ie, go bankrupt)?

If so, when a company on the S&P loses too much of its value, it is no longer one of the 500 largest companies and is replaced with another company. Is the replacement company factored in for the remaining time period? I would think that most of the 500 largest companies that lost half of their value (ie, those in the first column) would be removed from the index.

User avatar
SimpleGift
Posts: 3121
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Re: Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Post by SimpleGift » Sun Apr 29, 2018 12:20 am

celia wrote:
Sat Apr 28, 2018 11:53 pm
Question: The first two columns are for negative returns. What does -100% mean? Did the company lose all of its value (ie, go bankrupt)?
The first two columns on the left of the histogram chart I read as intervals: -100% to -50% and -50% to 0%. So the cumulative returns of companies in these first two columns for the 1997-2017 period were somewhere between 0% and -100% — and not many were likely completely worthless stocks at 100% loss.

Further, I believe that S&P accounts for survivorship bias in reporting their 500 Index returns — that is, as it drops companies for various reasons, it permanently retains the performance of those deficient companies up through the date that it drops them.
Cordially, Todd

User avatar
alpine_boglehead
Posts: 243
Joined: Fri Feb 17, 2017 9:51 am
Location: Austria

Re: Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Post by alpine_boglehead » Sun Apr 29, 2018 1:40 am

Thanks for posting this interesting chart and article. The term given to this skewed distribution by an earlier study is "The Capitalism Distribution". Seen simplistically, his sounds like the good old 80/20 rule.

Yet another good reason for indexing instead of taking chances active management.

AlohaJoe
Posts: 3925
Joined: Mon Nov 26, 2007 2:00 pm
Location: Saigon, Vietnam

Re: Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Post by AlohaJoe » Sun Apr 29, 2018 3:57 am

SimpleGift wrote:
Sat Apr 28, 2018 4:39 pm
Bogle once wrote "reversion to the mean is an Iron Law of stock investing".

I've often wondered how to square notions of reversion to the mean (which are nearly pervasive, at least at a gut feel "the market's too high" or "it'll bounce back" level) with charts like the above -- or Bessembinder's paper -- which clearly show that most stocks don't revert to any kind of mean.

User avatar
SimpleGift
Posts: 3121
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Re: Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Post by SimpleGift » Sun Apr 29, 2018 9:41 am

AlohaJoe wrote:
Sun Apr 29, 2018 3:57 am
Bogle once wrote "reversion to the mean is an Iron Law of stock investing".

I've often wondered how to square notions of reversion to the mean (which are nearly pervasive, at least at a gut feel "the market's too high" or "it'll bounce back" level) with charts like the above -- or Bessembinder's paper -- which clearly show that most stocks don't revert to any kind of mean.
Since Mr. Bogle makes the distinction between the market's speculative return (P/E changes) and fundamental return (earnings growth), I've always thought his references to mean reversion were mostly about the market's speculative periods of irrational exuberance and pessimism — the whole notion of investors' "animal spirits."

While mean reversion may have short-term significance on this level of animal spirits, I haven't seen much evidence that it holds over longer time periods — i.e., investors are clearly willing to pay different amounts for the same dollar of earnings in different eras. And there's not much evidence that mean reversion holds at all for individual stocks over the long term.
Cordially, Todd

User avatar
grayfox
Posts: 4939
Joined: Sat Sep 15, 2007 4:30 am

Re: Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Post by grayfox » Sun Apr 29, 2018 1:27 pm

SimpleGift wrote:
Sat Apr 28, 2018 4:39 pm
This topic has been discussed previously on the Forum, but I always find it fascinating. S&P recently published the chart below, showing the distribution of S&P 500 stocks' cumulative returns for the 20-year period from 1997-2017. Unlike in Lake Wobegon (where all children are above average), in the real world of positively skewed returns, most stock returns are below average.
My understanding is this skewness in returns is common in stock markets worldwide and is a primary reason why active managers underperform most of the time. Active managers hold only a minority of names in the index. Every stock they select has a 50/50 chance of being above median — but since returns are so skewed, less than a 50/50 chance of being above average.

Thoughts on this?
Inside of trying to pick the very few stocks that are above average, why not try to pick some of the worst and leave them out. It looks like about 50% of the stocks had -100% to 0%.

If you pick one stock at random, according to the chart, there should be 50% chance that it will have 0 or negative return.

Start with 500 stocks and pull out the some of the losers. Maybe remove 1, 2, 5, 10, 20 or 50 stocks with the worst prospects.

buylowbuyhigh
Posts: 48
Joined: Thu Dec 21, 2017 8:34 am

Re: Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Post by buylowbuyhigh » Sun Apr 29, 2018 2:18 pm

grayfox wrote:
Sun Apr 29, 2018 1:27 pm
Maybe remove 1, 2, 5, 10, 20 or 50 stocks with the worst prospects.
But aren't these the value stocks that used to outperform? Maybe delete the likely winners instead :D

Good analogue could maybe be the median and mean household income. For a random household the most likely income is the median one, which is almost always clearly smaller than the mean income of all households. This seems to be the case also in stock picking. Index funds guarantee the mean return, better than the median that one could expect with traditional concentrated stock portfolios or active funds.

User avatar
SimpleGift
Posts: 3121
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Re: Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Post by SimpleGift » Sun Apr 29, 2018 2:30 pm

grayfox wrote:
Sun Apr 29, 2018 1:27 pm
Inside of trying to pick the very few stocks that are above average, why not try to pick some of the worst and leave them out.
I'm sure this is the aim of many active fund managers. But just as one can't reliably predict the big winners, I imagine it's also difficult to reliably choose the big losers. And in the end, this isn't really necessary for someone willing to adopt a passive strategy, since the big winners have historically far overshadowed the big losers.

In the chart below, the blue line is the total net contribution of the 10 best and 10 worst S&P 500 stocks over the 1963-2015 period. Most of the time, the top 10 stocks pull the market up more than the bottom 10 stocks drag it down. The notable exceptions are during major bear markets. So historically, in order to reap positive market gains, there's been no need to avoid the worst losers.
This positive skew toward the top performers is likely due to the built-in structure of the market, since stocks can only go down -100%, but can go up any amount. Plus, investors also seem to have a built-in behavioral bias toward glamorous growth stocks.
Cordially, Todd

golfCaddy
Posts: 728
Joined: Wed Jan 10, 2018 10:02 pm

Re: Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Post by golfCaddy » Sun Apr 29, 2018 3:05 pm

That's an interesting chart, but most fund managers don't own just one stock. I think the average is about 90. If you took the average of 90 stocks, sampled at random, your distribution would start to look more like a bell curve.

b0B
Posts: 353
Joined: Fri Oct 19, 2018 11:39 am

Re: Chart: Skewed Distribution of S&P 500 Returns, 1997-2017

Post by b0B » Wed Dec 05, 2018 4:56 pm

golfCaddy wrote:
Sun Apr 29, 2018 3:05 pm
That's an interesting chart, but most fund managers don't own just one stock. I think the average is about 90. If you took the average of 90 stocks, sampled at random, your distribution would start to look more like a bell curve.
More diversification helps, but it would still have geo mean less than the whole market, so they are still hurt by concentrated bets. Even without fees/costs, most active traders/managers should lose compared to the market.

Post Reply