Bogleheads Home Bogleheads
Investing Advice Inspired by Jack Bogle
 
  WikiWiki    FAQFAQ    SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

S&P 500 Visualization: Worst. Year. Ever.
Go to page 1, 2  Next
 
Post new topic   Reply to topic    Bogleheads Forum Index -> Investing - Theory, News & General
View previous topic :: View next topic  
Author Message
Dan Kohn



Joined: 26 Jun 2007
Posts: 1504
Location: New York, NY

PostPosted: Wed Dec 03, 2008 9:32 am    Post subject: S&P 500 Visualization: Worst. Year. Ever. Reply with quote

I love this visualization of returns of the S&P 500. Who knew it's been around since 1825? And what a great illustration of the Normal Distribution.



Source is Daily Kos.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Roverdog



Joined: 02 May 2008
Posts: 457

PostPosted: Wed Dec 03, 2008 9:42 am    Post subject: Reply with quote

Dan,

Thanks for the graph (and reference).

I give a few lectures in a financial planning elective here at school, and that's a keeper! Smile

Bob
Back to top
View user's profile Send private message
Henry Sadovsky



Joined: 08 May 2008
Posts: 748

PostPosted: Wed Dec 03, 2008 9:57 am    Post subject: How normal is normal? Reply with quote

Hi Dan.

That is almost certainly not a normal distribution. You might take a look at: The Misbehavior of Markets: A Fractal View of Risk, Ruin & Reward by Benoit Mandelbrot.

H.
_________________
"What we can't say we can't say, and we can't whistle it either."
Frank P. Ramsey"


(f.k.a. Zalzel)
Back to top
View user's profile Send private message
mark500



Joined: 11 Oct 2008
Posts: 316

PostPosted: Wed Dec 03, 2008 10:18 am    Post subject: Reply with quote

That is almost certainly not a normal distribution.

Help me with statistics here. It looks similar to a bell curve.
Back to top
View user's profile Send private message
superlight



Joined: 31 Mar 2008
Posts: 1254

PostPosted: Wed Dec 03, 2008 10:24 am    Post subject: Reply with quote

mark500 wrote:
That is almost certainly not a normal distribution.

Help me with statistics here. It looks similar to a bell curve.


Basically it's the "similar" that gets us. It makes real risks harder (impossible) to compute. On the other hand, from [the] graph we can see that the S&P 500 pays off more often than not, which is why it's still here, after all those years.
_________________
"Simplicity is the ultimate sophistication."
Back to top
View user's profile Send private message
Rodc



Joined: 26 Jun 2007
Posts: 5395

PostPosted: Wed Dec 03, 2008 10:42 am    Post subject: Reply with quote

mark500 wrote:
That is almost certainly not a normal distribution.

Help me with statistics here. It looks similar to a bell curve.


Two things. I don't think that graphical depiction is centered on the mean. A minor nit.

Notice in particular that the right side tail is rather different from the left side tail.

That said, if you play with a normal random number generator, and pick say 100 samples and plot hem, repeat, repeat, repeat, you see lots of graphs that are not all that normal looking. Do it with 1000 or 10,000 and things start to smooth out a lot better.

Also to the OP, the S&P 500 does not extend back to 1825. I'd have to look it up, but prior to about the 1950s those data are reconstructed by people looking back at stock records. My understanding that much before 1900, maybe 1920s those data reconstructions are pretty shaky.

Still, none of that invalidates the graph showing that is year is truly horrible by any historical measure.
_________________
"all standard caveats apply"
Back to top
View user's profile Send private message
stratton



Joined: 04 Mar 2007
Posts: 8199
Location: Puget Sound

PostPosted: Wed Dec 03, 2008 10:54 am    Post subject: Reply with quote

Considering current daily volatility we could be at -25 or -30% by the end of the year. Or down more.

Paul
Back to top
View user's profile Send private message
ziggy29



Joined: 10 Mar 2008
Posts: 915
Location: Texas Hill Country

PostPosted: Wed Dec 03, 2008 10:56 am    Post subject: Reply with quote

stratton wrote:
Considering current daily volatility we could be at -25 or -30% by the end of the year. Or down more.

And wait until the tax loss harvesting starts up in earnest...
Back to top
View user's profile Send private message
jeffyscott



Joined: 27 Feb 2007
Posts: 3383
Location: Wisconsin

PostPosted: Wed Dec 03, 2008 11:13 am    Post subject: Reply with quote

Not sure that looking at discrete calendar years is all that meaningful in putting declines in perspective. 1973-74 saw about a 50% decline as did 2000-2002, I don't know that having it all in one calendar year (hopefully) this time really makes much difference.
_________________
Jeffy

press on, regardless - John C. Bogle
Back to top
View user's profile Send private message
maywood



Joined: 19 Jun 2007
Posts: 132

PostPosted: Wed Dec 03, 2008 11:22 am    Post subject: Reply with quote

Very interesting, thanks for posting!

The only good thing I could come up with was, if someone survived this year they should be able to survive anything!

Maywood
Back to top
View user's profile Send private message
TheEternalVortex



Joined: 27 Feb 2007
Posts: 1713
Location: San Jose, CA

PostPosted: Wed Dec 03, 2008 11:44 am    Post subject: Reply with quote

Where is the source for the data? The S&P500 doesn't go back that far...
Back to top
View user's profile Send private message AIM Address
dbr



Joined: 04 Mar 2007
Posts: 5397

PostPosted: Wed Dec 03, 2008 11:49 am    Post subject: Reply with quote

TheEternalVortex wrote:
Where is the source for the data? The S&P500 doesn't go back that far...


According to the blogger the S&P goes back to 1825!?!?
Back to top
View user's profile Send private message
tadamsmar



Joined: 07 May 2007
Posts: 2474

PostPosted: Wed Dec 03, 2008 11:49 am    Post subject: Reply with quote

mark500 wrote:
That is almost certainly not a normal distribution.

Help me with statistics here. It looks similar to a bell curve.


It is similar to a bell curve.

The OP called it an great illustration of a normal distribution. But it is really just a sample. It could be considered the sort of sample that one would get by sampling a normal distribution. Also, its the sort of sample that one would get from sampling any of an infinite number of other distributions.

The claim that it is almost certainly not a normal distribution is a claim about the underlying theoretical distribution from which the sample is drawn. But this has nothing to do with the sample, the sample does not prove it's not normal. It's a claim based on some other theory or observations.
Back to top
View user's profile Send private message
TheEternalVortex



Joined: 27 Feb 2007
Posts: 1713
Location: San Jose, CA

PostPosted: Wed Dec 03, 2008 11:52 am    Post subject: Reply with quote

I looked at it from 1950-2008 and it doesn't look normal, but a histogram is a poor way of judging normality.

Running an Anderson-Darling test gives a p-value of 0.442, which is consistent with the data being normal. However to disprove normality for something that is close to normal one normally needs more data points than ~50.
Back to top
View user's profile Send private message AIM Address
dbr



Joined: 04 Mar 2007
Posts: 5397

PostPosted: Wed Dec 03, 2008 11:56 am    Post subject: Reply with quote

PS With respect to the statistics, there are tests for normality and characterizations of properties of data distributions that reflect the shape of the distribution. In the end it is usually appropriate to find what sort of distribution is a good fit to the data, if any of the usual suspects is appropriate at all.

That data might not be normally distributed and that data might not be well described by any of a variety of mathematically tractable representations are two completely different issues. A different issue is the problem of how to represent rare events in a distribution that is generally well matched to a standard model overall. All of these problems are not novel and are much studied by those who need to deal with the relevant subjects. It may well be that success in this area when applied to markets leaves much to be desired.
Back to top
View user's profile Send private message
tadamsmar



Joined: 07 May 2007
Posts: 2474

PostPosted: Wed Dec 03, 2008 11:58 am    Post subject: Reply with quote

dbr wrote:
TheEternalVortex wrote:
Where is the source for the data? The S&P500 doesn't go back that far...


According to the blogger the S&P goes back to 1825!?!?


The Wikipedia says the 500 goes back to 1957. Everything before that is a reconstruction, or based on some similar index. There were S&P indexes before 1957, but not with 500 stocks.

I am not sure why the reconstructions stopped in 1825. Maybe there were just too few joint stock companies before that, or maybe there are no records.
Back to top
View user's profile Send private message
Rick Ferri



Joined: 26 Feb 2007
Posts: 3627
Location: Home on the range in Medina, Texas

PostPosted: Wed Dec 03, 2008 11:58 am    Post subject: Reply with quote

Dan,

Maybe the axiom that for every action there is an equal and opposite reaction holds true for the stock market next year.

Rick Ferri
Back to top
View user's profile Send private message Send e-mail Visit poster's website
regmac



Joined: 17 Dec 2007
Posts: 22

PostPosted: Wed Dec 03, 2008 11:59 am    Post subject: Re: S&P 500 Visualization: Worst. Year. Ever. Reply with quote

[quote="Dan Kohn"]I love this visualization of returns of the S&P 500. Who knew it's been around since 1825? And what a great illustration of the Normal Distribution.

The graph supports two points of view that we’ve been hearing for quite sometime: 1) Gary Schilling, who has been right on every call he’s made throughout this horrible year, places a fair value of “600” on the S&P; 2) Myriad analysts keep insisting that at these levels we are being presented with a buying opportunity that will not present itself again in our lifetime.
Back to top
View user's profile Send private message
tadamsmar



Joined: 07 May 2007
Posts: 2474

PostPosted: Wed Dec 03, 2008 12:01 pm    Post subject: Reply with quote

Rick Ferri wrote:
Dan,

Maybe the axiom that for every action there is an equal and opposite reaction holds true for the stock market next year.

Rick Ferri


Yes, but what if the 1990's is the action, and this is the reaction?
Back to top
View user's profile Send private message
dbr



Joined: 04 Mar 2007
Posts: 5397

PostPosted: Wed Dec 03, 2008 12:04 pm    Post subject: Re: S&P 500 Visualization: Worst. Year. Ever. Reply with quote

[quote="regmac"]
Dan Kohn wrote:
I love this visualization of returns of the S&P 500. Who knew it's been around since 1825? And what a great illustration of the Normal Distribution.

The graph supports two points of view that we’ve been hearing for quite sometime: 1) Gary Schilling, who has been right on every call he’s made throughout this horrible year, places a fair value of “600” on the S&P; 2) Myriad analysts keep insisting that at these levels we are being presented with a buying opportunity that will not present itself again in our lifetime.


It would certainly be interesting to debate who has made the wisest decision between those dumping everything into TIPS and those dumping everything into equities. Should we have another Swedroe vs Ferri Razz
Back to top
View user's profile Send private message
bobcat2



Joined: 20 Feb 2007
Posts: 1941

PostPosted: Wed Dec 03, 2008 12:22 pm    Post subject: Index is not down 50% Reply with quote

How did they arrive that the S&P 500 index is down 50% in 2008? According to M* the Vanguard S&P 500 Index fund is down about 41% year-to-date and also down about 41% over the last twelve months. Thru the end of November the Index was down almost 38%. Those are really bad numbers, but they are not 50%.

Bob K
_________________
The real difficulty in changing any enterprise lies not in developing new ideas, but in escaping from the old ones. - JM Keynes
Good questions outrank easy answers. - PA Samuelson
Back to top
View user's profile Send private message
tadamsmar



Joined: 07 May 2007
Posts: 2474

PostPosted: Wed Dec 03, 2008 12:27 pm    Post subject: Re: Index is not down 50% Reply with quote

bobcat2 wrote:
How did they arrive that the S&P 500 index is down 50% in 2008? According to M* the Vanguard S&P 500 Index fund is down about 41% year-to-date and also down about 41% over the last twelve months. Those are really bad numbers, but they are not 50%.

Bob K


It think that "-50" must mean "-40 to -50". Look at the X axis tick labels, that's all that makes sense.
Back to top
View user's profile Send private message
tadamsmar



Joined: 07 May 2007
Posts: 2474

PostPosted: Wed Dec 03, 2008 12:31 pm    Post subject: Re: S&P 500 Visualization: Worst. Year. Ever. Reply with quote

[quote="regmac"]
Dan Kohn wrote:
I love this visualization of returns of the S&P 500. Who knew it's been around since 1825? And what a great illustration of the Normal Distribution.

The graph supports two points of view that we’ve been hearing for quite sometime: 1) Gary Schilling, who has been right on every call he’s made throughout this horrible year, places a fair value of “600” on the S&P; 2) Myriad analysts keep insisting that at these levels we are being presented with a buying opportunity that will not present itself again in our lifetime.


Well, I think we got down to these P/E levels in 2003, and we will never see it again in our lifetimes? Is there an asteroid out there?
Back to top
View user's profile Send private message
bobcat2



Joined: 20 Feb 2007
Posts: 1941

PostPosted: Wed Dec 03, 2008 12:49 pm    Post subject: tadamsmar Reply with quote

Yes, you are right. -50% does not mean -50% or worse. Instead it means worse than -40%, but better than -50%. So if one draws the graph using end of November data, 2008 returns would not be in the left most column, but instead one column to the right. But if you draw the graph with data so far this week then 2008 gets dumped into the left most column. So where will it be by December 31? It could stay in the left column, move one column rightward, or find itself in a brand new -60% column. Laughing

Bob K

edit - It looks like the graph is correct (2008 worse than 1931) using returns from close of business on Monday, but incorrect as of close of business on Tuesday. Who knows how accurate by close of business today Question
_________________
The real difficulty in changing any enterprise lies not in developing new ideas, but in escaping from the old ones. - JM Keynes
Good questions outrank easy answers. - PA Samuelson


Last edited by bobcat2 on Wed Dec 03, 2008 12:56 pm; edited 1 time in total
Back to top
View user's profile Send private message
fluffyistaken



Joined: 04 Apr 2008
Posts: 1110

PostPosted: Wed Dec 03, 2008 12:52 pm    Post subject: Re: S&P 500 Visualization: Worst. Year. Ever. Reply with quote

[quote="tadamsmar"]
regmac wrote:
Dan Kohn wrote:
I love this visualization of returns of the S&P 500. Who knew it's been around since 1825? And what a great illustration of the Normal Distribution.

The graph supports two points of view that we’ve been hearing for quite sometime: 1) Gary Schilling, who has been right on every call he’s made throughout this horrible year, places a fair value of “600” on the S&P; 2) Myriad analysts keep insisting that at these levels we are being presented with a buying opportunity that will not present itself again in our lifetime.


Well, I think we got down to these P/E levels in 2003, and we will never see it again in our lifetimes? Is there an asteroid out there?


P/E is definitely lower now than in 2003: http://www2.standardandpoors.c....EPSEST.XLS
Back to top
View user's profile Send private message
tadamsmar



Joined: 07 May 2007
Posts: 2474

PostPosted: Wed Dec 03, 2008 2:52 pm    Post subject: Reply with quote

The comments section of the link in the OP is interesting.

Anyone yearning for a good economics thread with politics and crude humor mixed in should check it out, the stuff we can't have here.

In particular, the photo of the guy carrying the "Jump you &%$s!" sign past the NYSE.
Back to top
View user's profile Send private message
Kenster1



Joined: 28 Feb 2007
Posts: 2737

PostPosted: Wed Dec 03, 2008 2:58 pm    Post subject: Reply with quote

http://www.marketwatch.com/new....6DFD1EE%7D

The S&P's dividend yield now higher than the 10-year T-note's

Quote:
For the first time since 1958, the dividend yield on the Standard & Poor's 500 index has risen above the interest rate on the 10-year Treasury note. As of Tuesday's close, for example, the S&P's dividend yield was 3.3%, while the 10-year T-Note was yielding 2.7% -- a spread of 0.6 percentage points in favor of stocks.

To put this in historical context, the spread since 1958 has averaged 3.7 percentage points in favor of bonds.


Quote:
Cliff Asness, however, thinks he has come up with some clear answers. He is managing and founding principal at AQR Capital Management, a Greenwich, CT-based quantitative research firm. In the March/April 2000 issue of Financial Analysts Journal, Asness argued that neither the pre-1958 period nor the decades since are anomalous. On the contrary, he found that -- below the surface -- stock and bond yields have always been strongly positively correlated.

The reason that this strong correlation was hidden, according to Asness, is that investors' expectations have changed of the relative volatility of stocks and bonds. Prior to the last 50 years, investors expected the stock market's volatility to be much greater than bond market volatility. To entice investors to incur that greater volatility, the stock market had to provide a higher yield than bonds.

These expected relative volatilities changed early in the post-war period, according to Asness. And that is why -- beginning in 1958 -- the stock market's yield dropped below that of bonds, and stayed there for five decades.

The situation has reversed itself in recent months, according to Asness' theory, given stocks' extraordinary volatility.

To bet on whether stocks' dividend yield recent move above the T-Note yield will persist, therefore, you have to bet on whether investors will expect stocks' volatility, relative to bonds', to remain considerably higher than they have averaged over the last 50 years.

Note that the key element of this bet is investors' expectations. Asness points out that investors' memories live for a very long time. The memory of the Great Depression lingered for years after it ended, for example, which is one reason why stocks' dividend yields remained so high.

In contrast, the Baby Boom generation (at least possibly until now) had no traumatic memory similar to their parents' memory of the Great Depression. That in turn helps to explain why stocks' dividend yield slipped in the 1960s and beyond, relative to bonds' yields.

_________________
SURGEON GENERAL'S WARNING: Any overconfidence in your ability, willingness and need to take risk may be hazardous to your health.
Back to top
View user's profile Send private message
grayfox



Joined: 15 Sep 2007
Posts: 1936
Location: Anytown, USA

PostPosted: Wed Dec 03, 2008 3:09 pm    Post subject: Let's go for the record Reply with quote

Come one, people. How often do you get the chance to be part of something big like this? This is a rare opportunity--a once in a lifetime event.

So are we going to let 1931 beat us?

All they had back then was telephones, pencils, paper, ledgers and balance sheets.

We now have computers, high-speed networks, the internet, modern portfolio theory, ETFs, credit default swaps, and mortgage-backed securities.
We now have Financial Weapons of Mass Destruction--something that Wall Street professionals in 1931 couldn't have imagined in their wildest dreams!

I say go for the record. Let's bring it down 60% for 2008 !

Are you with me?
I said, are you with me?
Back to top
View user's profile Send private message Visit poster's website
jeffyscott



Joined: 27 Feb 2007
Posts: 3383
Location: Wisconsin

PostPosted: Wed Dec 03, 2008 3:16 pm    Post subject: Reply with quote

Cliff Asness
Quote:
To bet on whether stocks' dividend yield recent move above the T-Note yield will persist, therefore, you have to bet on whether investors will expect stocks' volatility, relative to bonds', to remain considerably higher than they have averaged over the last 50 years.


I don't understand that comment. Why can't I simply buy stocks and happily collect the (higher than bond) dividends, without it being a "bet" that they will remain above bonds?
_________________
Jeffy

press on, regardless - John C. Bogle
Back to top
View user's profile Send private message
bearwolf



Joined: 18 May 2008
Posts: 1089
Location: Oklahoma

PostPosted: Wed Dec 03, 2008 3:18 pm    Post subject: Re: Let's go for the record Reply with quote

grayfox wrote:
Come one, people. How often do you get the chance to be part of something big like this? This is a rare opportunity--a once in a lifetime event.

So are we going to let 1931 beat us?

...


Am I the only one seeing the good part of the graph? When I look at the graph I see the Up years that follow the down years. Let's just make it through the next year or so then start reaping the reward.

BearWolf
Back to top
View user's profile Send private message
Christine_NM



Joined: 20 Feb 2007
Posts: 862
Location: Albuquerque

PostPosted: Wed Dec 03, 2008 3:26 pm    Post subject: Reply with quote

It's the uptick rule, which began after 1937 and ended in July 2007. 1931, 1937, and 2008 all are years with no uptick rule.

Without the uptick rule, volatility will continue and there will be no "back to normal" for the market.

Facing this, in order to maintain an overall portfolio at an unchanged risk tolerance then one must permanently reduce the equity allocation. Or reduce it till the SEC comes to its senses and restores the uptick rule.

I'm not sure buy and hold makes sense without an uptick rule. So far, it doesn't. Something like Taleb's 5-10% stock trading and 90-95% Treasuries is more appropriate for a riskier (more volatile) market.
Back to top
View user's profile Send private message
Rodc



Joined: 26 Jun 2007
Posts: 5395

PostPosted: Wed Dec 03, 2008 3:29 pm    Post subject: Re: Let's go for the record Reply with quote

grayfox wrote:
Come one, people. How often do you get the chance to be part of something big like this? This is a rare opportunity--a once in a lifetime event.

So are we going to let 1931 beat us?

All they had back then was telephones, pencils, paper, ledgers and balance sheets.

We now have computers, high-speed networks, the internet, modern portfolio theory, ETFs, credit default swaps, and mortgage-backed securities.
We now have Financial Weapons of Mass Destruction--something that Wall Street professionals in 1931 couldn't have imagined in their wildest dreams!

I say go for the record. Let's bring it down 60% for 2008 !

Are you with me?
I said, are you with me?


I'm with you!

If I'm going to go through this much pain, I want a record dang it!

Then we can have a record recovery.
_________________
"all standard caveats apply"
Back to top
View user's profile Send private message
jeffyscott



Joined: 27 Feb 2007
Posts: 3383
Location: Wisconsin

PostPosted: Wed Dec 03, 2008 3:55 pm    Post subject: Reply with quote

I hope those of you begging for -60% for 2008 realize that this would mean 587 which would be -31% from yesterdays close.
_________________
Jeffy

press on, regardless - John C. Bogle
Back to top
View user's profile Send private message
TheEternalVortex



Joined: 27 Feb 2007
Posts: 1713
Location: San Jose, CA

PostPosted: Wed Dec 03, 2008 3:59 pm    Post subject: Reply with quote

Christine_NM wrote:
It's the uptick rule, which began after 1937 and ended in July 2007. 1931, 1937, and 2008 all are years with no uptick rule.

Without the uptick rule, volatility will continue and there will be no "back to normal" for the market.

Facing this, in order to maintain an overall portfolio at an unchanged risk tolerance then one must permanently reduce the equity allocation. Or reduce it till the SEC comes to its senses and restores the uptick rule.

I'm not sure buy and hold makes sense without an uptick rule. So far, it doesn't. Something like Taleb's 5-10% stock trading and 90-95% Treasuries is more appropriate for a riskier (more volatile) market.


The existence of volatile years while the rule was in place contradicts your hypothesis.
Back to top
View user's profile Send private message AIM Address
Kenster1



Joined: 28 Feb 2007
Posts: 2737

PostPosted: Wed Dec 03, 2008 4:21 pm    Post subject: Reply with quote

jeffyscott wrote:
I hope those of you begging for -60% for 2008 realize that this would mean 587 which would be -31% from yesterdays close.


And it would be difficult to take advantage of purchasing equities at those lower prices if you don't have a job because of the business downturn! Shocked
_________________
SURGEON GENERAL'S WARNING: Any overconfidence in your ability, willingness and need to take risk may be hazardous to your health.
Back to top
View user's profile Send private message
Rodc



Joined: 26 Jun 2007
Posts: 5395

PostPosted: Wed Dec 03, 2008 4:24 pm    Post subject: Reply with quote

jeffyscott wrote:
I hope those of you begging for -60% for 2008 realize that this would mean 587 which would be -31% from yesterdays close.


I'm willing to "win" by 0.01%
_________________
"all standard caveats apply"
Back to top
View user's profile Send private message
Christine_NM



Joined: 20 Feb 2007
Posts: 862
Location: Albuquerque

PostPosted: Wed Dec 03, 2008 4:46 pm    Post subject: Reply with quote

TheEternalVortex wrote:
Christine_NM wrote:
It's the uptick rule, which began after 1937 and ended in July 2007. 1931, 1937, and 2008 all are years with no uptick rule.

Without the uptick rule, volatility will continue and there will be no "back to normal" for the market.

Facing this, in order to maintain an overall portfolio at an unchanged risk tolerance then one must permanently reduce the equity allocation. Or reduce it till the SEC comes to its senses and restores the uptick rule.

I'm not sure buy and hold makes sense without an uptick rule. So far, it doesn't. Something like Taleb's 5-10% stock trading and 90-95% Treasuries is more appropriate for a riskier (more volatile) market.


The existence of volatile years while the rule was in place contradicts your hypothesis.


No year with an uptick rule has been as volatile in a bad way as the three outliers, 31, 37 and 08. The uptick rule will not cure all ills but will reduce volatility, which is the common measure of risk.

It's like walking a high wire with a net vs. without a net. You might do the former, but not so much the latter. Ignoring the fact that the net has been removed while we are in midwalk (invested in stocks) seems unwise to me.
Back to top
View user's profile Send private message
Pacific



Joined: 06 Mar 2007
Posts: 717
Location: Lost in the middle of the Pacific

PostPosted: Wed Dec 03, 2008 5:18 pm    Post subject: Reply with quote

tadamsmar said:

Quote:
The comments section of the link in the OP is interesting.

Anyone yearning for a good economics thread with politics and crude humor mixed in should check it out, the stuff we can't have here.

In particular, the photo of the guy carrying the "Jump you &%$s!" sign past the NYSE.


I especially liked this one:

Quote:
I kind of like the idea of throwing money at the problem till we get rid of it all. Money is the root of all evil. What we are doing is making sure future generations won't have to worry about it.
Back to top
View user's profile Send private message
tadamsmar



Joined: 07 May 2007
Posts: 2474

PostPosted: Wed Dec 03, 2008 5:31 pm    Post subject: Re: Let's go for the record Reply with quote

grayfox wrote:
Come one, people. How often do you get the chance to be part of something big like this? This is a rare opportunity--a once in a lifetime event.

So are we going to let 1931 beat us?

All they had back then was telephones, pencils, paper, ledgers and balance sheets.

We now have computers, high-speed networks, the internet, modern portfolio theory, ETFs, credit default swaps, and mortgage-backed securities.
We now have Financial Weapons of Mass Destruction--something that Wall Street professionals in 1931 couldn't have imagined in their wildest dreams!

I say go for the record. Let's bring it down 60% for 2008 !

Are you with me?
I said, are you with me?


They had airplanes and radios.
Back to top
View user's profile Send private message
madsinger



Joined: 29 Sep 2007
Posts: 372

PostPosted: Wed Dec 03, 2008 5:34 pm    Post subject: Reply with quote

What I don't like about the original graph in this post is the "arithmetic" as opposed to the "geometric" nature of it. There seems to be an implication that a "+10 to +20" year balances out a "-10 to -20" year. Or that the "+50 to +60" years even "surpass" the losses of the "-40 to -30" years. If the "x-axis" was spaced in a "log" fashion, then the "-50" block would have to be as far to the left as a "+100" block. A "-40" block would be "balanced" by a +67 block.

This would make those blocks on the left side, further to the left, and the "normalness" would be "less normal".

(That multiyear 1930-1932 "-90 to -80" block would need a "+500 to +1000" block on the right to counteract it!).

-Brad.
Back to top
View user's profile Send private message
peter71



Joined: 24 Jul 2007
Posts: 3247

PostPosted: Wed Dec 03, 2008 5:51 pm    Post subject: Reply with quote

madsinger wrote:
What I don't like about the original graph in this post is the "arithmetic" as opposed to the "geometric" nature of it. There seems to be an implication that a "+10 to +20" year balances out a "-10 to -20" year. Or that the "+50 to +60" years even "surpass" the losses of the "-40 to -30" years. If the "x-axis" was spaced in a "log" fashion, then the "-50" block would have to be as far to the left as a "+100" block. A "-40" block would be "balanced" by a +67 block.

This would make those blocks on the left side, further to the left, and the "normalness" would be "less normal".

(That multiyear 1930-1932 "-90 to -80" block would need a "+500 to +1000" block on the right to counteract it!).

-Brad.


Well, the graph is currently skewed to the right so arguably it'd actually make it MORE normal to somehow overweight losses, but there's really no perfect way to do this sort of thing as even the height and width of the boxes affects the appearance. All of that said, my sense is that a lot of the regression techniques used in the econ papers we read assume normality without even bothering to check that assumption as did Eternal Vortex above (and in this case the assumption checks out OK).

All best,
Pete
Back to top
View user's profile Send private message
Bharat



Joined: 18 Feb 2008
Posts: 291
Location: Somewhere between Appalachia and Atlantic

PostPosted: Wed Dec 03, 2008 6:00 pm    Post subject: Re: Let's go for the record Reply with quote

Rodc wrote:
grayfox wrote:
Come one, people. How often do you get the chance to be part of something big like this? This is a rare opportunity--a once in a lifetime event.

So are we going to let 1931 beat us?

All they had back then was telephones, pencils, paper, ledgers and balance sheets.

We now have computers, high-speed networks, the internet, modern portfolio theory, ETFs, credit default swaps, and mortgage-backed securities.
We now have Financial Weapons of Mass Destruction--something that Wall Street professionals in 1931 couldn't have imagined in their wildest dreams!

I say go for the record. Let's bring it down 60% for 2008 !

Are you with me?
I said, are you with me?


I'm with you!

If I'm going to go through this much pain, I want a record dang it!

Then we can have a record recovery.


I am with both of you. I hope that S&P stay under 800 for next couple of years, so that i can have a decent base of investment at good prices. I am willing to live with pain for next few years.
_________________
Everything that you own, owns piece of you.
Back to top
View user's profile Send private message
grayfox



Joined: 15 Sep 2007
Posts: 1936
Location: Anytown, USA

PostPosted: Thu Dec 04, 2008 2:42 am    Post subject: Reply with quote

Business idea for the recession: Make T-shirts that say:

I Lived Through The Crash of 2008
And All I Have Left is this Lousy T-Shirt
Back to top
View user's profile Send private message Visit poster's website
yobria



Joined: 20 Feb 2007
Posts: 2175
Location: SF CA USA

PostPosted: Thu Dec 04, 2008 3:02 am    Post subject: Reply with quote

grayfox wrote:
Business idea for the recession: Make T-shirts that say:

I Lived Through The Crash of 2008
And All I Have Left is this Lousy T-Shirt


Not sure that'd be appropriate - we all lost our shirts in 2008!

Nick
Back to top
View user's profile Send private message Visit poster's website
Murray Boyd



Joined: 19 Feb 2007
Posts: 745

PostPosted: Thu Dec 04, 2008 4:00 am    Post subject: Reply with quote

How can anyone know what the P/E is right now, what with the E crashing and all?
Back to top
View user's profile Send private message
sgr000



Joined: 28 Feb 2007
Posts: 292
Location: Boston

PostPosted: Fri Dec 05, 2008 11:44 am    Post subject: Reply with quote

mark500 wrote:
Henry Sadovsky wrote:
That is almost certainly not a normal distribution.
Help me with statistics here. It looks similar to a bell curve.

[NB: Before you take any of the following analysis seriously, be sure to read and understand the last paragraph.]

As to whether the original data is a histogram derived from a normal distribution, I can offer no opinion. There simply aren't enough data points to tell, now that it's been reduced to a histogram. Almost any test of normality (and I note with approval that somebody tried Anderson-Darling Smile) won't have enough to achieve significance if it were not normal.

However, we can take a look at Shiller's dataset of S&P composite returns from 1871-present. Now, he's done a lot of fiddling around to get a continuous series of index price levels, dividend values, and inflation measures that far back. His web site will tell you all about it; for purposes of this post, I'm going to take him at face value and assume that a highly competent economist such as himself has done a credible job.

So I took his yearly data series and simplified it down until it contained just what we need: Year, Price, Dividend, and CPI. I loaded those into R and repeated his spreadsheet calculations to derive the total return, as well as real (inflation-adjusted) index price level, dividend payment, and thus a real return. At the end of that, I was armed with a time series of year, nominal return, and real return from 1871 - 2006. (I had to fiddle with the 2007 data to supply enough information to calculate returns through 2006; obviously I can't get the 2008 numbers yet.)

R script and input data available from me upon PM request.

Fine. Now, are the returns normally distributed or not? To answer that, I ran 8 different tests of normality on the nominal and real returns. I won't bother you about the details of the differences between the tests, and their different sensitivities, because the verdict is unanimous: the returns are normally distributed.

In the tables that follow, the thing to keep your eye on is the p-value of the test, in the rightmost column. That's the probability of seeing a dataset like yours, with the deviations from normality that eventually creep in, even if the source of the data is really a normal distribution (the "null hypothesis" of the tests). That is, you should only believe the data is not normal if p<0.05 or so. Otherwise you should believe it's normal.

Here's the result for nominal returns, showing the test results in a table and a quantile-quantile plot. In the QQ plot, normal data will lie along the blue diagonal line, so this is a visual confirmation of what the table says. As you can see, it's pretty convincing.
Code:
Test Name                       Statistic  Value  P
=========                       =========  =====  =
Shapiro-Wilk normality          W          0.9970 0.991
Robust Jarque Bera              X-squared  0.0293 0.985
Standardized SJ                 SJ         0.0961 0.463
Anderson-Darling                A          0.1240 0.987
Cramer-von Mises                W          0.0185 0.979
Lilliefors (Kolmogorov-Smirnov) D          0.0379 0.903
Pearson chi-square              P         18.4000 0.104
Shapiro-Francia                 W          0.9960 0.955

Mean   = 10.5%
StdDev = 17.7%
N      = 136


Here's the result for the real returns:
Code:
Test Name                       Statistic  Value  P
=========                       =========  =====  =
Shapiro-Wilk normality          W          0.9950 0.893
Robust Jarque Bera              X-squared  0.0312 0.985
Standardized SJ                 SJ         0.1370 0.447
Anderson-Darling                A          0.1540 0.956
Cramer-von Mises                W          0.0192 0.974
Lilliefors (Kolmogorov-Smirnov) D          0.0385 0.892
Pearson chi-square              P          7.8200 0.799
Shapiro-Francia                 W          0.9960 0.930

Mean   = 8.25%
StdDev = 17.5%
N      = 136


Of course, the real question everybody wants to ask is, "What does that tell us about how crappy this year is?" Nominal returns have a mean of 10.5% and a standard deviation of 17.7%. This year's VFINX (Vanguard 500 Index) return through November was -37.7%. That's in the tail with probability less than 0.323%; figure about once every 300 years.

A cautionary note: This analysis is meaning-free; it's what happens when a statistics nerd gets bored. Impressive as Shiller's data is, it simply does not adequately sample the world to predict a black swan. In fact, that's essentially what Taleb and others are saying: due to the fat-tailed nature of the return distributions, no amount of historical samples can adequately fit a model which includes black swans. So my confident statement that this is a "once every 300 years" market is hogwash (especially when derived from a model with only 136 years of history! Sad).
Back to top
View user's profile Send private message Visit poster's website
grayfox



Joined: 15 Sep 2007
Posts: 1936
Location: Anytown, USA

PostPosted: Fri Dec 05, 2008 11:55 am    Post subject: Reply with quote

Outstanding work! This forum never ceases to amaze me.

I have a request. Would it be possible to post a table of nominal return and real return for each year from 1871 to present?
Back to top
View user's profile Send private message Visit poster's website
Henry Sadovsky



Joined: 08 May 2008
Posts: 748

PostPosted: Fri Dec 05, 2008 12:06 pm    Post subject: Assessing for normality Reply with quote

.
Hi sgr000.

That was very interesting. Does not the use of yearly data unnecessarily limit the power of the analysis? My understanding of the fat-tail question is that it is not a conjecture, but that fat-tails are seen in the analysis of daily data. I don't have the data at hand, but I seem to recall something to the effect that if (U.S. equity) market returns were normally distributed, the probability of seeing days like 10/19/1987, and all the +/- >3% days we are seeing this year would be vanishingly small.
_________________
"What we can't say we can't say, and we can't whistle it either."
Frank P. Ramsey"


(f.k.a. Zalzel)
Back to top
View user's profile Send private message
Dan Kohn



Joined: 26 Jun 2007
Posts: 1504
Location: New York, NY

PostPosted: Fri Dec 05, 2008 12:08 pm    Post subject: Outstanding Reply with quote

sgr000, thanks for the outstanding work.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
sgr000



Joined: 28 Feb 2007
Posts: 292
Location: Boston

PostPosted: Fri Dec 05, 2008 12:23 pm    Post subject: Reply with quote

grayfox wrote:
Outstanding work! This forum never ceases to amaze me.

I have a request. Would it be possible to post a table of nominal return and real return for each year from 1871 to present?

The best way would be to follow the link above to Shiller's yearly data spreadsheet. He calculates real returns only, but you can look at it in Excel to imitate a calculation for nominal returns if you want.

Presumably Shiller is updating his data. I won't be updating this. So what you have below is just a one-shot deal; you're better off going with Shiller.
  • P is the index price at the beginning of the year,
  • D is the dividend paid throughout the year,
  • CPI is the CPI level,
  • RealP is the inflation-adjusted index price (using the current year's CPI),
  • RealD is the inflation-adjusted dividend (using next year's CPI, essentially assuming with Shiller that dividends are paid at the end of the year),
  • RealReturn is the real return: (RealPnextyear - RealP + RealD) / RealP,
  • Return is the nominal return: (Pnextyear - P + D) / P

Code:

Year       P     D     CPI   RealP  RealD RealReturn    Return
1871    4.44  0.26  12.460   72.11  4.159  0.1358000  0.153200
1872    4.86  0.30  12.650   77.74  4.693  0.0886200  0.113200
1873    5.11  0.33  12.940   79.94  5.400  0.0215900 -0.023480
1874    4.66  0.33  12.370   76.26  5.802  0.1228000  0.045060
1875    4.54  0.30  11.510   79.82  5.599  0.1128000  0.048460
1876    4.46  0.30  10.850   83.23  5.550 -0.1443000 -0.136800
1877    3.55  0.19  10.940   65.67  4.167  0.1488000 -0.030990
1878    3.25  0.18   9.229   71.28  4.402  0.2899000  0.156900
1879    3.58  0.20   8.278   87.54  4.052  0.2290000  0.483200
1880    5.11  0.26   9.990  103.50  5.587  0.3387000  0.262200
1881    6.19  0.32   9.419  133.00  6.362 -0.0672900  0.008078
1882    5.92  0.32  10.180  117.70  6.484  0.0551900  0.035470
1883    5.81  0.33   9.990  117.70  7.238  0.0265900 -0.051640
1884    5.18  0.31   9.229  113.60  7.581 -0.0206600 -0.121600
1885    4.24  0.24   8.278  103.70  6.078  0.3288000  0.283000
1886    5.20  0.22   7.992  131.70  5.572  0.1154000  0.115400
1887    5.58  0.25   7.992  141.30  6.044 -0.0488800 -0.003584
1888    5.31  0.23   8.373  128.40  5.825  0.0791900  0.030130
1889    5.24  0.22   7.992  132.70  5.850  0.1221000  0.068700
1890    5.38  0.22   7.612  143.10  5.708 -0.0824200 -0.059480
1891    4.84  0.22   7.802  125.60  6.078  0.2608000  0.183900
1892    5.51  0.24   7.326  152.20  6.152 -0.0150400  0.061710
1893    5.61  0.25   7.897  143.80  7.387 -0.0609300 -0.185400
1894    4.32  0.21   6.850  127.60  6.475  0.0772900  0.032410
1895    4.25  0.19   6.565  131.00  5.774  0.0344200  0.049410
1896    4.27  0.18   6.660  129.80  5.631  0.0607500  0.030440
1897    4.22  0.18   6.470  132.00  5.471  0.1648000  0.199100
1898    4.88  0.20   6.660  148.30  5.993  0.2688000  0.286900
1899    6.08  0.21   6.755  182.20  5.383 -0.1122000  0.037830
1900    6.10  0.30   7.897  156.40  7.879  0.2380000  0.208200
1901    7.07  0.32   7.707  185.70  8.202  0.1650000  0.193800
1902    8.12  0.33   7.897  208.10  7.715 -0.0126500  0.082510
1903    8.46  0.35   8.658  197.80  8.559 -0.1308000 -0.169000
1904    6.68  0.31   8.278  163.30  7.410  0.2790000  0.308400
1905    8.43  0.33   8.468  201.50  7.888  0.2100000  0.210000
1906    9.87  0.40   8.468  235.90  9.150 -0.0342800  0.009119
1907    9.56  0.44   8.849  218.70 10.290 -0.2207000 -0.237400
1908    6.85  0.40   8.658  160.10  9.053  0.3370000  0.381000
1909    9.06  0.44   8.944  205.00  9.001  0.0495000  0.161100
1910   10.08  0.47   9.895  206.20 10.310  0.0360100 -0.033730
1911    9.27  0.47   9.229  203.30 10.420  0.0452900  0.034520
1912    9.12  0.48   9.134  202.10  9.914 -0.0005083  0.072370
1913    9.30  0.48   9.800  192.10  9.716 -0.0674200 -0.048390
1914    8.37  0.42  10.000  169.40  8.417 -0.0655000 -0.056150
1915    7.48  0.43  10.100  149.90  8.369  0.2672000  0.304800
1916    9.33  0.56  10.400  181.60  9.688 -0.0348900  0.085740
1917    9.57  0.69  11.700  165.60  9.976 -0.3101000 -0.174500
1918    7.21  0.57  14.000  104.20  6.993 -0.0091200  0.167800
1919    7.85  0.53  16.500   96.30  5.559  0.0193700  0.192400
1920    8.83  0.51  19.300   92.61  5.433 -0.1234000 -0.137000
1921    7.11  0.46  19.000   75.75  5.510  0.2270000  0.091420
1922    7.30  0.51  16.900   87.43  6.145  0.2967000  0.289000
1923    8.90  0.53  16.800  107.20  6.201  0.0212900  0.051690
1924    8.83  0.55  17.300  103.30  6.435  0.2605000  0.260500
1925   10.58  0.60  17.300  123.80  6.785  0.2104000  0.252400
1926   12.65  0.69  17.900  143.00  7.981  0.1393000  0.113800
1927   13.40  0.77  17.500  155.00  9.009  0.3815000  0.365700
1928   17.53  0.85  17.300  205.10 10.060  0.4838000  0.466600
1929   24.86  0.97  17.100  294.30 11.480 -0.0876900 -0.087690
1930   21.71  0.98  17.100  257.00 12.480 -0.1598000 -0.218800
1931   15.98  0.82  15.900  203.40 11.610 -0.3654000 -0.429300
1932    8.30  0.50  14.300  117.50  7.846  0.0137000 -0.085540
1933    7.09  0.44  12.900  111.30  6.747  0.5135000  0.548700
1934   10.54  0.45  13.200  161.60  6.698 -0.1058000 -0.078750
1935    9.26  0.47  13.600  137.80  6.894  0.5144000  0.536700
1936   13.76  0.72  13.800  201.80 10.340  0.3024000  0.330700
1937   17.59  0.80  14.100  252.50 11.400 -0.3164000 -0.311500
1938   11.31  0.51  14.200  161.20  7.374  0.1667000  0.150300
1939   12.50  0.62  14.000  180.70  9.029  0.0410400  0.033600
1940   12.30  0.67  13.900  179.10  9.618 -0.1007000 -0.087800
1941   10.55  0.71  14.100  151.50  9.154 -0.1794000 -0.086260
1942    8.93  0.59  15.700  115.10  7.067  0.1110000  0.196000
1943   10.09  0.61  16.900  120.90  7.096  0.1994000  0.234900
1944   11.85  0.64  17.400  137.90  7.278  0.1656000  0.192400
1945   13.49  0.66  17.800  153.40  7.340  0.3543000  0.384700
1946   18.02  0.71  18.200  200.40  6.684 -0.2521000 -0.116500
1947   15.21  0.84  21.500  143.20  7.174 -0.0653900  0.030240
1948   14.83  0.93  23.700  126.70  7.844  0.0847200  0.098450
1949   15.36  1.14  24.000  129.50  9.819  0.1981000  0.173200
1950   16.88  1.47  23.500  145.40 11.710  0.2431000  0.343600
1951   21.21  1.41  25.400  169.00 10.770  0.1569000  0.207000
1952   24.19  1.41  26.500  184.80 10.730  0.1363000  0.140600
1953   26.18  1.45  26.600  199.20 10.910  0.0164200  0.027880
1954   25.46  1.54  26.900  191.60 11.670  0.4697000  0.458800
1955   35.60  1.64  26.700  269.90 12.390  0.2814000  0.286200
1956   44.15  1.74  26.800  333.50 12.760  0.0374300  0.068400
1957   45.43  1.79  27.600  333.20 12.670 -0.0885000 -0.055470
1958   41.12  1.75  28.600  291.00 12.210  0.3759000  0.395200
1959   55.62  1.83  29.000  388.20 12.640  0.0652100  0.076230
1960   58.03  1.95  29.300  400.90 13.250  0.0449000  0.062730
1961   59.72  2.02  29.800  405.60 13.630  0.1825000  0.190400
1962   69.07  2.13  30.000  466.00 14.180 -0.0400200 -0.027220
1963   65.06  2.28  30.400  433.20 14.940  0.1905000  0.210100
1964   76.45  2.50  30.900  500.80 16.220  0.1480000  0.159200
1965   86.12  2.72  31.200  558.70 17.310  0.0941500  0.115200
1966   93.32  2.87  31.800  594.00 17.660 -0.0955800 -0.064290
1967   84.45  2.92  32.900  519.60 17.330  0.1192000  0.160000
1968   95.04  3.07  34.100  564.20 17.460  0.0593600  0.106000
1969  102.00  3.16  35.600  580.20 16.920 -0.1373000 -0.083990
1970   90.31  3.14  37.800  483.60 15.970  0.0162100  0.069980
1971   93.49  3.07  39.800  475.50 15.120  0.1018000  0.137800
1972  103.30  3.15  41.100  508.70 14.970  0.1354000  0.176900
1973  118.40  3.38  42.600  562.70 14.680 -0.2320000 -0.159900
1974   96.11  3.60  46.600  417.50 13.990 -0.2912000 -0.207600
1975   72.56  3.68  52.100  281.90 13.400  0.2984000  0.385600
1976   96.86  4.05  55.600  352.60 14.010  0.0583600  0.113600
1977  103.80  4.67  58.500  359.20 15.120 -0.1442000 -0.085640
1978   90.25  5.07  62.500  292.30 15.030  0.0624100  0.161000
1979   99.71  5.65  68.300  295.50 14.700  0.0258900  0.168600
1980  110.90  6.16  77.800  288.50 14.330  0.1222000  0.254900
1981  133.00  6.63  87.000  309.40 14.230 -0.1403000 -0.068140
1982  117.30  6.87  94.300  251.70 14.220  0.2426000  0.288700
1983  144.30  7.09  97.800  298.60 14.080  0.1541000  0.202500
1984  166.40  7.53 101.900  330.50 14.450  0.0398900  0.076630
1985  171.60  7.90 105.500  329.30 14.590  0.2121000  0.259200
1986  208.20  8.28 109.600  384.50 15.070  0.2914000  0.310300
1987  264.50  8.81 111.200  481.50 15.410 -0.0578600 -0.019730
1988  250.50  9.73 115.700  438.20 16.260  0.1258000  0.178300
1989  285.40 11.05 121.100  477.10 17.560  0.1691000  0.229900
1990  340.00 12.10 127.400  540.20 18.200 -0.0600900 -0.006971
1991  325.50 12.20 134.600  489.50 17.880  0.2824000  0.315800
1992  416.10 12.38 138.100  609.90 17.570  0.0418300  0.075780
1993  435.20 12.58 142.600  617.80 17.420  0.0881900  0.115700
1994  473.00 13.18 146.200  654.90 17.750 -0.0160900  0.011500
1995  465.20 13.79 150.300  626.60 18.080  0.3144000  0.350300
1996  614.40 14.90 154.400  805.50 18.960  0.2338000  0.271300
1997  766.20 15.50 159.100  974.80 19.410  0.2578000  0.277500
1998  963.40 16.20 161.600 1207.00 19.960  0.2915000  0.313100
1999 1249.00 16.69 164.300 1538.00 20.010  0.1242000  0.155000
2000 1426.00 16.27 168.800 1709.00 18.810 -0.0889900 -0.054990
2001 1331.00 15.74 175.100 1539.00 17.990 -0.1413000 -0.131500
2002 1140.00 16.07 177.100 1303.00 17.900 -0.2205000 -0.200200
2003  895.80 17.39 181.700  998.00 19.010  0.2025000  0.225700
2004 1081.00 19.44 185.200 1181.00 20.630  0.0952200  0.127700
2005 1199.00 22.22 190.700 1273.00 22.680  0.0299100  0.070960
2006 1262.00 24.88 198.300 1288.00 24.880  0.1188000  0.142000
2007 1416.42 27.73 202.416 NA      NA      NA         NA
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    Bogleheads Forum Index -> Investing - Theory, News & General All times are GMT - 4 Hours
Go to page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group