Accuracy of the "Monte Carlo" Simulations?

Have a question about your personal investments? No matter how simple or complex, you can ask it here.
User avatar
vineviz
Posts: 14456
Joined: Tue May 15, 2018 1:55 pm
Location: Baltimore, MD

Re: Accuracy of the "Monte Carlo" Simulations?

Post by vineviz »

doobiedoo wrote: Tue Nov 22, 2022 3:53 am Monte Carlo simulations for financial planning typically rely on repeated random sampling of past results to get the statistical likelihood of achieving financial goals.

A basic assumption is that the random sampling is independent, e.g. next year's return does not depend on last year's return or the return of any other year.

Now suppose stock market returns for 5 consecutive years were over 20%. Monte Carlo simulation will say the odds of a 20+% return in the 6th year is the same as any of the previous years.
This last sentence is not true.

A Monte Carlo analysis can incorporate any model of stock returns that you can imagine. The distribution of returns need not be "normal" nor must they be i.i.d.
"Far more money has been lost by investors preparing for corrections than has been lost in corrections themselves." ~~ Peter Lynch
User avatar
HomerJ
Posts: 19897
Joined: Fri Jun 06, 2008 12:50 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by HomerJ »

vineviz wrote: Tue Nov 22, 2022 12:22 pm
doobiedoo wrote: Tue Nov 22, 2022 3:53 am Monte Carlo simulations for financial planning typically rely on repeated random sampling of past results to get the statistical likelihood of achieving financial goals.

A basic assumption is that the random sampling is independent, e.g. next year's return does not depend on last year's return or the return of any other year.

Now suppose stock market returns for 5 consecutive years were over 20%. Monte Carlo simulation will say the odds of a 20+% return in the 6th year is the same as any of the previous years.
This last sentence is not true.

A Monte Carlo analysis can incorporate any model of stock returns that you can imagine. The distribution of returns need not be "normal" nor must they be i.i.d.
You are correct that a Monte Carlo analysis need not use independent and identically distributed returns.

But many do. And most mainstream Monte-Carlo tools available from websites don't tell us how the inner workings are set up.

You are correct that Monte Carlo simulations can be set up with much more sophisticated parameters, but our general "BEWARE" of taking most Monte Carlo simulations at face-value is also correct.
"The best tools available to us are shovels, not scalpels. Don't get carried away." - vanBogle59
nigel_ht
Posts: 4742
Joined: Tue Jan 01, 2019 10:14 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by nigel_ht »

vineviz wrote: Tue Nov 22, 2022 9:44 am
nigel_ht wrote: Tue Nov 22, 2022 9:33 am
vineviz wrote: Tue Nov 22, 2022 7:49 am
nigel_ht wrote: Mon Nov 21, 2022 6:33 pm. I can provide the same input into two different Monte Carlo simulations...say one with mean reversion and one without...and get two different results.
But those two simulations DO NOT HAVE THE SAME INPUTS!

You’ve input different assumptions about the strength of mean reversion so of course you get different results.
Nope. The two parts of a Monte Carlo are the inputs and the equation that does the computation. You can use the same input values for the same uncertainty variables in different eqautions.
I think anyone who has ever constructed a Monte Carlo simulation will understand how nonsensical this statement is.
nigel_ht wrote: Tue Nov 22, 2022 9:33 am Simulation 1:

Tomorrow Stock Price = Today's Stock Price x e^((Average Daily Return - Variance/2) + random input value)

Simulation 2:

Tomorrow Stock Price = Today's Stock Price x e^((Average Daily Return - Variance) + random input value)
I think part of your confusion might lie in your belief that these two equations are different. They aren't: you've merely made an explicit assumption about one of the variables in the first equation and a different implicit assumption in the second.

Both can be expressed in a common form:

Tomorrow Stock Price = Today's Stock Price x e^((Average Daily Return - Variance/X) + random input value)

In your "Simulation 1" you set X=2 and in the second you set X=1. It's that simple, unless you try to draw an arbitrary disnticion between "input" and "equation".

EVERYTHING in a model is an input, including the form of the model.
Nope.

The point is that explicit assumptions can be inserted into the equation to create biases...either intentionally or not.

Sometimes this simplifies the calculation so make it more performant. As long as the bias isn't a lot the trade off in performance vs accuracy is useful.

Sometimes the bias conciously or unconciously reflects the biases of the researcher and you get results that fit what they expect to see. After all if you see results wildly off from what you expect you go back in to take a look to see if you made a mistake and sometimes you tinker your way into a model that looks sorta right but ends up being completely wrong.

Sometime the bias is there so you can sell a product through fear, uncertainty or doubt. 2% SWR is the example above. You can't live on 2% so come to us and our financial professionals using our proprietary analysis systems designed by quants from MIT will help you! Yes, yes, all the financial professionals you know are as pure as the driven snow but certainly there are folks out there in the industry who aren't so ethical.

There isn't an "anti-intellectual" bent here in the forums as much as an "anti-snake oil" bent here. Is MC a useful tool for researchers? Yes. Has it been abused in the past to sell financial services? Yes.

And nope, the equation/algorithm/program isn't an input. The equation/algorithm/program operates on the inputs. Sometimes the model is expressed a mathematical equation...in this case a fairly trivial one. For others, you may have a very complex model implemented in software.
MathWizard
Posts: 5735
Joined: Tue Jul 26, 2011 1:35 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by MathWizard »

https://youtu.be/Q5Fw2IRMjPQ

is a YouTube video that shows how to perform Monte Carlo simulations yourself in Excel.

You can then enter the inputs yourself.

In the video the yearly returns are assumed to come from a gaussian (normal or bell curve) distribution, so all that is required is the mean(average) and the standard deviation.

You have to supply those two inputs and the amount you have in your portfolio and annual contributions.

As stated above, the distribution of returns may or not be a guassian distribution, it could be a much more complicated than that, but the guassian is what is commonly used.
Last edited by MathWizard on Tue Nov 22, 2022 12:45 pm, edited 1 time in total.
nigel_ht
Posts: 4742
Joined: Tue Jan 01, 2019 10:14 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by nigel_ht »

vineviz wrote: Tue Nov 22, 2022 12:22 pm
doobiedoo wrote: Tue Nov 22, 2022 3:53 am Monte Carlo simulations for financial planning typically rely on repeated random sampling of past results to get the statistical likelihood of achieving financial goals.

A basic assumption is that the random sampling is independent, e.g. next year's return does not depend on last year's return or the return of any other year.

Now suppose stock market returns for 5 consecutive years were over 20%. Monte Carlo simulation will say the odds of a 20+% return in the 6th year is the same as any of the previous years.
This last sentence is not true.

A Monte Carlo analysis can incorporate any model of stock returns that you can imagine. The distribution of returns need not be "normal" nor must they be i.i.d.
And I can imagine one with a bias that drives the results to a conclusion I want to support. Can you not?
User avatar
vineviz
Posts: 14456
Joined: Tue May 15, 2018 1:55 pm
Location: Baltimore, MD

Re: Accuracy of the "Monte Carlo" Simulations?

Post by vineviz »

nigel_ht wrote: Tue Nov 22, 2022 12:38 pm And nope, the equation/algorithm/program isn't an input. The equation/algorithm/program operates on the inputs. Sometimes the model is expressed a mathematical equation...in this case a fairly trivial one. For others, you may have a very complex model implemented in software.
Subsequent comment removed.

Discussions are about issues, not people. If you disagree with an idea, go ahead and marshal all your forces against it. But do not confuse ideas with the person posting them. At all times we must conduct ourselves in a respectful manner to other posters. Attacks on individuals, insults, name calling, trolling, baiting or other attempts to sow dissension are not acceptable.

Moderator Pops1860.
"Far more money has been lost by investors preparing for corrections than has been lost in corrections themselves." ~~ Peter Lynch
nigel_ht
Posts: 4742
Joined: Tue Jan 01, 2019 10:14 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by nigel_ht »

HomerJ wrote: Tue Nov 22, 2022 12:13 pm
alex_686 wrote: Tue Nov 22, 2022 6:53 am
doobiedoo wrote: Tue Nov 22, 2022 3:53 am Now suppose stock market returns for 5 consecutive years were over 20%. Monte Carlo simulation will say the odds of a 20+% return in the 6th year is the same as any of the previous years. [Because the sampling is independent!] Common sense might say that after 5 years of abnormally large returns, the market is overvalued and due for a correction. Monte Carlo ignores that. Some might say that's good. Some might say it's a flaw.

And the same is true for 5 consecutive big down years.

P.S. SP500 returns for 1995-1999 were 34%, 20%, 31%, 27%, 20% (rounded). And were followed by 3 negative years.
I would not say this is a flaw but a fairly accurate representation of reality. Humans like to find patterns and antidotal evidence. I understand this. On the other hand I have been exposed to lots of crunchy stats. And while I wouldn’t say the results are exactly independent they are not time dependent. The length of a bull market contains no information on when it is going to end.
This is where I disagree.. This year's stock market returns may be mostly independent of LAST year's returns (some researchers just look year-to-year and see mostly randomness), but this year's stock market returns are NOT independent of the last 3-5-10 years of returns. Outside variables that can affect future stock market returns do change based on past stock market returns...

A very easy example is when a government steps in and changes economic variables after a recession or a depression.
Macro event modelling is where you can play all sorts of shenanigans if you want to. Even if you dont want to there is likely vast disagreements on how to realistically model these kinds of events.

Sometimes the government does the right thing and averts disaster and you get the Global Financial Crisis instead of the Global Financial Crash + Depression 2.0.

Sometimes you get Nikkei and become a cautionary tale.
MathWizard
Posts: 5735
Joined: Tue Jul 26, 2011 1:35 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by MathWizard »

nigel_ht wrote: Tue Nov 22, 2022 12:48 pm
HomerJ wrote: Tue Nov 22, 2022 12:13 pm
alex_686 wrote: Tue Nov 22, 2022 6:53 am
doobiedoo wrote: Tue Nov 22, 2022 3:53 am Now suppose stock market returns for 5 consecutive years were over 20%. Monte Carlo simulation will say the odds of a 20+% return in the 6th year is the same as any of the previous years. [Because the sampling is independent!] Common sense might say that after 5 years of abnormally large returns, the market is overvalued and due for a correction. Monte Carlo ignores that. Some might say that's good. Some might say it's a flaw.

And the same is true for 5 consecutive big down years.

P.S. SP500 returns for 1995-1999 were 34%, 20%, 31%, 27%, 20% (rounded). And were followed by 3 negative years.
I would not say this is a flaw but a fairly accurate representation of reality. Humans like to find patterns and antidotal evidence. I understand this. On the other hand I have been exposed to lots of crunchy stats. And while I wouldn’t say the results are exactly independent they are not time dependent. The length of a bull market contains no information on when it is going to end.
This is where I disagree.. This year's stock market returns may be mostly independent of LAST year's returns (some researchers just look year-to-year and see mostly randomness), but this year's stock market returns are NOT independent of the last 3-5-10 years of returns. Outside variables that can affect future stock market returns do change based on past stock market returns...

A very easy example is when a government steps in and changes economic variables after a recession or a depression.
Macro event modelling is where you can play all sorts of shenanigans if you want to. Even if you dont want to there is likely vast disagreements on how to realistically model these kinds of events.

Sometimes the government does the right thing and averts disaster and you get the Global Financial Crisis instead of the Global Financial Crash + Depression 2.0.

Sometimes you get Nikkei and become a cautionary tale.
Do you have a suggestion on how one could build an approximate model besides using Monte Carlo with a guassian distribution?

Government policy of course can mess with any model, but I'm not sure how I could predict that.

This is not a criticism. It sounds like you have lots of experience in statistical modeling and macro economics, at least more than I do.
Or is this just a hopeless exercise. I hope not.

One must plan, not because the plan will be perfect, but because one can follow a plan, adjust according to new circumstances rather than struggling blindly.
User avatar
HomerJ
Posts: 19897
Joined: Fri Jun 06, 2008 12:50 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by HomerJ »

nigel_ht wrote: Tue Nov 22, 2022 12:38 pm There isn't an "anti-intellectual" bent here in the forums as much as an "anti-snake oil" bent here. Is MC a useful tool for researchers? Yes. Has it been abused in the past to sell financial services? Yes.
This.
"The best tools available to us are shovels, not scalpels. Don't get carried away." - vanBogle59
User avatar
LilyFleur
Posts: 2820
Joined: Fri Mar 02, 2018 10:36 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by LilyFleur »

MathWizard wrote: Tue Nov 22, 2022 12:59 pm
nigel_ht wrote: Tue Nov 22, 2022 12:48 pm
HomerJ wrote: Tue Nov 22, 2022 12:13 pm
alex_686 wrote: Tue Nov 22, 2022 6:53 am
doobiedoo wrote: Tue Nov 22, 2022 3:53 am Now suppose stock market returns for 5 consecutive years were over 20%. Monte Carlo simulation will say the odds of a 20+% return in the 6th year is the same as any of the previous years. [Because the sampling is independent!] Common sense might say that after 5 years of abnormally large returns, the market is overvalued and due for a correction. Monte Carlo ignores that. Some might say that's good. Some might say it's a flaw.

And the same is true for 5 consecutive big down years.

P.S. SP500 returns for 1995-1999 were 34%, 20%, 31%, 27%, 20% (rounded). And were followed by 3 negative years.
I would not say this is a flaw but a fairly accurate representation of reality. Humans like to find patterns and antidotal evidence. I understand this. On the other hand I have been exposed to lots of crunchy stats. And while I wouldn’t say the results are exactly independent they are not time dependent. The length of a bull market contains no information on when it is going to end.
This is where I disagree.. This year's stock market returns may be mostly independent of LAST year's returns (some researchers just look year-to-year and see mostly randomness), but this year's stock market returns are NOT independent of the last 3-5-10 years of returns. Outside variables that can affect future stock market returns do change based on past stock market returns...

A very easy example is when a government steps in and changes economic variables after a recession or a depression.
Macro event modelling is where you can play all sorts of shenanigans if you want to. Even if you dont want to there is likely vast disagreements on how to realistically model these kinds of events.

Sometimes the government does the right thing and averts disaster and you get the Global Financial Crisis instead of the Global Financial Crash + Depression 2.0.

Sometimes you get Nikkei and become a cautionary tale.
Do you have a suggestion on how one could build an approximate model besides using Monte Carlo with a guassian distribution?

Government policy of course can mess with any model, but I'm not sure how I could predict that.

This is not a criticism. It sounds like you have lots of experience in statistical modeling and macro economics, at least more than I do.
Or is this just a hopeless exercise. I hope not.

One must plan, not because the plan will be perfect, but because one can follow a plan, adjust according to new circumstances rather than struggling blindly.
Short of a crystal ball, this is what we must do.
As each year passes, we know a little more; we have another year under our belts.
User avatar
HomerJ
Posts: 19897
Joined: Fri Jun 06, 2008 12:50 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by HomerJ »

MathWizard wrote: Tue Nov 22, 2022 12:40 pm In the video the yearly returns are assumed to come from a gaussian (normal or bell curve) distribution, so all that is required is the mean(average) and the standard deviation.
And that's the problem with the standard Monte Carlo simulation. Real world returns aren't independent.
As stated above, the distribution of returns may or not be a guassian distribution, it could be a much more complicated than that, but the guassian is what is commonly used.
Correct.
"The best tools available to us are shovels, not scalpels. Don't get carried away." - vanBogle59
abc132
Posts: 1258
Joined: Thu Oct 18, 2018 1:11 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by abc132 »

nigel_ht wrote: Mon Nov 21, 2022 9:01 pm
abc132 wrote: Mon Nov 21, 2022 6:42 pm
nigel_ht wrote: Mon Nov 21, 2022 6:33 pm The limitations of Monte Carlo isn't just garbage in and garbage out. Sometimes is good data in and garbage out. Or more accurately...I can put good data in and get (almost) any answer I want out if I play with the model enough...like 2% SWR in the JP Morgan example.

It's just a tool and only as good or as honest as the person who builds it...Fildelity's seems okay.
That's a result of the inability to accurately predict the future historically.

That is what the model is telling you, along with what might give you your desired level of safety and at what cost.
Nope.

In the case of JP Morgan they didn't just put a finger on the scale but a whole body to make it come out the way they wanted...arguably it's not their model that produces their misleading 2% result but their other shenanigans. However using MC in their "analysis" attempts to hide those shenanigans.

There is also a high probabilty your model is broken somewhere if you deviate that much from real world results. That someone like JP Morgan can push a 2% result and have it accepted it as the "inability to accurately predict the future" as opposed to a broken model means that rather than an adversion to trusting monte carlo there is too MUCH trust in it as a tool.

2% is a possible outcome but requires significant departure from some of the basic assumptions that underpins index investing. I recall in another thread that the 30 year SWR for the Nikkei crash cohort was 3%. If your MC is producing results below 3% SWR imma gonna say something is wrong with your model and its your job to show otherwise.
This all has nothing to do with Monte Carlo. One of the basic tools for extracting money from clients is showing that the issues are complex and then showing that they can reduce this uncertainty. The goal is for the client to want them to take on the complex task. This practice does not make all of the financial and statistical tools useless.

Monte Carlo showed me that there is little financial reason to hold bonds in my first two decades of investing. Additions are likely to be sufficient to overcome and even benefit from down sequences. It is only about a decade before retirement that down sequences start to become a risk. The difference here is I learned from the tool by using the tool to look at many results from many different inputs. I held 100% stocks until I came close to my objectives.

You can skip Monte Carlo altogether and could have used -1% real bonds and 1/CAPE stocks to get 3% stock and -1% bonds. Using the expected values a 50/50 portfolio would have around 1% real returns. That is not sufficient for 4% SWR. Its pretty easy to get to 3% by simply recognizing the average value is for those that want a 50% failure rate. The math is/was fine, but the problem was always those that put -1% real for bonds into their models. That's where the 2-3% SWR's came from, and many of us objected to that assumption at the time these estimates were given.
abc132
Posts: 1258
Joined: Thu Oct 18, 2018 1:11 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by abc132 »

HomerJ wrote: Tue Nov 22, 2022 1:12 pm And that's the problem with the standard Monte Carlo simulation. Real world returns aren't independent.
Nobody needs to use the standard model and its not a problem as long as the user understands the limitations.

I use block chains of historic data and set a probability of grabbing the next years stock, bond, and inflation returns. With an 80% chance I get 3 year sequences on average, but also shorter and longer chains. By doing this I capture some of the historical dependence of the returns. With a 100% chance I get historic data. I also try lower values by using historical returns minus 1, 2 and 3%.

The only thing I don't do is try to correct for mean and deviation to match historical values. I do get this result from the 100% historical tests but I want a bigger range of trials for non-historical data. Fitting the historical mean and deviation seems like unnecessary over-fitting to me. The important thing here is I understand the implications of using historical segments when looking at outcomes. Not being a perfect model is not a problem when the limitations are understood.

The limitations of the MC method are actually very small compared to the range of outcomes. It is the uncertainty of the future that causes uncertainty, not the issues with the MC method. It is bonds going from -1% real to 1-2% real that have change SWRs. A good MC user would have looked at all of these outcomes instead of focusing on -1% real.
User avatar
HomerJ
Posts: 19897
Joined: Fri Jun 06, 2008 12:50 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by HomerJ »

abc132 wrote: Tue Nov 22, 2022 1:58 pmYou can skip Monte Carlo altogether and could have used -1% real bonds and 1/CAPE stocks to get 3% stock and -1% bonds.
Except that 1/CAPE is totally made-up recent construct, so one would be foolish to use it.

And -1% real bonds doesn't look like it's going to pan out either.
Using the expected values a 50/50 portfolio would have around 1% real returns. That is not sufficient for 4% SWR. Its pretty easy to get to 3% by simply recognizing the average value is for those that want a 50% failure rate. The math is/was fine, but the problem was always those that put -1% real for bonds into their models. That's where the 2-3% SWR's came from, and many of us objected to that assumption at the time these estimates were given.
Exactly... I agree with you.
"The best tools available to us are shovels, not scalpels. Don't get carried away." - vanBogle59
abc132
Posts: 1258
Joined: Thu Oct 18, 2018 1:11 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by abc132 »

HomerJ wrote: Tue Nov 22, 2022 6:58 pm
abc132 wrote: Tue Nov 22, 2022 1:58 pmYou can skip Monte Carlo altogether and could have used -1% real bonds and 1/CAPE stocks to get 3% stock and -1% bonds.
Except that 1/CAPE is totally made-up recent construct, so one would be foolish to use it.
The point was that alternate methods also predicted low results and that this was not due to using a Monte Carlo simulation. Anyone blaming Monte Carlo for unrealistically low SWR predictions is proven incorrect as the Monte Carlo method was not the source of the issue.

HomerJ wrote: Tue Nov 22, 2022 6:58 pm And -1% real bonds doesn't look like it's going to pan out either.
Agreed, and I said so at the time. I do find it odd that there are those that believe stocks return to historic pricing metrics but don't assume the same thing about bond prices. I think one had to rationally choose between:

1) CAPE being somewhat predictive and future bond rates expected to be above -1%
2) knowing nothing about future stock or bond returns

Those that believed in CAPE and insisted on using -1% bond rates believed in the accuracy of current pricing of bonds but not in the accuracy of current pricing of stocks. I'm not sure it makes sense to assume only one asset has a tendency towards historical norms but I would be interested in hearing why there is the apparent discrepancy.
JasonHutt
Posts: 160
Joined: Wed Oct 26, 2022 1:47 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by JasonHutt »

nigel_ht wrote: Tue Nov 22, 2022 12:38 pm
vineviz wrote: Tue Nov 22, 2022 9:44 am
nigel_ht wrote: Tue Nov 22, 2022 9:33 am
vineviz wrote: Tue Nov 22, 2022 7:49 am
nigel_ht wrote: Mon Nov 21, 2022 6:33 pm. I can provide the same input into two different Monte Carlo simulations...say one with mean reversion and one without...and get two different results.
But those two simulations DO NOT HAVE THE SAME INPUTS!

You’ve input different assumptions about the strength of mean reversion so of course you get different results.
Nope. The two parts of a Monte Carlo are the inputs and the equation that does the computation. You can use the same input values for the same uncertainty variables in different eqautions.
I think anyone who has ever constructed a Monte Carlo simulation will understand how nonsensical this statement is.
nigel_ht wrote: Tue Nov 22, 2022 9:33 am Simulation 1:

Tomorrow Stock Price = Today's Stock Price x e^((Average Daily Return - Variance/2) + random input value)

Simulation 2:

Tomorrow Stock Price = Today's Stock Price x e^((Average Daily Return - Variance) + random input value)
I think part of your confusion might lie in your belief that these two equations are different. They aren't: you've merely made an explicit assumption about one of the variables in the first equation and a different implicit assumption in the second.

Both can be expressed in a common form:

Tomorrow Stock Price = Today's Stock Price x e^((Average Daily Return - Variance/X) + random input value)

In your "Simulation 1" you set X=2 and in the second you set X=1. It's that simple, unless you try to draw an arbitrary disnticion between "input" and "equation".

EVERYTHING in a model is an input, including the form of the model.
Nope.

The point is that explicit assumptions can be inserted into the equation to create biases...either intentionally or not.

Sometimes this simplifies the calculation so make it more performant. As long as the bias isn't a lot the trade off in performance vs accuracy is useful.

Sometimes the bias conciously or unconciously reflects the biases of the researcher and you get results that fit what they expect to see. After all if you see results wildly off from what you expect you go back in to take a look to see if you made a mistake and sometimes you tinker your way into a model that looks sorta right but ends up being completely wrong.

Sometime the bias is there so you can sell a product through fear, uncertainty or doubt. 2% SWR is the example above. You can't live on 2% so come to us and our financial professionals using our proprietary analysis systems designed by quants from MIT will help you! Yes, yes, all the financial professionals you know are as pure as the driven snow but certainly there are folks out there in the industry who aren't so ethical.

There isn't an "anti-intellectual" bent here in the forums as much as an "anti-snake oil" bent here. Is MC a useful tool for researchers? Yes. Has it been abused in the past to sell financial services? Yes.

And nope, the equation/algorithm/program isn't an input. The equation/algorithm/program operates on the inputs. Sometimes the model is expressed a mathematical equation...in this case a fairly trivial one. For others, you may have a very complex model implemented in software.
Subsequent comment deleted. Please remember forum ploicy:
Discussions are about issues, not people. If you disagree with an idea, go ahead and marshal all your forces against it. But do not confuse ideas with the person posting them. At all times we must conduct ourselves in a respectful manner to other posters. Attacks on individuals, insults, name calling, trolling, baiting or other attempts to sow dissension are not acceptable.


Moderator Pops1860
User avatar
vineviz
Posts: 14456
Joined: Tue May 15, 2018 1:55 pm
Location: Baltimore, MD

Re: Accuracy of the "Monte Carlo" Simulations?

Post by vineviz »

HomerJ wrote: Tue Nov 22, 2022 6:58 pm
abc132 wrote: Tue Nov 22, 2022 1:58 pmYou can skip Monte Carlo altogether and could have used -1% real bonds and 1/CAPE stocks to get 3% stock and -1% bonds.
Except that 1/CAPE is totally made-up recent construct, so one would be foolish to use it.
I'm not sure where this idea came from, but it is factually incorrect.

The concept of earnings yields (which is simply E/P instead of P/E) as a valuation tool date back at least to the writings of Benjamin Graham in the 1930s.

It's certainly not "recent" nor is it a "made-up construct", since it explicitly measures directly relevant measure of valuation and (therefore) expected return. The term is defined in the 1934 edition of Graham and Dodd's "Security Analysis" text book. It shows up occasionally as early as 1907.
"Far more money has been lost by investors preparing for corrections than has been lost in corrections themselves." ~~ Peter Lynch
User avatar
HomerJ
Posts: 19897
Joined: Fri Jun 06, 2008 12:50 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by HomerJ »

vineviz wrote: Wed Nov 23, 2022 7:41 am
HomerJ wrote: Tue Nov 22, 2022 6:58 pm
abc132 wrote: Tue Nov 22, 2022 1:58 pmYou can skip Monte Carlo altogether and could have used -1% real bonds and 1/CAPE stocks to get 3% stock and -1% bonds.
Except that 1/CAPE is totally made-up recent construct, so one would be foolish to use it.
I'm not sure where this idea came from, but it is factually incorrect.

The concept of earnings yields (which is simply E/P instead of P/E) as a valuation tool date back at least to the writings of Benjamin Graham in the 1930s.

It's certainly not "recent" nor is it a "made-up construct", since it explicitly measures directly relevant measure of valuation and (therefore) expected return. The term is defined in the 1934 edition of Graham and Dodd's "Security Analysis" text book. It shows up occasionally as early as 1907.
The concept that 1/CAPE gives an good estimate of 10-year returns going forward is a recent construct.

The trend-line did not support that until very recently, but many writers seem to think it's some kind of natural economic law these days, maybe confusing it with the general valuations tool you reference.

A CAPE of 20 did not indicate an expected 10-year return of 5% over the past 120 years of U.S. history, nor did a CAPE of 25 indicate a 10-year return of 4%.

The expected return trend-line only started indicating that rough correlation recently, when the last 10 or 15 years of data was added.

Someone noticed that that new trend-line is NOW roughly approximated by 1/CAPE, and it seems so simple and intuitive that many have latched onto it as a common natural economic law.

But it's not. It's a coincidental data-construct.
"The best tools available to us are shovels, not scalpels. Don't get carried away." - vanBogle59
coffeeblack
Posts: 531
Joined: Wed Jun 19, 2019 10:20 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by coffeeblack »

Rick Ferri wrote: Mon Nov 21, 2022 10:54 am
vineviz wrote: Sun Nov 20, 2022 4:09 pm
BufordTJustice wrote: Sun Nov 20, 2022 4:02 pm What is consensus on the validity of these types of protection tools? I have been using the "significantly below average market" approach to (presumably) be safe.
I suspect the consensus answer you receive here will be to dismiss the simulations out-of-hand. There's a distinct anti-intellectual streak among some (though definitely not all) frequent participants.

If you asked a group of people with actual professional expertise, you'd get the opposite answer I'm sure.
I have 35 years of actual professional expertise and I dismiss this tool also. It's used mainly by less-experienced advisers who wish to create an illusion of control via a precise output when all the inputs they use are imprecise or incomplete samples.

Rick Ferri
I think they are all just basic tools to get a general idea. But what makes your tools better? Less-experienced? So, are you saying your experience predicts the future better?
The financial advisor industry loves to make predictions to get clients.
User avatar
vineviz
Posts: 14456
Joined: Tue May 15, 2018 1:55 pm
Location: Baltimore, MD

Re: Accuracy of the "Monte Carlo" Simulations?

Post by vineviz »

HomerJ wrote: Wed Nov 23, 2022 9:32 am
The concept that 1/CAPE gives an good estimate of 10-year returns going forward is a recent construct.

The trend-line did not support that until very recently, but many writers seem to think it's some kind of natural economic law these days, maybe confusing it with the general valuations tool you reference.

I'll repeat: the idea the an earnings yield provides an estimate of expected return is absolutely not new.

It's true that we've learned a thing or two since 1934, but that's a good thing not a bad thing.
"Far more money has been lost by investors preparing for corrections than has been lost in corrections themselves." ~~ Peter Lynch
Ron
Posts: 6966
Joined: Fri Feb 23, 2007 7:46 pm
Location: Allentown–Bethlehem–Easton, PA-NJ Metropolitan Statistical Area

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Ron »

Just for giggles, I just compared FIDO's planning tool report created on May 1, 2007 (my retirement date) vs today's report.

What I found was that the total value of my/wife's retirement portfolios is 66% higher today than forecast 15 years ago, at the same age on the report. It also shows (correctly) that RMD's forecast wayback then are much lower than what we are actually withdrawing today. Joint SS income is 25% higher at the same age than what was forecast back then.

All this shows is that the primary forecast tool we used years ago to help us make retirement decisions greatly underestimated the actual income and portfolio values we have today. It also didn't match reality to decisions on SS (Forecast - both take at age 66 vs. reality - both took at age 70, since we were still under the file/suspend rules, which I was unaware of when I retired). Additionally, while initially we both were to retire at age 59, my wife had "cold feet" when her 59th birthday approached. While I did retire at age 59, she remained on the job for an additional 5 years - until age 64. She continued to contribute to her 401(k) but ceased contributions to her Roth IRA.

Did the simulations align forecast vs reality in our case? No. But it was "good enough" to make guesstimates of the direction and possibility of the financial future for us. Regardless of any forecast, the reality is that things change, "Der mentsh trakht un got lakht".

FWIW,

- Ron
nigel_ht
Posts: 4742
Joined: Tue Jan 01, 2019 10:14 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by nigel_ht »

Ron wrote: Wed Nov 23, 2022 10:50 am Just for giggles, I just compared FIDO's planning tool report created on May 1, 2007 (my retirement date) vs today's report.

What I found was that the total value of my/wife's retirement portfolios is 66% higher today than forecast 15 years ago, at the same age on the report. It also shows (correctly) that RMD's forecast wayback then are much lower than what we are actually withdrawing today. Joint SS income is 25% higher at the same age than what was forecast back then.

All this shows is that the primary forecast tool we used years ago to help us make retirement decisions greatly underestimated the actual income and portfolio values we have today. It also didn't match reality to decisions on SS (Forecast - both take at age 66 vs. reality - both took at age 70, since we were still under the file/suspend rules, which I was unaware of when I retired). Additionally, while initially we both were to retire at age 59, my wife had "cold feet" when her 59th birthday approached. While I did retire at age 59, she remained on the job for an additional 5 years - until age 64. She continued to contribute to her 401(k) but ceased contributions to her Roth IRA.

Did the simulations align forecast vs reality in our case? No. But it was "good enough" to make guesstimates of the direction and possibility of the financial future for us. Regardless of any forecast, the reality is that things change, "Der mentsh trakht un got lakht".

FWIW,

- Ron
Well, we did have a great decade soooo...count your blessings. :)
JasonHutt
Posts: 160
Joined: Wed Oct 26, 2022 1:47 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by JasonHutt »

I'm sorry that Rick F. didn't explain his view on MC after dismissing it.
User avatar
HomerJ
Posts: 19897
Joined: Fri Jun 06, 2008 12:50 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by HomerJ »

vineviz wrote: Wed Nov 23, 2022 10:17 am
HomerJ wrote: Wed Nov 23, 2022 9:32 am
The concept that 1/CAPE gives an good estimate of 10-year returns going forward is a recent construct.

The trend-line did not support that until very recently, but many writers seem to think it's some kind of natural economic law these days, maybe confusing it with the general valuations tool you reference.

I'll repeat: the idea the an earnings yield provides an estimate of expected return is absolutely not new.

It's true that we've learned a thing or two since 1934, but that's a good thing not a bad thing.
And I repeat, the idea that 1/CAPE gives a good estimate absolutely is new.

Show me anywhere, in any literature, from 1934 - 2010 where someone says a CAPE of 25 gives an "expected" return of 4% real.

We're obviously talking about two different things... Just like this whole Monte Carlo discussion, where you are technically correct about the inputs and the design of various Monte Carlo systems being the important factors, but missing the big picture of how they are used in the real world.

You're missing the difference between academic researchers and excellent financial advisors such as yourself and poor financial advisors/general financial blogs/average investors.

You're right, when talking in generalities, that financial experts used earnings yield in the past in their calculations. That's not new. But I'm talking directly about someone using 1/CAPE as a predictor.

There are plenty of blogs and websites that claim that 1/CAPE can used to accurately describe the relationship between CAPE and historical 10-year returns. People do it routinely here as well now. It's almost an article of faith now.

But this wasn't very accurate until about 5 years ago.

You won't find blogs or websites talking about 1/CAPE in 2005, or even 2011, because it didn't fit the trendline as it existed back then. I'd be interested if anyone on Bogleheads mentioned it before 2012.

It's a recent data construct, now that we've had numerous 10-year periods starting with a CAPE of 20-25 that returned 4%-5% real. But those didn't exist until the 2002-2012, 2003-2013, 2004-2014, 2005-2015, 2006-2016, etc. data points showed up.

In 2011, experts were still saying a CAPE of 22 predicted 1% real, not 4.5% real, because that's what the data up to 2011 was still saying.
"The best tools available to us are shovels, not scalpels. Don't get carried away." - vanBogle59
student
Posts: 8083
Joined: Fri Apr 03, 2015 6:58 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by student »

Ron wrote: Wed Nov 23, 2022 10:50 am Just for giggles, I just compared FIDO's planning tool report created on May 1, 2007 (my retirement date) vs today's report.

What I found was that the total value of my/wife's retirement portfolios is 66% higher today than forecast 15 years ago, at the same age on the report.
Which forecast from 15 years ago did you compare to? Average, below average, or significantly leave below average?
User avatar
vineviz
Posts: 14456
Joined: Tue May 15, 2018 1:55 pm
Location: Baltimore, MD

Re: Accuracy of the "Monte Carlo" Simulations?

Post by vineviz »

HomerJ wrote: Thu Nov 24, 2022 12:44 am And I repeat, the idea that 1/CAPE gives a good estimate absolutely is new.
Then you're repeatedly incorrect.

Go find copy of Graham and Dodd's 1934 book and look it up for yourself if you don't want to take my word for it.
"Far more money has been lost by investors preparing for corrections than has been lost in corrections themselves." ~~ Peter Lynch
Ron
Posts: 6966
Joined: Fri Feb 23, 2007 7:46 pm
Location: Allentown–Bethlehem–Easton, PA-NJ Metropolitan Statistical Area

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Ron »

student wrote: Thu Nov 24, 2022 7:54 am
Ron wrote: Wed Nov 23, 2022 10:50 am Just for giggles, I just compared FIDO's planning tool report created on May 1, 2007 (my retirement date) vs today's report.

What I found was that the total value of my/wife's retirement portfolios is 66% higher today than forecast 15 years ago, at the same age on the report.
Which forecast from 15 years ago did you compare to? Average, below average, or significantly leave below average?
Back then, the FIDO tool defaulted to a "extremely conservative market performance" measurement. They did not have the breakdown as is available in today's tool.

From the report:

"You have a 90% historical likelihood that your money may last at least to the age you planned for, 94.
This was determined from your input, using an extremely conservative market performance assumption
(95% confidence level) and the factors outlined below.* As a target, Fidelity believes you should plan for
a 90% likelihood your money will last to the end of your retirement whether the market is up or down.
Your current plan suggests you have a 90% likelihood that your money will last until age 94, with an
extremely conservative market performance assumption derived from the historical performance
analysis."


- Ron
Ron
Posts: 6966
Joined: Fri Feb 23, 2007 7:46 pm
Location: Allentown–Bethlehem–Easton, PA-NJ Metropolitan Statistical Area

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Ron »

nigel_ht wrote: Wed Nov 23, 2022 3:24 pm
Well, we did have a great decade soooo...count your blessings. :)
Again, just to show that forecasts don't truly predict the actual future, but just a "maybe" view.

- Ron
student
Posts: 8083
Joined: Fri Apr 03, 2015 6:58 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by student »

Ron wrote: Thu Nov 24, 2022 9:20 am
student wrote: Thu Nov 24, 2022 7:54 am
Ron wrote: Wed Nov 23, 2022 10:50 am Just for giggles, I just compared FIDO's planning tool report created on May 1, 2007 (my retirement date) vs today's report.

What I found was that the total value of my/wife's retirement portfolios is 66% higher today than forecast 15 years ago, at the same age on the report.
Which forecast from 15 years ago did you compare to? Average, below average, or significantly leave below average?
Back then, the FIDO tool defaulted to a "extremely conservative market performance" measurement. They did not have the breakdown as is available in today's tool.

From the report:

"You have a 90% historical likelihood that your money may last at least to the age you planned for, 94.
This was determined from your input, using an extremely conservative market performance assumption
(95% confidence level) and the factors outlined below.* As a target, Fidelity believes you should plan for
a 90% likelihood your money will last to the end of your retirement whether the market is up or down.
Your current plan suggests you have a 90% likelihood that your money will last until age 94, with an
extremely conservative market performance assumption derived from the historical performance
analysis."


- Ron
Thanks for the info. In this case, I am not surprised as I can see a big difference between the projections for the average assumption and the significantly below average assumption in my report. Personally I would also like to see the number for a 95% historical likelihood.
Post Reply