Accuracy of the "Monte Carlo" Simulations?

Have a question about your personal investments? No matter how simple or complex, you can ask it here.
Topic Author
Archie Bunker
Posts: 52
Joined: Sun Nov 20, 2022 12:12 pm

Accuracy of the "Monte Carlo" Simulations?

Post by Archie Bunker »

All-

This is a great forum and in my short time here Ive already started to "know what I didnt know", if that makes sense. There are alot of very knowledgeable folks here who seem very willing to help others (who are less adept in financial areas, such as myself).

A basic question I have (as my employer uses Fidelity, and therefore I have a 401k there. Fidelity provides a planning tool where various current investments can be entered, along with planned contributions, as well as specifying timelines, etc. They provide various "what if" scenarios presumably based on historical statistics. The methodology is as below. What is consensus on the validity of these types of protection tools? I have been using the "significantly below average market" approach to (presumably) be safe.

Significantly Below Average Market
A significantly below average market is defined as the 90% confidence level of estimated future balances and/or estimated future income. The 90% confidence level represents significantly below average market conditions with 10% of all hypothetical scenarios tested performing worse. This means that in 90 out of 100 market scenarios tested a hypothetical portfolio similar to yours performed at least as well as the results shown and 10 out of 100 performed worse than the results shown.

Below Average Market
A below average market is defined as the 75% confidence level of estimated future balances and/or estimated future income. The 75% confidence level represents below average market conditions with 25% of all hypothetical scenarios tested performing worse. This means that in 75 out of 100 market scenarios tested a hypothetical portfolio similar to yours performed at least as well as the results shown and 25 out of 100 performed worse than the results shown.

Average Market
An average market is defined as the 50% confidence level of estimated future balances and/or estimated future income. The 50% confidence level represents average market conditions with 50% of all hypothetical scenarios tested performing worse. This means that in 50 out of 100 market scenarios tested a hypothetical portfolio similar to yours performed at least as well as the results shown and 50 out of 100 performed worse than the results shown.
User avatar
vineviz
Posts: 14180
Joined: Tue May 15, 2018 1:55 pm
Location: Baltimore, MD

Re: Accuracy of the "Monte Carlo" Simulations?

Post by vineviz »

BufordTJustice wrote: Sun Nov 20, 2022 4:02 pm What is consensus on the validity of these types of protection tools? I have been using the "significantly below average market" approach to (presumably) be safe.
I suspect the consensus answer you receive here will be to dismiss the simulations out-of-hand. There's a distinct anti-intellectual streak among some (though definitely not all) frequent participants.

If you asked a group of people with actual professional expertise, you'd get the opposite answer I'm sure.

Monte Carlo analysis is a completely legitimate tool but, as with any planning tool, it's only as useful as the inputs are good. I don't know the details of the assumptions that Fidelity includes in their analysis, but if they are using raw historical data as inputs then their conclusions are likely to be overly optimistic. On the other hand, if you're using the "significantly below average market" results then you may very well be offsetting that bias to end up with a conservative-but-not-excessively-so final estimate.

If you're more than 10 years from retirement, this kind of generic tool is probably "good enough" since there is so much uncertainty in financial planning that doing better would be a lot of work AND still not necessarily give you a result that is actionably different.

If you're trying to decide whether to retire now or in the immediate future, I'd do a more complete analysis of the whole picture.
"Far more money has been lost by investors preparing for corrections than has been lost in corrections themselves." ~~ Peter Lynch
Shallowpockets
Posts: 2503
Joined: Fri Nov 20, 2015 10:26 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Shallowpockets »

vineviz wrote: Sun Nov 20, 2022 4:09 pm
BufordTJustice wrote: Sun Nov 20, 2022 4:02 pm What is consensus on the validity of these types of protection tools? I have been using the "significantly below average market" approach to (presumably) be safe.
I suspect the consensus answer you receive here will be to dismiss the simulations out-of-hand. There's a distinct anti-intellectual streak among some (though definitely not all) frequent participants.

If you asked a group of people with actual professional expertise, you'd get the opposite answer I'm sure.

Monte Carlo analysis is a completely legitimate tool but, as with any planning tool, it's only as useful as the inputs are good. I don't know the details of the assumptions that Fidelity includes in their analysis, but if they are using raw historical data as inputs then their conclusions are likely to be overly optimistic. On the other hand, if you're using the "significantly below average market" results then you may very well be offsetting that bias to end up with a conservative-but-not-excessively-so final estimate.

If ignoring a Monte Carlo simulation because you do not know the inputs of the base data at Fidelity what would you use?
Marseille07
Posts: 12192
Joined: Fri Nov 06, 2020 1:41 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Marseille07 »

BufordTJustice wrote: Sun Nov 20, 2022 4:02 pm All-

This is a great forum and in my short time here Ive already started to "know what I didnt know", if that makes sense. There are alot of very knowledgeable folks here who seem very willing to help others (who are less adept in financial areas, such as myself).

A basic question I have (as my employer uses Fidelity, and therefore I have a 401k there. Fidelity provides a planning tool where various current investments can be entered, along with planned contributions, as well as specifying timelines, etc. They provide various "what if" scenarios presumably based on historical statistics. The methodology is as below. What is consensus on the validity of these types of protection tools? I have been using the "significantly below average market" approach to (presumably) be safe.

Significantly Below Average Market
A significantly below average market is defined as the 90% confidence level of estimated future balances and/or estimated future income. The 90% confidence level represents significantly below average market conditions with 10% of all hypothetical scenarios tested performing worse. This means that in 90 out of 100 market scenarios tested a hypothetical portfolio similar to yours performed at least as well as the results shown and 10 out of 100 performed worse than the results shown.

Below Average Market
A below average market is defined as the 75% confidence level of estimated future balances and/or estimated future income. The 75% confidence level represents below average market conditions with 25% of all hypothetical scenarios tested performing worse. This means that in 75 out of 100 market scenarios tested a hypothetical portfolio similar to yours performed at least as well as the results shown and 25 out of 100 performed worse than the results shown.

Average Market
An average market is defined as the 50% confidence level of estimated future balances and/or estimated future income. The 50% confidence level represents average market conditions with 50% of all hypothetical scenarios tested performing worse. This means that in 50 out of 100 market scenarios tested a hypothetical portfolio similar to yours performed at least as well as the results shown and 50 out of 100 performed worse than the results shown.
They're all valid metrics, but I only look at Significantly Below Average Market (aka p10).

As far as retirement planning, you care about near the worst cases, not the average or slightly below average, and certainly not the best case.
US & FM (5% seed) | 100K EF
livesoft
Posts: 81604
Joined: Thu Mar 01, 2007 8:00 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by livesoft »

I think I was aware of that tool back when I had a 401(k) at Fidelity. It didn't impact what I was going to do at all for retirement, so I am wondering how you think it will have an impact on your life.

For instance, would you save and invest more based on the simulation? Or would you spend more now and not save as much as you could?

In any event, I would not bet my life on anything that it told me. But nor would I get any extra "peace of mind" from the output either.
Wiki This signature message sponsored by sscritic: Learn to fish.
Topic Author
Archie Bunker
Posts: 52
Joined: Sun Nov 20, 2022 12:12 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Archie Bunker »

vineviz wrote: Sun Nov 20, 2022 4:09 pm
BufordTJustice wrote: Sun Nov 20, 2022 4:02 pm What is consensus on the validity of these types of protection tools? I have been using the "significantly below average market" approach to (presumably) be safe.
I suspect the consensus answer you receive here will be to dismiss the simulations out-of-hand. There's a distinct anti-intellectual streak among some (though definitely not all) frequent participants.

If you asked a group of people with actual professional expertise, you'd get the opposite answer I'm sure.

Monte Carlo analysis is a completely legitimate tool but, as with any planning tool, it's only as useful as the inputs are good. I don't know the details of the assumptions that Fidelity includes in their analysis, but if they are using raw historical data as inputs then their conclusions are likely to be overly optimistic. On the other hand, if you're using the "significantly below average market" results then you may very well be offsetting that bias to end up with a conservative-but-not-excessively-so final estimate.

If you're more than 10 years from retirement, this kind of generic tool is probably "good enough" since there is so much uncertainty in financial planning that doing better would be a lot of work AND still not necessarily give you a result that is actionably different.

If you're trying to decide whether to retire now or in the immediate future, I'd do a more complete analysis of the whole picture.
I am 50....and planning for retirement at ~62 or so. Id like sooner, but I dont think it would be wise financially.

I wish I could show the tool to everyone here and have them play with it. It does allow asset allocation mixes to be adjusted as well. There appear to be 9 mixes (ranging from 100% short term to a 70/30 domestic/foreign stock split) with corresponding varying projections.
User avatar
ApeAttack
Posts: 692
Joined: Wed Dec 23, 2020 8:28 pm
Location: Gorillatown

Re: Accuracy of the "Monte Carlo" Simulations?

Post by ApeAttack »

I view Monte Carlo simulations as a tool that helps you get a feel for the range of outcomes that you can typically expect. For example, if a set of simulations show that 1/4 of the time you earn a CARG of 5% of lower over a 20 year period using your preferred investment strategy, but you need to have 5% or higher to meet your retirement goals, you may decide to save more as a hedge against low returns.
Just another lazy index investor.
Topic Author
Archie Bunker
Posts: 52
Joined: Sun Nov 20, 2022 12:12 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Archie Bunker »

livesoft wrote: Sun Nov 20, 2022 4:13 pm I think I was aware of that tool back when I had a 401(k) at Fidelity. It didn't impact what I was going to do at all for retirement, so I am wondering how you think it will have an impact on your life.

For instance, would you save and invest more based on the simulation? Or would you spend more now and not save as much as you could?

In any event, I would not bet my life on anything that it told me. But nor would I get any extra "peace of mind" from the output either.
Excellent points........I am not trying to use it to change behavior, but rather to (hopefully) provide some peace of mind about the outcome.

We do save (what I think anyway) a good amount each year (about 50% of gross) and there probably not room for much more without a big change in lifestyle. I would be OK with that....IF it was statistically likely to be needed to reach the desired end goal. Of course, this is the usual balance of living today but expecting tomorrow. We cant all just singularly live for tomorrow (unless thats actually what makes one happiest of course).
exodusing
Posts: 48
Joined: Thu Oct 13, 2022 7:32 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by exodusing »

vineviz wrote: Sun Nov 20, 2022 4:09 pmMonte Carlo analysis is a completely legitimate tool but, as with any planning tool, it's only as useful as the inputs are good.
In my experience, the reason many dismiss Monte Carlo simulations in this context is precisely because they are only as useful as the inputs are good. Given the wide range of plausible inputs over the period relevant to an investor, the results may not be very useful.

They are useful for showing the relation between inputs and possible outputs.

It appears one reason financial advisor professionals like them is that they look very sophisticated and can be a great marketing tool. Depends on the particular circumstances and context.
User avatar
vineviz
Posts: 14180
Joined: Tue May 15, 2018 1:55 pm
Location: Baltimore, MD

Re: Accuracy of the "Monte Carlo" Simulations?

Post by vineviz »

exodusing wrote: Sun Nov 20, 2022 4:29 pm
vineviz wrote: Sun Nov 20, 2022 4:09 pmMonte Carlo analysis is a completely legitimate tool but, as with any planning tool, it's only as useful as the inputs are good.
In my experience, the reason many dismiss Monte Carlo simulations in this context is precisely because they are only as useful as the inputs are good. Given the wide range of plausible inputs over the period relevant to an investor, the results may not be very useful.

They are useful for showing the relation between inputs and possible outputs.

It appears one reason financial advisor professionals like them is that they look very sophisticated and can be a great marketing tool. Depends on the particular circumstances and context.
[Personal attack removed by Moderator ClaycordJCA]

If there are professionals using Monte Carlo analysis merely because it "looks cool" or whatever, I don't know them.
"Far more money has been lost by investors preparing for corrections than has been lost in corrections themselves." ~~ Peter Lynch
Shallowpockets
Posts: 2503
Joined: Fri Nov 20, 2015 10:26 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Shallowpockets »

livesoft wrote: Sun Nov 20, 2022 4:13 pm I think I was aware of that tool back when I had a 401(k) at Fidelity. It didn't impact what I was going to do at all for retirement, so I am wondering how you think it will have an impact on your life.

For instance, would you save and invest more based on the simulation? Or would you spend more now and not save as much as you could?

In any event, I would not bet my life on anything that it told me. But nor would I get any extra "peace of mind" from the output either.
Curious. If you doubt the validity of input and that doubt extends to all Monte Carlo simulations, what do you use?
Do you do no projection at all of your assets into the future? Do you, as some have said in past like discussions, create your own spreadsheet with your own inputs?

If you cannot trust the input of historical data inside the Fidelity simulation, how could you trust any projection? The huge long time organization of Fidelity has got to have the best they can do in their data input.
Don’t you think?
User avatar
Horton
Posts: 1187
Joined: Mon Jan 21, 2008 3:53 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Horton »

I personally use the Fidelity tool and think it’s pretty useful.
User avatar
vineviz
Posts: 14180
Joined: Tue May 15, 2018 1:55 pm
Location: Baltimore, MD

Re: Accuracy of the "Monte Carlo" Simulations?

Post by vineviz »

Shallowpockets wrote: Sun Nov 20, 2022 4:12 pm If ignoring a Monte Carlo simulation because you do not know the inputs of the base data at Fidelity what would you use?
I wouldn't necessarily dismiss the Fidelity implementation merely because I don't know the inputs and methodology they are using. But generally speaking - as someone with training and experience in finance and economics - my preference is always going to be for a model with clear and explicit assumptions rather than a black box.

That said, I'd certainly expect a tool published by Fidelity, T. Rowe Price, Vanguard, or any other firm with expertise in retirement planning to be trustworthy in at least a general sense of the word.
"Far more money has been lost by investors preparing for corrections than has been lost in corrections themselves." ~~ Peter Lynch
livesoft
Posts: 81604
Joined: Thu Mar 01, 2007 8:00 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by livesoft »

Shallowpockets wrote: Sun Nov 20, 2022 4:39 pm
livesoft wrote: Sun Nov 20, 2022 4:13 pmI think I was aware of that tool back when I had a 401(k) at Fidelity.
[...]
Curious. If you doubt the validity of input and that doubt extends to all Monte Carlo simulations, what do you use?
I am now fully retired and so is my spouse. I guess back in the day* I was saving and investing all that was possible, so that I could not save and invest anything more --- and I wasn't going to change. So in that case any and all projections would not change my behavior nor my outlook on life. As the OP noted, they are already saving and investing "about 50% of gross" which I think is all that one needs to do. With that kind of rate either one retires earlier or later no matter what.

*And before then there was no internet, no personal computers, and it was easy to tune out the noise.
Wiki This signature message sponsored by sscritic: Learn to fish.
JasonHutt
Posts: 119
Joined: Wed Oct 26, 2022 1:47 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by JasonHutt »

I was not aware that Fidelity's tool allowed one to tweek the asset allocation, except for allowing comparison to what Fido thinks you should hold. That said, I think it's worth using. HOWEVER, Fidelity has a conflict of interest in that it does better when you invest more money, and it pushes this often.

I happen to think that if you can figure it out, Portfolio Visualizer is a much superior tool, as you can easily change allocations for MC simulations and for backtesting (no backtesting for the Fidelity instrument).

But as for MC in general? It's a extremely valuable tool that is widely used in various fields. It acknowledges the uncertainty of predictions instead of giving a single predicted value. We must care about variability in retirement planning.
rbd789
Posts: 94
Joined: Fri Jul 24, 2020 2:58 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by rbd789 »

vineviz wrote: Sun Nov 20, 2022 4:42 pm
Shallowpockets wrote: Sun Nov 20, 2022 4:12 pm If ignoring a Monte Carlo simulation because you do not know the inputs of the base data at Fidelity what would you use?
I wouldn't necessarily dismiss the Fidelity implementation merely because I don't know the inputs and methodology they are using. But generally speaking - as someone with training and experience in finance and economics - my preference is always going to be for a model with clear and explicit assumptions rather than a black box.

That said, I'd certainly expect a tool published by Fidelity, T. Rowe Price, Vanguard, or any other firm with expertise in retirement planning to be trustworthy in at least a general sense of the word.
Google "fidelity retirement planner methodology". I'm no expert, for sure, but their model appears to me to be far from a black box. I do use it as one of my checks on progress, paying attention mostly to the "significantly below average" outputs.
livesoft
Posts: 81604
Joined: Thu Mar 01, 2007 8:00 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by livesoft »

rbd789 wrote: Sun Nov 20, 2022 5:35 pmI do use it as one of my checks on progress, paying attention mostly to the "significantly below average" outputs.
OK, but let me ask again: You check on progress, but has it ever changed your behavior -- either making you spend less/invest more or the other way spend more/invest less? Or perhaps change asset allocation drastically?
Wiki This signature message sponsored by sscritic: Learn to fish.
User avatar
HMSVictory
Posts: 1427
Joined: Sun Nov 01, 2020 7:02 am
Location: Lower Gun Deck

Re: Accuracy of the "Monte Carlo" Simulations?

Post by HMSVictory »

Spending drives the bus with retirement planning not investment returns.

For instance if you can live off of SS, pensions, fixed annuity income to pay all of your necessities you will survive almost any conceivable retirement scenario. The ability to spend more when times are good (returns) and less (none is ideal) when times are bad will surely survive. I'd add being totally debt free (including the house) as critical as well. Being able to sell the house and downsize is a common tactic.

Where things start to get a little sketchy is if your fixed expenses (your needs) require a 4-5% (or more) draw inflation adjusted on your portfolio indefinitely. Even just skipping the inflation adjustment will be huge to ensure portfolio survive ability. Retirees who have flexibility will have peace and the ability to spend more when times are good.
Stay the course!
chassis
Posts: 1123
Joined: Tue Mar 24, 2020 4:28 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by chassis »

BufordTJustice wrote: Sun Nov 20, 2022 4:02 pm All-

This is a great forum and in my short time here Ive already started to "know what I didnt know", if that makes sense. There are alot of very knowledgeable folks here who seem very willing to help others (who are less adept in financial areas, such as myself).

A basic question I have (as my employer uses Fidelity, and therefore I have a 401k there. Fidelity provides a planning tool where various current investments can be entered, along with planned contributions, as well as specifying timelines, etc. They provide various "what if" scenarios presumably based on historical statistics. The methodology is as below. What is consensus on the validity of these types of protection tools? I have been using the "significantly below average market" approach to (presumably) be safe.

Significantly Below Average Market
A significantly below average market is defined as the 90% confidence level of estimated future balances and/or estimated future income. The 90% confidence level represents significantly below average market conditions with 10% of all hypothetical scenarios tested performing worse. This means that in 90 out of 100 market scenarios tested a hypothetical portfolio similar to yours performed at least as well as the results shown and 10 out of 100 performed worse than the results shown.

Below Average Market
A below average market is defined as the 75% confidence level of estimated future balances and/or estimated future income. The 75% confidence level represents below average market conditions with 25% of all hypothetical scenarios tested performing worse. This means that in 75 out of 100 market scenarios tested a hypothetical portfolio similar to yours performed at least as well as the results shown and 25 out of 100 performed worse than the results shown.

Average Market
An average market is defined as the 50% confidence level of estimated future balances and/or estimated future income. The 50% confidence level represents average market conditions with 50% of all hypothetical scenarios tested performing worse. This means that in 50 out of 100 market scenarios tested a hypothetical portfolio similar to yours performed at least as well as the results shown and 50 out of 100 performed worse than the results shown.
Monte Carlo simulation has validity. The Fidelity version is sadly limited. As with all financial custodians, the Fidelity calculator answers suggest to the investor that "you don't have enough money" and "give us your money, keep it with us, and don't take it out."

Use portfoliovisualizer. It gives more control over input parameters, richer output, and allows the investor to decide for him/herself what the future may look like.

Be very clear that Monte Carlo is not a predictor. Rather, a Monte Carlo simulation gives a range of outcomes that are more or less likely to occur, with some degree of certainty and uncertainty. I find MC tools useful.
rbd789
Posts: 94
Joined: Fri Jul 24, 2020 2:58 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by rbd789 »

livesoft wrote: Sun Nov 20, 2022 5:46 pm
rbd789 wrote: Sun Nov 20, 2022 5:35 pmI do use it as one of my checks on progress, paying attention mostly to the "significantly below average" outputs.
OK, but let me ask again: You check on progress, but has it ever changed your behavior -- either making you spend less/invest more or the other way spend more/invest less? Or perhaps change asset allocation drastically?
No, I have an allocation that was chosen not long before retirement, based mainly on what I'd learned here. I maintain that allocation, and asset levels are high enough for me to be comfortable having lived below our means. Withdraw and spending are at low enough levels that ad hoc works. I simply use various tools to validate my choices and that they continue to perform as hoped/expected. My only point was to say that Fidelity's detailed methodology description is there for the asking.
JasonHutt
Posts: 119
Joined: Wed Oct 26, 2022 1:47 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by JasonHutt »

Here is a link to a Fidelity explanation for its retirement planner. If you skip to the Monte Carlo simulator you will notice that it's clear. Frankly, the whole document is surprisingly clear.

Unfortunately, if you slice and dice equity indexes with tilts and so forth, your projections will not reflect this, apparently. It uses S&P for most years, transitioning to a total US market index for recent decades. Three-Fund Portfolios are fine.

https://www.fidelity.com/bin-public/060 ... DOLOGY.pdf
User avatar
JoeRetire
Posts: 14498
Joined: Tue Jan 16, 2018 2:44 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by JoeRetire »

BufordTJustice wrote: Sun Nov 20, 2022 4:02 pmWhat is consensus on the validity of these types of protection tools?
What does "validity" mean for you in this context?
Oh, noooooo! I'm so sorry, it's the moops! The correct answer is 'the moops'.
Topic Author
Archie Bunker
Posts: 52
Joined: Sun Nov 20, 2022 12:12 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Archie Bunker »

JoeRetire wrote: Sun Nov 20, 2022 6:02 pm
BufordTJustice wrote: Sun Nov 20, 2022 4:02 pmWhat is consensus on the validity of these types of protection tools?
What does "validity" mean for you in this context?
Are the underlying assumptions used in these tools reasonable, and if so, have people found them to match their own personal outcomes?
Topic Author
Archie Bunker
Posts: 52
Joined: Sun Nov 20, 2022 12:12 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Archie Bunker »

JasonHutt wrote: Sun Nov 20, 2022 6:01 pm Here is a link to a Fidelity explanation for its retirement planner. If you skip to the Monte Carlo simulator you will notice that it's clear. Frankly, the whole document is surprisingly clear.

Unfortunately, if you slice and dice equity indexes with tilts and so forth, your projections will not reflect this, apparently. It uses S&P for most years, transitioning to a total US market index for recent decades. Three-Fund Portfolios are fine.

https://www.fidelity.com/bin-public/060 ... DOLOGY.pdf
Great find!

The table on page 12 is very helpful as that shows the types of investments in each of the nine generalized portfolio types. Thank you!
User avatar
JoeRetire
Posts: 14498
Joined: Tue Jan 16, 2018 2:44 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by JoeRetire »

BufordTJustice wrote: Sun Nov 20, 2022 6:05 pm
JoeRetire wrote: Sun Nov 20, 2022 6:02 pm
BufordTJustice wrote: Sun Nov 20, 2022 4:02 pmWhat is consensus on the validity of these types of protection tools?
What does "validity" mean for you in this context?
Are the underlying assumptions used in these tools reasonable, and if so, have people found them to match their own personal outcomes?
I haven't used the Fidelity tool, so I can't speak to that.
I have used the cFIREsim tool. The underlying assumptions used in this tool seem well thought out and completely reasonable.

My personal outcome cannot be known for 30 years or so. So far, so good.
Of course, that's the nature of simulation tools.
Oh, noooooo! I'm so sorry, it's the moops! The correct answer is 'the moops'.
trallium
Posts: 78
Joined: Wed Dec 29, 2021 7:53 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by trallium »

livesoft wrote: Sun Nov 20, 2022 5:46 pm
rbd789 wrote: Sun Nov 20, 2022 5:35 pmI do use it as one of my checks on progress, paying attention mostly to the "significantly below average" outputs.
OK, but let me ask again: You check on progress, but has it ever changed your behavior -- either making you spend less/invest more or the other way spend more/invest less? Or perhaps change asset allocation drastically?
I had access to this tool for something like 15 years. The investments office would send out an annual summary mailer and reminder to check the online tool.

I would not be retired now if I hadn't played around with it. It convinced me to double max out, fully funding both my 457b and 403b for several years.

It is nice that I can put things in there like when my house will be paid off, and you can model what will happen if you have long term care for the last years. I have not figured out if I can model Roth conversions with it.

https://www.fidelity.com/bin-public/060 ... DOLOGY.pdf
dcdowden
Posts: 255
Joined: Sun Sep 28, 2014 11:42 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by dcdowden »

When I retired over 20 years ago, I used the Quicken planning tool which was extremely limited since you simply picked an average annual rate of return for the projections. The Fidelity tool is far more sophisticated as is the one provided by Personal Capital and others. But in the end, these are just tools to help you make decisions about your savings, expenses and investment asset allocations. I look at their projections to understand the impact of possible decisions, but recognize that none of them can really project the future.
User avatar
Wiggums
Posts: 5474
Joined: Thu Jan 31, 2019 8:02 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Wiggums »

BufordTJustice wrote: Sun Nov 20, 2022 4:02 pm All-

This is a great forum and in my short time here Ive already started to "know what I didnt know", if that makes sense. There are alot of very knowledgeable folks here who seem very willing to help others (who are less adept in financial areas, such as myself).

A basic question I have (as my employer uses Fidelity, and therefore I have a 401k there. Fidelity provides a planning tool where various current investments can be entered, along with planned contributions, as well as specifying timelines, etc. They provide various "what if" scenarios presumably based on historical statistics. The methodology is as below. What is consensus on the validity of these types of protection tools? I have been using the "significantly below average market" approach to (presumably) be safe.

Significantly Below Average Market
A significantly below average market is defined as the 90% confidence level of estimated future balances and/or estimated future income. The 90% confidence level represents significantly below average market conditions with 10% of all hypothetical scenarios tested performing worse. This means that in 90 out of 100 market scenarios tested a hypothetical portfolio similar to yours performed at least as well as the results shown and 10 out of 100 performed worse than the results shown.

Below Average Market
A below average market is defined as the 75% confidence level of estimated future balances and/or estimated future income. The 75% confidence level represents below average market conditions with 25% of all hypothetical scenarios tested performing worse. This means that in 75 out of 100 market scenarios tested a hypothetical portfolio similar to yours performed at least as well as the results shown and 25 out of 100 performed worse than the results shown.

Average Market
An average market is defined as the 50% confidence level of estimated future balances and/or estimated future income. The 50% confidence level represents average market conditions with 50% of all hypothetical scenarios tested performing worse. This means that in 50 out of 100 market scenarios tested a hypothetical portfolio similar to yours performed at least as well as the results shown and 50 out of 100 performed worse than the results shown.
I created a spreadsheet before I discovered the Fidelity tool which you described. Interestingly, the results match my spreadsheet results. I used conservative variables in my spreadsheet, so I recommend the Fidelity tool for people looking for something free.

The Fidelity provides tables for average, below average, and significantly below average. I use the average table.

In my personal experience, the Fidelity gives you a good estimate. Like any future projection, the further out you go, the results are just a reasonable approximation for planning purposes only.

I manually enter our balances monthly which takes maybe 10 minutes. I export all three reports in PDF format.

My wife has been using Quicken over 20 years. I generally like the product, but it can’t handle proprietary funds. Whereas the Fidelity tool allows you to specify the stock/bond/cash ratio for each account/investment.

I hope this answers your question.
"I started with nothing and I still have most of it left."
DIYtrixie
Posts: 206
Joined: Sun Sep 13, 2020 12:11 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by DIYtrixie »

trallium wrote: Sun Nov 20, 2022 6:37 pm
livesoft wrote: Sun Nov 20, 2022 5:46 pm
rbd789 wrote: Sun Nov 20, 2022 5:35 pmI do use it as one of my checks on progress, paying attention mostly to the "significantly below average" outputs.
OK, but let me ask again: You check on progress, but has it ever changed your behavior -- either making you spend less/invest more or the other way spend more/invest less? Or perhaps change asset allocation drastically?
I had access to this tool for something like 15 years. The investments office would send out an annual summary mailer and reminder to check the online tool.

I would not be retired now if I hadn't played around with it. It convinced me to double max out, fully funding both my 457b and 403b for several years.

It is nice that I can put things in there like when my house will be paid off, and you can model what will happen if you have long term care for the last years. I have not figured out if I can model Roth conversions with it.

https://www.fidelity.com/bin-public/060 ... DOLOGY.pdf
The largest benefit I have realized to date with my annual suggested meeting with a rep at the local Fidelity office is poking around with this tool in real time with someone who understands all its bells and whistles. This helped me find some functions I was looking for but struggled to find on my own, like projected per-year, per-account spend down in retirement.
FIRWYW
Posts: 102
Joined: Thu Dec 23, 2021 9:11 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by FIRWYW »

I like the fidelity estimator. I am not sure I have changed anything I do because of it, but I like that you can easily adjust scenarios from linked accounts. It does not recognize back door roths correctly so you have to massage the inputs but still good. I also have used firecalc and vanguards nest calculator. I like firecalc the most but fidelity is still good
User avatar
Artful Dodger
Posts: 1646
Joined: Thu Oct 20, 2016 12:56 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Artful Dodger »

I use the Fidelity Planner and have for years. Most of my investments are with Fidelity and they feed automatically into the planner. I can link it to bank accounts and if needed can set up other funds manually. As others have noted, it’s a tool to help assess if your current investments, asset allocation, future investments, etc will cover your future spending when the time comes. One concern I have is how it handles inflation and is it adjusting for the current higher levels. I expect it is continuing to use their average inflation expectations which I believe are 2.5% overall and 5% for health care costs. There is a Fidelity hosted forum. I may post a question there.

I use the significantly below market return in my projection. OP, my planner estimates what I will have left at the end of plan. In my case, it’s close to $1M. Does yours do the same? I figure it’s just hedging the bet. I also don’t include my home equity in the plan. I’ll probably be one of those with $XXX,XXX left over when I kick the bucket. :?
Buford T Justice
Posts: 112
Joined: Mon Dec 15, 2014 5:33 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Buford T Justice »

Welcome to Bogleheads
livesoft
Posts: 81604
Joined: Thu Mar 01, 2007 8:00 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by livesoft »

trallium wrote: Sun Nov 20, 2022 6:37 pm[...]

I would not be retired now if I hadn't played around with it. It convinced me to double max out, fully funding both my 457b and 403b for several years.

....
Now that's a testimonial! Thanks!
Wiki This signature message sponsored by sscritic: Learn to fish.
Topic Author
Archie Bunker
Posts: 52
Joined: Sun Nov 20, 2022 12:12 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Archie Bunker »

Buford T Justice wrote: Sun Nov 20, 2022 7:16 pm Welcome to Bogleheads
OMG.....I PROMISE I did not try and steal your screen name😵‍💫
Topic Author
Archie Bunker
Posts: 52
Joined: Sun Nov 20, 2022 12:12 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Archie Bunker »

Artful Dodger wrote: Sun Nov 20, 2022 7:14 pmOP, my planner estimates what I will have left at the end of plan. In my case, it’s close to $1M. Does yours do the same?
Yes it does. It also gives the choice of seeing the projections based on income or total assets (either monthly or yearly) using either current day dollars or future (inflated dollars), and of course the end of plan age is adjustable.
Topic Author
Archie Bunker
Posts: 52
Joined: Sun Nov 20, 2022 12:12 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Archie Bunker »

Wiggums wrote: Sun Nov 20, 2022 6:49 pm I created a spreadsheet before I discovered the Fidelity tool which you described. Interestingly, the results match my spreadsheet results. I used conservative variables in my spreadsheet, so I recommend the Fidelity tool for people looking for something free.

The Fidelity provides tables for average, below average, and significantly below average. I use the average table.

In my personal experience, the Fidelity gives you a good estimate. Like any future projection, the further out you go, the results are just a reasonable approximation for planning purposes only.

I manually enter our balances monthly which takes maybe 10 minutes. I export all three reports in PDF format.

My wife has been using Quicken over 20 years. I generally like the product, but it can’t handle proprietary funds. Whereas the Fidelity tool allows you to specify the stock/bond/cash ratio for each account/investment.

I hope this answers your question.
Yes it does....thank you for the reply.

I have been trying to use the worse case (significantly below avg), but in the back of mind I am hoping for average.....who wouldnt :-) ). The results are obviously much much better (and my children will be far wealthier than I ever will be if that average holds). One of my retirement goals is to leave more than I ever had while I was still breathing.

I did see that the Fidelity tool does allow that rather discrete ration to be set for each of the invested line items (but thus far I have been lazy and just selected one choice overall for everything). I didnt want to spend too much time on the tool if folks here (aka those 10x smarter than me on this stuff) thought it was a useless pursuit.
Topic Author
Archie Bunker
Posts: 52
Joined: Sun Nov 20, 2022 12:12 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Archie Bunker »

trallium wrote: Sun Nov 20, 2022 6:37 pm
I would not be retired now if I hadn't played around with it. It convinced me to double max out, fully funding both my 457b and 403b for several years.

I am happy to hear you state this as in my short time messing with the tool, the "subjective" feedback it offers (regardless of whether or not it objectively holds 100% true) reinforces rather clearly the value of saving and doing so over time.

As for the 403b/457b......those things are AWESOME! My wifes employer offers both (I have only a 401k...albeit with a very generous company match from a very kind employer), so on her account we can go to the IRS limits (plus the catch up as we are both 50) x 2. I originally thought this was not possibly allowed, but a quick search online revealed it is possible. We havent been able to quite afford to max out both (while also maxing out my 401k and the Roths), but being able to do so is on my radar for the immediate future to get our savings rates up a bit higher (now that our childrens college costs are covered).
Mr. Buzzkill
Posts: 145
Joined: Mon Mar 28, 2022 8:23 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Mr. Buzzkill »

Keep in mind that Monte Carlo simulation is not a forecasting tool. It allows you to test and compare various scenarios using some randomization over multiple iterations.

In some ways it is like backtesting but backtesting on actual historical results for a specific period of time is deterministic, i.e., each scenario (in this case, specified portfolio) yields the same result for the same period every single time. No randomness.

Both allow you to test the general sanity of your portfolios and plans and reveal weaknesses relative to the inputs (historical or random) but always remember that neither the past nor random inputs predict the future. Future could be much or a little better or worse than simulations and backtesting would indicate.

I’ve found the best use of both to be testing your risk tolerance. If you can’t stomach or endure the maximum drawdown of a simulated or backtested portfolio or the probability of ending portfolio value of zero or less than whatever you plan to leave to heirs, then you probably should consider a less risky portfolio, because it could be worse in the future.
A strategy that works only in bull markets isn’t much of a strategy.
Topic Author
Archie Bunker
Posts: 52
Joined: Sun Nov 20, 2022 12:12 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Archie Bunker »

Artful Dodger wrote: Sun Nov 20, 2022 7:14 pm One concern I have is how it handles inflation and is it adjusting for the current higher levels. I expect it is continuing to use their average inflation expectations which I believe are 2.5% overall and 5% for health care costs. There is a Fidelity hosted forum. I may post a question there.
I just logged in and checked-----

"We assume a general average annual inflation rate of 2.5%, with the exception of health care costs. The inflation rate for those is based on a schedule of rates that starts at 4.9% and gradually declines until it matches the general inflation rate."
Topic Author
Archie Bunker
Posts: 52
Joined: Sun Nov 20, 2022 12:12 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Archie Bunker »

Artful Dodger wrote: Sun Nov 20, 2022 7:14 pm I use the significantly below market return in my projection. OP, my planner estimates what I will have left at the end of plan. Does yours do the same? I figure it’s just hedging the bet. I also don’t include my home equity in the plan.
Yes, mine works the same way I believe. I also didnt include anything re: home equity (as although I know it is tangible, to me the last thing Id want to do is ever rely on a reverse mortgage, etc unless I really had to). At that point, any port in a storm......

Using this mix and an age 62 retirement (we are 50 now), a $10k/mo income requirement (we have no debts and dont plan to have any so this is living expenses) and including SS estimation and a small pension my wife will get (but hey....well take it)---

Image
pic uploader

The simulator says the below is for significantly below average returns-----

Image
Image

If the financial universe shines on us all-----


Image
Image
pic uploader
Topic Author
Archie Bunker
Posts: 52
Joined: Sun Nov 20, 2022 12:12 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Archie Bunker »

And now that Ive left dreamland........

Image
pic uploader

Image
Image
pic uploader
FellsGuy
Posts: 189
Joined: Thu Dec 23, 2021 1:30 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by FellsGuy »

Mr. Buzzkill wrote: Sun Nov 20, 2022 8:16 pm Keep in mind that Monte Carlo simulation is not a forecasting tool. It allows you to test and compare various scenarios using some randomization over multiple iterations.

In some ways it is like backtesting but backtesting on actual historical results for a specific period of time is deterministic, i.e., each scenario (in this case, specified portfolio) yields the same result for the same period every single time. No randomness.

Both allow you to test the general sanity of your portfolios and plans and reveal weaknesses relative to the inputs (historical or random) but always remember that neither the past nor random inputs predict the future. Future could be much or a little better or worse than simulations and backtesting would indicate.

I’ve found the best use of both to be testing your risk tolerance. If you can’t stomach or endure the maximum drawdown of a simulated or backtested portfolio or the probability of ending portfolio value of zero or less than whatever you plan to leave to heirs, then you probably should consider a less risky portfolio, because it could be worse in the future.


I'm trying to understand your post and feel like I'm missing something. So you're saying Monte Carlo simulations are not forecasting but backtesting tools? And backtesting or Monte Carlo simulations may or may not predict the future accurately?
“Annual income twenty pounds, annual expenditure nineteen nineteen and six , result happiness. | Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery”
JasonHutt
Posts: 119
Joined: Wed Oct 26, 2022 1:47 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by JasonHutt »

FellsGuy wrote: Sun Nov 20, 2022 9:33 pm
Mr. Buzzkill wrote: Sun Nov 20, 2022 8:16 pm Keep in mind that Monte Carlo simulation is not a forecasting tool. It allows you to test and compare various scenarios using some randomization over multiple iterations.

In some ways it is like backtesting but backtesting on actual historical results for a specific period of time is deterministic, i.e., each scenario (in this case, specified portfolio) yields the same result for the same period every single time. No randomness.

Both allow you to test the general sanity of your portfolios and plans and reveal weaknesses relative to the inputs (historical or random) but always remember that neither the past nor random inputs predict the future. Future could be much or a little better or worse than simulations and backtesting would indicate.

I’ve found the best use of both to be testing your risk tolerance. If you can’t stomach or endure the maximum drawdown of a simulated or backtested portfolio or the probability of ending portfolio value of zero or less than whatever you plan to leave to heirs, then you probably should consider a less risky portfolio, because it could be worse in the future.


I'm trying to understand your post and feel like I'm missing something. So you're saying Monte Carlo simulations are not forecasting but backtesting tools? And backtesting or Monte Carlo simulations may or may not predict the future accurately?
Of course, in general English, MC simulation is a forecasting tool. However, it forecasts a distribution of results and lets you see this distribution. You can then decide what risks you want to take.

Let me add one interesting thing I've learned. The bond proportion, if too high, can lead to a GREATER number of bad outcomes (specifically portfolio going to zero). We need to think carefully about what "risk" really means.
Mr. Buzzkill
Posts: 145
Joined: Mon Mar 28, 2022 8:23 am

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Mr. Buzzkill »

MC is not inherently a forecasting technology and should not be used as such. Is is a What-if tool.

If MC was a reliable forecasting tool, then economists would not have predicted 17 of the last 11 recessions. :)

The distinction is subtle to some, maybe a matter of semantics to others. I studied Operations Research in university and used MC in a number of projects. We didn’t predict with it. We tested assumptions with many variables that could not be speedily done by hand. The outputs are only as good as the inputs including the model itself, not just the parameters fed into it.

Some might make personal forecasts based on the kinds of MC models that the OP mentioned and gain comfort. But read the disclaimers for those MC web pages. It’s easy to plug numbers into a model. Less easy to make predictions on which one can rely with high degree of confidence. Ask active and passive fund managers who have ridden Meta stock down to its current level.

Mathematical techniques of forecasting always seem to miss the black swan events. They are called black swans for a reason: you can’t anticipate them.

Surely if the MC on personal finance websites could forecast the future, they would have been used by economists, fund managers, and individual investors to sell Meta ahead of its crash, and reveal the disproportionate affect on market cap weighted index funds. It should have been used to forecast and predict COVID economic effects and the Fed’s slow response to inevitable inflationary pressures from over a decade of historically low interest rates, and the USA government overstimulating effect of cash relief benefits, right?

Does MC on the Financial firms websites forecast the growth prospects from the underfunding of social security and other pension systems? Drawdown effects of medical catastrophes? I’m guessing not.

MC models are only as good for forecasting as the assumptions behind them. Which is to say, FC doesn't consider things not explicitly baked into the model.

Dr. Bill Bernstein, often cited on BH, hasn’t written much on MC that I recall. But he has studied and written much about financial and economic history and he seems to suggest that those who don’t study history are doomed to repeat it. Even those who study it let recency bias cause discounting of historical lessons.

And the good doctor also seems to suggest not confusing good portfolio results with investing skill. Even a stopped analog clock is correct twice per day.
A strategy that works only in bull markets isn’t much of a strategy.
FellsGuy
Posts: 189
Joined: Thu Dec 23, 2021 1:30 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by FellsGuy »

JasonHutt wrote: Sun Nov 20, 2022 10:23 pm
FellsGuy wrote: Sun Nov 20, 2022 9:33 pm
Mr. Buzzkill wrote: Sun Nov 20, 2022 8:16 pm Keep in mind that Monte Carlo simulation is not a forecasting tool. It allows you to test and compare various scenarios using some randomization over multiple iterations.

In some ways it is like backtesting but backtesting on actual historical results for a specific period of time is deterministic, i.e., each scenario (in this case, specified portfolio) yields the same result for the same period every single time. No randomness.

Both allow you to test the general sanity of your portfolios and plans and reveal weaknesses relative to the inputs (historical or random) but always remember that neither the past nor random inputs predict the future. Future could be much or a little better or worse than simulations and backtesting would indicate.

I’ve found the best use of both to be testing your risk tolerance. If you can’t stomach or endure the maximum drawdown of a simulated or backtested portfolio or the probability of ending portfolio value of zero or less than whatever you plan to leave to heirs, then you probably should consider a less risky portfolio, because it could be worse in the future.


I'm trying to understand your post and feel like I'm missing something. So you're saying Monte Carlo simulations are not forecasting but backtesting tools? And backtesting or Monte Carlo simulations may or may not predict the future accurately?
Of course, in general English, MC simulation is a forecasting tool. However, it forecasts a distribution of results and lets you see this distribution. You can then decide what risks you want to take.

Let me add one interesting thing I've learned. The bond proportion, if too high, can lead to a GREATER number of bad outcomes (specifically portfolio going to zero). We need to think carefully about what "risk" really means.
That Bond proportion comment is interesting but isnt that simply a result of lower growth and yield over longer periods of time?
“Annual income twenty pounds, annual expenditure nineteen nineteen and six , result happiness. | Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery”
User avatar
vineviz
Posts: 14180
Joined: Tue May 15, 2018 1:55 pm
Location: Baltimore, MD

Re: Accuracy of the "Monte Carlo" Simulations?

Post by vineviz »

FellsGuy wrote: Sun Nov 20, 2022 9:33 pm
Mr. Buzzkill wrote: Sun Nov 20, 2022 8:16 pm Keep in mind that Monte Carlo simulation is not a forecasting tool. It allows you to test and compare various scenarios using some randomization over multiple iterations.

In some ways it is like backtesting but backtesting on actual historical results for a specific period of time is deterministic, i.e., each scenario (in this case, specified portfolio) yields the same result for the same period every single time. No randomness.

Both allow you to test the general sanity of your portfolios and plans and reveal weaknesses relative to the inputs (historical or random) but always remember that neither the past nor random inputs predict the future. Future could be much or a little better or worse than simulations and backtesting would indicate.

I’ve found the best use of both to be testing your risk tolerance. If you can’t stomach or endure the maximum drawdown of a simulated or backtested portfolio or the probability of ending portfolio value of zero or less than whatever you plan to leave to heirs, then you probably should consider a less risky portfolio, because it could be worse in the future.


I'm trying to understand your post and feel like I'm missing something. So you're saying Monte Carlo simulations are not forecasting but backtesting tools? And backtesting or Monte Carlo simulations may or may not predict the future accurately?
Monte Carlo analyses of the kind we are discussing are absolutely NOT "backtesting tools". A backtest is simply an examination of how a certain scenario (asset allocation, timing strategy, etc.) would have played out given the ACTUAL set of historical data. The comparison to backtesting is completely unhelpful, so no wonder you are confused.

It is nearly as unhelpful, IMHO, to say that a Monte Carlo simulation is "not a forecasting tool" since the word "forecast" is incredibly ambiguous. I also don't like the term "forecast", but essentially that's what the Monte Carlo analysis produces: a large number of discrete forecasts/estimates/predictions each of which is conditioned on a particular sample of inputs. Most weather "forecasts", for instance, are built using some form of Monte Carlo analysis.

Think of Monte Carlo analysis as giving you answer to the following question: IF we assume the following range of inputs, WHAT is the range of possible outcomes?

It's not telling you which particular outcome you will get, but it WILL tell you the probability of getting a particular outcome conditional upon the model inputs being correct. The more comfortable you are about the inputs being right, the more confident you can be about the output.
"Far more money has been lost by investors preparing for corrections than has been lost in corrections themselves." ~~ Peter Lynch
Greg in Idaho
Posts: 209
Joined: Tue Dec 27, 2016 12:59 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by Greg in Idaho »

vineviz wrote: Mon Nov 21, 2022 7:35 am
FellsGuy wrote: Sun Nov 20, 2022 9:33 pm
Mr. Buzzkill wrote: Sun Nov 20, 2022 8:16 pm Keep in mind that Monte Carlo simulation is not a forecasting tool. It allows you to test and compare various scenarios using some randomization over multiple iterations.

In some ways it is like backtesting but backtesting on actual historical results for a specific period of time is deterministic, i.e., each scenario (in this case, specified portfolio) yields the same result for the same period every single time. No randomness.

Both allow you to test the general sanity of your portfolios and plans and reveal weaknesses relative to the inputs (historical or random) but always remember that neither the past nor random inputs predict the future. Future could be much or a little better or worse than simulations and backtesting would indicate.

I’ve found the best use of both to be testing your risk tolerance. If you can’t stomach or endure the maximum drawdown of a simulated or backtested portfolio or the probability of ending portfolio value of zero or less than whatever you plan to leave to heirs, then you probably should consider a less risky portfolio, because it could be worse in the future.


I'm trying to understand your post and feel like I'm missing something. So you're saying Monte Carlo simulations are not forecasting but backtesting tools? And backtesting or Monte Carlo simulations may or may not predict the future accurately?
Monte Carlo analyses of the kind we are discussing are absolutely NOT "backtesting tools". A backtest is simply an examination of how a certain scenario (asset allocation, timing strategy, etc.) would have played out given the ACTUAL set of historical data. The comparison to backtesting is completely unhelpful, so no wonder you are confused.

It is nearly as unhelpful, IMHO, to say that a Monte Carlo simulation is "not a forecasting tool" since the word "forecast" is incredibly ambiguous. I also don't like the term "forecast", but essentially that's what the Monte Carlo analysis produces: a large number of discrete forecasts/estimates/predictions each of which is conditioned on a particular sample of inputs. Most weather "forecasts", for instance, are built using some form of Monte Carlo analysis.

Think of Monte Carlo analysis as giving you answer to the following question: IF we assume the following range of inputs, WHAT is the range of possible outcomes?

It's not telling you which particular outcome you will get, but it WILL tell you the probability of getting a particular outcome conditional upon the model inputs being correct. The more comfortable you are about the inputs being right, the more confident you can be about the output.
Thanks Mr. B...very helpful way of framing how to think about and us MC simulations.

The pedantic attempts at correcting Mr. B. - not so helpful.
User avatar
vineviz
Posts: 14180
Joined: Tue May 15, 2018 1:55 pm
Location: Baltimore, MD

Re: Accuracy of the "Monte Carlo" Simulations?

Post by vineviz »

Greg in Idaho wrote: Mon Nov 21, 2022 8:19 am Thanks Mr. B...very helpful way of framing how to think about and us MC simulations.

The pedantic attempts at correcting Mr. B. - not so helpful.
What's the alternative to correcting misinformation? Letting it stand as-is?

Either nothing related to financial planning involves "forecasting" or everything does.
"Far more money has been lost by investors preparing for corrections than has been lost in corrections themselves." ~~ Peter Lynch
sixtyforty
Posts: 640
Joined: Tue Nov 25, 2014 12:22 pm
Location: USA

Re: Accuracy of the "Monte Carlo" Simulations?

Post by sixtyforty »

I use Fidelity Retirement Planning tool w/ "Significantly below average" returns. I think it's a useful tool. I also run other retirement planner tools and find FRP to be a bit more on the conservative side.
"Simplicity is the ultimate sophistication" - Leonardo Da Vinci
trallium
Posts: 78
Joined: Wed Dec 29, 2021 7:53 pm

Re: Accuracy of the "Monte Carlo" Simulations?

Post by trallium »

Mr. Buzzkill wrote: Mon Nov 21, 2022 4:31 am ...
It should have been used to forecast and predict COVID economic effects and the Fed’s slow response to inevitable inflationary pressures from over a decade of historically low interest rates, and the USA government overstimulating effect of cash relief benefits, right?
I don't see how this follows at all. You don't think some of the random simulations went through downturns that were comparable or worse? My pension is inflation adjusted and my mortgage won't increase, so I just run them in nominal dollars mode. The PDF that was shared on here discusses some of the limitations of inflation modeling.

Nate Silver (who makes Baysean models for sports and politics) does not maintain the Fidelity tool. I guess he probably does not have the reputation he once did.
Mr. Buzzkill wrote: Mon Nov 21, 2022 4:31 am Does MC on the Financial firms websites forecast the growth prospects from the underfunding of social security and other pension systems? Drawdown effects of medical catastrophes? I’m guessing not.
...
I had to copy the social security estimates from the SS website. I could discount those to 70% or 0% if I wanted to model that.

There is a toggle for "long term care" that I can adjust the price of and start year for.

These are just tools people! If you understand the basic methodology and limitations, I think a lot of people find they provide useful information.
Post Reply