Refining the failure rate, when it fails matters, not just how often

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
Post Reply
Topic Author
larryswedroe
Posts: 16020
Joined: Thu Feb 22, 2007 8:28 am
Location: St Louis MO

Refining the failure rate, when it fails matters, not just how often

Post by larryswedroe » Mon Mar 06, 2017 9:26 am

There's a big difference in failing in the 29th year vs the 15th year.

http://www.etf.com/sections/index-inves ... es-failure

In planning, it's important to analyze not just the failure rate but when it tends to fail
Larry

Greg in Idaho
Posts: 117
Joined: Tue Dec 27, 2016 12:59 pm

Re: Refining the failure rate, when it fails matters, not just how often

Post by Greg in Idaho » Mon Mar 06, 2017 9:48 am

Often a surprise to hear about what people aren't considering on this front...but I have a 0% failure rate on the current plan, and look at the charts and tables to see where the lows are under plans that do have a failure rate (which is pretty easy). I'd never just rely on a single number any model spit out...

User avatar
willthrill81
Posts: 15058
Joined: Thu Jan 26, 2017 3:17 pm
Location: USA

Re: Refining the failure rate, when it fails matters, not just how often

Post by willthrill81 » Mon Mar 06, 2017 10:31 am

Nice article Larry.

Do the models that BAM uses provide guidance on when adjustments are needed during the withdrawal phase or only at the beginning? It seems that most people concerned with determining a SWR are only interested in getting it 'right' at the beginning of the withdrawal phase, but I think that it's potentially more useful to know when adjustments need to made during the withdrawal phase.
“It's a dangerous business, Frodo, going out your door. You step onto the road, and if you don't keep your feet, there's no knowing where you might be swept off to.” J.R.R. Tolkien,The Lord of the Rings

User avatar
ERMD
Posts: 291
Joined: Thu Dec 26, 2013 11:26 am

Re: Refining the failure rate, when it fails matters, not just how often

Post by ERMD » Mon Mar 06, 2017 11:07 am

Greg in Idaho wrote:but I have a 0% failure rate on the current plan
does not exist
between scotch and nothing, i'll take scotch. -- faulkner

User avatar
willthrill81
Posts: 15058
Joined: Thu Jan 26, 2017 3:17 pm
Location: USA

Re: Refining the failure rate, when it fails matters, not just how often

Post by willthrill81 » Mon Mar 06, 2017 11:31 am

ERMD wrote:
Greg in Idaho wrote:but I have a 0% failure rate on the current plan
does not exist
In the context of historical data and/or simulations, it absolutely does.

In the context of being able to predict the future, you are correct. But no one can predict the future, so that's why we use "failure rates" as our next best option currently. But for each person, there is no "failure rate"; you either succeed or you fail.
“It's a dangerous business, Frodo, going out your door. You step onto the road, and if you don't keep your feet, there's no knowing where you might be swept off to.” J.R.R. Tolkien,The Lord of the Rings

MoonOrb
Posts: 1004
Joined: Thu Jan 24, 2013 6:58 pm

Re: Refining the failure rate, when it fails matters, not just how often

Post by MoonOrb » Mon Mar 06, 2017 11:38 am

FIRECalc is a nice complement to the Monte Carlo simulations for this type of planning, I think.

User avatar
ERMD
Posts: 291
Joined: Thu Dec 26, 2013 11:26 am

Re: Refining the failure rate, when it fails matters, not just how often

Post by ERMD » Mon Mar 06, 2017 1:28 pm

willthrill81 wrote:But for each person, there is no "failure rate"; you either succeed or you fail.
bingo
between scotch and nothing, i'll take scotch. -- faulkner

lack_ey
Posts: 6701
Joined: Wed Nov 19, 2014 11:55 pm

Re: Refining the failure rate, when it fails matters, not just how often

Post by lack_ey » Mon Mar 06, 2017 1:32 pm

Who'd have thunk that binarizing outcomes leads to loss of information?

Topic Author
larryswedroe
Posts: 16020
Joined: Thu Feb 22, 2007 8:28 am
Location: St Louis MO

Re: Refining the failure rate, when it fails matters, not just how often

Post by larryswedroe » Mon Mar 06, 2017 2:04 pm

willthrill
Our system allows us, and our clients therefore, to look at MCS output literally on daily basis. It can also be set up to have alerts to when a failure rate reaches a certain level. Now typically we review these only every few years but also ANY TIME a life event that impacts one's ability, willingness and/or need to take risk occurs, or major market moves that are well above or way below the "expected" return occurs. That allows adjustments to be made along the way. Financial planning is not a "one and done" endeavor, or at least it should not be
Larry

james22
Posts: 1488
Joined: Tue Aug 21, 2007 2:22 pm

Re: Refining the failure rate, when it fails matters, not just how often

Post by james22 » Tue Mar 07, 2017 1:26 am

How do these relate?

Current valuation metrics should be used.

The Monte Carlo simulator *randomly* selects a return for each year...

Is there no assumption of RTM? Are not each year's returns influenced by the prior?

Topic Author
larryswedroe
Posts: 16020
Joined: Thu Feb 22, 2007 8:28 am
Location: St Louis MO

Re: Refining the failure rate, when it fails matters, not just how often

Post by larryswedroe » Tue Mar 07, 2017 9:44 am

James
So we use the current valuations to forecast beta return and then historical factor returns and haircut them by 25%.
Then add correlations and SD.
Then the model runs 3k different scenarios out to 30 years
Larry

Random Walker
Posts: 4222
Joined: Fri Feb 23, 2007 8:21 pm

Re: Refining the failure rate, when it fails matters, not just how often

Post by Random Walker » Tue Mar 07, 2017 10:57 am

I'm just going to make a plug here for MCS. Once I had a good MCS run for me, I found that I don't want to make another future major financial decision without evaluating it with MCS. And at this time in particular, after an almost 9 year run up in the market I think it's a valuable tool to prove to many investors that they can reach their financial goals with a significantly less aggressive AA.

Dave

james22
Posts: 1488
Joined: Tue Aug 21, 2007 2:22 pm

Re: Refining the failure rate, when it fails matters, not just how often

Post by james22 » Wed Mar 08, 2017 12:43 am

larryswedroe wrote:James
So we use the current valuations to forecast beta return and then historical factor returns and haircut them by 25%.
Then add correlations and SD.
Then the model runs 3k different scenarios out to 30 years
Larry
Thanks, Larry.

But is your randomly selected return for each year generated from the same probability distribution?

Or does it recognize path dependency, with each selected return skewing positively or negatively the probability distribution of the following year?

Topic Author
larryswedroe
Posts: 16020
Joined: Thu Feb 22, 2007 8:28 am
Location: St Louis MO

Re: Refining the failure rate, when it fails matters, not just how often

Post by larryswedroe » Wed Mar 08, 2017 3:39 pm

James
Randomly draws from the initial input assumptions.
While there is some logic to what you are suggesting there are no tools that do that, at least that we are aware of.
Larry

james22
Posts: 1488
Joined: Tue Aug 21, 2007 2:22 pm

Re: Refining the failure rate, when it fails matters, not just how often

Post by james22 » Thu Mar 09, 2017 12:59 am

Too bad.

Each selected return impacts the future, of course, and random generation excludes the previous period:
larryswedroe wrote:I always find it somewhat amusing that people say there's no small premium when you exclude the 75-83 period. All that shows you is all factors/asset classes can have long periods of relatively poor performance, and they can have long and short periods of very strong performance. The reason small did so poorly after that period is because the small bubble led to prices going so high that future returns were virtually doomed to be lower, and that is the same thing that happened to growth stocks in late 90s, dooming returns after to a lower return fate. You cannot exclude one period without understanding how it impacted the future.
RTM would narrow the distribution of outcomes. But I suppose you focus on the central outcome anyway?

Thanks again.

lack_ey
Posts: 6701
Joined: Wed Nov 19, 2014 11:55 pm

Re: Refining the failure rate, when it fails matters, not just how often

Post by lack_ey » Thu Mar 09, 2017 1:18 am

Seems like it shouldn't be that difficult to do (to account for distributions shifting over the course of a simulation based on what prior returns were or some other evolution), even accounting for the correlation between assets. Though I guess if you're using some canned solution from somebody else you couldn't add something like that.

That said, by that point you're making so many assumptions any changes you're doing may not really be improving things anyway. Hard to tell the truth of the underlying asset behaviors. And I don't think RTM is as strong as a lot of people think. Though if nothing else a sharp equity drop (spike) should probably influence valuations and thus probably there is some justification to think future returns will be higher (lower).

Of course with bonds higher returns earlier should mean lower yields and lower returns later and vice versa. That much should be accounted for, it should seem.

Topic Author
larryswedroe
Posts: 16020
Joined: Thu Feb 22, 2007 8:28 am
Location: St Louis MO

Re: Refining the failure rate, when it fails matters, not just how often

Post by larryswedroe » Thu Mar 09, 2017 9:06 am

James
Few thoughts
As you note lower valuations do produce higher future returns. But it's also possible that lower returns don't produce lower valuations because earnings fall just as fast.

Also lower returns when they do produce lower valuations don't mean that you will get those higher returns--see Japan. So you want to see the tails, and that is what you want to focus on as well as the mean. You need to have a Plan B which allows you to address the risk of the tails should they show up.

That's why it's important to look at the entire distribution, and not just the mean

And finally even if you get lower returns and higher valuations it could be that bond yields collapse too and that can offset much of the benefit of lower valuations of stocks.

Larry

Random Walker
Posts: 4222
Joined: Fri Feb 23, 2007 8:21 pm

Re: Refining the failure rate, when it fails matters, not just how often

Post by Random Walker » Thu Mar 09, 2017 12:18 pm

James,
I'm not a statistician, financial economist, or anything close to that. But I do think it's fair to say that there is almost no predictive value of one year's returns in determining the next year's returns. So I think it makes good sense in these simulations to make an initial assumption for mean long run asset class returns based on valuations and historic premiums, and then let a computer randomly assign yearly returns around a mean, SD, +/- fat tails. I think it's reasonable to say that your valuation issue is addressed up front in the initial assumptions. And if valuations change substantially at some point, the MCS can be run again in the future with new assumptions.

Dave

james22
Posts: 1488
Joined: Tue Aug 21, 2007 2:22 pm

Re: Refining the failure rate, when it fails matters, not just how often

Post by james22 » Sat Mar 11, 2017 3:16 am

Thanks again, Larry. You're right, protecting against the left tail maybe most important.

And you're right too, Dave, simply re-running the MCS addresses most of this. I'd overlooked:

Now typically we review these only every few years but also ANY TIME a life event that impacts one's ability, willingness and/or need to take risk occurs, or major market moves that are well above or way below the "expected" return occurs.

Phil DeMuth
Posts: 152
Joined: Thu Mar 06, 2008 7:02 pm

Re: Refining the failure rate, when it fails matters, not just how often

Post by Phil DeMuth » Sat Mar 11, 2017 10:52 am

My guess: the highest Sharpe ratio (in the sense of return/risk) portfolio wins, not the highest Geometric Mean portfolio. This could be an important lesson in target practice for anyone who might retire one day.

User avatar
HomerJ
Posts: 13550
Joined: Fri Jun 06, 2008 12:50 pm

Re: Refining the failure rate, when it fails matters, not just how often

Post by HomerJ » Sat Mar 11, 2017 11:19 am

larryswedroe wrote:There's a big difference in failing in the 29th year vs the 15th year.

http://www.etf.com/sections/index-inves ... es-failure

In planning, it's important to analyze not just the failure rate but when it tends to fail
Larry
It's also important to understand what "failure" means. For many here on these boards, it means you cut back to 2 vacations a year instead of 4.

People hear 5% chance of "failure" and they think 5% chance of starving is too high. But it's not 5% chance of starving. It's a 5% chance that you may only visit Europe 15 times in retirement instead of 25.

Post Reply