Backtesting Results

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
Post Reply
Topic Author
justsomeguy2018
Posts: 277
Joined: Wed Oct 03, 2018 8:11 pm

Backtesting Results

Post by justsomeguy2018 » Wed Jan 09, 2019 7:59 pm

I've recently used Portfolio Visualizer to backtest different AAs of asset classes (bonds, stock, international, small cap, mid cap, etc.). Seems like the results are entirely dependent on the time period chosen to backtest.

How do you get around cherry picking data to decide upon a "best" allocation (e.g. % to midcap, growth stocks, bonds, emerging markets, etc.)?

02nz
Posts: 2102
Joined: Wed Feb 21, 2018 3:17 pm

Re: Backtesting Results

Post by 02nz » Wed Jan 09, 2019 8:11 pm

One of the ways is to test adding more money periodically, you can set how much and whether it goes up with inflation. This is much more like how real people invest than looking at the growth of $10,000 (or whatever), with nothing ever added, and it makes it less critical which starting point you choose.

User avatar
Tyler9000
Posts: 425
Joined: Fri Aug 21, 2015 11:57 am

Re: Backtesting Results

Post by Tyler9000 » Wed Jan 09, 2019 8:32 pm

My personal strategy to avoid cherry picking is to not just look at one investing timeframe but to model all of them simultaneously. Any individual timeframe can unfairly hurt or favor a particular portfolio, but I've found you can learn a lot by studying the full spread of returns. Portfolio Charts is my effort to share that idea with other investors, so hopefully you'll find it as interesting as I do. And if you're looking for one place to start, you might like this article: The Avoidable Mistake of Cherry Picking Data
Last edited by Tyler9000 on Thu Jan 10, 2019 11:23 am, edited 1 time in total.

venkman
Posts: 1023
Joined: Tue Mar 14, 2017 10:33 pm

Re: Backtesting Results

Post by venkman » Wed Jan 09, 2019 11:51 pm

justsomeguy2018 wrote:
Wed Jan 09, 2019 7:59 pm
How do you get around cherry picking data to decide upon a "best" allocation (e.g. % to midcap, growth stocks, bonds, emerging markets, etc.)?
The "best" allocation is unknowable, because your investment horizon is just another time period on the data charts (that we don't have data for yet). The optimal strategy for the next 30 years will only be evident 30 years from now. Ex-ante, the optimal strategy is broad diversification and minimizing costs.

User avatar
Portfolio7
Posts: 587
Joined: Tue Aug 02, 2016 3:53 am

Re: Backtesting Results

Post by Portfolio7 » Thu Jan 10, 2019 12:30 am

You might start by reading about the three fund portfolio in the Wiki. https://www.bogleheads.org/wiki/Three-fund_portfolio

You may also read in the wiki how owning the entire market (at market proportions) is considered to be the most efficient form of investing. Bogleheads tend to favor the three fund portfolio for that reason, though there are many 'acceptable' portfolios. Though some people who frequent the site do maintain different portfolios for a variety of reasons, you may want to start with simple and relatively foolproof. Many experienced investors stay with simple and never complicate their portfolio; there are many very good reasons for this behavior described in the wiki. Warren Buffet's investment partner, Charlie Munger, likes to emphasize that a critical factor in successful investing is avoiding dumb mistakes (I have not always succeeded in doing so). Following the guidance in creating a 3-Fund portfolio will help you do so.

There will always be thousands of portfolios better than yours for any particular sequence of years; It's impossible to pick the best performing asset classes in advance. For that reason it makes sense to pick a portfolio that will tend to do well in most time periods.
An investment in knowledge pays the best interest.

User avatar
hdas
Posts: 1019
Joined: Thu Jun 11, 2015 8:24 am

Re: Backtesting Results

Post by hdas » Thu Jan 10, 2019 1:17 am

Here’s an illuminating article about the general case of your query. Cheers :greedy
"whenever there is a randomized way of doing something, then there is a nonrandomized way that delivers better performance but requires more thought" ET Jaynes

User avatar
Taylor Larimore
Advisory Board
Posts: 28488
Joined: Tue Feb 27, 2007 8:09 pm
Location: Miami FL

Re: Backtesting Results

Post by Taylor Larimore » Thu Jan 10, 2019 9:01 pm

justsomeguy2018 wrote:
Wed Jan 09, 2019 7:59 pm
I've recently used Portfolio Visualizer to backtest different AAs of asset classes (bonds, stock, international, small cap, mid cap, etc.). Seems like the results are entirely dependent on the time period chosen to backtest.

How do you get around cherry picking data to decide upon a "best" allocation (e.g. % to midcap, growth stocks, bonds, emerging markets, etc.)?
justsomeguy2018:

Be very careful about using Portfolio Visualizer for "backtesting." Our mentor, Jack Bogle, wrote:

"The biggest mistake investors make is looking backward at performance and thinking it’ll recur in the future."

Experienced investors agree.

Best wishes.
Taylor
"Simplicity is the master key to financial success." -- Jack Bogle

User avatar
arcticpineapplecorp.
Posts: 4252
Joined: Tue Mar 06, 2012 9:22 pm

Re: Backtesting Results

Post by arcticpineapplecorp. » Thu Jan 10, 2019 9:23 pm

investing by looking backwards is like driving only looking through the rear view mirror.

there are 150 portfolios (or more). See below. One of them is bound to be the best. Problem is, you'll never know until after it's already happened.

https://www.whitecoatinvestor.com/150-p ... han-yours/

Best to pick an allocation and stick with it. If you change too often, you'll never get the returns you should have gotten (it's called "chasing performance").

Better yet, just own the total market (stocks and bonds, in the allocation that is appropriate to your need, ability and willingness to take risk) and call it a day. You could do worse. Don't let the perfect be the enemy of the good enough. When you own the market you get the return of the market. When you own something else, you get something else. Could be better, could be worse. Why take unecessary risk?
"May you live as long as you want and never want as long as you live" -- Irish Blessing | "Invest we must" -- Jack Bogle

User avatar
CaliJim
Posts: 3050
Joined: Sun Feb 28, 2010 8:47 pm
Location: California, near the beach

Re: Backtesting Results

Post by CaliJim » Thu Jan 10, 2019 9:49 pm

I started my boglehead education by doing backtesting with Simba's Spreadsheet.

Eventually I internalized what others here were saying about unpossibility of predicting the future. There is so much diffusion of results the effort is futility. monte carlo simulation is probably a better way to do it. save as much as you can. live well below your means, don't overestimate your risk tolerance, always own a good amount of safe assets (ie. bonds) so you can respond by rebalancing aggresively when equities go on sale which they inevitably be during your lifetime markets. Having a balanced stock AND bond port will cushion the blow when stocks decline 50%
If only 50/50 port suffers only a 25% decline when equities decline by half. This is a huge benefit and can be the difference between panic selling at the bottom vs rebalancing and buying at the bottom. it is very good advice to buy when prices are low!!!!!!
IMHO, The three fund portfolio advocated by tayloe is a very good fit for most investors. 30 years from now it WILL BE POSSIBLE to identify the slice and dice portfolio that will outperform the 3 fund port. but the chances are that the "winning" slice and dice port will not be the one you picked based on back testing. if only forward testing was possible.
Last edited by CaliJim on Fri Jan 18, 2019 10:10 pm, edited 6 times in total.
-calijim- | | For more info, click this Wiki

Topic Author
justsomeguy2018
Posts: 277
Joined: Wed Oct 03, 2018 8:11 pm

Re: Backtesting Results

Post by justsomeguy2018 » Thu Jan 10, 2019 9:53 pm

Taylor Larimore wrote:
Thu Jan 10, 2019 9:01 pm
justsomeguy2018 wrote:
Wed Jan 09, 2019 7:59 pm
I've recently used Portfolio Visualizer to backtest different AAs of asset classes (bonds, stock, international, small cap, mid cap, etc.). Seems like the results are entirely dependent on the time period chosen to backtest.

How do you get around cherry picking data to decide upon a "best" allocation (e.g. % to midcap, growth stocks, bonds, emerging markets, etc.)?
justsomeguy2018:

Be very careful about using Portfolio Visualizer for "backtesting." Our mentor, Jack Bogle, wrote:

"The biggest mistake investors make is looking backward at performance and thinking it’ll recur in the future."

Experienced investors agree.

Best wishes.
Taylor
Mr. Larimore, any AA in particular you recommend?

I am 37 years old. I am generally following the current AA, at the moment:

67% U.S. Equity
13% Intl Equity
20% Total US Bond

Thank you

User avatar
Taylor Larimore
Advisory Board
Posts: 28488
Joined: Tue Feb 27, 2007 8:09 pm
Location: Miami FL

Re: Backtesting Results

Post by Taylor Larimore » Thu Jan 10, 2019 10:23 pm

justsomeguy2018 wrote:
Mr. Larimore, any AA in particular you recommend?

I am 37 years old. I am generally following the current AA, at the moment:

67% U.S. Equity
13% Intl Equity
20% Total US Bond

Thank you
justsomeguy2018:

Your Asset-Allocation appears reasonable to me. I like the fact that it reflects The Three-Fund Portfolio. On the other hand, it is impossible for me to recommend your best asset-allocation. Only you know your goal(s), your time-frame, your risk-tolerance, and your personal financial situation.

Use this link to the Vanguard Investor Questionnaire to help you decide your best Asset-Allocation:

https://personal.vanguard.com/us/FundsI ... unds/tools

Best wishes.
Taylor
"Simplicity is the master key to financial success." -- Jack Bogle

User avatar
abuss368
Posts: 14403
Joined: Mon Aug 03, 2009 2:33 pm
Location: Where the water is warm, the drinks are cold, and I don't know the names of the players!

Re: Backtesting Results

Post by abuss368 » Fri Jan 11, 2019 6:59 pm

justsomeguy2018 wrote:
Thu Jan 10, 2019 9:53 pm

I am 37 years old. I am generally following the current AA, at the moment:

67% U.S. Equity
13% Intl Equity
20% Total US Bond

Thank you
I think you have a very simple and sophisticated investment portfolio. The portfolio is low cost and very diversified.

The most important decision an investor will make is asset allocation. That is the allocation between stocks and bonds based on goals, timeframe, and tolerance for risk.

Keep investing simple.
John C. Bogle: "You simply do not need to put your money into 8 different mutual funds!"

User avatar
CaliJim
Posts: 3050
Joined: Sun Feb 28, 2010 8:47 pm
Location: California, near the beach

Re: Backtesting Results

Post by CaliJim » Fri Jan 18, 2019 10:11 pm

abuss368 wrote:
Fri Jan 11, 2019 6:59 pm
justsomeguy2018 wrote:
Thu Jan 10, 2019 9:53 pm

I am 37 years old. I am generally following the current AA, at the moment:

67% U.S. Equity
13% Intl Equity
20% Total US Bond

Thank you
Keep investing simple.
simplicity!!!!! absolutely great advice.
-calijim- | | For more info, click this Wiki

User avatar
abuss368
Posts: 14403
Joined: Mon Aug 03, 2009 2:33 pm
Location: Where the water is warm, the drinks are cold, and I don't know the names of the players!

Re: Backtesting Results

Post by abuss368 » Fri Jan 18, 2019 11:30 pm

CaliJim wrote:
Fri Jan 18, 2019 10:11 pm
abuss368 wrote:
Fri Jan 11, 2019 6:59 pm
justsomeguy2018 wrote:
Thu Jan 10, 2019 9:53 pm

I am 37 years old. I am generally following the current AA, at the moment:

67% U.S. Equity
13% Intl Equity
20% Total US Bond

Thank you
Keep investing simple.
simplicity!!!!! absolutely great advice.
:beer
John C. Bogle: "You simply do not need to put your money into 8 different mutual funds!"

User avatar
willthrill81
Posts: 11823
Joined: Thu Jan 26, 2017 3:17 pm
Location: USA

Re: Backtesting Results

Post by willthrill81 » Fri Jan 18, 2019 11:37 pm

justsomeguy2018 wrote:
Wed Jan 09, 2019 7:59 pm
I've recently used Portfolio Visualizer to backtest different AAs of asset classes (bonds, stock, international, small cap, mid cap, etc.). Seems like the results are entirely dependent on the time period chosen to backtest.

How do you get around cherry picking data to decide upon a "best" allocation (e.g. % to midcap, growth stocks, bonds, emerging markets, etc.)?
You can't. It's as simple as that. You don't know which mix is going to be optimal for the sake of helping you reach your goals in the way in which you want to reach them. This is also true of your asset allocation (i.e. stocks vs. fixed income).

That's a serious problem with factor investing: you are guaranteed to underperform the market at some point, and this underperformance can go on for years. Unless you're prepared to stick with it, then don't invest in factors and buy the whole market.
“It's a dangerous business, Frodo, going out your door. You step onto the road, and if you don't keep your feet, there's no knowing where you might be swept off to.” J.R.R. Tolkien,The Lord of the Rings

furnace
Posts: 325
Joined: Tue Oct 20, 2015 3:38 pm

Re: Backtesting Results

Post by furnace » Fri Jan 18, 2019 11:49 pm

venkman wrote:
Wed Jan 09, 2019 11:51 pm
justsomeguy2018 wrote:
Wed Jan 09, 2019 7:59 pm
How do you get around cherry picking data to decide upon a "best" allocation (e.g. % to midcap, growth stocks, bonds, emerging markets, etc.)?
The "best" allocation is unknowable, because your investment horizon is just another time period on the data charts (that we don't have data for yet). The optimal strategy for the next 30 years will only be evident 30 years from now. Ex-ante, the optimal strategy is broad diversification and minimizing costs.
That's a good way of explaining it. Predicting the past is so easy, any historian can do it. But when it comes to the future, we'd have to turn to someone more knowledgeable - like an astrologer. Hopefully one with long hair who can chant :P

User avatar
Ben Mathew
Posts: 747
Joined: Tue Mar 13, 2018 11:41 am
Location: Seattle
Contact:

Re: Backtesting Results

Post by Ben Mathew » Sat Jan 19, 2019 12:30 am

Using recent performance to determine AA can lead you to overweight asset classes that are overpriced (a recent run-up in price = looks good in a backtest). Focusing on valuations and yields, in conjunction with long term performance (over many decades), can help you develop better forward-looking return estimates.

Post Reply