[Back testing programs]

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
Post Reply
Topic Author
pvguy
Posts: 27
Joined: Mon Sep 23, 2013 7:54 pm
Location: Austin, TX
Contact:

[Back testing programs]

Post by pvguy » Mon Sep 23, 2013 8:15 pm

I'm a long time reader of the bogleheads.org web site, and several discussions around portfolio backtesting, Fama-French factor analysis (e.g. factor loadings of small cap value ETFs) and asset class correlations inspired me to create some web based analytical tools for these purposes, which I made available at http://www.portfoliovisualizer.com/. The web based portfolio backtesting tool is essentially identical to Simba's spreadsheet in that it allows you to specify the asset allocation and compare the results against common lazy portfolios. There is also a version that allows you to enter specific ticker symbols (MF, ETF or stock) for the portfolio assets, in which case the results are typically limited based on the available history for each asset. In both cases the results assume yearly rebalancing.

User avatar
LadyGeek
Site Admin
Posts: 55793
Joined: Sat Dec 20, 2008 5:34 pm
Location: Philadelphia
Contact:

Re: [Back testing programs]

Post by LadyGeek » Mon Sep 23, 2013 9:09 pm

Welcome! pvguy has received permission from the Advisory Board to post a link to his website.

The back testing links are in the wiki: Simba's backtesting spreadsheets ( JonoJono1: Improve, Revise, Update, Debug Simba's Backtester )

This post was originally in Back testing programs, I moved it into a stand-alone thread.
Wiki To some, the glass is half full. To others, the glass is half empty. To an engineer, it's twice the size it needs to be.

User avatar
camontgo
Posts: 324
Joined: Tue Oct 27, 2009 1:46 pm
Location: Chicago, IL
Contact:

Re: [Back testing programs]

Post by camontgo » Mon Sep 23, 2013 9:17 pm

Wow....very nice tools. Thanks for posting!
"Essentially, all models are wrong, but some are useful." - George E. P Box

User avatar
GTF
Posts: 53
Joined: Thu Jun 20, 2013 8:13 pm

Re: [Back testing programs]

Post by GTF » Mon Sep 23, 2013 10:14 pm

I agree bookmarked :greedy

scone
Posts: 1457
Joined: Wed Jul 11, 2012 4:46 pm

Re: [Back testing programs]

Post by scone » Tue Sep 24, 2013 10:23 am

This thing is lots of fun. I'd suggest you also need to credit Trev H, Robert T, et.al. for their contributions. Are you looking for bug reports?
"My bond allocation is the amount of money that I cannot afford to lose." -- Taylor Larimore

Topic Author
pvguy
Posts: 27
Joined: Mon Sep 23, 2013 7:54 pm
Location: Austin, TX
Contact:

Re: [Back testing programs]

Post by pvguy » Tue Sep 24, 2013 5:35 pm

scone wrote:This thing is lots of fun. I'd suggest you also need to credit Trev H, Robert T, et.al. for their contributions. Are you looking for bug reports?
Thank you for the feedback, and I hope you find the tools useful. If there are any ideas or requests for additional features, or bug reports, those are all very welcome. For bug reports you can PM me on this forum or use the email address/contact form on the site.

User avatar
LadyGeek
Site Admin
Posts: 55793
Joined: Sat Dec 20, 2008 5:34 pm
Location: Philadelphia
Contact:

Re: [Back testing programs]

Post by LadyGeek » Tue Sep 24, 2013 6:54 pm

Hi,

I was wondering if you could update your factor analysis? An often overlooked aspect of software development is the user interface (UI), as it's the first thing your viewers see. If they don't understand your intentions, or it contains errors, all of your hard work goes for zilch. You never get a 2nd chance to make a first impression...

Getting the terminology consistent is very important. With that being said, could you retitle your factor results to align with the Fama-French variables? I'm not sure what "Market", "Small," and "Value" refer to. (I do, but I want to make a point here.)

Take a look at this wiki article: Fama-French three-factor model analysis, in the "Fama-French Parameters" table. That's the terminology I would expect.

Could you display the name of the data file you imported from Kenneth French's web page? It would provide a way to check your results. Getting the data aligned is somewhat tricky, as explained in Bill Bernstein's comment here. When is "live" data used vs. the library?

Your date validation is good, but it might be a little more flexible. IOW, allow me to enter 9/24/2013 instead of 09/24/2013.

BTW, I get an error for VTSMX, from 09/22/2013 to 09/23/2013, returns "Unexpected error: no data"
Wiki To some, the glass is half full. To others, the glass is half empty. To an engineer, it's twice the size it needs to be.

User avatar
Taylor Larimore
Advisory Board
Posts: 28494
Joined: Tue Feb 27, 2007 8:09 pm
Location: Miami FL

"Issues Related to Back Testing"

Post by Taylor Larimore » Tue Sep 24, 2013 7:28 pm

Bogleheads:

A word of caution:

Issues Related to Back Testing

Best wishes
Taylor
"Simplicity is the master key to financial success." -- Jack Bogle

Topic Author
pvguy
Posts: 27
Joined: Mon Sep 23, 2013 7:54 pm
Location: Austin, TX
Contact:

Re: [Back testing programs]

Post by pvguy » Wed Sep 25, 2013 1:55 am

LadyGeek wrote:Hi,

I was wondering if you could update your factor analysis? An often overlooked aspect of software development is the user interface (UI), as it's the first thing your viewers see. If they don't understand your intentions, or it contains errors, all of your hard work goes for zilch. You never get a 2nd chance to make a first impression...

Getting the terminology consistent is very important. With that being said, could you retitle your factor results to align with the Fama-French variables? I'm not sure what "Market", "Small," and "Value" refer to. (I do, but I want to make a point here.)

Take a look at this wiki article: Fama-French three-factor model analysis, in the "Fama-French Parameters" table. That's the terminology I would expect.

Could you display the name of the data file you imported from Kenneth French's web page? It would provide a way to check your results. Getting the data aligned is somewhat tricky, as explained in Bill Bernstein's comment here. When is "live" data used vs. the library?
Hi LadyGeek,

Thank you for the detailed feedback. I updated the factor analysis page per your suggestion to clarify the terminology, and I also linked the monthly factor returns file used from Kenneth French's web page. For the monthly factor returns I'm using the U.S. research returns data file since it has the risk free rate explicitly unlike the benchmark returns file. I'll update the factor analysis page later this week so that the user can choose between the research returns and benchmark returns to make that more explicit.
BTW, I get an error for VTSMX, from 09/22/2013 to 09/23/2013, returns "Unexpected error: no data"
The "no data" message occurred when the selected data range was insufficient for regression analysis (no full month data for September yet). I clarified the error message to indicate that at least four data points are required for the regression analysis, i.e., 05/01/2013 to 08/31/2013 would be the current minimum interval for four monthly data points (not that the results for a minimum period would necessarily be very useful). In addition to error message clarifications I also fixed a browser compatibility bug impacting non-WebKit based browsers that was reported by another forum member.

User avatar
DueDiligence
Posts: 99
Joined: Mon Sep 09, 2013 3:57 pm

Re: [Back testing programs]

Post by DueDiligence » Wed Sep 25, 2013 7:25 am

Very nice, valuable, and much appreciated!
"Personal preferences, circumstances, and abilities affect portfolio construction in a profound manner..." David Swensen

User avatar
InvestorNewb
Posts: 1609
Joined: Mon Sep 03, 2012 11:27 am

Re: [Back testing programs]

Post by InvestorNewb » Wed Sep 25, 2013 10:28 am

This is really cool - thanks for sharing.

If someone started with $10,000 in 1972 with VTI, VXUS, and VNQ - at a ratio of 60/23/17, (approximating my own portfolio), they would have $637,685 today. I hope the numbers are accurate. :happy
My Portfolio: VTI [US], VXUS [Int'l], VNQ [REIT], VCN [Canada] (largest to smallest)

technovelist
Posts: 2918
Joined: Wed Dec 30, 2009 9:02 pm
Contact:

Re: [Back testing programs]

Post by technovelist » Wed Sep 25, 2013 10:54 am

Very cool! Would it be possible to add foreign currencies, e.g., the Swiss Franc, and metals, e.g., silver, to the available asset types? That would make it easier to analyze the results of funds like the "Permanent Portfolio" fund (PRPFX).
In theory, theory and practice are identical. In practice, they often differ.

User avatar
LadyGeek
Site Admin
Posts: 55793
Joined: Sat Dec 20, 2008 5:34 pm
Location: Philadelphia
Contact:

Re: [Back testing programs]

Post by LadyGeek » Wed Sep 25, 2013 3:46 pm

pvguy wrote:The "no data" message occurred when the selected data range was insufficient for regression analysis (no full month data for September yet). I clarified the error message to indicate that at least four data points are required for the regression analysis, i.e., 05/01/2013 to 08/31/2013 would be the current minimum interval for four monthly data points (not that the results for a minimum period would necessarily be very useful). In addition to error message clarifications I also fixed a browser compatibility bug impacting non-WebKit based browsers that was reported by another forum member.
Much better, but I think you left out a key word - "monthly" - from the error message.
The selected date range contains insufficient data. At least 4 data points are required for the regression analysis (1 selected).
One final nit - R^2 isn't defined. No need to define, just state that it's the correlation coefficient.
Wiki To some, the glass is half full. To others, the glass is half empty. To an engineer, it's twice the size it needs to be.

bargainhuntingking
Posts: 211
Joined: Wed May 14, 2008 3:48 am

Re: [Back testing programs]

Post by bargainhuntingking » Thu Sep 26, 2013 4:51 pm

Thank you for this.

If only I had invested $10k in Emerging Markets in 1972. ;)

gtwhitegold
Posts: 433
Joined: Fri Sep 21, 2012 1:55 pm

Re: [Back testing programs]

Post by gtwhitegold » Mon Jun 02, 2014 2:45 am

pvguy, I really appreciate your website and the effort that you put into it. I would like to know if you have any intentions of adding the Quality Minus Junk (QMJ) and Bet Against Beta (BAB) factors to your website if Andrea Frazzini or Lasse Pedersen decide to update them?

Allen

User avatar
ThePrune
Posts: 909
Joined: Wed Nov 10, 2010 9:38 am
Location: Midland, MI

Re: [Back testing programs]

Post by ThePrune » Mon Jun 02, 2014 9:09 pm

pvguy, a very nice website and suite of tools. Please consider the following comment purely as constructive criticism.

A basic "reality check" for any Monte Carlo model that uses a Normal Distribution is to compare projected mean and median values against the values calculated using the exact equations, using the same starting distribution mean and standard deviation in both. My personal experience with Monte Carlo models and large numbers of simulations (30,000+) is that these come out quite close. You can find the closed form equations in the Appendix to the article The Misuse of Expected Returns by Hughson, Stutzer and Yung. This is a great article in its own right!

When I used your Monte Carlo Tool (Normal Distribution, no additions or subtractions, 30+ year projections), the mean (average) projected values and the median values seemed to always be high. Not by a lot, but by more than I was expecting. This may be an artifact of the limited number of simulations (the web page indicated that only 10,000 were used).

I suggest you explore this on your own, and see if running larger numbers of simulations brings the Monte Carlo results into better agreement with theory.

Also, I'd like to exchange some useful Monte Carlo articles with you and get some of your favorites in return. I'm always looking for Monte Carlo references that I might have overlooked. I'll initiate this via PM.

Art
Investment skill is often just luck in sheep's clothing.

Topic Author
pvguy
Posts: 27
Joined: Mon Sep 23, 2013 7:54 pm
Location: Austin, TX
Contact:

Re: [Back testing programs]

Post by pvguy » Tue Jun 03, 2014 3:36 pm

gtwhitegold wrote:pvguy, I really appreciate your website and the effort that you put into it. I would like to know if you have any intentions of adding the Quality Minus Junk (QMJ) and Bet Against Beta (BAB) factors to your website if Andrea Frazzini or Lasse Pedersen decide to update them?

Allen
I would certainly like to include those data sets if they were be updated. Data for both QMJ and BAB currently ends in 2012, which is why I only included a link to those factor data sources in the FAQ. Ideally it would be nice to just have a good source for all the raw data so you could calculate the factor data automatically.

heavywhether
Posts: 26
Joined: Sat May 24, 2014 8:59 pm

Re: [Back testing programs]

Post by heavywhether » Wed Jun 04, 2014 12:28 pm

Beautifully executed - well done - and thanks.
I hope you might consider including a write-up on interpreting the output, and/or, pointers to other sources of articles that might serve as a primer, at some point in the future.

Post Reply