Simba's backtesting spreadsheet [a Bogleheads community project]

[gummy]

Mamma mia!
Simba, you is so fast!
I wuz still workin' on rev5j when you done finished rev5k with the probability stuff included.

About Ito, the lognormal distribution of prices at time T (starting with price Po) is:

I just stuck in the parameters r, s and Po, using your data.

[jms969]

Mamma mia!
Simba, you is so fast!
I wuz still workin' on rev5j when you done finished rev5k with the probability stuff included.

About Ito, the lognormal distribution of prices at time T (starting with price Po) is:

I just stuck in the parameters r, s and Po, using your data.

WOW I do love having a PhD in mathmatics on the board. My little engineering background just does not hold a candle to you

[jms969]

jms - I updated the spreadsheet with gummy's changes.

Latest version rev5k can be downloaded here

Honestly I can't say I understand everything he did there. I am still reading about Ito's calculus and sortino ratio.

Yeah Gummy gets out ahead of me pretty quickly as well (but that is very good!!!). But what an amazing group of people here!!! I learn more here than I ever dreamed.


The following site is Aswath Damodaran's, Professor of Finance and David Margolis Teaching Fellow at the Stern School of Business at New York University. He has an amazing array of spreadsheets and business data that is available for your use. It is free and regularly updated.

Enjoy...

http://pages.stern.nyu.edu/~adamodar/New_Home_Page/

Peace,

JMS

[gummy]

jms969:
Thanks for the link to all them thar spreadsheets.
I reckon I gotta steal some things to add to this collection:**
http://www.gummy-stuff.org/stock-charts.htm

** Thievery is my favourite hobby ...

[jms969]

jms969:
Thanks for the link to all them thar spreadsheets.
I reckon I gotta steal some things to add to this collection:**
http://www.gummy-stuff.org/stock-charts.htm

** Thievery is my favourite hobby ...

Great collection of charts!!!

We are one sick bunch

[edge]

I think the past few exchanges illustrate the need for a collaborative tool to help streamline updates and availability - google spreadsheets could work here. If any more assistance is required to understand how to set this up I could volunteer to help.

In any case, the changes are pretty cool.

[jms969]

I think the past few exchanges illustrate the need for a collaborative tool to help streamline updates and availability - google spreadsheets could work here. If any more assistance is required to understand how to set this up I could volunteer to help.

In any case, the changes are pretty cool.

Agreed!!!

But one person, Simba, (or small committee) still needs ultimate say over what is and is not accepted for the release version... We also need to reduce the load on Simba...

...and we of course need to be able to suck the release version off of google ss into excel.

[gummy]

But one person, Simba, (or small committee) still needs ultimate say over what is and is not accepted for the release version...

Agreed! Now if we can only reduce the pressure (and sweat) on Simba.

P.S.
The google spreadsheets can't handle all of Excel's functionality ... but it's cheeep, eh?

If Simba agrees, I can collect revisions, stick them on my website then PM Simba so he can check them out and decide it they're acceptable.

[norm]

gummy wrote:
After I entered the % of my allocations nothing appeared on the graphs. Am I missing a step?
Do your allocations add to 100%?

Yes they do. In addition on Row 45 under Total it shows Error and under CAGR it shows #VALUE!.

[edge]

But one person, Simba, (or small committee) still needs ultimate say over what is and is not accepted for the release version...

Agreed! Now if we can only reduce the pressure (and sweat) on Simba.

P.S.
The google spreadsheets can't handle all of Excel's functionality ... but it's cheeep, eh?

If Simba agrees, I can collect revisions, stick them on my website then PM Simba so he can check them out and decide it they're acceptable.

The permissions/sharing policy available in the Google SS should support the kind of collaboration that JMS was talking about. Gummy's point needs addressing - I think the lack of functionality may have more to do with the fact that the product is very new and definitely still beta. The graphing capability is still very beta in terms of features.

Which critical features are missing (Gummy)?

[SoonerSunDevil]

Gummy/Simba,

Could either of you explain what the graph showing the "Probability of achieving less than 1000$P in 15 years" means?

I'm sorry that my mathematical abilities are limited; I only had Calculus I and II during college

Thanks,

John

[simba]

Thanks to Gummy - I updated the spreadsheet to calculate Sortino ratio.

The formula I used is

=((AVERAGE(AB5:AB39))-(AVERAGE(Data_72_06!$V$5:$V$39)))/STDEV(IF(AB5:AB39<AVERAGE(Data_72_06!$V$5:$V$39),AB5:AB39))

Need to enter that formula with Ctrl-Shift-enter.

Latest version rev5m can be downloaded here

[sterjs]

I posted a link to the sheet on another messageboard and one poster thinks that the sheet's calculation of Sharpe Ratios is off:

"yea the sharpe ratio calculations don't take compounding into effect

they use Average(portfolio returns)-Average(tbill returns) to get the excess returns. it should be subtracting the tbill return from the portfolio return in each time period and then calculating the compound returns.

their sharpe ratios are historical underestimates in this case (sheets SP-72-06 and SP-85-06). "

[norm]

Simba,

After I entered the % of my allocations nothing appeared on the graphs. Am I missing a step? My total allocation comes to 100%

[gummy]

Could either of you explain what the graph showing the "Probability of achieving less than 1000$P in 15 years" means?

The magic formula I gave above, namely

uses the historical mean return r and standard deviation s and the initial portfolio value Po.

The spreadsheet calculates the probability (based upon these parameters) that, in T years, your portfolio will be less than P.
Actually, the horizontal axis is in units of $1K so, for the example shown in the left chart,
there's a 50% probability that your portfolio will be less than $100K in T = 15 years.

Change the allocation and/or the time and get different probabilities.
That's assuming you believe in Ito's magic equation.

After I entered the % of my allocations nothing appeared on the graphs. Am I missing a step? My total allocation comes to 100%

To relieve Simba I'll take a try at explaining. Just email the "faulty" spreadsheet to:
pjponzo@golden.net

the sharpe ratio calculations don't take compounding into effect ... they use Average(portfolio returns)-Average(tbill returns) to get the excess returns. it should be

That's the "standard" definition. Indeed, the numerator is the "Expected" excess which is the same as the Average excess which is the same as the difference in the two Averages.

[Eric White]

Simba:

ERROR FOR 85-06 CALCULATIONS:

Cell Z3 on sheet Returns_85_06 uses the following formula:
=Portfolio!$K$28

Cell AA3 on sheet Returns_85_06 uses the following formula:
=Portfolio!$K$30

It looks like Cell AA3 needs to use the formula =Portfolio!$K$29, and the remaining cells in the row need new transposed links.

ERROR CHECKING REQUEST
Also, if you can eliminate this error check in the calculations and instead use simple 100% summary visual sanity checks, that reduces rework for each rev to enable efficient frontier Excel Solver calculations. I have redone the Solver work twice now and it's been obsoleted a third time. I will not be supporting efficient frontier additions moving forward if we can't find a solution that will be incorporated or planned with compatibility in mind.

CHANGE PLANNING
Although I'm Pi Tau Sigma and Tau Beta Pi and enjoy mathematical acrobatics, simplicity is our friend until the basic portfolio options and datasets get nailed down. It will let us focus on data quality and basic calculation integrity. I think after the portfolio request additions and resulting structures settle down we should initiate discussions (and polls!) around metrics that are most useful to Diehards overall. I'm a little worried that as we add metrics into the tool, we may start to lose the forest for the trees.

Of course, I'm a guilty party in this process as well, shooting for the efficient frontier calcs too early Portfolio Process Capability (Cp), Kurtosis, Monte Carlo analysis, discrete event simulations, and system dynamics models should all have their day in the sun, just maybe not yet (insert your "horse before the cart" jokes here).

Thanks for all the hard work Simba! This is the foundation for expansion work down the road.

Cheers,
Eric

[Eric White]

Do fellow Diehards feel it would be useful to add costs for each investment choice (absolute return - cost - inflation = real investor return) and include them in calculations? Sorry if it's already in there...

In addition, does anyone have datasets we can add for the following asset subclasses:

High Priority:
- Stable value/fixed (e.g. GICs, etc. offered via insurance to 401ks/403bs)
- Currencies? (to separate currency effect from International, European, Pacific, and EM returns; similar to inflation for separating investor returns)

Medium Prioriy:
- Int SV (no current Vanguard availability; ETF availability)
- EM SV (no current Vanguard availability; future ETF availability)
- Int REIT (no current Vanguard availability; mutual fund & ETF availability; new asset subclass may not have sufficient data)

Low Prioriy:
- Private Equity (no current Vanguard availability; future ETF availability?)
- International Bonds (no current Vanguard availability; mutual funds available)
- EM Bonds (no current Vanguard availability; mutual funds available)

Cheers,
Eric

[BornInBoston]

Amazing job, and superb collaberative work on this spreadsheet.

Question: is there any rebalancing involved in these calculations? Can't seem to find a reference to rebalancing. Sorry if I missed it. Lee

[Cb]

Do fellow Diehards feel it would be useful to add costs for each investment choice (absolute return - cost - inflation = real investor return) and include them in calculations? Sorry if it's already in there...
Eric

I agree with you wrt to expene ratios, Eric. At present, in many (most?) of the asset classes, there is a bit of both. The 1972 - 199x returns are most often from MSCII-Barra or CRSP or another source that does NOT include typical fund ER's, and the returns used after the launh of the corresponding Vanguard fund in 199x - 2006 DO include the fund's ER.

I think it would be preferable to subtract the Vanguard funds's Investor Class ER from the earlier returns of the corresponding raw index also.

My preference would be to continue using nominal returns however, rather than subtracting one inflation index or another.

Cb

[gummy]

Question: is there any rebalancing involved in these calculations?

Yup! Annual rebalancing to maintain the prescribed allocations.

[edge]

Maybe I am missing something but it seems that nearly every calcuation of correlation is using a wrong column or all the columns in many of the correlation calcutaions are wrong in 5m.

[gummy]

The latest-and-greatest is available here:
http://www.gummy-stuff.org/Simba.htm

Simba has made a number of changes.
Check it out.

[edge]

I think the correlation numbers are wrong in this one too :p (?)

[simba]

I think the correlation numbers are wrong in this one too :p (?)

edge - I sent a PM to gummy this morning to update the SS on his website.
The latest version rev5p should be okay.

Regards,
Simba

[johnb]

Any chance of an unprotected version? I'm not able to change the blue cells on the newest version of the file. I'm on a Mac.

[gummy]

Simba's rev5p is now here:
http://www.gummy-stuff.org/Excel/Simba-Backtest.xls

The "portfolio" sheet is unprotected.

[johnb]

Thanks Gummy!

[edge]

I think the data source for EM has a developed market index listed - this does not seem correct.

[simba]

I think the data source for EM has a developed market index listed - this does not seem correct.

edge - thanks for pointing it out but that's just a typo error.

The EM returns for the duration 1972-87 were compiled by Robert T and I listed the source of the returns (These are not true EM returns either, they are a combination of 50% Intl Value & 50% Intl Small combination, check the IFA website for more info).

MSCI EM returns are available from 1987 and VG EM returns are available from 1995.

Regards,
Simba

[edge]

Simba,

Ya, I checked it myself and the actual data seemed fine

[gbs]

Simba,

I think it will be interesting to add the results for the un-rebalanced portfolio as well.

I did some testing on my own and in most cases for an all stock portfolio the returns as well as stdev go up.

gbs

[fundtalker123]

Ok, math-stat wizzes, I've got a question:

Suppose you've got this marvelous spreadsheet going and you're compulsively diddling around your asset allocation (heaven forbid, maybe you're even started messin' with SCV, EM, REITS, and Commodities). After diddling for 5 hours you get your Std. Dev. to PLUMMET from 13.8% to 12.8% and, at the same time, your CAGR SKYROCKETS from 10.8% to 11%!

How do you judge whether this change is statistically significant? It seems like some statistical test is in order. If there's a 55%/45% chance that my new portfolio is better/worse than my old one, I probably wouldn't bother making a change. But if it's 80%/20%, that might get me thinkin'. Any way to estimate this?

[BornInBoston]

When I try to put 100% in TSM (first item in asset list) I get error messages. Why won't the SS let me do this? Other combinations work fine, by the way.

[simba]

Simba,

I think it will be interesting to add the results for the un-rebalanced portfolio as well.

I did some testing on my own and in most cases for an all stock portfolio the returns as well as stdev go up.

gbs

gbs - Thanks for the suggestion

The latest version rev6a can be downloaded here.

Some additions include (Thanks Gummy. Your help is very much appreciated).
- One can now compare 5 different portfolios
- Chart shows only the selected time periods
- SS shows returns for both rebalanced & un-rebalanced portfolios.

Regards,
Simba

[gbs]

Thanks Simba!

gbs

[fundtalker123]

Countour plots of sharpe ratio and compound annual return calculated from the 35 yr data, comparing various mixes of bonds and stocks, with various international, REIT, small cap value tilts, or with slice-dice are posted here:

http://www.gummy-stuff.org/FundTalker.htm

(the plots were done in Matlab and I don't know how to incorporate that in the Excel spreadsheet)

[edge]

Hi,

I think some of the "real" returns are wrong, looks like one of the columns used in the equation are wrong.

It would be better if the inflation column was $'ed so the equation could just be copied/pasted.

A minor issue, one of the graph legend has "rebalanced" portfolio when it should be "unrebalanced"

Edited:

P2 P3 P4 P5 in the portfolio comparision section

Real returns are the same as nominal returns.

[sterjs]

Interesting Result: Commodities don't improve Sharpe Ratios much for diversified portfolios, but they improve Sortino Ratios substantially.

I also messed around with the allocations and produced a portfolio that never had a losing year:

65% ST Treasuries
15% Commodities
10% U.S. Small Value
05% EAFE Value
05% Emerging Markets

CAGR 10.01%
Std. Dev. 4.62%
Sharpe 0.87
Sortino 2.97

P.S. The Intl Value looks strange to me... was there really only a .27% Value premium in EAFE from 72-2006?

[simba]

Hi,

I think some of the "real" returns are wrong, looks like one of the columns used in the equation are wrong.

It would be better if the inflation column was $'ed so the equation could just be copied/pasted.

A minor issue, one of the graph legend has "rebalanced" portfolio when it should be "unrebalanced"

Edited:

P2 P3 P4 P5 in the portfolio comparision section

Real returns are the same as nominal returns.

Edge - Thanks for the correction. The latest version rev6b can be downloaded here

[simba]

Interesting Result: Commodities don't improve Sharpe Ratios much for diversified portfolios, but they improve Sortino Ratios substantially.

I also messed around with the allocations and produced a portfolio that never had a losing year:

65% ST Treasuries
15% Commodities
10% U.S. Small Value
05% EAFE Value
05% Emerging Markets

CAGR 10.01%
Std. Dev. 4.62%
Sharpe 0.87
Sortino 2.97

P.S. The Intl Value looks strange to me... was there really only a .27% Value premium in EAFE from 72-2006?

sterjis,

I went back and verified the data for Intl Value and looked up VG's SEC filing. The data matched but Vanguard/Trustees' Equity Fund-International Portfolio became VG Intl Value (VTRIX) in 1997 so I went back and used the MSCI EAFE Value from 1975-1996 and VTRIX from 1997-2006 (instead of 1984-2006 that I was using earlier) now the value premium shows upto about 2% which is about the same for US Mkt as well.

Updated revision rev6c can be downloaded here

Regards,
Simba

[jms969]

Here is a nice link to some excellent excel add-ins.

I am trying to apply the linear programming example to your spreadsheet to be able to change constraints and have it find an optimal solution. Solver only goes so far

http://www.me.utexas.edu/~jensen/ORMM/frontpage/jensen.lib/index.html

PS. Gummy you should like these

[gummy]

Alas, I can never seem to run Simba's spreadsheets after downloading.

Hmm ... gotta take a look at them add-ins.

[sterjs]

1985*-2006 Mid Cap Value, Growth

MC Value
0.32
0.18
-0.02
0.25
0.23
-0.16
0.38
0.22
0.16
-0.02
0.35
0.2
0.34
0.05
0.02
0.19
0.02
-0.1
0.38
0.24
0.13
0.2

MC Growth
0.32
0.18
0.03
0.13
0.32
-0.05
0.47
0.09
0.11
-0.02
0.34
0.18
0.23
0.18
0.51
-0.12
-0.2
-0.27
0.43
0.16
0.12
0.11

* MC Blend returns were used for 1985.

Source 1

Source 2

[simba]

Latest version rev6d can be downloaded here.

This includes Mid-Cap growth & Mid-Cap Value.

[Peppe]

Can you add VG extended market or other S&P completion type index? My 401k doesn't have total stock market, but has S&P 500 and the Russell small cap completion index. I used mid cap blend in place of it, but the completion index is more volatile than the VG mid cap.

Thanks for all your hard work this spreadsheet is fun to tinker with and is very helpful.

[simba]

Can you add VG extended market or other S&P completion type index? My 401k doesn't have total stock market, but has S&P 500 and the Russell small cap completion index. I used mid cap blend in place of it, but the completion index is more volatile than the VG mid cap.

Thanks for all your hard work this spreadsheet is fun to tinker with and is very helpful.

Peppe,

I added VG extended Mkt returns to the spreadsheet. Latest version rev6e can be downloaded here

Regards,
Simba

[Zapped]

Has anyone sucessfully loaded the spreadsheet in OpenOffice? I'm using release 2.0.2 and even after enabling macros, it's not working correctly. I can't edit the blue boxes (OO says their protected cells), and the two graphs on the Portfolio sheet ("Portfolio Growth (Nominal) 19XX-2006") have nothing drawn in them.

I have access to Excel but I'm not a registered owner myself so I prefer to use OpenOffice. If anyone has experience getting Excel spreadsheets to work in OO I'd appreciate a conversion of Simba's latest rev6e.

[kaesler]

Regarding OpenOffice I think I found that if I used Excel to remove the cell locking I could then use it in OpenOffice. Give that a try?

[CyberBob]

I can't edit the blue boxes (OO says their protected cells)

Just select Tools --> Protect Document and uncheck Sheet...

Bob

[LH]

I did as bob said, and OO.org spreadsheet allowed me to enter values, and the spreadsheet generated a graph and everything.

Often times, as with gummys spreadsheets, openoffice does not work with excel macros.

Also, I am using the Novell form of openoffice, which has extra excel macro support versus the regular openoffice. Often times, even with Novell openoffice, gummys spreadsheets do not work : ( But they are specifically working towards increasing the macro capability as I understand it.

If gummy and simba excel macro type spreadsheets are what you are trying to run using openoffice.org, I recommend downloading the novell form of openoffice.org from below. You will get less macro errors.

http://www.novell.com/products/desktop/ ... s/ooo.html

http://www.novell.com/news/press/novell ... perability